AI Recommendation Systems
Ranking that has to beat “most popular” in a live test before anyone gets to call it a success.
Measure the lift ↗
What is a recommendation system?
A recommendation system decides what to show next — products, articles, jobs, tracks, shifts — by scoring a large catalogue for one person in the time it takes a page to render. In practice it’s two problems stacked: generating a shortlist of candidates cheaply, then ranking that shortlist well.
The only number that matters is lift over what you do today, and “most popular this week” is a much stronger opponent than most teams expect. So the baseline ships first, offline metrics are treated as a filter rather than proof, and the model is judged in a live A/B test with guardrails — because a ranker that lifts clicks while quietly shrinking basket size isn’t a win.
What a recommender build includes
Trustworthy event data
Impressions, clicks and conversions tracked consistently. Without knowing what was shown and not clicked, nothing can be trained or honestly measured.
A popularity baseline
Live in the first fortnight. It usually captures a surprising share of the available value and gives every later model something real to beat.
Candidate generation
A cheap first pass — co-visitation, embeddings, category rules — that cuts a large catalogue to a few hundred items in single-digit milliseconds.
Ranking model
A learned ranker over behavioural and item features, served inside a p95 budget and versioned so any result can be traced back to a model.
Cold-start rules
Explicit behaviour for new users and new items, so a fresh catalogue addition isn’t effectively invisible for its first month.
A/B harness
Traffic splitting, guardrail metrics and a stopping rule agreed before the test starts, so results get read rather than argued over.
How a recommender gets proven
Audit the events
We check whether your tracking can support ranking at all and fix the gaps first. It’s usually the longest step and always the one people want to skip.
Ship the baseline
A popularity or rules-based recommender live behind the same interface the model will later use, with measurement already running against it.
Train the ranker
Offline evaluation on historical sessions to shortlist approaches, with the standing caveat that offline gains routinely shrink once real users are involved.
A/B against the baseline
A real split test with pre-agreed primary and guardrail metrics, run long enough to survive a weekend, a payday and a promotion.
Keep, tune or roll back
If the lift holds we harden and widen it. If it doesn’t, we say so and the baseline stays — a far cheaper outcome than pretending otherwise.
AI Recommendation Systems FAQ
Typically $20k to $60k to a tested first version, and more when event tracking has to be rebuilt along the way. We price the data work separately, because on a lot of projects it’s half the engagement.
The tools we build with
Everything is built so the model can be switched off in one click and compared with the dumb baseline it replaced, because that comparison is the whole product.
Modelling
Python
PyTorch
scikit-learn
PandasFeatures & events
Kafka
PostgreSQL
Redis
SnowflakeServing
FastAPIpgvector
Docker
AWSMeasure
Metabase
Grafana
Weights & Biases
OpenTelemetryRelated work
Related reading
Want to know what better ranking is worth?
Tell us your catalogue size and weekly traffic. We’ll come back with a baseline plan, a p95 latency target and the A/B test we’d run against it.