0

Design a nearby restaurant recommendation system

by Frontier Interview· 1 day ago· ⑂ 0 remixes

Question: Design a nearby restaurant recommendation system

Companies: Uber

Difficulty: Hard

Problem statement

Recommend restaurants a user is likely to order from right now, given location, time of day, delivery time and their history.

Problem framing

Business goal. More completed orders per app open from a food delivery home feed, without hurting delivery reliability or starving new restaurants. The feed is the main entry point: most orders start from a restaurant card on the home page, not from search.

ML task. When a user opens the app (or scrolls, or changes the address), rank the restaurants that can deliver to that address right now. The core prediction is P(order from restaurant r in this session | user u, context c, r shown at slot k). Context is delivery address, local time and day, weather, live ETA and fee, and whether the store is open and has couriers nearby.

Constraints (assumed numbers, state them in the interview).

  • 30M daily active eaters, ~60M feed loads a day, peak ~5k feed requests/s (dinner rush, 6-8 pm local time, which is spread across time zones).
  • Feed p99 latency 200 ms end to end; the ML part gets ~80 ms.
  • Per request, 300-3,000 restaurants can deliver (dense city vs suburb); the user sees ~20 in the first screens.
  • The deliverable set changes every minute: stores open and close, pause when the kitchen is busy, and ETAs move with courier supply. A recommendation for a store that cannot deliver now is worse than no recommendation.

ML objective vs business metric. We train for order probability per impression (a logloss on session-level order labels), then rank by expected value: score = P(order) * (1 + a * normalized_basket_value) - b * eta_penalty + exploration_bonus. The business metrics are session conversion (sessions with an order / sessions), orders per DAU, and GMV. Guardrails are actual delivery time, late-order rate, cancellations, and the share of orders going to new or small merchants. Clicks are an auxiliary signal only. Optimizing CTR alone promotes good-looking photos of stores 50 minutes away.

Data and labels

Sources.

  • Impression logs: request id, user, address geohash, timestamp, slot, carousel, every restaurant shown with the features served at that moment (logged at serve time).
  • Engagement: card click, store page view, add to cart, checkout, order placed, order cancelled.
  • Orders: items, basket value, promised ETA vs actual delivery time, rating, refunds, reorders.
  • Restaurant catalog: cuisine, menu, price level, photos, hours, prep time history, ratings.
  • Marketplace state: live courier supply per hexagon, current ETA and delivery fee per (store, address), store pause events.
  • Search queries and the cuisine filter chips the user taps.

Label. Positive = the user placed an order from restaurant r within the same session (30 min of inactivity closes it) after seeing r in the feed. Attribution goes to the last feed impression of r before the order, including orders that went through the store page after a search for the same store name. Negatives are impressions in sessions with no order from r. Auxiliary labels: click (instant), add to cart, and "order not cancelled and rated >= 4" (known 1-2 hours later).

Delay. Order labels close 30 minutes after the session; quality labels need up to 24 h. Daily training uses data that is at least 24 h old so every label is final.

Imbalance. Click rate ~4% per impression, order rate ~0.4%. Keep all positives, downsample negatives 10x, and correct the predicted probability: p = p' / (p' + (1 - p') / w) with w = 0.1. Calibration matters because the score multiplies P(order) by value terms.

Leakage traps.

  • Restaurant rating, order counts and "popular now" must be point-in-time. A daily snapshot joined by date leaks today's orders into the features of today's impressions. Use the serve-time logged features or an as-of join in the feature store.
  • The live ETA after the order is placed reflects the order itself. Use the ETA quoted at impression time.
  • Stores that were closed or paused but appeared in logs (bugs, scheduled orders) are not real negatives. Drop them.
  • Reorders from the "Order again" carousel convert at many times the base rate. Keep the carousel id as a feature, or the model learns that "was ordered yesterday" means "show first" everywhere.

Sampling and splits. Train on 28 days and validate on the next day, never randomly, because menus, prices and promos drift weekly. Report the dinner peak as its own validation slice. Also keep a 1% exploration bucket (randomized order within the top 50) for unbiased evaluation and position-bias estimates.

Features

User (offline, daily, plus real time).

  • Cuisine and price affinity: order share by cuisine over 7/30/180 days, with time decay.
  • Restaurant-level history: number of past orders from r, days since the last one, last rating given to r.
  • Time-of-day habits: breakfast/lunch/dinner/late-night order shares, weekday vs weekend.
  • Address type: home vs work (from the saved label and from order times). One user is two contexts.
  • Sensitivity: mean basket value, share of orders with a promo, share with a delivery fee above $3, tolerance for ETA (median ETA of past orders).
  • Session (real time, from the event stream): stores viewed and cuisines tapped in the last 10 minutes, search queries in this session, items left in the cart.

Restaurant (offline daily, plus near real time).

  • Cuisine, price level, menu embedding (from item names and descriptions), photo quality score.
  • Quality: Bayesian-smoothed rating, cancel rate, late rate, refund rate over 30 days.
  • Popularity: orders in the last 1 h / 24 h in this city zone (streaming counters, 1-minute freshness).
  • Prep time: p50/p90 over the last 2 h.
  • Age of the store on the platform (for cold start).

User x restaurant x context (computed at request time).

  • Quoted ETA, delivery fee, distance, and the ETA relative to the user's own median ETA.
  • Open now, minutes until close, paused flag, busy multiplier.
  • Active promo for this user, and the promo depth.
  • Two-tower similarity: user embedding dot restaurant embedding.
  • Local time, day of week, holiday, weather (rain raises delivery demand and ETAs).

Freshness and storage. A feature store with an offline store (training, as-of joins) and an online key-value store (p99 < 10 ms for a batch read of 1,000 keys). User history: daily batch. Session and popularity counters: Flink jobs, under 1 minute old. ETA and fee: called live from the ETA and pricing services, cached 60 s per (store, address hexagon).

Model

Baseline. Filter to deliverable and open stores, then sort by popularity_in_zone * rating_prior * exp(-eta / 30 min), and put the user's 5 most recent stores in an "Order again" row. This is hard to beat at first and is the fallback when the ranker is down.

Stage 1: candidate generation (~2,000 to ~400). No ANN over the whole catalog is needed: the hard filter is geographic. A geo index (H3 hexagon resolution 8 to the set of stores that deliver there) returns all deliverable stores, pre-filtered by open/paused state from a cache. Then merge several sources:

  • Two-tower model (user tower: history and context; restaurant tower: content and quality), scored with a dot product over the deliverable set, top 250.
  • Past orders and saved favorites (all).
  • Session intent: stores matching cuisines viewed or searched in the last 10 minutes.
  • Zone popularity and new stores (an exploration quota of ~20). Two-tower loss: sampled softmax on (user, ordered store) pairs, with in-batch negatives and a log-Q correction for popularity, plus extra negatives from the same zone so the model does not just learn geography.

Stage 2: ranking (~400 to ordered list). A multi-task deep model (DLRM-style: embeddings for user, store, cuisine and zone; dense features; feature crosses via DCN-v2 layers) with heads:

  • P(click), P(order | impression), P(good order | order): binary cross-entropy for each, weighted 1 : 3 : 1 in the total loss.
  • Training on impressions only means the model learns P(order | shown). Position bias is handled with a shallow position tower whose logit is added in training and set to a fixed slot at serving. GBDT (LightGBM, ~300 features) is the first production ranker. Move to the DNN once there are enough IDs and session sequences that embeddings beat hand-made counts (usually +1-3% relative in session conversion).

Stage 3: re-ranking and page assembly.

  • Expected-value score as in Problem framing, with a and b set by online tuning.
  • Hard rules: remove stores that just paused, cap ETA at the user's limit, and dedupe chains (no three locations of the same chain in the top 10).
  • Diversity: no more than 2 consecutive stores of one cuisine (MMR-style penalty).
  • Fairness to new stores: a guaranteed small quota, and an uncertainty bonus (Thompson-style) on stores with under 50 orders.
  • Sponsored slots come from the ads auction in fixed positions; they are scored by the same P(order) so low-relevance ads lose.

Evaluation

Offline.

  • Primary: session-level NDCG@10 and recall@10 with the ordered store as the relevant item, on the next-day holdout. Weight by session, not by impression, so heavy users don't dominate.
  • Per head: AUC and normalized logloss. Expected calibration error for P(order) by decile, and per city and per daypart. Miscalibration breaks the expected-value score.
  • Candidate generation: recall@400 of the eventually ordered store (target > 97%; misses are mostly new stores and searches by name).
  • Counterfactual check on the 1% randomized bucket with inverse propensity scoring, since logged data only contains what the old model showed.
  • Slices: new users (< 3 orders), new stores, suburbs with < 50 deliverable stores, breakfast and late night, rain.

Online A/B. Randomize by user (not by request, so the feed is stable), run 2 weeks to cover two weekend cycles, with about 5% of users per arm.

  • Primary: session conversion and orders per user per week.
  • Secondary: GMV per user, 7-day and 28-day retention, time to first click.
  • Guardrails: actual delivery time and late-order rate (ranking can shift demand to busy kitchens), cancellation rate, order rating, concentration of orders on top 1% of stores, new-store order share, feed latency p99.
  • Marketplace interference: if treatment sends many more users to the same popular kitchens, control's ETAs get worse too. For changes that move a lot of demand, use a switchback test by city and time window, or a city-level test.

Serving and monitoring

The request path, from opening the app to the ranked feed:

{"type":"excalidraw","elements":[
 {"type":"rectangle","id":"app","x":0,"y":0,"width":280,"height":100,"label":{"text":"App: open feed\n(user, address, time)"}},
 {"type":"rectangle","id":"feed","x":360,"y":0,"width":280,"height":100,"label":{"text":"Feed service"}},
 {"type":"rectangle","id":"geo","x":720,"y":0,"width":280,"height":100,"label":{"text":"Geo index\nH3 cell to\ndeliverable stores"}},
 {"type":"rectangle","id":"avail","x":1080,"y":0,"width":280,"height":100,"label":{"text":"Availability cache\nopen, paused,\nETA, fee"}},
 {"type":"rectangle","id":"cand","x":1080,"y":180,"width":280,"height":100,"label":{"text":"Candidate merge\ntwo-tower, past orders,\nsession, popular, new"}},
 {"type":"rectangle","id":"fs","x":720,"y":180,"width":280,"height":100,"label":{"text":"Feature store\nonline lookup"}},
 {"type":"rectangle","id":"ranker","x":360,"y":180,"width":280,"height":100,"label":{"text":"Ranker\nmulti-task DNN\non GPU"}},
 {"type":"rectangle","id":"rerank","x":0,"y":180,"width":280,"height":100,"label":{"text":"Re-rank\nEV score, diversity,\nads, exploration"}},
 {"type":"rectangle","id":"resp","x":0,"y":360,"width":280,"height":100,"label":{"text":"Feed response"}},
 {"type":"ellipse","id":"emb","x":1060,"y":350,"width":320,"height":120,"label":{"text":"Restaurant\nembeddings index"}},
 {"type":"arrow","id":"app-feed","x":280,"y":50,"width":80,"height":0,"start":{"id":"app"},"end":{"id":"feed"}},
 {"type":"arrow","id":"feed-geo","x":640,"y":50,"width":80,"height":0,"start":{"id":"feed"},"end":{"id":"geo"}},
 {"type":"arrow","id":"geo-avail","x":1000,"y":50,"width":80,"height":0,"start":{"id":"geo"},"end":{"id":"avail"}},
 {"type":"arrow","id":"avail-cand","x":1220,"y":100,"width":1,"height":80,"start":{"id":"avail"},"end":{"id":"cand"}},
 {"type":"arrow","id":"cand-fs","x":1080,"y":230,"width":-80,"height":0,"start":{"id":"cand"},"end":{"id":"fs"}},
 {"type":"arrow","id":"fs-ranker","x":720,"y":230,"width":-80,"height":0,"start":{"id":"fs"},"end":{"id":"ranker"}},
 {"type":"arrow","id":"ranker-rerank","x":360,"y":230,"width":-80,"height":0,"start":{"id":"ranker"},"end":{"id":"rerank"}},
 {"type":"arrow","id":"rerank-resp","x":140,"y":280,"width":1,"height":80,"start":{"id":"rerank"},"end":{"id":"resp"}},
 {"type":"arrow","id":"emb-cand","x":1220,"y":350,"width":1,"height":-70,"start":{"id":"emb"},"end":{"id":"cand"}}
]}

What the feed logs feeds back into features and the models:

{"type":"excalidraw","elements":[
 {"type":"rectangle","id":"resp","x":0,"y":0,"width":280,"height":100,"label":{"text":"Feed response"}},
 {"type":"rectangle","id":"logs","x":360,"y":0,"width":280,"height":100,"label":{"text":"Impression and\nevent logs"}},
 {"type":"rectangle","id":"kafka","x":720,"y":0,"width":280,"height":100,"label":{"text":"Kafka and Flink"}},
 {"type":"rectangle","id":"fs","x":1080,"y":0,"width":280,"height":100,"label":{"text":"Feature store"}},
 {"type":"rectangle","id":"offline","x":360,"y":180,"width":280,"height":100,"label":{"text":"Offline store\nas-of joins,\nlabels after 24h"}},
 {"type":"rectangle","id":"train","x":720,"y":180,"width":280,"height":100,"label":{"text":"Daily training\ntwo-tower and ranker"}},
 {"type":"rectangle","id":"gates","x":1080,"y":180,"width":280,"height":100,"label":{"text":"Validation gates\nAUC, calibration,\nslices"}},
 {"type":"rectangle","id":"canary","x":1080,"y":360,"width":280,"height":100,"label":{"text":"Shadow then canary"}},
 {"type":"rectangle","id":"ranker","x":720,"y":360,"width":280,"height":100,"label":{"text":"Ranker"}},
 {"type":"ellipse","id":"emb","x":340,"y":350,"width":320,"height":120,"label":{"text":"Restaurant\nembeddings index"}},
 {"type":"rectangle","id":"cand","x":-120,"y":360,"width":280,"height":100,"label":{"text":"Candidate merge"}},
 {"type":"arrow","id":"resp-logs","x":280,"y":50,"width":80,"height":0,"start":{"id":"resp"},"end":{"id":"logs"}},
 {"type":"arrow","id":"logs-kafka","x":640,"y":50,"width":80,"height":0,"start":{"id":"logs"},"end":{"id":"kafka"}},
 {"type":"arrow","id":"kafka-fs","x":1000,"y":50,"width":80,"height":0,"start":{"id":"kafka"},"end":{"id":"fs"}},
 {"type":"arrow","id":"logs-offline","x":500,"y":100,"width":1,"height":80,"start":{"id":"logs"},"end":{"id":"offline"}},
 {"type":"arrow","id":"offline-train","x":640,"y":230,"width":80,"height":0,"start":{"id":"offline"},"end":{"id":"train"}},
 {"type":"arrow","id":"train-gates","x":1000,"y":230,"width":80,"height":0,"start":{"id":"train"},"end":{"id":"gates"}},
 {"type":"arrow","id":"gates-canary","x":1220,"y":280,"width":1,"height":80,"start":{"id":"gates"},"end":{"id":"canary"}},
 {"type":"arrow","id":"canary-ranker","x":1080,"y":410,"width":-80,"height":0,"start":{"id":"canary"},"end":{"id":"ranker"}},
 {"type":"arrow","id":"train-emb","x":760,"y":280,"width":-200,"height":70,"start":{"id":"train"},"end":{"id":"emb"}},
 {"type":"arrow","id":"emb-cand","x":340,"y":410,"width":-180,"height":0,"start":{"id":"emb"},"end":{"id":"cand"}}
]}

Latency budget (p99, ML part ~80 ms).

  • Geo index and availability: 10 ms (in-memory, cache per hexagon).
  • Candidate scoring (user tower once, dot products over ~2,000 cached store vectors): 10 ms.
  • Feature fetch for ~400 stores: 15 ms (one batched read; ETA comes from the 60 s cache).
  • Ranker on ~400 items: 25 ms (GPU, batched across requests; on CPU, cut to ~200 items or use GBDT).
  • Re-rank and ads merge: 10 ms; 10 ms of slack.
  • Throughput: 5k req/s x 400 items = 2M item scores/s at peak, which fits ~20 GPUs with batching plus headroom.
  • Fallbacks: if the ranker times out, return the stage 1 order; if the feature store is down, use the baseline heuristic. Never return an empty feed.

Monitoring.

  • Feature health: null rates, value distribution (PSI > 0.2 alerts) per feature, freshness lag of streaming counters, training/serving skew by comparing logged serve-time features with the offline recomputation.
  • Prediction health: mean predicted P(order) vs observed order rate per city each hour; a gap above 15% pages the on-call.
  • Business: conversion per city per hour against a seasonal forecast; delivery-time and late-rate drift.
  • Live-ops: a bad ETA service or a promo misconfiguration looks like model drift; dashboards split by cause.

Feedback loops. The model only learns from what it showed, so popular stores get richer. Counter it with the 1% randomized bucket, the new-store quota, inverse-propensity weighting from logged propensities, and the position tower.

Retraining. Ranker: daily incremental training from the last checkpoint, full retrain weekly. Two-tower: weekly, with store embeddings refreshed daily so new stores get content-only vectors on day one. Real-time adaptation comes from session features, not from retraining.

Trade-offs

  • Order probability vs click probability as the main label: order labels are 10x sparser and noisier per example but track the business; clicks give more data but reward attractive photos and give up conversion.
  • Geo filter instead of ANN retrieval: exact and simple, every deliverable store is scored; gives up sub-linear cost, which only matters in dense cities with >5,000 stores.
  • Expected-value re-rank with hand-tuned weights: transparent and easy to adjust for business goals; gives up joint optimization, and the weights need re-tuning when prices or promos change.
  • Daily retraining: cheap and stable; gives up fast reaction to sudden events (a store going viral, a storm). Streaming session and popularity features cover most of that gap.
  • User-level A/B: clean per-user metrics; gives up correct measurement of marketplace effects (shared couriers and kitchens), which needs switchbacks with lower statistical power.
  • Exploration and new-store quota: healthier supply and less popularity bias; gives up ~0.5-1% short-term conversion.
  • GPU DNN vs GBDT: better use of IDs and sequences; gives up simplicity, debuggability and cost.
  • Penalizing long ETA in ranking: better delivery experience and less late-order risk; gives up some orders from users who would wait for a favorite store, which the personal ETA-tolerance feature partly recovers.

Interviewer expectations

Mid-level (L4)

  • Hard-filters to deliverable, open stores before ranking.
  • Uses order (not click) as the label, with a time-based split and point-in-time features.
  • Proposes a two-stage system with a GBDT ranker and AUC/NDCG offline, conversion online.

Senior (L5)

  • Treats ETA, fee and availability as live request-time features with a cache, and states a latency budget.
  • Handles negative downsampling with calibration correction, position bias, and label delay.
  • Defines guardrails (delivery time, cancellations, new-store share) and a cold-start plan for new users and new stores.

Staff (L6+)

  • Sees the marketplace effect: ranking moves demand to kitchens and couriers, so it changes ETAs and the A/B needs switchbacks; may suggest supply-aware ranking.
  • Designs the feedback-loop controls (exploration bucket, propensity logging, IPS evaluation) and the multi-objective trade-off between conversion, GMV, delivery quality and merchant health.
  • Plans the system's evolution: heuristic, then GBDT, then multi-task DNN, with fallbacks, skew monitoring and ownership across ETA, pricing and ads teams.

Sign in to comment, vote or remix.