Design a bot detection system

Business goal. Keep automated abuse (scraping, credential stuffing, fake signups, inventory hoarding, ad and review fraud) off a large site without adding friction for real people. Bots cost money directly (stolen pricing, account takeovers, wasted infra) and indirectly (polluted analytics, ranking signals, ad billing). Good bots (verified search crawlers, monitoring, partner APIs) must keep working.

ML task. For every request, and for every session as it unfolds, predict P(automated and unwanted) and map it to an action: allow, log-only, throttle, challenge (invisible JS proof, then CAPTCHA), or block. Two decision points:

• Per-request, at the edge, before the origin is hit: cheap features only.
• Per-session / per-entity, streaming, every few seconds over a rolling window: richer behavioral features, writes a score back to the edge so the next request is handled differently.
• Offline, batch, hourly/daily: graph and cluster analysis to find botnets and produce labels.

Constraints (assumed). 1M requests/s average, 3M peak; ~200M sessions/day; edge decision budget p99 under 5 ms added latency; streaming scorer within 2 s of the event; fail open (allow) if the scorer is down, except on endpoints already under attack (login, checkout), which fail to "challenge".

Objective vs business metric. The model optimizes a cost-weighted log loss; the policy picks thresholds per endpoint. Business metrics are: bot-attributed losses (ATO count, scraped pages, fake accounts surviving 7 days), human friction rate (challenges shown to real users, challenge abandonment, conversion), and good-bot breakage. False positives on checkout cost revenue now; false negatives on login cost accounts later — so thresholds differ by endpoint, not one global cutoff.

Adversary. Bots adapt within hours of any visible change. That shapes everything: hide which signal fired, respond with graded friction instead of hard blocks, keep a randomized holdout, and retrain fast.

Sources.

• Edge logs: IP, ASN, TLS ClientHello (JA3/JA4 fingerprint), HTTP/2 settings and header order, user-agent, path, timing.
• Client telemetry from an obfuscated JS/SDK challenge: canvas/WebGL/audio hashes, navigator properties, automation artifacts (navigator.webdriver, CDP traces), pointer/scroll/keystroke timing aggregates.
• Application events: login success/failure, signup, add-to-cart, search, checkout, account age.
• Downstream outcomes: account takedowns, chargebacks, ATO reports, password resets, abuse reports.
• Threat intel: datacenter/residential-proxy IP lists, known bad fingerprints, verified crawler IP ranges (reverse+forward DNS).

Labels (weak, multiple, delayed).

• Strong positive: honeypot hits (hidden links/form fields humans never see), failed proof-of-work or impossible client telemetry, credential-stuffing patterns confirmed by many accounts failing with breached passwords, accounts later banned for abuse. Delay: seconds (honeypot) to 30 days (bans, chargebacks).
• Strong negative: sessions that solved a hard challenge and later produced a real purchase that did not charge back; logged-in users with long tenure and verified payment. Delay: 1–30 days.
• Analyst labels: security analysts label sampled sessions and attack campaigns; ~5k/week, used mainly for evaluation.
• Good bots: a separate class via verification, not learned — never let the model decide on Googlebot.
• Combine weak sources with a label model (Snorkel-style weighting) into a soft label; keep source provenance to audit.

Imbalance. By request, 30–50% of traffic may be automated, but on any one endpoint the harmful fraction is often 0.1–5%, and most bot traffic comes from a few heavy campaigns. Weight by session or by entity, not by request, or one scraper making 50M requests dominates training. Cap per-campaign contribution (for example, at most 1% of positives per cluster).

Leakage traps.

• Features computed from the enforcement outcome (a "was challenged" flag, CAPTCHA result) leak the old model's decision. Use them only as labels, never as features of the same request.
• Labels from bans exist only for traffic the old system let through, and analysts look at what was flagged — selection bias. Fix with a 0.5–1% random holdout that gets log-only treatment and is always eligible for labeling.
• IP reputation lists built from future bans: compute every aggregate as of the event timestamp (point-in-time joins).

Splits. Strictly time-based: train on weeks 1–6, validate on week 7, test on week 8. Also hold out whole attack campaigns (cluster IDs) to measure generalization to new bots, which is the real job.

Request-level (edge, sub-millisecond lookup).

• Network: ASN type (residential, mobile, datacenter, hosting), IP reputation score, proxy/VPN/Tor flags, geo vs account geo.
• Protocol fingerprint: JA4 hash, HTTP/2 frame settings, header order, consistency of claimed UA vs TLS stack (a "Chrome 128" UA with a Python requests TLS fingerprint is a strong tell).
• Path: endpoint class, sensitivity, query-parameter entropy.

Entity aggregates (online feature store, rolling windows of 1 min / 10 min / 1 h / 24 h).

• Per IP, /24 subnet, ASN, device fingerprint, account, cookie: request rate, distinct accounts tried, login failure ratio, distinct paths, fraction of requests for static assets (bots often skip images/CSS/JS), 4xx ratio.
• Novelty: first-seen age of the fingerprint, count of distinct IPs per fingerprint (fingerprint shared across 5,000 IPs = farm).

Behavioral (session, from client telemetry).

• Inter-request timing distribution (coefficient of variation; bots are too regular or perfectly random), navigation graph (deep link to checkout without browsing), mouse path curvature and velocity, keystroke dwell times, scroll events per page.
• Telemetry integrity: missing or self-contradictory values, evidence of headless browsers, replayed telemetry blobs (same hash on many sessions).

Graph/batch features (hourly). Cluster membership from a graph of shared IPs, fingerprints, accounts, payment instruments; cluster size and growth; campaign label from analysts.

Feature store. Streaming jobs (Flink) update counters in a low-latency KV store (approximate counters, HyperLogLog for distinct counts) replicated to each edge region. The same feature definitions are used for offline training via point-in-time logging: the edge logs the exact feature vector it used, so training uses served values and there is no skew.

Freshness. Rate counters: seconds. Reputation and cluster features: hourly. Fingerprint blocklists: pushed in under a minute.

Baseline. Rules plus rate limits: per-IP and per-account token buckets on login and search, datacenter-ASN challenge on sensitive endpoints, honeypots, known-bad fingerprint blocklist. This catches most unsophisticated traffic, provides labels, and is the fallback when models fail. Keep it — models sit on top.

Production, stage 1 (edge, per request). A gradient-boosted tree model (~200 trees, depth 6) on ~80 features. Trees handle mixed categorical/numeric features, missing telemetry and fast CPU inference (~50 µs). Hashed fingerprints go in as reputation scores, not raw IDs, so the model generalizes to new hashes. Loss: binary cross-entropy on soft labels with per-entity weights, plus a cost weight per endpoint.

Production, stage 2 (streaming, per session/entity). A sequence model over the last N=200 events of a session or fingerprint (small transformer or GRU on event type, path class, inter-arrival time, telemetry summary) concatenated with aggregate features, trained with cross-entropy. It emits a session score every few seconds and writes it to the edge KV store; the next request's stage-1 model reads it as a feature. This is where low-and-slow bots that look normal per request get caught.

Unsupervised layer. New attacks have no labels. Run an isolation forest / autoencoder on fingerprint-level aggregates and a density-based clustering on the entity graph; sudden dense clusters (10k sessions sharing an unusual JA4 + identical timing) get routed to analysts and to "challenge" automatically if they hit a sensitive endpoint. Confirmed clusters become labels within hours.

Policy layer. Score → action via per-endpoint thresholds from a cost matrix (cost of friction to a human vs loss from a bot on that endpoint). Mid-band scores get an invisible challenge first; only failures escalate to visible CAPTCHA. Actions are delayed or randomized for some bots (serve stale prices, silently drop fake signups later) so attackers cannot tell which signal fired.

Offline.

• PR-AUC and recall at fixed human-FPR (for example, recall at 0.1% of human sessions challenged), per endpoint. ROC-AUC alone is misleading with this imbalance.
• Cost-weighted loss using the same cost matrix as the policy, so offline wins translate to dollars.
• Recall on held-out campaigns (generalization to new bots) reported separately from in-distribution recall.
• Calibration (reliability plots, ECE) per endpoint — thresholds and the cost-matrix policy assume calibrated probabilities; recalibrate with isotonic regression on the latest week.
• Slices: mobile app vs web, country, residential vs mobile carrier IPs (CGNAT puts thousands of humans behind one IP), assistive-technology users (screen readers and keyboard-only navigation look "non-human" on mouse features).

Online.

• A/B by session hash on the action policy, not just the model. Primary metrics: bot-attributed loss per 1M sessions (ATOs, fake accounts surviving 7 days, scrape volume), human challenge rate, challenge solve rate, conversion.
• The random holdout gives an unbiased estimate of how much bot traffic exists and how much the system catches.
• Guardrails: checkout conversion drop under 0.1% relative, challenge rate for logged-in long-tenure users under 0.05%, good-bot crawl success at 100%, edge p99 latency, support tickets about CAPTCHAs.
• Adversarial caveat: an A/B win decays as bots adapt; track the metric weekly after launch, not just over the test window.

The request path at the edge, and the stream that keeps its features current:

{"type":"excalidraw","elements":[
 {"type":"rectangle","id":"client","x":0,"y":0,"width":180,"height":90,"label":{"text":"Client request"}},
 {"type":"rectangle","id":"edge","x":260,"y":0,"width":200,"height":90,"label":{"text":"Edge: verify\ngood bots, rules,\nrate limits"}},
 {"type":"rectangle","id":"feat","x":540,"y":0,"width":200,"height":90,"label":{"text":"Feature lookup:\nedge KV, entity\ncounters"}},
 {"type":"rectangle","id":"m1","x":820,"y":0,"width":180,"height":90,"label":{"text":"Stage 1 GBDT,\nabout 1 ms"}},
 {"type":"rectangle","id":"policy","x":1080,"y":0,"width":230,"height":90,"label":{"text":"Policy: allow,\nthrottle, challenge,\nblock"}},
 {"type":"rectangle","id":"origin","x":1390,"y":0,"width":180,"height":90,"label":{"text":"Origin"}},
 {"type":"rectangle","id":"log","x":1080,"y":240,"width":230,"height":90,"label":{"text":"Event log with\nserved features"}},
 {"type":"rectangle","id":"stream","x":760,"y":240,"width":220,"height":90,"label":{"text":"Stream processor:\ncounters, session\nstate"}},
 {"type":"rectangle","id":"m2","x":380,"y":240,"width":180,"height":90,"label":{"text":"Stage 2\nsession model"}},
 {"type":"arrow","id":"c-edge","x":180,"y":45,"width":80,"height":0,"start":{"id":"client"},"end":{"id":"edge"}},
 {"type":"arrow","id":"edge-feat","x":460,"y":45,"width":80,"height":0,"start":{"id":"edge"},"end":{"id":"feat"}},
 {"type":"arrow","id":"feat-m1","x":740,"y":45,"width":80,"height":0,"start":{"id":"feat"},"end":{"id":"m1"}},
 {"type":"arrow","id":"m1-policy","x":1000,"y":45,"width":80,"height":0,"start":{"id":"m1"},"end":{"id":"policy"}},
 {"type":"arrow","id":"policy-origin","x":1310,"y":45,"width":80,"height":0,"start":{"id":"policy"},"end":{"id":"origin"}},
 {"type":"arrow","id":"policy-log","x":1195,"y":90,"width":1,"height":150,"start":{"id":"policy"},"end":{"id":"log"}},
 {"type":"arrow","id":"log-stream","x":1080,"y":285,"width":-100,"height":0,"start":{"id":"log"},"end":{"id":"stream"}},
 {"type":"arrow","id":"stream-feat","x":810,"y":240,"width":-110,"height":-150,"start":{"id":"stream"},"end":{"id":"feat"}},
 {"type":"arrow","id":"stream-m2","x":760,"y":285,"width":-200,"height":0,"start":{"id":"stream"},"end":{"id":"m2"}},
 {"type":"arrow","id":"m2-feat","x":490,"y":240,"width":110,"height":-150,"start":{"id":"m2"},"end":{"id":"feat"},"label":{"text":"session score"}}
]}

The offline loop that turns logs into labels and ships new models:

{"type":"excalidraw","elements":[
 {"type":"rectangle","id":"log","x":0,"y":0,"width":180,"height":90,"label":{"text":"Event log"}},
 {"type":"rectangle","id":"wh","x":260,"y":0,"width":180,"height":90,"label":{"text":"Warehouse"}},
 {"type":"rectangle","id":"labels","x":560,"y":0,"width":220,"height":90,"label":{"text":"Labels: honeypots,\nbans, chargebacks,\nanalysts"}},
 {"type":"rectangle","id":"graph","x":380,"y":200,"width":220,"height":90,"label":{"text":"Graph clustering,\nanomaly detection"}},
 {"type":"rectangle","id":"train","x":860,"y":0,"width":200,"height":90,"label":{"text":"Training: daily\nstage 1, weekly\nstage 2"}},
 {"type":"rectangle","id":"eval","x":1140,"y":0,"width":200,"height":90,"label":{"text":"Offline eval,\nshadow, canary"}},
 {"type":"rectangle","id":"m1","x":1420,"y":-130,"width":180,"height":90,"label":{"text":"Stage 1 GBDT,\nabout 1 ms"}},
 {"type":"rectangle","id":"m2","x":1420,"y":130,"width":180,"height":90,"label":{"text":"Stage 2\nsession model"}},
 {"type":"arrow","id":"log-wh","x":180,"y":45,"width":80,"height":0,"start":{"id":"log"},"end":{"id":"wh"}},
 {"type":"arrow","id":"wh-labels","x":440,"y":45,"width":120,"height":0,"start":{"id":"wh"},"end":{"id":"labels"}},
 {"type":"arrow","id":"wh-graph","x":350,"y":90,"width":80,"height":110,"start":{"id":"wh"},"end":{"id":"graph"}},
 {"type":"arrow","id":"graph-labels","x":550,"y":200,"width":120,"height":-110,"start":{"id":"graph"},"end":{"id":"labels"}},
 {"type":"arrow","id":"labels-train","x":780,"y":45,"width":80,"height":0,"start":{"id":"labels"},"end":{"id":"train"}},
 {"type":"arrow","id":"train-eval","x":1060,"y":45,"width":80,"height":0,"start":{"id":"train"},"end":{"id":"eval"}},
 {"type":"arrow","id":"eval-m1","x":1340,"y":30,"width":80,"height":-115,"start":{"id":"eval"},"end":{"id":"m1"}},
 {"type":"arrow","id":"eval-m2","x":1340,"y":60,"width":80,"height":115,"start":{"id":"eval"},"end":{"id":"m2"}}
]}

Latency budget (edge, p99 5 ms). Good-bot verification and rule checks 0.5 ms (cached reverse DNS), feature KV lookups 2 ms (batched, regional replica), GBDT 0.3 ms, policy 0.2 ms, remaining ~2 ms headroom. If the KV times out, score with request-only features (a second small model trained on that subset) rather than defaulting to allow.

Capacity. 3M peak QPS × 0.3 ms model CPU ≈ 900 core-seconds per second, so ~900 cores for inference across all edge regions, plus feature lookups. Stage 2 runs on sessions, not requests: 200M sessions/day × ~5 min average ≈ 700k concurrently active sessions; rescoring each every 5 s of activity is ~140k scores/s, batched on a GPU pool and skipped for sessions with no new events.

Rollout. Every model goes shadow (score, no action) for 24 h, then canary at 1% traffic, then ramp. Thresholds ship as config so the response to an attack is minutes, not a retrain.

Monitoring.

• Score distribution and action rates per endpoint, ASN, country, every minute; alert on sudden shifts (a 3× jump in challenges from one mobile carrier usually means a false-positive incident, not an attack).
• Feature drift (PSI per feature), especially fingerprint mix after browser releases — a new Chrome version changes TLS and header fingerprints for everyone overnight.
• Challenge solve rates by score band: rising solve rates in the high-score band means either CAPTCHA farms or false positives; check purchase follow-through to tell them apart.
• Holdout-measured bot rate and catch rate, weekly.

Feedback loops. The model's own blocks remove the evidence that would have produced labels (blocked bots never get banned later). The holdout and log-only bands keep a label stream for the traffic the model would block. Analysts sample across score bands, not just the top.

Retraining. Stage 1 daily on a 6-week window with recent weeks up-weighted; stage 2 weekly; blocklists and cluster labels continuously. Emergency path: an analyst-confirmed campaign becomes a fingerprint rule in minutes and labeled training data in the next daily run.

• Edge per-request model vs session model only: the edge model stops attacks on the first request but sees thin features; session models catch subtle bots but act only after several requests. Use both; give up simplicity.
• Hard block vs graded friction: blocking is cheap and decisive but tells the attacker exactly what tripped; challenges and silent degradation slow adaptation but cost humans friction and add complexity.
• Client telemetry depth vs privacy and performance: rich behavioral signals improve recall but add JS weight, raise privacy/consent obligations (GDPR), and get spoofed once attackers replay real telemetry. Keep collection to aggregates, rotate and obfuscate the collector.
• Trees vs deep models at the edge: GBDT is fast, robust to missing data and easy to debug; it gives up modeling raw sequences, which stage 2 handles.
• Fast retraining vs stability: daily retrains track adaptation but can learn from a poisoned label stream (attackers deliberately solve challenges to look human); require label sources to agree and cap any cluster's weight.
• Fail open vs fail closed: fail open protects revenue during outages but leaves a window attackers can trigger by overloading the scorer; fail-to-challenge on login and checkout only.
• IP features vs fairness: IP and ASN are strong signals but punish CGNAT mobile users, shared offices and VPN users; cap the weight of network-only evidence and require a behavioral signal before a hard block.

Mid-level (L4)

• Frames it as binary classification with real thresholds and actions, not just "detect bots".
• Lists sensible features (rates, IP reputation, UA, timing) and a GBDT model.
• Knows to use PR-AUC and to watch false positives on real users.

Senior (L5)

• Splits per-request edge scoring from session-level streaming scoring with a real latency budget.
• Treats labels as weak, delayed and biased by enforcement; adds a random holdout and time-based splits; avoids leakage from challenge outcomes.
• Sets per-endpoint thresholds from a cost matrix, graded challenges, good-bot verification, shadow and canary rollout, drift monitoring.

Staff (L6+)

• Designs for the adversary: hidden signals, delayed or silent actions, held-out-campaign evaluation, unsupervised detection of new clusters feeding labels within hours.
• Reasons about the closed feedback loop, label poisoning, and why A/B wins decay.
• Addresses org and policy issues: fairness for CGNAT and assistive-tech users, privacy limits on telemetry, fail-open policy, ownership split between ML, security analysts and edge infra.