Design a feature flag service

Source: https://www.hack2hire.com/question-bank/questions/system-design/69d47d8679bf03c0074bd2bd
Company: Stripe
Difficulty: Easy

Problem statement

Let teams turn features on for a percentage of users or specific accounts, with changes reaching every service within seconds and no per-request network call.

Requirements

We are building a feature flag service: teams define flags with targeting rules (specific accounts, percentage rollouts), and an SDK inside every service evaluates them locally from an in-memory copy of the rules, so a flag check never makes a network call and a change reaches every process within a few seconds.

Functional requirements

• Create, edit, archive flags per environment (dev, staging, prod). A flag is boolean or multivariate (e.g. "control" | "v2" | "v3").
• Target by explicit account or user ID allow/deny lists, by attributes (country, plan, API version), and by a sticky percentage of a chosen unit (account, user, request).
• Evaluate in-process: flags.enabled("new_checkout", ctx) reads memory only.
• Changes propagate to all SDK instances in seconds; a kill switch turns a flag off everywhere fast.
• Full audit trail: who changed what, when, and why; one-click revert to any prior version.
• Optional approvals for prod changes on sensitive flags; flag ownership and stale-flag reports.

Clarifying questions worth asking: Is evaluation server-side only, or also in browsers and mobile (which can't hold the full rule set or secret targeting lists)? Do we need experiment exposure logging, or just rollout? Is "within seconds" p99 or p50? Server-side only and p99 under 5 s is assumed below.

Non-functional requirements

• Evaluation: under 1 µs median, zero network I/O, no allocation on the hot path.
• Propagation: p99 under 5 s from save to every connected SDK.
• Availability of evaluation is effectively 100%: if the flag service is down, SDKs keep serving the last known rules. The control plane can be 99.9%.
• Consistency: an admin write is durable and totally ordered per environment; SDKs are eventually consistent but never apply versions out of order.
• Deterministic: the same flag version and context give the same answer in every process and language.

Back-of-the-envelope estimation

• 2,000 services, 50,000 SDK processes (pods), 20,000 active flags in prod.
• Rules per flag ≈ 1 KB on average; a few flags carry 50k-account lists (~1 MB each). Full prod ruleset ≈ 20 MB + lists ≈ 50 MB; keep lists as sorted ID sets or Bloom-backed sets in the SDK.
• Evaluations: 50,000 pods × 20,000 evals/s ≈ 1B evals/s, all local. The service sees none of them.
• Writes: ~5,000 flag changes/day ≈ 0.06/s, peaking at a few per second during deploy windows.
• Fan-out per change: 50,000 streams × ~2 KB delta = 100 MB burst, fine for a tier of ~20 stream servers holding 2,500 connections each.
• Cold start: 50,000 pods restarting in a fleet-wide deploy over 10 min ≈ 80 snapshot fetches/s × 20 MB → served from a CDN/object store, not the database.

API design

Admin (control plane, authenticated, audited):

POST /v1/envs/{env}/flags
  {key: "new_checkout", type: "boolean", owner: "payments-team",
   variations: [false, true], default_off: false, default_on: true,
   rules: [], description: "...", expires_at: "2026-12-31"}
  → 201 {key, version: 1}

PATCH /v1/envs/{env}/flags/{key}
  Headers: If-Match: <version>        (optimistic concurrency)
  {rules: [...], rollout: {unit: "account_id", weights: {true: 1000, false: 9000}},
   comment: "10% of accounts", approval_id?: "apr_..."}
  → 200 {version: 8} | 409 version_conflict | 403 approval_required

POST /v1/envs/{env}/flags/{key}/kill          → 200 {version: 9}  (forces off, bypasses approval)
POST /v1/envs/{env}/flags/{key}/revert {to_version: 6}
GET  /v1/envs/{env}/flags/{key}/history?cursor=...

SDK (data plane, SDK key scoped to one env, read-only):

GET /v1/sdk/snapshot?env=prod
  → 302 to CDN: snapshot-prod-v184233.json.gz   {env_version: 184233, flags: {...}, segments: {...}}

GET /v1/sdk/changes?env=prod&since=184200
  → 200 {from: 184200, to: 184233, changes: [{flag, version, env_version, body|deleted}]}
  → 410 too_old  (log compacted; refetch snapshot)

GET /v1/sdk/stream?env=prod&since=184233     (Server-Sent Events)
  event: put   data: {env_version: 184234, flag: "new_checkout", version: 8, body: {...}}
  event: ping  data: {env_version: 184234}     (every 15 s, also detects silent gaps)

Recovery contract: every message carries a strictly increasing env_version. The SDK applies only env_version = last + 1; on a gap it calls /changes?since=last; on 410 it reloads the snapshot. On reconnect it always passes since, so nothing is lost while disconnected.

Supporting surfaces: POST /v1/sdk/exposures (batched, sampled evaluation counts for stale-flag detection), GET /v1/flags/{key}/evaluate?ctx=... (debug "why did account X get true?"), segment CRUD for reusable account lists.

Data model

Postgres is the source of truth: writes are rare, need transactions, unique keys, and a strict per-environment order. Nothing on the evaluation path ever reads it.

One write is one transaction:

BEGIN;
UPDATE environments SET current_version = current_version + 1
  WHERE id = $env RETURNING current_version;           -- serializes writers per env
UPDATE flags SET rules = $rules, version = version + 1, env_version = $ev
  WHERE env_id = $env AND key = $key AND version = $expected;   -- 0 rows → 409
INSERT INTO flag_versions (...) VALUES (...);
INSERT INTO env_log (env_id, env_version, flag_key, flag_version, body) VALUES (...);
INSERT INTO audit_events (...) VALUES (...);
COMMIT;

Access patterns: by (env, key) for admin reads; env_log WHERE env_id = ? AND env_version > ? ORDER BY env_version for catch-up (covered by the PK); history by (env, key, version DESC); stale-flag scans by flag_usage.last_seen_at and flags.expires_at. Snapshots are immutable gzipped JSON in object storage keyed by (env, env_version), served through a CDN.

High-level design

flowchart LR
  UI[Admin UI and CLI] --> API[Flag API]
  API --> PG[(Postgres)]
  PG --> RELAY[Change relay]
  RELAY --> BUS[Pub sub fanout]
  RELAY --> SNAP[Snapshot builder]
  SNAP --> OBJ[(Object store and CDN)]
  BUS --> STREAM[Stream servers]
  STREAM -->|SSE| SDK[SDK in each service]
  OBJ -->|bootstrap| SDK
  STREAM -->|"catch up /changes"| SDK
  SDK --> EXP[Exposure collector]
  EXP --> USAGE[(Usage rollups)]

1. Admin write. An engineer edits a rule in the UI. The Flag API checks permission and approval, validates the rule (unknown attributes, weights sum to 10,000), then runs the transaction above. The commit assigns env_version 184234.
2. Relay. The relay tails env_log (logical replication or a poll every 200 ms on the PK) and publishes each entry, in order, to the pub/sub channel for that env. It also asks the snapshot builder to write a fresh snapshot every 60 s or every 500 changes.
3. Fan-out. Each stream server subscribes to the env channels its clients need and pushes the event over every open SSE connection. Save to SDK is typically 300–800 ms.
4. SDK read path. At boot the SDK downloads the latest snapshot from the CDN, then opens the stream with since=<snapshot env_version>. Each event replaces one flag in an immutable map and swaps a pointer atomically. enabled(key, ctx) reads the current map, walks rules and hashes, and returns; no locks, no I/O.
5. Feedback. The SDK counts evaluations per (flag, variation) in memory and flushes a sampled batch every 60 s to the exposure collector, which feeds stale-flag reports and the debug view.

The two paths only meet at env_log: the write path ends by appending to it, and the read path is nothing more than ordered replay of it, starting from a snapshot.

Deep dives

Rule evaluation engine. A flag compiles into an ordered list evaluated top to bottom, first match wins:

1. Flag off (or killed) → off_variation.
2. Prerequisite flags (e.g. new_checkout requires new_cart = true); depth capped at 3 and cycles rejected at write time.
3. Individual targets: hash sets of account IDs per variation, O(1).
4. Rules: clauses over context attributes (country in [DE, FR], plan = "enterprise", api_version >= 2024-06-01, in_segment("beta")), ANDed within a rule; each rule serves a fixed variation or a rollout.
5. Fallthrough: default rollout.

The SDK compiles rules at load time (pre-parsed semver/date values, interned strings, sets instead of arrays) so evaluation is a few hundred nanoseconds. Missing attribute means the clause is false, never an error. Any exception returns the caller's coded default and increments an error metric; a flag check must never throw into business code. A shared conformance test suite (JSON fixtures of context → expected variation) runs against every language SDK so Go, Ruby, Java and Python agree bit for bit.

Sticky percentage rollout via deterministic hashing. bucket = int(sha1(flag_key + "." + salt + "." + unit_id)[0:15], 16) mod 10000, then walk cumulative weights: with {true: 1000, false: 9000}, buckets 0–999 get true. Properties that matter:

No assignment is stored anywhere, so 1B evals/s need no lookup; any process computes the same answer.

Change propagation and SDK resilience. Options for getting a change to 50,000 processes:

The SDK holds three layers: in-memory map (serving), a local disk cache of the last snapshot plus env_version, and the network. If the stream drops, it reconnects with jittered backoff (1 s → 30 s) and uses since to catch up; after 60 s without a stream it falls back to polling /changes every 10 s. Gap detection uses env_version plus the 15 s ping, so a silently missed event is noticed within 15 s. Stream servers are stateless; a redeploy drains connections over 5 minutes so 50,000 clients don't reconnect at once.

SDK cold-start and bootstrap resilience. On boot the SDK blocks up to 2 s for the CDN snapshot. If that fails it loads the disk cache; if that's missing (fresh container) it can read a snapshot baked into the image at build time; failing all of those, every call returns the coded default and the SDK reports initialized=false so the service can decide whether to take traffic. Defaults in code must be the safe value (usually the old behavior). Because snapshots come from a CDN and are immutable, a fleet-wide restart puts no load on Postgres, and the flag service being fully down only blocks changes, not evaluations.

Audit, approvals and stale-flag cleanup. Every change writes an audit_events row with a JSON diff, actor, comment and approval, in the same transaction as the change, so the audit log can't miss a write. Sensitive flags (tagged payments, compliance) require a second approver in prod; the kill switch is exempt but pages the owner. A daily job marks a flag stale when it has served one variation to 100% of traffic for 30 days, has no evaluations for 14 days, or passed expires_at; it opens a ticket for the owning team with the code references found by a repo search. Archived flags stay in history but drop out of snapshots, keeping the payload small.

Trade-offs

• Local evaluation gives sub-microsecond checks and survives a service outage, but every SDK holds the whole ruleset (tens of MB) and targeting lists ship to every process; client-side SDKs would instead need server-side evaluation, losing the "no network call" property.
• Streaming deltas gives sub-second propagation at the cost of 50,000 long-lived connections and ordering logic; polling is simpler but stuck at 5–30 s.
• A single per-environment sequence gives total order and trivial gap detection, but serializes writes per env; at a few writes per second that limit is irrelevant.
• Hash-based buckets need no storage and are consistent everywhere, but you can't hand-pick who lands in the 10%; explicit targets cover that.
• Fail-to-last-known-rules keeps services running during an outage, but a kill switch issued during that outage won't arrive; the fallback is a deploy with a changed code default.
• Approvals reduce risky prod changes but slow incident response, so kill is exempt.

Interviewer expectations

Mid-level (L4)

• Flag table plus an SDK that caches rules and refreshes them, with a code default when nothing is loaded.
• Percentage rollout by hashing the user ID so a user's answer is stable.
• A working admin API with an audit log.

Senior (L5)

• Local evaluation as the core decision, with snapshot + stream + catch-up and a versioned recovery contract.
• Sticky, monotonic rollouts (flag-salted hash, growing buckets) and an ordered rule engine with first-match semantics.
• SDK behavior when the service, the CDN or the stream is down; safe defaults; never throwing into callers.

Staff (L6+)

• Separates control plane (consistent, audited, low volume) from data plane (eventual, read-only, massive fan-out) and explains why evaluation availability doesn't depend on the service.
• Cross-language determinism via conformance tests, connection draining for 50k clients, sidecar vs. direct streaming at fleet scale.
• Organizational lifecycle: ownership, approvals for sensitive flags, stale-flag detection tied to code references, and the risk of flags as permanent hidden config.