FSD Frontend System Design

Part II — The Shield · Lesson 4.4

A/B Testing

In one lineShip two versions to real users, measure one metric, and let the data decide.

Where you've seen itNetflix showing different artwork for the same title. Same content, different thumbnail, measurably different play rate.

  • Time15 min
  • DiagramTraffic split + metric funnel + the frontend's job
  • Chapter4 · Testing

Diagram

diagram
THE SPLIT — assignment must be sticky and server-decided

  incoming user
       │
       ▼
  ┌──────────────────┐  hash(userId + experimentId) % 100
  │  assignment      │  ← deterministic: same user, same bucket,
  └────────┬─────────┘    every session, every device
           │
     ┌─────┴─────┐
     ▼           ▼
  ┌──────┐   ┌──────┐
  │  A   │   │  B   │
  │ 50%  │   │ 50%  │
  │ old  │   │ new  │
  └──┬───┘   └──┬───┘
     │          │
     ▼          ▼
   exposure event fired once, on RENDER — not on assignment
     │          │
     └────┬─────┘
          ▼
   ┌──────────────┐
   │ metric funnel│  view → click → add to cart → purchase
   └──────────────┘  pick ONE primary metric before you start
diagram
FRONTEND FAILURE MODES

  ✗ assign in the client        different bucket per device/session
  ✗ flicker (CSS swap)          user sees A, then B — biases the result
  ✗ log exposure on assignment  counts users who never saw the variant
  ✗ 6 experiments, same surface interaction effects, unreadable result
  ✗ stop when it looks good     peeking inflates false positives

How it works

  1. Assign server-side or at the edge, keyed on a stable user id, and send the variant with the document.
  2. Render the variant on first paint. Client-side swapping causes flicker, which itself changes behaviour.
  3. Fire an exposure event when the variant is actually rendered, not when the flag was read.
  4. Pick one primary metric and a guardrail metric (e.g. conversion up, but page latency and error rate must not regress).
  5. Fix the duration in advance — full weekly cycles — and don't stop early because the numbers look nice.

Trade-offs

A/B test whenDon't when
The change is measurable and reversibleIt's a bug fix or a legal requirement
You have enough traffic for significanceSample size gives 3-week runtimes
The decision is genuinely contestedThe answer is already obvious

Feature flags vs experiments: the same delivery mechanism, different intent. A flag controls release (kill switch, gradual rollout); an experiment measures effect. Reuse the infrastructure, keep the vocabulary separate — and delete both once the decision is made.

Interview angle

"How do you A/B test without hurting performance?"

  • Decide at the edge so the variant ships in the HTML — no flicker, no extra round trip, no layout shift.
  • Keep the payload as one small config object, not two bundles; lazy-load only genuinely divergent code.
  • Add latency and error rate as guardrail metrics, so a winning variant that slows the page still loses.

Recap

  • Sticky, deterministic, server-side assignment; render on first paint.
  • Log exposure on render, not on assignment.
  • One primary metric, one guardrail, a fixed duration, and no peeking.