Part II β The Shield Β· Lesson 4.3
E2E and Automation Testing
In one lineA real browser walking a real user journey β few of them, and none of them flaky.
Where you've seen itThe checkout test that runs before every release, because that flow breaking costs money per minute.
Diagram
ONE JOURNEY, FOUR ASSERTION POINTS
ββββββββββ ββββββββββ ββββββββββ βββββββββββ ββββββββββ
β Search ββββΊβ ProductββββΊβ Cart ββββΊβ Payment ββββΊβ Order β
ββββββββββ ββββββββββ ββββββββββ βββββββββββ ββββββββββ
β β β β β
results price total test card order id
visible matches updates accepted shown
Cover the money path end to end.
Do NOT enumerate 30 coupon edge cases here β that's 4.2's job.WHERE FLAKE COMES FROM β and the actual fix
cause β bad fix β real fix
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
element not ready yet sleep(2000) wait for the
assertion itself
animation still running sleep disable animations
in test env
shared/leftover data re-run seed fresh data
per test
real 3rd-party API retry stub at the network
parallel tests colliding run serially isolate accounts
time/timezone dependence ignore freeze the clockWHERE E2E RUNS β cost climbs left to right
local dev βββΊ PR (smoke, ~5) βββΊ main (full, ~30) βββΊ prod
(synthetic
monitors)How it works
- Pick journeys by revenue and blast radius, not by page count. Five to fifteen for most products.
- Seed state through the API, not the UI β logging in through the form on every test triples the runtime and adds a flake source.
- Wait on assertions, never on time. Modern runners (Playwright, Cypress) auto-retry assertions;
sleepis always a bug. - Stub third parties at the network layer; a payment sandbox outage should not fail your build.
- Run smoke on PRs, the full suite on main, and reuse the same specs as production synthetic monitors.
// Playwright: no sleeps, no manual retries
await page.getByRole('button', { name: 'Place order' }).click();
await expect(page.getByTestId('order-id')).toBeVisible(); // auto-waitsTrade-offs
| Strength | Cost |
|---|---|
| Catches wiring, routing, auth, and build issues | Slowest layer by 100Γ |
| Closest to real user experience | Flake risk is structural |
| Doubles as production monitoring | Failures point at a screen, not a line |
Visual regression belongs here too β screenshot diffing catches CSS breakage that no assertion describes. Budget for the review workload it creates.
Interview angle
"Your E2E suite is flaky. How do you fix it?"
- Measure first: track per-test failure rate and quarantine anything under 99% within a day.
- Attack the structural causes β shared data, real third parties,
sleep-based waits, animation timing. - Then shrink the suite; most flaky E2E suites are covering things that belong two layers down.
Recap
- Few tests, high value, real browser, money paths only.
- Every
sleepis a latent flake; wait on assertions instead. - Seed via API, stub third parties, isolate data per test.