End-to-End Testing Strategy
End-to-end testing exercises a complete journey across a defined system boundary. Choose journeys whose failure would matter to users, then state which components are real, substituted, or outside the test. The label alone does not tell you what a passing test proves.
Use a small set of durable journeys to check integration while faster tests cover detailed rules. A browser test that stubs every backend response can be useful UI evidence, but it cannot verify that the server actually saved the order.
TL;DR
- Define the tested boundary and the user outcome before choosing a runner.
- Prioritize critical journeys, meaningful failure paths, and hard-to-test integrations.
- Use unit, component, API, and contract tests for detailed combinations.
- Isolate data and wait for observable state instead of arbitrary sleeps.
- Retain first-failure evidence and distinguish application defects from test-environment failures.
Quick Example
Save this complete Python standard-library fixture as journey.py, run it locally, and open http://127.0.0.1:8876. It is a disposable teaching server with in-memory state, no login, no real database, and no payment. Stop it with Ctrl+C. It accepts only integer quantities 1–10.
Enter quantity 2 and save. Confirm Order 1 saved. and inspect /orders to verify server state contains quantity 2. This crosses the browser, HTTP handler, validation, and process-local storage boundary. It does not test authentication, durable persistence, concurrent writes, or a production deployment.
Core Concepts
Journey boundary
Name the start, end, actors, data, and infrastructure included. “Create order” may stop at a confirmation screen or include durable storage and downstream fulfillment. Choose the boundary from the risk being tested.
Evidence of completion
Assert the outcome users depend on. A click event or a 200 response is weaker evidence than the expected record and visible confirmation. Playwright recommends testing user-visible behavior and isolating tests; those principles apply regardless of the chosen runner. Browser testing practices
Test portfolio
Place combinations where they are cheapest to diagnose. A pure price rule needs detailed unit cases; an API contract needs schema/semantic checks; one representative browser journey verifies the pieces connect. No fixed test-count ratio fits every product.
Choose journeys by consequence and coverage
List the product’s important outcomes, the components involved, and the failure that each test should expose. Include denial and recovery paths where they determine whether the product works safely.
Prioritize high-consequence and frequently used paths, plus integrations whose wiring cannot be established elsewhere. Avoid reproducing every unit-test combination in a browser. Larger suites cost execution time and diagnosis effort; each test should have a clear reason to exist.
Keep substitutes visible. A sandbox payment provider exercises a different boundary from a fully mocked payment response. Both can be useful, but the report should say which behavior was real. Never use live financial transactions simply to make an example feel more complete.
Make failures actionable
Give each run isolated data and a known starting state. For the teaching server, restarting the process resets the order list; a real system needs approved fixture setup and cleanup that cannot affect another run’s records. Use unique identifiers and scoped cleanup rather than deleting broadly by timestamp.
Wait for a meaningful UI or service condition with a bounded timeout. Arbitrary sleeps can be too short under load and unnecessarily long when the condition is already true. Keep first-failure screenshots, relevant logs, request identifiers, and environment versions without exposing credentials or private data.
When a test fails, determine whether the user journey is broken, the test’s assumptions changed, or the environment could not support execution. Preserve that category even if a retry passes. A final green result should not erase an intermittent failure that users may also encounter.
The fixture is intentionally small enough to inspect completely. For an actual product, add durable-state checks, authentication and denial cases, supported browsers/devices, and failure recovery according to the product’s requirements. The local demonstration provides evidence for one journey in one environment, not a compatibility matrix.
Best Practices
Assign an owner and purpose
Document the user consequence and tested boundary with each journey. Retire or relocate tests whose coverage has become redundant, while preserving important integration checks.
Keep setup separate from the action
Use an approved fixture/API to prepare unrelated state, then exercise the specific user path being tested. Repeating a long setup journey in every test makes unrelated failures harder to interpret.
Validate the test with a meaningful defect
In an isolated fixture, introduce a known failure at the intended boundary and verify the assertion catches it. Restore the fixture afterward. Passing against a healthy application alone does not show the test can detect its target defect.
Common Mistakes
Bad: Calling a browser-only mocked flow a persistence test.
Correct: State the substituted boundary and add real storage integration evidence where required.
Bad: Using fixed sleeps to wait for completion.
Correct: Wait for a bounded observable condition that represents the outcome.
Bad: Making all tests depend on one earlier test’s data.
Correct: Give tests independent setup and scoped data ownership.
FAQ
How many end-to-end tests should we have?
Enough to cover important integration risks with clear ownership and useful diagnosis. Choose from consequence and coverage rather than a universal percentage.
Does the fixture prove production readiness?
No. It demonstrates a local browser-to-server journey with process memory only; production identity, persistence, concurrency, and recovery need separate tests.
Should retries be enabled?
They can help collect evidence, but retain the first failure and track flakiness. A passing retry is not a repair.
Related Topics
- Testing & Quality Assurance — Design the broader test portfolio.
- Unit vs Integration vs End-to-End Tests — Place assertions at useful boundaries.
- Flaky Test Diagnosis — Investigate intermittent failures.