Test by Design
← All work

Dashboard and test suites

Three repositories with no shared code, meeting at three points. The dashboard dispatches either suite — API or UI — and each meets it the same way. One run, end to end.

Open the dashboard
User browser Machine API key Dashboard, repo 1 Test suite, repo 2 or 3 Front end React Backend Worker · the only writer File storage R2 · private Database D1 · run rows CI pipeline GitHub Actions Test suite API or UI · one per run Test report Allure · one file picks + Run POST /runs session status, polled POST /runs no browser, same rules workflow_dispatch run_id · scope · style · workers queued row out, result back POST /webhook HMAC signed runs the branch builds PUT scoped storage token report, read back

Scroll sideways to see the whole diagram.

Blue marks a crossing that has to prove who it is. Only the backend writes run rows, the suite writes only its own report, and reports are served through the backend — the storage is never public.
Repo 1 playwright-run-dashboard the dashboard and the Worker
Repo 2 playwright-api-automation-patterns the API suite it dispatches
Repo 3 playwright-ui-automation-patterns the UI suite, same three points

Inside a suite run

What “the test suite runs” means: one command, a pool of workers, and tests that do not depend on each other.

Dispatch on-demand.yml One suite package, repo 2 or 3 playwright test --workers=N <slice> Worker pool, N processes worker 1 own process fixtures set up · per worker test · test · test own context each own data each worker 2 own process fixtures set up · per worker test · test · test own context each own data each worker N own process fixtures set up · per worker test · test · test own context each own data each allure-results → one report every worker writes into the same run One shared target API suite → bundled mock 127.0.0.1:4010/api/v1 the runner boots it, then tears it down UI suite → the real site www.saucedemo.com nobody here controls it one target, N workers — which is where the ceiling comes from

Scroll sideways to see the whole diagram.

Blue is the part the suite owns. The API suite signs in once per worker; the UI suite signs in for every test, because its cart lives in the browser. Everything a test changes is its own.

What makes the tests independent

API suite: each test makes its own data

One mock serves every worker and is never reset. Each test creates uniquely named data and checks only what it created.

UI suite: each test signs in fresh

The cart lives in the browser, so a shared session would carry one test’s cart into the next.

What parallelism buys

The API suite on a ten-core machine, as its Playwright config records it:

WorkersWall clock
52.9sthe setting — 50% of ten cores
103.5sslower
164.1sslower still

More workers made it slower. One mock serves every worker and becomes the bottleneck, so past half the cores the extra workers only queue. That is why the setting is 50%, written as a fraction so a bigger runner is still used.

Before a suite is called stable, it runs at 12 workers, five times over, with nothing flaky.

The same suite, built twice

Each repository builds the same tests two ways, with identical settings, so the two styles can be compared fairly.

API

functional-style and class-style — 111 tests each.

UI

locator-first and page-first — the same journeys, held level by a check.

← All work
Test by Design