perf(tests): evaluate pytest-xdist safe parallelism with isolated worker databases #238

New issue

Open

opened 2026-10-03 08:11:06 +00:00 by gabogg · 0 comments

gabogg commented

2026-10-03 08:11:06 +00:00

Owner

Goal

Evaluate whether pytest-xdist improves this suite safely. Parallel execution is not adopted by #217; retain the serial full-suite gate until measurements and isolation checks justify a change.

Measured research

Source: docs/research/217-test-performance-and-agent-workflows.md (PR #218, research #217).

The research reports three real runs each of -n 2, -n 4 and -n auto, with commands, timings, load and worker receipts. These local observations do not establish CI speed or universal safety. The old ~88s to ~25–35s promise and SQLite lock-contention claim are withdrawn: no lock failure was reproduced. Each process already creates a distinct TemporaryDirectory database; worker-ID filenames are not required.

Shuffle experiments exposed order dependence tracked in #269; #264 tracks the shuffle hook. Resolve isolation prerequisites before enabling parallel execution.

Acceptance criteria

Audit worker isolation for databases, lifecycle logs, mutable singleton state and any shared output. Preserve conftest collection bootstrap, network guard, #243's opt-in repository fixture and #233's occupancy reset.
Resolve or demonstrate isolation of the order-dependent failure tracked in #269 before adoption.
Compare serial, -n 2, -n 4 and auto on the same pinned tree locally and in CI. Record commands, environment/load, worker counts, complete results and wall times; report actual failure tracebacks instead of inferring SQLite locking.
If adopting xdist, add the dependency and pass five consecutive full-suite -n auto runs, as required by #217. Select an invocation from the measurements, without a predetermined speed target.
Document the adoption verdict and keep the serial full-suite validation gate unless the measured result and confirmed validation policy justify changing it.

## Goal Evaluate whether pytest-xdist improves this suite safely. Parallel execution is not adopted by #217; retain the serial full-suite gate until measurements and isolation checks justify a change. ## Measured research Source: [docs/research/217-test-performance-and-agent-workflows.md](https://git.gaboggamer.online/gabogg/hikcentral/src/commit/4abfff3f690b29ed075020192b4fbb492880f9b2/docs/research/217-test-performance-and-agent-workflows.md) (PR #218, research #217). The research reports three real runs each of `-n 2`, `-n 4` and `-n auto`, with commands, timings, load and worker receipts. These local observations do not establish CI speed or universal safety. The old ~88s to ~25–35s promise and SQLite lock-contention claim are withdrawn: no lock failure was reproduced. Each process already creates a distinct TemporaryDirectory database; worker-ID filenames are not required. Shuffle experiments exposed order dependence tracked in #269; #264 tracks the shuffle hook. Resolve isolation prerequisites before enabling parallel execution. ## Acceptance criteria 1. Audit worker isolation for databases, lifecycle logs, mutable singleton state and any shared output. Preserve conftest collection bootstrap, network guard, #243's opt-in repository fixture and #233's occupancy reset. 2. Resolve or demonstrate isolation of the order-dependent failure tracked in #269 before adoption. 3. Compare serial, `-n 2`, `-n 4` and auto on the same pinned tree locally and in CI. Record commands, environment/load, worker counts, complete results and wall times; report actual failure tracebacks instead of inferring SQLite locking. 4. If adopting xdist, add the dependency and pass five consecutive full-suite `-n auto` runs, as required by #217. Select an invocation from the measurements, without a predetermined speed target. 5. Document the adoption verdict and keep the serial full-suite validation gate unless the measured result and confirmed validation policy justify changing it.

gabogg added this to the Fast, hermetic agent test loop milestone

2026-10-03 08:11:06 +00:00

gabogg added the

ready-for-agent

label

2026-10-03 08:11:06 +00:00

gabogg referenced this issue

2026-10-03 08:11:56 +00:00

fix(tests): keep tests inside their sandbox and cut suite time #218

gabogg referenced this issue

2026-10-03 08:12:18 +00:00

fix(tests): keep tests inside their sandbox and cut suite time #218