gabogg/hikcentral

Watch

Fork 0

perf(tests): profile statistics and occupancy fixture costs before optimization #240

New issue

Open

opened 2026-10-03 08:11:19 +00:00 by gabogg · 0 comments

gabogg commented

2026-10-03 08:11:19 +00:00

Owner

Goal

Measure fixture and synthetic-data costs in the statistics and occupancy tests, then optimize only a demonstrated bottleneck without changing domain semantics. No specific saving or batching implementation is established yet.

Measured research

Source: docs/research/217-test-performance-and-agent-workflows.md (PR #218, research #217).

For the selected 19-test API/summary workload, median cumulative time over three profiles was 3.077s for init_db, 2.880s for seeding, 1.310s for authentication and 0.091s for record_event. Seeding is nested inside initialization; these rows cannot be added independently. This subset does not profile the whole suite.

The former claim that 80%+ of wall time is unbatched INSERTs and the promised 15–20s saving are withdrawn. record_event_async also updates camera counters and telemetry: inserting only event rows would lose those semantics. Closed/checkpointed template-copy measurements are candidates, not proof of suite-wide savings.

Next steps / acceptance criteria

Profile the intended statistics/occupancy subset on a pinned tree, separating initialization, seeding, authentication, event recording and endpoint computation. Preserve reproducible commands, phase durations and component receipts.
Choose an optimization only after identifying a material cost. Evaluate fixture reuse or consistent template snapshots where starting state is identical; batch event creation only if equivalent counter and telemetry semantics are retained.
Keep per-test isolation, existing assertions and real SQLite/ASGI flows. Never copy only the main file of a live WAL database; use a consistent snapshot.
Compare repeated before/after subset and full-suite runs under comparable conditions, report measured savings and run the complete suite. There is no predetermined 0.2s per-test or 15–20s suite target.

Needs triage: measurements must determine the implementation scope before this is ready for an agent.

## Goal Measure fixture and synthetic-data costs in the statistics and occupancy tests, then optimize only a demonstrated bottleneck without changing domain semantics. No specific saving or batching implementation is established yet. ## Measured research Source: [docs/research/217-test-performance-and-agent-workflows.md](https://git.gaboggamer.online/gabogg/hikcentral/src/commit/4abfff3f690b29ed075020192b4fbb492880f9b2/docs/research/217-test-performance-and-agent-workflows.md) (PR #218, research #217). For the selected 19-test API/summary workload, median cumulative time over three profiles was 3.077s for init_db, 2.880s for seeding, 1.310s for authentication and 0.091s for record_event. Seeding is nested inside initialization; these rows cannot be added independently. This subset does not profile the whole suite. The former claim that 80%+ of wall time is unbatched INSERTs and the promised 15–20s saving are withdrawn. record_event_async also updates camera counters and telemetry: inserting only event rows would lose those semantics. Closed/checkpointed template-copy measurements are candidates, not proof of suite-wide savings. ## Next steps / acceptance criteria 1. Profile the intended statistics/occupancy subset on a pinned tree, separating initialization, seeding, authentication, event recording and endpoint computation. Preserve reproducible commands, phase durations and component receipts. 2. Choose an optimization only after identifying a material cost. Evaluate fixture reuse or consistent template snapshots where starting state is identical; batch event creation only if equivalent counter and telemetry semantics are retained. 3. Keep per-test isolation, existing assertions and real SQLite/ASGI flows. Never copy only the main file of a live WAL database; use a consistent snapshot. 4. Compare repeated before/after subset and full-suite runs under comparable conditions, report measured savings and run the complete suite. There is no predetermined 0.2s per-test or 15–20s suite target. Needs triage: measurements must determine the implementation scope before this is ready for an agent.

gabogg added this to the Fast, hermetic agent test loop milestone

2026-10-03 08:11:19 +00:00

gabogg added the

ready-for-agent

label

2026-10-03 08:11:19 +00:00

gabogg referenced this issue

2026-10-03 08:11:56 +00:00

fix(tests): keep tests inside their sandbox and cut suite time #218

gabogg referenced this issue

2026-10-03 08:12:18 +00:00

fix(tests): keep tests inside their sandbox and cut suite time #218

gabogg referenced this issue

2026-10-03 09:49:57 +00:00

fix(tests): keep tests inside their sandbox and cut suite time #218

gabogg referenced this issue

2026-10-03 12:06:23 +00:00

WIP: docs(tests): record confirmed local validation policy #265

gabogg referenced this issue

2026-10-03 12:46:25 +00:00

research(tests): faster test suite and right-sized testing for agentic workflows #217

gabogg referenced this issue

2026-10-03 13:12:55 +00:00

fix(tests): keep tests inside their sandbox and cut suite time #218

gabogg referenced this issue

2026-10-03 18:01:08 +00:00

docs(tests): pass-2 P3 follow-ups from PR #265 (policy clauses, #238/#240 reconciliation) #271

gabogg added

needs-triage

and removed

ready-for-agent

labels

2026-10-03 18:10:28 +00:00

gabogg changed title from ~~perf(tests): batch synthetic time-series insertions in statistics test fixtures~~ to perf(tests): profile statistics and occupancy fixture costs before optimization

2026-10-03 18:11:40 +00:00

gabogg referenced this issue

2026-10-03 18:16:29 +00:00

fix(tests): keep tests inside their sandbox and cut suite time #218