fix(tests): two tests reach real HikCentral endpoints and wait out network timeouts (45 s per run) #216

New issue

Open

opened 2026-10-02 18:17:11 +00:00 by gabogg · 1 comment

gabogg commented

2026-10-02 18:17:11 +00:00

Owner

Agent Brief

Category: bug
Summary: Two tests reach real HikCentral network endpoints and wait out their timeouts. That accounts for 45 s of every full test run, about a third of the suite, and breaks the rule that external endpoints are mocked.

Current behavior (measured on master a7e89c6: 520 passed, 1 skipped, 134 s, pytest --durations):

tests/test_docs.py::test_artemis_execution_endpoint_admin: 25.1 s. It posts to /api/docs/execute/artemis. That executes a real signed Artemis request (acsDoorList) against the configured server, and the test waits for the HTTP timeout. The autouse Artemis mock in tests/conftest.py doesn't cover this execution path.
tests/test_api.py::test_probe_endpoints_typed_responses: 20.1 s. It monkeypatches scan_hikcentral_ports, but /api/status and /api/probe/status still make other real network probes, which time out.
AGENTS.md §3: "Mock external HikCentral hardware/network endpoints (ArtemisClient, BumblebeeClient) so tests remain fully deterministic and offline." On a machine that can reach the server, these tests would send real requests to it.

Desired behavior:

Both tests run offline and deterministically, in well under 1 s each. Every outbound call they trigger is mocked: Artemis execution, Bumblebee, and any status or probe check.
Each test still asserts what it asserts today, including the debug.signature field and the typed probe responses.
A guard stops this from coming back. For example, a session-level fixture in tests/conftest.py fails any test that opens a real outbound socket or HTTP connection, except to localhost and the ASGI test transport. The guard names the test and the target.

Acceptance criteria:

Neither test makes a real network call. With the network unavailable, both pass in under 1 s.
The full suite's runtime drops by about 45 s. Record before and after in the PR, with --durations=25.
A network guard fails a test that tries a real outbound connection. Include one test proving the guard trips.
No other test fails under the guard. Mock any other test the guard catches.

Out of scope:

Broader test-speed work, such as parallelism or test selection. That is research issue #217.

Found while measuring the suite for #217 (2026-10-02).

🤖 Generated with Claude Code

## Agent Brief **Category:** bug **Summary:** Two tests reach real HikCentral network endpoints and wait out their timeouts. That accounts for 45 s of every full test run, about a third of the suite, and breaks the rule that external endpoints are mocked. **Current behavior** (measured on master `a7e89c6`: 520 passed, 1 skipped, **134 s**, `pytest --durations`): - **`tests/test_docs.py::test_artemis_execution_endpoint_admin`: 25.1 s.** It posts to `/api/docs/execute/artemis`. That executes a real signed Artemis request (`acsDoorList`) against the configured server, and the test waits for the HTTP timeout. The autouse Artemis mock in `tests/conftest.py` doesn't cover this execution path. - **`tests/test_api.py::test_probe_endpoints_typed_responses`: 20.1 s.** It monkeypatches `scan_hikcentral_ports`, but `/api/status` and `/api/probe/status` still make other real network probes, which time out. - AGENTS.md §3: *"Mock external HikCentral hardware/network endpoints (`ArtemisClient`, `BumblebeeClient`) so tests remain fully deterministic and offline."* On a machine that *can* reach the server, these tests would send real requests to it. **Desired behavior:** - Both tests run offline and deterministically, in well under 1 s each. Every outbound call they trigger is mocked: Artemis execution, Bumblebee, and any status or probe check. - Each test still asserts what it asserts today, including the `debug.signature` field and the typed probe responses. - A guard stops this from coming back. For example, a session-level fixture in `tests/conftest.py` fails any test that opens a real outbound socket or HTTP connection, except to localhost and the ASGI test transport. The guard names the test and the target. **Acceptance criteria:** - [ ] Neither test makes a real network call. With the network unavailable, both pass in under 1 s. - [ ] The full suite's runtime drops by about 45 s. Record before and after in the PR, with `--durations=25`. - [ ] A network guard fails a test that tries a real outbound connection. Include one test proving the guard trips. - [ ] No other test fails under the guard. Mock any other test the guard catches. **Out of scope:** - Broader test-speed work, such as parallelism or test selection. That is research issue #217. Found while measuring the suite for #217 (2026-10-02). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

gabogg added the

ready-for-agent

label

2026-10-02 18:17:11 +00:00

gabogg referenced this issue

2026-10-02 18:17:28 +00:00

research(tests): faster test suite and right-sized testing for agentic workflows #217

gabogg referenced this issue from a commit

2026-10-02 18:23:10 +00:00

chore: open draft for #216 and #210 (tests stay inside their sandbox)

gabogg referenced this issue from a pull request that will close it,

2026-10-02 18:23:43 +00:00