gabogg/hikcentral
Watch
1
0
Fork
You've already forked hikcentral
 0
Code Issues 113 Pull requests 8 Projects Releases Packages Wiki Activity Actions

refactor(occupancy): group-join cutover for multi-camera camera groups (deferred until the first multi-camera group) #62

New issue
Open
opened 2026-09-23 13:54:15 +00:00 by gabogg · 2 comments
gabogg commented 2026-09-23 13:54:15 +00:00
Owner
Copy link

Deferred from the grilling session of 2026-09-22/23 on draft PRs #52, #53 and #54. Do not start this until its trigger fires.

Trigger

The first time a HikCentral people-counting resource group contains more than one camera. #53 detects this: the sync flags the group on counting_camera_groups and emits a warning / operator indicator. Until then, the system relies on the one-camera-per-group invariant (CONTEXT.md §3 Camera Group, ADR 0005).

Why this exists

HikCentral reports passenger flow per resource group, never per camera (people/resourceGroupRealTimeCount); there is no per-camera counting source, neither in the query catalog nor in the event subscription (131585–131588 are door alarms, not passages). The sync manufactures per-camera rows by an even split (occupancy_service.py:1901-1923).

In this deployment every group has exactly one camera, so the split divides by 1 and per-camera figures are exact. That makes the full group-join cutover originally drafted in #54 pure cost today. It was cut from #54 and parked here. Once a multi-camera group appears, the even split fabricates per-camera attribution again (the #28 failure mode), and this work becomes necessary.

Scope (from the original #54 draft, carried over)

  1. Group-level ingestion. Delete the even split and the in_cams/out_cams derivation (:1814-1832). One event per group per direction per tick. Reconciliation baseline reads the group directly.
  2. Event schema. Add people_counting_events.resource_group_code; make camera_index_code nullable. SQLite cannot relax NOT NULL in place — this is a table rebuild (copy → drop → rename → recreate the four idx_counting_events_* indexes). Decide whether the group is resolved at write time (stable history if a camera moves group) or at query time.
  3. PassengerFlowEvent (occupancy_models.py:234): camera_index_code becomes optional; add group identity. Update the flux stream / recent_events consumers (TelemetryEngine, CommandDeckAdapter).
  4. Join key. Move every counting query onto the group: get_counts_in_range_async, get_timespan_aggregates_async (incl. the per-camera LEFT JOIN breakdowns at :774, :917), get_cycle_peak_occupancy_async, get_cycle_average_occupancy_async, get_hourly_flow_distribution_async, get_bucketed_cycle_flow_async, get_passenger_flow_telemetry_async.
  5. Group-level exclusion. Decide what is_excluded / is_active mean once counting is per group (a camera's traffic can no longer be subtracted from its group's total).
  6. direction_type / needs_confirmation. direction_type becomes operator-maintained camera metadata; infer_camera_direction_and_zone stays discovery-only and writes needs_confirmation; backfill existing rows.
  7. Simulator (occupancy_controller.py:365): decide what it emits post-cutover (group rows vs camera rows) and drop its or cams fallback.
  8. Deck. Relabel the attribution panel "Group Attribution"; flow_share_pct across groups; member cameras as metadata with no per-camera %. Coordinate with PR #20 (RFC §4.1.4 / §4.2.4 / §4.3.4).
  9. Glossary. Revise CONTEXT.md §3 Camera Group: drop the one-camera-per-group invariant, state that per-camera figures are not available for multi-camera groups, and review whether CameraEntry still belongs to exactly one group.
  10. ADR. Supersede or amend ADR 0005.
  11. User-facing explanation of the panel rename and the loss of per-camera percentages.

Acceptance criteria

  • No per-camera figure is fabricated for a multi-camera group.
  • All counting queries share one join key and one exclusion filter; trust rules R1/R2 read the same dataset as the published KPIs.
  • No KPI reads zero across the merge.
  • CONTEXT.md §3 and ADR 0005 updated to the post-cutover model.
  • Full suite green (pytest + node --test).

Related: #28, #35, #53, #54, PR #20.

> Deferred from the grilling session of 2026-09-22/23 on draft PRs #52, #53 and #54. **Do not start this until its trigger fires.** ## Trigger The first time a HikCentral people-counting resource group contains **more than one camera**. #53 detects this: the sync flags the group on `counting_camera_groups` and emits a warning / operator indicator. Until then, the system relies on the **one-camera-per-group invariant** (`CONTEXT.md` §3 *Camera Group*, ADR 0005). ## Why this exists HikCentral reports passenger flow **per resource group**, never per camera (`people/resourceGroupRealTimeCount`); there is no per-camera counting source, neither in the query catalog nor in the event subscription (131585–131588 are door alarms, not passages). The sync manufactures per-camera rows by an even split (`occupancy_service.py:1901-1923`). In this deployment every group has exactly one camera, so the split divides by 1 and per-camera figures are exact. That makes the full group-join cutover originally drafted in #54 pure cost today. It was cut from #54 and parked here. Once a multi-camera group appears, the even split fabricates per-camera attribution again (the #28 failure mode), and this work becomes necessary. ## Scope (from the original #54 draft, carried over) 1. **Group-level ingestion.** Delete the even split and the `in_cams`/`out_cams` derivation (`:1814-1832`). One event per group per direction per tick. Reconciliation baseline reads the group directly. 2. **Event schema.** Add `people_counting_events.resource_group_code`; make `camera_index_code` nullable. SQLite cannot relax `NOT NULL` in place — this is a **table rebuild** (copy → drop → rename → recreate the four `idx_counting_events_*` indexes). Decide whether the group is resolved at write time (stable history if a camera moves group) or at query time. 3. **`PassengerFlowEvent`** (`occupancy_models.py:234`): `camera_index_code` becomes optional; add group identity. Update the flux stream / `recent_events` consumers (`TelemetryEngine`, `CommandDeckAdapter`). 4. **Join key.** Move every counting query onto the group: `get_counts_in_range_async`, `get_timespan_aggregates_async` (incl. the per-camera `LEFT JOIN` breakdowns at `:774`, `:917`), `get_cycle_peak_occupancy_async`, `get_cycle_average_occupancy_async`, `get_hourly_flow_distribution_async`, `get_bucketed_cycle_flow_async`, `get_passenger_flow_telemetry_async`. 5. **Group-level exclusion.** Decide what `is_excluded` / `is_active` mean once counting is per group (a camera's traffic can no longer be subtracted from its group's total). 6. **`direction_type` / `needs_confirmation`.** `direction_type` becomes operator-maintained camera metadata; `infer_camera_direction_and_zone` stays discovery-only and writes `needs_confirmation`; backfill existing rows. 7. **Simulator** (`occupancy_controller.py:365`): decide what it emits post-cutover (group rows vs camera rows) and drop its `or cams` fallback. 8. **Deck.** Relabel the attribution panel **"Group Attribution"**; `flow_share_pct` across groups; member cameras as metadata with no per-camera %. Coordinate with PR #20 (RFC §4.1.4 / §4.2.4 / §4.3.4). 9. **Glossary.** Revise `CONTEXT.md` §3 *Camera Group*: drop the one-camera-per-group invariant, state that per-camera figures are not available for multi-camera groups, and review whether *CameraEntry* still belongs to exactly one group. 10. **ADR.** Supersede or amend ADR 0005. 11. **User-facing explanation** of the panel rename and the loss of per-camera percentages. ## Acceptance criteria - [ ] No per-camera figure is fabricated for a multi-camera group. - [ ] All counting queries share one join key and one exclusion filter; trust rules R1/R2 read the same dataset as the published KPIs. - [ ] No KPI reads zero across the merge. - [ ] `CONTEXT.md` §3 and ADR 0005 updated to the post-cutover model. - [ ] Full suite green (pytest + `node --test`). Related: #28, #35, #53, #54, PR #20.
gabogg changed title from Group-join cutover for multi-camera camera groups (deferred; trigger: first multi-camera group) to refactor(occupancy): group-join cutover for multi-camera camera groups (deferred until the first multi-camera group) 2026-09-24 10:15:32 +00:00
gabogg commented 2026-09-24 12:56:33 +00:00
Author
Owner
Copy link

Triaged 2026-09-24 → wontfix for now, kept open on purpose.

Every deployment in sight keeps camera groups 1:1 with cameras (verified against production 2026-09-23), and ADR 0005 makes that the rule. So this cutover is not planned. The issue stays open so the scope above isn't lost: if a deployment ever needs a multi-camera group, #53's runtime guard flags the group (is_multi_camera, GET /api/occupancy/camera-groups) and logs a warning that points here. That's the moment to re-triage.

**Triaged 2026-09-24 → `wontfix` for now, kept open on purpose.** Every deployment in sight keeps camera groups 1:1 with cameras (verified against production 2026-09-23), and ADR 0005 makes that the rule. So this cutover is not planned. The issue stays **open** so the scope above isn't lost: if a deployment ever needs a multi-camera group, #53's runtime guard flags the group (`is_multi_camera`, `GET /api/occupancy/camera-groups`) and logs a warning that points here. That's the moment to re-triage.
gabogg commented 2026-09-24 22:50:50 +00:00
Author
Owner
Copy link

Note (2026-09-24): PR #20 is closed because the statistics deck is being redesigned from scratch. References to PR #20 / RFC-ARCH-2026-004 here now mean the future deck. The deck-side items stay valid, but the redesign will decide where they are shown. The backend contracts a deck should consume are listed in PR #20, comment 1835.

Note (2026-09-24): **PR #20 is closed** because the statistics deck is being redesigned from scratch. References to PR #20 / RFC-ARCH-2026-004 here now mean *the future deck*. The deck-side items stay valid, but the redesign will decide where they are shown. The backend contracts a deck should consume are listed in PR #20, comment 1835.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
docs/test-policy-217
feat/historical-flow-publication
feat/historical-flow-curation
fix/holiday-stamping-followups
chore/schedule-p3-followups-186
fix/test-isolation
test/silent-camera-trace-199
feat/historical-flow-statistics
No results found.
Labels
Clear labels   
blocked
Waits on the issues listed in its "Blocked by:" line before implementation

  
bug
Category: something is broken; behaviour differs from the spec or docs

  
enhancement
Category: new feature or improvement, including refactors, chores and docs

  
high-priority
Address before ordinary-priority work; independent of triage readiness and blockers

  
low-priority
Address after ordinary-priority work; independent of triage readiness and blockers

  
needs-info
Waiting on reporter for more information

  
needs-triage
Maintainer needs to evaluate this issue

  
ready-for-agent
Fully specified, ready for an AFK agent

  
ready-for-human
Requires human implementation

  
referenced
Previously closed issue reopened or referenced by an open issue; visible history marker

  
research
Needs investigation (internet or local resources) before it can be triaged down; used alongside a category and a triage role

  
wontfix
Will not be actioned

No labels
blocked
bug
enhancement
high-priority
low-priority
needs-info
needs-triage
ready-for-agent
ready-for-human
referenced
research
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
gabogg/hikcentral#62
No description provided.