test(cli): no execution coverage for Windows PATH registration (AC4/AC5 rest on string assertions)

gabogg commented

2026-09-22 19:09:34 +00:00

Owner

Follow-up from the round-5 review of #47 (comment). #47 states this caveat in its own body; this issue tracks closing it.

Problem

Every Windows assertion in tests/test_cli_adapters.py is a substring check against generated PowerShell text. Nothing runs it. The suite confirms the script says DoNotExpandEnvironmentNames, ExpandString, 0x001A; it cannot confirm the script works.

Consequences already observed on this branch — each found by reading, not by a failing test:

idempotency compared an unexpanded PATH entry against a resolved target, so registration appended a duplicate on every run and unregistration never matched (fixed in bcfd1a4);
the CLM fallback is unreachable in the case it names (see the CLM issue);
a trailing ; turned a trailing empty PATH element into an interior one (fixed in 5240167).

String assertions also pin shape, not behaviour: they pass just as happily on a script that is syntactically valid and semantically wrong.

Constraint

CI is runs-on: native — a single Linux runner. There is no Windows execution path today, so this needs infrastructure, not just test code.

Options

Windows Server 2019 runner in the Forgejo CI. Highest fidelity, needs a host and an agent.
Manual validation checklist, run once per release against a staging Windows host, results recorded on the PR. Cheap; catches the current class of bug; not regression-proof.
Extract the PATH algebra into pure Python (parse → decide → render) and unit-test that exhaustively on Linux, leaving PowerShell as a thin, near-logic-free shell. Gets most of the value with no Windows host — but does not validate the registry or broadcast calls, which is where the CLM bug lives.

(3) and (2) compose well and could land first; (1) is the durable answer.

Acceptance

A decision recorded on which of the above we adopt
Registration/unregistration validated against a real Windows Server 2019 machine PATH containing %SystemDrive%-style entries and a trailing ;
REG_EXPAND_SZ confirmed intact after both operations (reg query before/after)
Result recorded so the next PR touching this code is not guessing again

Related: #23, #47.

Triage resolution — 2026-09-23

Adopt portable behavioral tests for PATH transformation logic together with a
human-run Windows Server 2019 checklist. The user owns human validation; the
agent implementing the change prepares commands and an evidence template.
An automated Windows runner remains a separate infrastructure commitment.

Acceptance:

Portable tests exercise PATH behavior, including repeated registration
and unregistration, unexpanded variables, spaces and trailing separators.
Supply a reproducible checklist and evidence template for real Windows
Server 2019 execution in both normal and constrained-language modes.
Record PATH and registry value type before and after registration and
unregistration, including %SystemDrive%-style entries and a trailing ;.
Confirm REG_EXPAND_SZ and unexpanded values remain intact, repeated
operations are idempotent, and unrelated PATH entries are preserved.
Record actual language mode, command outcomes and broadcast/fallback
behavior. Failures must not claim that registry writes succeeded.
Human evidence passes before merging #59 or subsequent PATH changes.

Keep this issue ready-for-human; portable tests and checklist preparation can
be delivered by the agent alongside #59.

Follow-up from the round-5 review of #47 ([comment](https://git.gaboggamer.online/gabogg/hikcentral/pulls/47#issuecomment-1117)). #47 states this caveat in its own body; this issue tracks closing it. ## Problem Every Windows assertion in `tests/test_cli_adapters.py` is a **substring check against generated PowerShell text**. Nothing runs it. The suite confirms the script *says* `DoNotExpandEnvironmentNames`, `ExpandString`, `0x001A`; it cannot confirm the script *works*. Consequences already observed on this branch — each found by reading, not by a failing test: - idempotency compared an unexpanded PATH entry against a resolved target, so registration appended a duplicate on every run and unregistration never matched (fixed in `bcfd1a4`); - the CLM fallback is unreachable in the case it names (see the CLM issue); - a trailing `;` turned a trailing empty PATH element into an interior one (fixed in `5240167`). String assertions also pin *shape*, not *behaviour*: they pass just as happily on a script that is syntactically valid and semantically wrong. ## Constraint CI is `runs-on: native` — a single Linux runner. There is no Windows execution path today, so this needs infrastructure, not just test code. ## Options 1. **Windows Server 2019 runner** in the Forgejo CI. Highest fidelity, needs a host and an agent. 2. **Manual validation checklist**, run once per release against a staging Windows host, results recorded on the PR. Cheap; catches the current class of bug; not regression-proof. 3. **Extract the PATH algebra into pure Python** (parse → decide → render) and unit-test *that* exhaustively on Linux, leaving PowerShell as a thin, near-logic-free shell. Gets most of the value with no Windows host — but does not validate the registry or broadcast calls, which is where the CLM bug lives. (3) and (2) compose well and could land first; (1) is the durable answer. ## Acceptance - [ ] A decision recorded on which of the above we adopt - [ ] Registration/unregistration validated against a real Windows Server 2019 machine PATH containing `%SystemDrive%`-style entries and a trailing `;` - [ ] `REG_EXPAND_SZ` confirmed intact after both operations (`reg query` before/after) - [ ] Result recorded so the next PR touching this code is not guessing again Related: #23, #47. --- ### Triage resolution — 2026-09-23 Adopt portable behavioral tests for PATH transformation logic together with a human-run Windows Server 2019 checklist. The user owns human validation; the agent implementing the change prepares commands and an evidence template. An automated Windows runner remains a separate infrastructure commitment. Acceptance: - [ ] Portable tests exercise PATH behavior, including repeated registration and unregistration, unexpanded variables, spaces and trailing separators. - [ ] Supply a reproducible checklist and evidence template for real Windows Server 2019 execution in both normal and constrained-language modes. - [ ] Record PATH and registry value type before and after registration and unregistration, including `%SystemDrive%`-style entries and a trailing `;`. - [ ] Confirm `REG_EXPAND_SZ` and unexpanded values remain intact, repeated operations are idempotent, and unrelated PATH entries are preserved. - [ ] Record actual language mode, command outcomes and broadcast/fallback behavior. Failures must not claim that registry writes succeeded. - [ ] Human evidence passes before merging #59 or subsequent PATH changes. Keep this issue `ready-for-human`; portable tests and checklist preparation can be delivered by the agent alongside #59.

gabogg added the

ready-for-human

label

2026-09-22 19:09:35 +00:00

gabogg referenced this issue

2026-09-22 19:10:07 +00:00

feat(cli): system PATH registration across Linux and Windows Server 2019 (#23) #47

gabogg referenced this issue from a commit

2026-09-22 19:18:27 +00:00

feat(cli): system PATH registration across Linux and Windows Server 2019 (#47)

gabogg referenced this issue

2026-09-23 14:31:10 +00:00