Skip to content

test(windows): isolate fake app roots from the harness console - #900

Merged
coneilen merged 2 commits into
mainfrom
coneilen-harness-fake-app-roots
Oct 20, 2026
Merged

coneilen merged 2 commits into
mainfrom
coneilen-harness-fake-app-roots

Conversation

@coneilen

@coneilen coneilen commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Keep the app-child negative controls independent of the runner's console host, and await their dedicated held roots before asserting exit. The fixture now preserves separate native-kernel and actual-stream closure proofs within one existing cleanup deadline.

Changes

  • Give the unrelated-held-parent negative and forged-advisory-parent sibling their own spawned, held, job-owned harmless app root. Reuse the existing fixture's post-FreeConsole stdin gate without releasing child creation.
  • Assert actual native ancestry differs from the scan root; retain the existing non-adoption/non-termination and forged-parent refusal assertions.
  • Record the extra root's native generation; await its original held handle within the single existing 5000 ms window anchored before job termination, then require separate actual kernel and stream EOF proofs.
  • Test-only change within Invoke-CapAppChildCase; no production module, worker, supervisor, allowlist, native guard, timeout growth, registration identity, or subcase removal. Registration delta 0, current 177 default / 180 real.

Test plan

RED: pwsh -NoProfile -File C:\Users\coneilen.copilot\session-state\efadadd2-466e-4d8f-ac1e-5f4c5b534b0a\files\Invoke-P2rComponent.ps1 -Name component-red -DelayMilliseconds 1000 -Expected Reject -> old f4d49 actual cleanup refused: Dedicated unrelated app root did not prove held kernel exit and both actual stream EOFs; kernel=false, stdoutEOF=true, stderrEOF=true; injected separate-empty-job component boundary, not a native whole-suite claim.
GREEN: pwsh -NoProfile -File C:\Users\coneilen.copilot\session-state\efadadd2-466e-4d8f-ac1e-5f4c5b534b0a\files\Invoke-P2rComponent.ps1 -Name component-green -DelayMilliseconds 1000 -Expected Pass -> actual new cleanup passed with held kernel and both actual EOFs; component-past-window -DelayMilliseconds 6500 -Expected Reject also observed strict refusal within shared 5000ms window, later stream wait budgets 0/0.
REGRESSION: pwsh -NoProfile -File C:\Users\coneilen.copilot\session-state\efadadd2-466e-4d8f-ac1e-5f4c5b534b0a\files\Invoke-P2r.ps1 -RunName green-owned -Mode Focus -Condition Owned -Manifest D:\p2r-efadadd2\green-prepare\suite\fixtures.json -> 9 executed, 9 passed, 0 failed; actual owned-console case proof, unaltered real jobs, held kernel exit/job0/both actual EOF/sourceFrozen. Whole hosted wrapper on new head remains pending.

2026-10-19 reopened P2 qualification — current head 160a96041bc050dc9a35da9b10fd9344a4f2abee

This signed follow-up descends from the actual server-rebased head f4d49b4b16aa173e8c0c4b185e979ada2913dd5b, on merged S2 base a786b140b086742d9ea0e6c44dc78c608ec74e23. The owner synchronized the clean local branch non-destructively; no force-push, amendment, or further rebase was performed. The current inventory is 177 default / 180 real, delta 0. All 112 literal Invoke-CapTest registration identities and their order are preserved. The only new source hunk is the existing app-child fixture cleanup, in Tools\windows\Tests\UiCaptureDriver.Tests.ps1.

Hosted failure classification, not a pipe leak: attempt-1 artifacts from all three specified completed jobs record unrelatedRoot.kernelExited=false, stdoutEof=true, stderrEof=true, and safetyJobEmpty=true. The forged-parent refusal itself succeeded. The saved false kernel observation then failed the strict fixture assertion, and the rehearsal report gate cascaded. The source sampled IsExited immediately after job accounting reached zero, without waiting on the original held root, and retained that result before the subsequent stream waits. The artifacts contain no per-wait timestamps, so historical exit latency is unknown; a slower-runner or tight-bound explanation is not proved.

Attempt-1 job Run Artifact UI result
Deterministic Windows 2022, 118326731621 39330357227 12224188664 177 executed / 174 passed / 3 failed / 0 skipped
Deterministic Windows 2025, 118326731578 39330357227 12224363610 177 executed / 175 passed / 2 failed / 0 skipped
Spikes Swift Windows 2022, 118326759583 39330356779 12224124119 177 executed / 175 passed / 2 failed / 0 skipped

All three job metadata records bind full head f4d49b4b16aa173e8c0c4b185e979ada2913dd5b and shell pwsh. Their four source hashes match local source after the hosted CRLF checkout conversion. Operator-supplied attempt-2 outcomes were mixed: Windows 2022 passed, while Windows 2025 and spikes failed again. The fixture defect is therefore intermittent, not universally deterministic. The separate Windows-2022 S2 spent-window failure is watch-only; this follow-up does not change that group or production code.

Fix and existing budget: one absolute monotonic deadline starts before job termination, using the existing 5000 ms cleanup window. Job-empty waiting consumes its remaining time. The fixture then reuses the unchanged merged-S2 CapNative.WaitForExit on the same held handle, PID, and creation generation with that absolute deadline. Kernel observation remains separate from actual stdout and stderr EOF. Each stream wait is capped by both its existing 2000 ms cap and the shared remaining budget. No deadline reset, per-stream extra window, optional assertion, allowlist change, or production/native-helper edit was added.

Controlled RED/GREEN boundary: the private component harness AST-extracts and executes the exact checked-out Invoke-CapAppChildCase finally block; it does not duplicate or replace that cleanup logic. A real held root belongs to a separate real owned job and sleeps after its stdin gate is released. The cleanup receives a different, genuinely empty job. This is an explicitly injected component boundary, not a claim that a real job is empty while its member is still running, and not proof of historical hosted exit latency. A duplicate of the original kernel handle retains the same generation for independent final closure. These component cases are separate from the nine-case integration denominator.

Component Actual cleanup result Independent owned closure
Old f4d49b4b cleanup, 1000 ms signal delay Expected refusal; saved kernel=false, both actual EOF=true; about 1418 ms Held generation exited, real held job zero, both EOF, source frozen
New cleanup, same 1000 ms delay Pass; native held wait configured 4966 ms from shared remaining window; kernel and both EOF=true; about 1216 ms Held generation exited, real held job zero, both EOF, source frozen
New cleanup, 6500 ms delay Expected strict refusal; held wait configured 4979 ms, kernel=false; both later stream waits configured 0 ms, EOF=false; about 5170 ms including script/observation overhead Root then terminated through its separate owned job; held exit, real job zero, both EOF, source frozen

The component command exits successfully when the expected refusal is observed; that is not mislabeled as a passing cleanup or a full-suite pass. The preserved raw component ErrorRecords, actual cleanup source, generations, snapshots, configured waits, and original/deadline/observation timestamps are under D:\p2r-efadadd2\component-red, component-green, and component-past-window.

Real registered focused integration: Invoke-P2r.ps1 -RunName green-owned -Mode Focus -Condition Owned -Manifest D:\p2r-efadadd2\green-prepare\suite\fixtures.json invoked UiCaptureDriver.Tests.ps1 -AppChildContractOnly with a fresh source-bound fixture manifest. It executed 9 groups / 9 passed / 0 failed on the new source, using unaltered real native jobs, not the component boundary. The unrelated-parent child was not adopted or terminated, the forged-parent refusal remained intact, and both dedicated roots proved generation-bound held exit and actual stdout/stderr EOF. The target was assigned to its owned job before resume; its original native generation, final kernel exit, actual redirected-pipe EOFs, job-zero closure, and source freeze are recorded.

The real focus ran 22:34:28.7912721Z–22:36:10.3243421Z. At both actual negative-case executions, suite 25176 / 134369228688085707 had console process count 1 and owned held console host 45556 / 134369228692530849, with native parent and parent-generation binding. Dedicated roots were 59704 / 134369229457200619 and 20088 / 134369229498253259; all six kernel/stream closure booleans for the dedicated roots were true. This is owned-console evidence only.

No-console local reproduction remains unavailable, not green: the first explicit DETACHED_PROCESS | CREATE_SUSPENDED | EXTENDED_STARTUPINFO_PRESENT attempt (flags 524300) exited 0 with empty output and no fixture execution; its cause is unknown. The authorized CREATE_NO_WINDOW | CREATE_SUSPENDED | EXTENDED_STARTUPINFO_PRESENT alternative (flags 134742020) positively prepared fixtures and ran the old source 9/9, but case-time observations proved suite 46200 / 134369224749960600 still owned live console host 46128 / 134369224754157969 with console process count 1. Flags alone did not establish absence. The breakpoint action's thrown refusal did not propagate to the test verdict, so explicit observation readback—not exit 0—governs this classification. These attempts and all original earlier failures remain immutable; no FreeConsole or NonInteractive fallback was pursued in this reopened lane. The operator then explicitly stopped local no-console work and authorized the controlled signal-delay RED.

Pending gates and limits: the new head's hosted whole ValidationRunner.Tests.ps1 and the operator-designated second/no-console condition are pending train gates, not local passes. The reopened local coverage is the exact cleanup component plus the registered nine-group direct-driver focus, not the whole 177-group driver or whole wrapper. Real 180 was not run. No CI polling, watch, dispatch, rerun, merge, desktop input, capture, C8R preparation, beta, or S2 follow-up work was performed. The prior single normal review remains the only review; the operator-directed scoped fix did not start another review loop.

Fresh dispatch start 2026-10-19T22:19:04.1221233Z; immutable hard deadline 2026-10-19T23:19:04.1221233Z, a new authorization rather than an extension of the historical box. Native work is closed and the serialized slot released.

Historical original P2 evidence below remains retained; it is not qualification of the new head.

Historical command: GREEN: & 'C:\Program Files\PowerShell\7\pwsh.exe' -NoProfile -File 'C:\Users\coneilen.copilot\session-state\efadadd2-466e-4d8f-ac1e-5f4c5b534b0a\files\Invoke-P2Qualification.ps1' -RunName commit-owned -Mode AppChild -Condition Owned -FixtureManifest 'D:\p2-efadadd2\commit-prep\suite\fixtures.json' -> 9 executed, 9 passed, 0 failed on the signed implementation; both dedicated roots proved kernel exit and actual stdout/stderr EOF.

Historical command: RED: & 'C:\Program Files\PowerShell\7\pwsh.exe' -NoProfile -File 'C:\Users\coneilen.copilot\session-state\efadadd2-466e-4d8f-ac1e-5f4c5b534b0a\files\Invoke-P2Qualification.ps1' -RunName red-owned -Mode AppChild -Condition Owned -FixtureManifest 'D:\p2-efadadd2\red-prep02\suite\fixtures.json' -> 9 executed, 8 passed, 1 failed; DescendantUnrecognizedCommand: Unrecognized live ConPTY conhost child PID 65848; refusing.

Historical command: REGRESSION: & 'C:\Program Files\PowerShell\7\pwsh.exe' -NoProfile -File 'C:\Users\coneilen.copilot\session-state\efadadd2-466e-4d8f-ac1e-5f4c5b534b0a\files\Invoke-P2Qualification.ps1' -RunName full-wrapper -Mode Wrapper -Condition Owned -> BLOCKED by the private-controller harness slip, process exit 1: The property 'Count' cannot be found on this object. PesterVersion 14 passed; UiCaptureDriver 175 was not reached. No product/test failure or full-suite pass is claimed.

Historical original evidence and limits

The local source is signed head 67a4ccebddaaea360478c089dc4f7217d2e27a95 on frozen pre-S2 base 782659f765c099adeef0f2003d0c219ab7b96164. The operator explicitly directed the whole wrapper to run on this base without a mid-run rebase, followed by a PR on this base; the train will server-rebase after S2. Local inventory is 175 default / 178 real, versus post-S2 177 default / 180 real, with this change contributing delta 0. A local old-base result is not evidence for the server-rebased source.

The original negative fails behaviorally, not at compile time. Its held suite generation 1708 / 134369127740880413 owns live console host 65848 / 134369127741008766, born after the suite and natively parent-bound to it. The actual app fixture and child are different generations. This confirms the pre-existing harness-root collision, not a product allowlist defect.

The private controller uses the repository's existing Start-CapJobLauncher / journal-before-gate / target-wait / job-close custody path. RED and committed-source GREEN receipts prove held target and launcher kernel exit, owned job Active=0, launcher stdout/stderr EOF, and frozen source bytes. Dedicated-root EOF is additionally recorded by the test itself; outer owned-file proofs do not claim target raw-pipe EOF.

Evidence Location Result
Behavioral RED D:\p2-efadadd2\red-owned\outer-result.json Exit 1; 9 executed / 8 passed / 1 failed; kernel/job/EOF closure proved
Committed-source focused GREEN D:\p2-efadadd2\commit-owned\outer-result.json Exit 0; 9 executed / 9 passed / 0 failed; kernel/job/EOF closure proved
Whole registered wrapper attempt D:\p2-efadadd2\full-wrapper\outer-result.json 1 wrapper invocation / 0 passed / 1 failed; UiCaptureDriver 175 not reached; kernel/job/EOF closure proved
Review Session artifact p2-review.json One normal default code-review; no significant issues

Local full-wrapper qualification is BLOCKED by the owner's private-controller harness slip, not claimed as a product or test failure. The attempt started 2026-10-19T20:05:11.9821061Z and closed 2026-10-19T20:05:41.1861101Z. The complete unmodified wrapper was invoked, not an extracted allocator or direct-driver substitute, but the invocation failed before the capture suite. The private entry invokes the wrapper in its own StrictMode Latest context; that differs from the documented independent pwsh -NoProfile -File invocation. StrictMode inheritance is a likely but unproven cause. The raw stdout/stderr, source pins, target/launcher generations and closure are frozen; the entry did not retain a full original ErrorRecord/stack, so its exact failing line is not claimed. The operator directed shipment without further local attempts. Future C8R preparation must use the documented fresh PowerShell invocation, not & from this StrictMode entry.

Second-condition gate: no-owned-console-host local GREEN was not established. Three private controller setup failures are preserved separately under red-fixtures, green-none, and detach-check; none ran the test suite and none is claimed as test RED or GREEN. In the final setup attempt, FreeConsole succeeded and the attachment count became zero, but the same held own console-host generation remained live beyond the existing 5000 ms bound. The operator stopped further condition work; the proposed target -NonInteractive observation was not run. Hosted CI on the exact P2 head is the operator-designated second-condition gate and remains pending until the train reports it.

Final gate: the hosted full ValidationRunner.Tests.ps1 on the operator's post-S2 server-rebased P2 head is pending and belongs to the train, not to this local result. Neither local real 178 nor real 180 qualification was run here; the operator schedules real 180 as the first future C8R preparation step. No desktop activation, input, screenshots, capture, CI dispatch, CI polling, or merge was performed. macOS make test / make check were not run on this Windows host. The existing negative supplied the pre-fix behavioral RED without adding a registration.

Immutable work box: start 2026-10-19T19:42:23.6681446Z; hard deadline 2026-10-19T20:42:23.6681446Z, without extension.

Checklist

  • I have read the Contributing Guidelines
  • I have signed off my commits (git commit -s) per the DCO
  • Tests pass locally (make test)
  • Code follows the existing style (make check)
  • I added the test/contract before the implementation and observed the intended RED failure

@coneilen
coneilen force-pushed the coneilen-harness-fake-app-roots branch from 67a4cce to f4d49b4 Compare October 19, 2026 21:48
Give unrelated-parent and forged-advisory app child controls their own held job-owned fixture root instead of relabeling the suite PowerShell process. Preserve all registrations and native refusal rules.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: efadadd2-466e-4d8f-ac1e-5f4c5b534b0a
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Keep job drain, generation-bound held-root exit, and actual stream EOF observations within one existing 5000 ms cleanup deadline anchored before termination. Preserve strict refusal after the remaining window expires.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: efadadd2-466e-4d8f-ac1e-5f4c5b534b0a
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
@coneilen
coneilen force-pushed the coneilen-harness-fake-app-roots branch from 160a960 to dcd5e3a Compare October 19, 2026 23:44
@coneilen
coneilen merged commit 0c18452 into main Oct 20, 2026
25 checks passed
@coneilen
coneilen deleted the coneilen-harness-fake-app-roots branch October 20, 2026 21:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant