Skip to content

Fix ETXTBSY flake in hatch version-gate test - #1760

Merged
Mikola Lysenko (mikolalysenko) merged 1 commit into
mainfrom
ci-janitor/hatch-probe-etxtbsy
Oct 28, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 1 commit into
mainfrom
ci-janitor/hatch-probe-etxtbsy

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

vendor::pypi_hatch::tests::hatch_on_an_absolute_path_entry_passes_the_version_gate flakes in the coverage (1) leg, which ci-ok needs. That makes it a merge-queue eviction risk.

Root cause

The test writes a fake hatch script, and the first exec of that script happens inside the production version probe. A sibling test thread that forks while fs::write holds the script open keeps a write fd until its own exec. Exec'ing the script in that window fails with ETXTBSY. require_environment_context_support_with turns any spawn error (.ok()) into the same refusal as a Hatch that is too old, so the test sees an Err. This is the same class as exec_freshly_written in the CLI tests and output_of_fresh_shim in utils::process.

Fix

fake_hatch now execs the script once through the existing utils::process::tests::output_of_fresh_shim helper, which retries ETXTBSY only. It then deletes the marker that run created. Once one exec succeeds, no process holds a write fd and none can gain one, so the probe's own spawn can't race. The only other change is making that helper pub(crate). Production code is unchanged and no assertion is weakened: planted_hatch_in_the_project_is_never_executed still checks the marker after the probe.

Proof

Stress run with --test-threads=64 on pypi_hatch::tests::hatch_on pypi_hatch::tests::planted utils::process (skipping the 10s hung/timeout probes):

build failures
origin/main (b787454) 6 / 1500, each with the exact CI panic
this branch 0 / 3000

The full vendor::pypi_hatch and utils::process modules pass (33 tests). rustfmt --check is clean on the touched file. cargo clippy -p socket-patch-core --all-targets reports only errors that already exist on main in other files; CI's clippy job (--workspace --all-features, no test targets) is unaffected by cfg(test) code.

Removed or moved tests

None.


Generated by Claude Code


Note

Low Risk
Test-only changes (cfg(test) / test helpers); production Hatch probing is unchanged.

Overview
Fixes flaky pypi_hatch Hatch version-gate tests on Unix by avoiding a fork/exec race (ETXTBSY) when a freshly written fake hatch script is first executed inside the real version probe (spawn failures are treated like “Hatch too old”).

fake_hatch now runs the script once through the existing output_of_fresh_shim helper (ETXTBSY-only retries), clears the marker from that warmup run, and documents why the probe must not be the first exec. utils::process exposes that helper and its test module as pub(crate) so the vendor test can reuse it—no production behavior changes.

Reviewed by Cursor Bugbot for commit 1a61d16. Configure here.


Generated by Claude Code

hatch_on_an_absolute_path_entry_passes_the_version_gate flaked in
coverage: the fake `hatch` script is written and then first exec'd by
the production version probe. A sibling test thread that forks while
fs::write holds the file open keeps a write fd until its own exec, so
the probe's exec can fail with ETXTBSY. The probe turns any spawn
error into the "Hatch too old" refusal, so the test saw an Err.

fake_hatch now execs the script once through the existing
output_of_fresh_shim helper (retries ETXTBSY only), then deletes the
marker that run created. Once one exec succeeds, no process holds a
write fd and none can gain one, so the probe can't race. Production
code is unchanged.

Stress (64 threads, hatch + utils::process tests): 6/1500 failures on
main, 0/3000 with this change.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011f9aeP9eHg6uhzuSY2Qg8D
@mikolalysenko Mikola Lysenko (mikolalysenko) added the ci-janitor Opened by the CI janitor routine (flakes, redundant tests, CI perf) label Oct 28, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

bugbot run


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 1a61d16. Configure here.

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Oct 28, 2026
@mikolalysenko

Mikola Lysenko (mikolalysenko) commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator Author

Labeled Ready for review at 1a61d16.

  • CI: 31/31 non-skipped checks green on the head (19 skipped).
  • Bugbot: reviewed 1a61d16, no findings.
  • Look at: test-only change. The hatch version-gate test now runs the fake hatch script once through output_of_fresh_shim (made pub(crate) in utils/process.rs) so the probe never hits the ETXTBSY race; production code is unchanged.
  • Slack announcement not sent this run (Slack send tool unavailable); a later run will retry.

Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Ready for review at 1a61d16ff75e001f7e737b04b78a9a918c27ef27 (burn-down agent label sweep).

  • CI: 50/50 check runs green or skipped on this head; mergeable, no conflicts.
  • Bugbot: reviewed 1a61d16, no unresolved findings.
  • Slack announcement not sent this run (Slack send unavailable); a later run will retry.

Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Final review brief

What it does — Test-only fix for an ETXTBSY flake in hatch_on_an_absolute_path_entry_passes_the_version_gate. The fake hatch script is now exec'd once (retrying only ETXTBSY) right after it is written, so the production version probe never performs the first, racy exec. No production code changes.

Risk — low. #[cfg(test)] code only; the one visibility change (mod tests / output_of_fresh_shim → pub(crate)) is inside a test module.

Look here

Verified — read the full diff and surrounding tests; the marker is removed after the settle run, so the "planted hatch never runs" assertion still tests the probe. Ran cargo test -p socket-patch-core --lib pypi_hatch locally: 15/15 pass. CI 31/31 green on head; Cursor Bugbot: no issues. CHANGELOG.md untouched.

Changes I made — none.

Open questions — none.

Auto-merge is armed: approving sends it straight to the merge queue.


Generated by Claude Code

Merged via the queue into main with commit 5a07275 Oct 28, 2026
50 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the ci-janitor/hatch-probe-etxtbsy branch October 28, 2026 14:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-janitor Opened by the CI janitor routine (flakes, redundant tests, CI perf) Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants