Skip to content

fix(cache): restore cached outputs atomically so concurrent cache hits don't fail - #866

Open
tlq5l wants to merge 1 commit into
voidzero-dev:mainfrom
tlq5l:fix/cache-concurrent-restore
Open

tlq5l wants to merge 1 commit into
voidzero-dev:mainfrom
tlq5l:fix/cache-concurrent-restore

Conversation

@tlq5l

@tlq5l tlq5l commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #846

Motivation

When two vp run processes hit the same cache entry at the same time, restoring outputs can fail with File exists (os error 17): tar's unpack removes an existing file and then recreates it with create_new, so another process can create the file in between. The same window lets a reader see a missing or half-written file.

extract_output_archive now writes each regular file to a temporary file in the target's directory and renames it over the target, with no lock. This was suggested by @everton-dgn in #846. A failed extraction leaves no temporary file. Entries that are not regular files, which create_output_archive never writes, still go through Entry::unpack_in.

  • Readers: one test reads every restored file on two threads while two others restore 30 times each; another runs two concurrent restores for 50 rounds. Both fail on main.
  • Windows: the final step uses std::fs::rename, which retries with a POSIX-semantics rename when MoveFileExW is denied; tempfile's persist does not. A target opened without FILE_SHARE_DELETE can still fail. Only CI covers Windows.
  • Paths and metadata: entry paths must be plain relative components, and entries resolving outside the workspace through a symlinked directory are rejected before any directory is created. Permission bits and modification time are restored as before.

Ran live: 300 rounds of two concurrent vt run gen cache hits (8 output files), fresh builds. main: 98 of 300 rounds failed, each with File exists (os error 17). This branch: 0 of 300.

@tlq5l
tlq5l force-pushed the fix/cache-concurrent-restore branch from c66fd7f to 07327e7 Compare October 19, 2026 22:27
…s don't fail

Fixes voidzero-dev#846

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@tlq5l
tlq5l force-pushed the fix/cache-concurrent-restore branch from 07327e7 to f79683a Compare October 21, 2026 17:46

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Concurrent cache hits for the same task fail to restore outputs with "File exists (os error 17)"

1 participant