Skip to content

Refresh the Julia name map only from the advertised registry - #565

Merged
andrew merged 1 commit into
git-pkgs:mainfrom
simonchrz:fix/julia-registry-refresh
Oct 23, 2026
Merged

andrew merged 1 commit into
git-pkgs:mainfrom
simonchrz:fix/julia-registry-refresh

Conversation

@simonchrz

Copy link
Copy Markdown
Contributor

Every request for a General registry tarball with a hash other than the one behind the current name map started a goroutine. That goroutine downloaded the whole registry again from upstream instead of using the copy just cached. Nothing limited how many of these loads ran at once, and they had no deadline. Alternating requests between two valid snapshots therefore cost a full download and parse each time, and the map ended up at whichever snapshot was requested last, older ones included.

What changes:

  • The name map is only reloaded for the hash /registries currently lists. /registries is read through the metadata cache that already serves it to clients. Older snapshots are still served, but they no longer trigger a download or replace the map.
  • One reload runs at a time. A request that finds a reload running skips its own instead of waiting.
  • The reload uses the request's context without its cancellation and stops after five minutes.
  • Registry.toml is read up to 64 MB.

Tests cover old snapshots not being downloaded or rolling back the map, the current one loading once, skipping while a load runs, and the size limit.

Each request for a General registry tarball whose hash differed from
the loaded name map started a goroutine that downloaded the whole
registry from upstream again, with no limit on concurrent loads and no
deadline. Alternating between two valid snapshots made every request
cost a full download and parse, and left the map at whichever snapshot
came last, older ones included.

The refresh now only loads the hash /registries currently lists, read
through the metadata cache, runs one load at a time and skips while one
is running, and stops after five minutes. Registry.toml is read up to
64 MB.

@andrew andrew left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two follow-ups:

  • The new tests all call refreshNamesFromRegistry directly, and nothing covers handleRegistry. A test that requests an old snapshot and then the advertised one through GET /registry/{uuid}/{hash}, and checks the name map and download counts (polling for the goroutine), would cover the path clients hit.
  • loadRegistryTarball still downloads the tarball from upstream right after handleRegistry cached it, so each new General snapshot costs two full upstream downloads. Reading Registry.toml from the cached artifact would drop the second one.

@andrew
andrew merged commit 183b07e into git-pkgs:main Oct 23, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants