Skip to content

Cache cooldown metadata rewrites for a minute - #566

Merged
andrew merged 1 commit into
git-pkgs:mainfrom
simonchrz:perf/cooldown-rewrite-cache
Oct 23, 2026
Merged

andrew merged 1 commit into
git-pkgs:mainfrom
simonchrz:perf/cooldown-rewrite-cache

Conversation

@simonchrz

Copy link
Copy Markdown
Contributor

With cooldown on, the rewrite cache was skipped, so every npm, Composer and Chef metadata request rewrote the whole document. A cached rewrite could have kept hiding a version after its cooldown ended. The cost was:

  • Time and memory: a rewrite on every request.
  • No gzip: responses went out uncompressed.
  • No Last-Modified: Composer revalidates p2 files with If-Modified-Since only, so with cooldown on it never got a 304.

The fix:

  • Cooldown rewrites now go into the rewrite cache with a one-minute expiry. A version leaving cooldown shows up at most a minute late.
  • Concurrent requests share one rewrite, clients get the gzip copy, and the rewrite is dated through metadataDate like any other.
  • The ETag is the hash of the output, so replicas agree on it.
  • The cache is keyed by the digest the metadata cache already recorded, instead of hashing the document a second time. That also lets storedRewrite serve cooldown rewrites without reading the stored document.
  • Without the rewrite cache (metadata_rewrite_cache_size: 0), cooldown rewrites behave as before.

I chose a fixed expiry over computing when the next filtered version leaves cooldown. The exact time would mean threading it out of every handler's filter. One rewrite per package per minute already removes almost all of the cost.

Benchmark with a 24.5 MB synthetic packument (benchPackument(5000)), cooldown 3d, 20 requests with the same upstream digest:

time/request allocated/request
main 45.4 ms 31.1 MB
this PR 3.8 ms 1.6 MB

The 3.8 ms includes the first rewrite, spread over the 20 requests, plus dating each response in the database.

The tests that pinned the old behaviour now check the new one:

  • Last-Modified and a 304 for If-Modified-Since with cooldown on;
  • a gzip copy with cooldown on;
  • the ETag following the kept versions once the entry has expired;
  • the expiry itself;
  • an expired entry freeing its room.

The ETag-from-inputs test now runs without the rewrite cache, which is where that path remains.

With cooldown on, every npm, Composer and Chef metadata request rewrote
the whole document, because a cached rewrite could keep hiding a
version after its cooldown ended. For a 24.5 MB packument that is about
45 ms and 31 MB of allocations per request, and the result was served
uncompressed and without Last-Modified, so Composer, which revalidates
p2 files with If-Modified-Since only, never got a 304.

The rewrite cache now keeps cooldown rewrites with an expiry of one
minute, so a version leaving cooldown shows up at most that late.
Concurrent requests share one rewrite, clients get the gzip copy, and
the rewrite is dated through metadataDate like any other. Its ETag is
the hash of its output, so replicas agree on it.

The cache is keyed by the digest the metadata cache already recorded,
instead of hashing the document again, which also lets storedRewrite
serve cooldown rewrites without reading the stored document.

Without the rewrite cache, cooldown rewrites behave as before.
@andrew
andrew merged commit d87f857 into git-pkgs:main Oct 23, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants