Skip to content

Serve cached Composer releases without a metadata lookup - #568

Merged
andrew merged 1 commit into
git-pkgs:mainfrom
simonchrz:perf/composer-download-cache-first
Oct 23, 2026
Merged

andrew merged 1 commit into
git-pkgs:mainfrom
simonchrz:perf/composer-download-cache-first

Conversation

@simonchrz

Copy link
Copy Markdown
Contributor

Every Composer archive download looked up its dist in the package metadata before checking the artifact cache. With cache_metadata off, which is the default, each download of an archive the proxy already had cost an uncoalesced upstream GET of the p2 document: three downloads meant three metadata requests. With the cache on, each download after the TTL revalidated the document and scanned it again.

The download only needs that lookup for the publish time it checks cooldown against, because lock files name archive URLs directly. Like npm, the handler now stores that time in the versions table when the dist is first resolved. Later downloads of a tagged release then work like this:

  • The handler reads the stored time and checks cooldown against it.
  • The cached archive is served directly.
  • If no time is stored, for example for an archive cached by an older proxy, the download takes the metadata path, which records the time.
  • The time is stored even with cooldown off, so turning cooldown on later keeps the fast path.

Guarantees that still hold:

  • The cache still enforces the denylist and verifies the stored digest.
  • An archive only gets cached under a name that was checked against the metadata.

Downloads that stay on the metadata path:

  • Branches, whose archive and time change with every push.
  • Routed packages: as with npm, the versions row is keyed by the plain PURL and would not say which repository the time came from.
  • Requests with a query string, such as ?sha=.

One behaviour change: a cached release is now served even if upstream has since removed or re-tagged the version. With cache_metadata off, that request used to get 404. npm already behaves the new way.

Tests:

  • TestComposerCachedReleaseSkipsMetadata: with metadata caching off, on with the TTL expired, and with cooldown, three downloads now make one metadata request instead of three.
  • TestComposerDownloadStoresPublishTime.
  • TestComposerCachedReleaseCooldownFromStoredTime: cooldown enabled or raised after caching still withholds the archive.
  • TestComposerCachedReleaseWithoutStoredTime.
  • TestComposerCachedDevVersionUsesMetadata and TestComposerCachedRoutedReleaseUsesMetadata.
  • TestComposerCachedReleaseDenied.

Every Composer archive download resolved its dist from the package
metadata before checking the artifact cache. With cache_metadata off,
the default, that meant an uncoalesced upstream GET of the p2 document
for every download of an archive the proxy already had: three downloads
of a cached archive made three metadata requests. With the cache on,
each download after the TTL revalidated and rescanned the document.

The lookup is only needed for the publish time cooldown checks
downloads against, since lock files name archive URLs directly. Like
the npm handler, record that time in the versions table when the dist
is first resolved, and on later downloads of a tagged release read it
from there and serve the cached archive straight away. Without a stored
time, as for archives cached by an older proxy, the download takes the
metadata path, which then records it. The time is stored even with
cooldown off, so enabling cooldown later keeps the fast path.

The cache still enforces the denylist and verifies the stored digest,
and an archive is only ever cached under a name that was checked
against the metadata, so those guarantees hold. Branches, whose
archive and time change with every push, routed packages, whose
versions row would not say which repository the time came from, and
requests with a query string such as ?sha= keep resolving from the
metadata.
@andrew
andrew merged commit 0ef7312 into git-pkgs:main Oct 23, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants