[Medformatik]

What changed and why

This is PR 5a of the plan to stop mounting the whole checkout into the operations agent. Built-in services now mount the files of the release the stack runs, not the checkout's services/. 5b (#635) is merged into this branch, so this PR lands both on main. 5b adds overrides, the one-time import of checkout edits, the host-render rule and unusable-override errors in the admin panel; its description has the details. 5a never reaches main alone: on its own, it would silently drop an operator's edits, a Traefik basic-auth gate included.

The release's files.

• scripts/service-files.mjs (pnpm generate:service-files) writes embedded-service-files.bundle.ts.
  • It lists, with a literal new URL(…), every file a built-in mount names (relative and @service: sources, directories expanded through git ls-files) and Traefik's configuration: 21 files.
  • Each entry records the file's executable bit.
  • The ops-agent's bundle swaps this module in, as it does for the manifests. The bundler copies each file into assets/, so the bundle digest covers them. The Dockerfile's bundle stage copies them (check-dockerfile-sync verifies that).
• The same script writes services/service-file-history.json: the sha256 of every version each file had in Git history. 5b's import uses it.
  • CI runs pnpm check-service-files in the lint job, unshallowing first. It fails when the module is stale, or the history lacks any version, the current one included. The file is generated, never edited by hand.
• The release authority digest (release1_…) covers each file's path, executable bit and bytes, and the history.

Where a generation's containers find them. They go in content-addressed trees next to the generations, infra/docker/state/ops-agent/service-files/<id>/<sha256>/<path>, not in G/service-files/:

• The release before this one runs an unchanged safeGenerationTree on every generation. It refuses any directory that isn't 0700 and any file that isn't 0400 or 0600. A generation holding the copies would leave a rolled-back agent and CLI unable to read current.
  • The generation records its trees in service-files.json (0600), so it still fully identifies its bytes.
  • A test runs that function, kept verbatim from d4da20793 in a fixture, on generations this release writes.
• Copies are 0444, scripts 0555, directories 0755. Containers that run as neither root nor the deployment user (PostGIS's 999, nginx's workers) can read them. Docker mounts the file or directory itself, so the 0700 state/ops-agent above it doesn't stand in their way.
• A tree's name is the hash of what it holds. An unchanged render mounts the same path, and Compose recreates nothing for it. A changed file gets a new path, and compose up recreates exactly that service.
  • A tree already there is reused only if it still holds exactly its files and modes. Otherwise the render fails with what to do.
• Cleanup removes a tree only when no retained generation records it and no container, running or stopped, mounts it. The CLI and the agent list mounts with docker inspect. When Docker can't tell, nothing is removed.
• The renderer points built-in relative and @service: sources into the tree. The OTP graph build (build-config.json) and the Pelias build (pelias.json) mount the applied generation's copy, through appliedServiceFile.

OPENMAPX_RELEASE=local. The checkout's files are the release's, for the bundled agent and its fixed CLI as for the host CLI. Before, a bundled process rendered its image's files whatever the selector said.

Traefik. traefik.yml (in the generation) and the dynamic directory (in the agent's state, #633) now take the release's files, instead of mirroring the checkout. Running from source, a checkout file render may not use fails the render. Before, it was skipped and its last copy kept.

Valhalla's writable copy.

• The manifest now marks its valhalla.json mount with writableCopy: "@infra:data/valhalla/config/valhalla.json". This is a new optional bind-mount field in @openmapx/extension-contracts; built-ins only, since community services can't declare bind mounts.
• Render seeds the copy from the release's file when it is missing, and again only when that seed changes. The seed hash is recorded in state/ops-agent/writable-copies.json. In between, configure_valhalla.sh's rewrite stays.
• A missing copy is seeded before the generation commits. If that fails, or a directory or a planted link is in its place, the render fails with the reason and nothing is committed. Otherwise Docker would create a root-owned directory at the copy's path and Valhalla couldn't start. An existing copy that can't be re-seeded is still only reported, and Valhalla keeps it.
• Containers write data/, so render never writes through a planted link:
  • each directory below data/ is created as a real directory;
  • on Linux, each is opened relative to the one above it through /proc/self/fd, and the copy is written and renamed there;
  • elsewhere (development), the path is checked again before the rename.
• check-ops-authority refuses a writable copy that reaches the agent's state.

Verification

• Real containers (service-files.integration.test.ts, run in CI's api-privacy group after pulling the pinned well-known and Valhalla images; PostGIS's image is that job's service image):

  • Neither root nor the deployment user. With state/ops-agent at 0700:
    • the pinned nginx's workers, running as nginx, serve assetlinks.json from the tree's 0755 directory;
    • the pinned PostGIS image, run as 999, finds the entrypoint wrapper executable and byte-identical.
  • Valhalla rewrites its copy. The pinned valhalla-scripted runs its own build_tiles start, with tiles present and serving off:
    • it merges its defaults into the seeded copy in place (same inode, concurrency: "12", the release's drain_seconds kept);
    • the next render keeps that rewrite;
    • a changed seed replaces it.
  • Locally (Docker Desktop) both pass; CI is their first Linux run.

• Store (service-file-store.test.ts):

  • tree paths and modes for the repository's real manifests, @service: resolved into pelias's tree;
  • generations pass the previous release's check;
  • an unchanged render reuses the tree (same inode), and a changed martin.yaml gets a new path while PostGIS's stays;
  • cleanup keeps recorded and mounted trees, and removes nothing when Docker can't tell;
  • a tampered tree fails the render;
  • Valhalla's seed: the rewrite is kept and re-seeded when the seed changes. An existing copy that can't be re-seeded is reported. A missing copy that can't be written, a directory in its place, or a planted link fails the render and leaves the applied generation as it was, and the link isn't followed;
  • release-service-files-local.test.ts, with the bundle's modules swapped in: the bundle's files by default, the checkout's under OPENMAPX_RELEASE=local;
  • builds get the applied generation's copy, or are told to render first;
  • the bundle module covers every file a manifest names, with matching executable bits;
  • the history knows every current version;
  • the digest changes with a byte, an executable bit or a path.

• Gate C: a full real-manifest render mounts each service's files from its own tree, read-only. Nothing is mounted from services/, and Valhalla mounts its copy.

• Image: I built the ops-agent image locally from this branch. Its bundle embeds the 21 files, and the fixed CLI starts.

• Checks:

  • the full Vitest suite from the repository root (1,924 files passed);
  • tsc --noEmit for core, CLI, ops-agent, API and extension-contracts;
  • Biome;
  • check-service-files, check-ops-authority, check-dockerfile-sync, check-openapi and check-api-sdk. The OpenAPI document and generated admin model gain the optional writableCopy.

• Not performed:

  • a real stack recreated on these mounts;
  • a panel self-update;
  • a host update.

  Nothing on the production host was changed.

Release and upgrade impact

Product impact: migration, operator-action

release-notes/service-files-from-the-release.md, from 5b, covers both layers. It covers the one-time import and reviewing its list, overrides from now on (under umask 022), and selecting the release before a host render. It also says that the first stack-wide compose up recreates every service whose mounts moved, PostGIS and Traefik included, and covers the PR 8 wait and rollback.

.changeset/service-bind-mount-writable-copy.md: @openmapx/extension-contracts minor (the writableCopy field), @openmapx/extension-cli patch.