Skip to content

Emit absolute creative rewrite URLs - #917

Open
ChristianPavilonis wants to merge 6 commits into
mainfrom
feature/absolute-creative-rewrite-urls
Open

ChristianPavilonis wants to merge 6 commits into
mainfrom
feature/absolute-creative-rewrite-urls

Conversation

@ChristianPavilonis

@ChristianPavilonis ChristianPavilonis commented Jul 15, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Emit absolute proxy, click, CSS, srcset, creative-runtime, sign-response and rebuild-response URLs using validated, normalized publisher.public_origin.
  • Capture the injected classic script's origin for runtime endpoints and retain absolute click authority across repeated repairs. Preserve existing signing, rewrite policy, legacy inputs, failure behavior and credential modes.
  • Refresh onto current main, including its separate click switch and asset-host allowlist. Fix runtime-specific integration origins, foreign-parent browser coverage and the browser CI generator dependency.

Closes #915

Spec

Absolute creative rewrite URLs, with two independent source-backed review rounds recorded in its review log.

Changes

Area Change
Configuration Optional publisher.public_origin; normalization at Settings/app-config loading and construction; TOML/JSON/blob and present/absent-leaf CLI overlay coverage.
Creative/proxy output Shared absolute signed-URL construction and runtime tag generation; absolute rebuild JSON/redirects; existing input parser and original-token validation retained.
Creative runtime Immutable script-origin endpoint selection; normalized legacy fallback; exact proxy recognition; absolute rebuilt click persistence; source and emitted-IIFE regression tests.
Integration fixtures Derive public origin from each actual listener, independently of upstream origin; preserve custom Viceroy resources/config; keep the legacy source fixture without the optional key.
Browser/CI Real first-party and foreign-parent sandbox recovery checks; explicitly build the native config generator on the browser runner.
Documentation Absolute output examples, configuration normalization, environment-overlay caveat and coordinated rollout/rollback.

GET rebuild routing and absolute-click input support already exist on main; they are not new routes in this PR. Useful compatibility/route tests remain. The inline SSAT/page-bids path keeps its request-origin override and no runtime injection. TSJS source selection and hashing are unchanged. No CORS expansion or signing-format migration.

Compatibility and rollout

An omitted public_origin stays omitted in serialized configuration, but generated general-purpose URLs now use https://{publisher.domain} rather than following the visitor's scheme, host and port. Verify the actual browser-facing endpoint, including local HTTP ports. Old binaries reject the new key, so coordinate binary/config activation and rollback when the fallback cannot serve traffic. CLI environment overrides only replace an existing TOML scalar leaf.

Test plan

Validation uses Rust 1.98.1, Node 24.20.0 and standalone Viceroy 0.21.0. Local artifacts and logs are retained outside the worktree.

  • cargo test-fastly && cargo test-fastly-reuse && cargo test-axum && cargo test-cloudflare && cargo test-spin
  • Cross-adapter parity tests.
  • Focused Settings/app-config, config-generator, generated Cloudflare and actual CLI environment-overlay tests.
  • All eight target-matched Clippy aliases, native build-digest test/lint, integration-test lint and wasm-clock lint.
  • cargo fmt --all -- --check
  • JS build, lint and formatting; full Vitest suite: 52 files, 1,260 tests passed, no type-assertion errors.
  • Emitted independent core+creative IIFE regression, including captured origin and repeated click repair/navigation.
  • Docs lint, formatting and build; root Markdown formatting.
  • Fastly release WASM build.
  • Spin release WASM build; native core API documentation build.
  • Full real Chromium suites: Next.js 23 passed/1 framework skip; WordPress 13 passed/11 framework skips. Both include first-party and actual foreign-parent signed GET rebuild/click redirect checks, with browser security unchanged.
  • Authoritative committed-state URL-host lint: ts dev lint domains --changed-vs origin/main.
  • Workflow actionlint, CI Node-pin guard and documentation-proposal script checks.
  • Manual fastly compute serve session. Automated Viceroy/Chromium checks cover the runtime path instead.

All local gates above are complete. Native core rustdoc succeeds with 31 link warnings on unchanged main documentation; it is not warning-clean. All GitHub checks passed on 99d2109c2, including the fresh-runner browser config-generator build, both browser framework suites, integration matrix and Fastly EC lifecycle tests.

Review

Four independent final lanes covered correctness, architecture, code smells and style. The missing browser-CI generator and changed-line example-host issues were fixed and re-reviewed. No blocking production findings remain. One nonblocking maintenance suggestion, sharing the two integration-fixture retargeting helpers, is deferred to avoid merging their distinct upstream/timestamp/error contracts in this fix.

Checklist

  • Feature spec linked, ending in two review rounds.
  • Changes follow AGENTS.md conventions.
  • No new production unwrap() calls or plaintext production credentials.
  • No new logging API or identity/consent policy.
  • New behavior and compatibility cases have tests.

Retain main's sanitization controls, bounded CSS rewriting and opaque-origin
click recovery while keeping configured absolute creative URLs.

Keep the inline renderer's explicit origin override and both branches'
regression coverage. Adapt root-relative test assumptions to absolute
signing responses.
@ChristianPavilonis
ChristianPavilonis marked this pull request as ready for review October 29, 2026 18:26

@aram356 aram356 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Adds [publisher].public_origin and emits absolute proxy, click, runtime, sign, and rebuild URLs on it, with the creative runtime capturing its script origin for endpoints. The template's new active public_origin placeholder bypasses the deploy placeholder guard, so an operator following the guard's error message ships a config that points every rewritten creative at ads.example.com. The absent-key fallback and the new currentScript precedence also deserve a second look.

4 of the inline comments below carry a one-click GitHub suggestion. Use Commit suggestion (or Add suggestion to batch for several at once) to apply them as commits on the PR branch. The remaining comments describe the fix in prose because the change touches multiple files or lines outside the diff and can't be auto-applied.

Blocking

🔧 wrench

  • Template public_origin placeholder escapes the deploy guard: see inline at trusted-server.example.toml:70

Non-blocking

🤔 thinking

  • Absent-key fallback moves every existing deployment to https://{apex}: see inline at crates/trusted-server-core/src/settings.rs:275
  • currentScript precedence bypasses the pinned-stamp hardening: see inline at crates/trusted-server-js/lib/src/integrations/creative/first_party.ts:8

♻️ refactor

  • public_origin errors drop the reason: see inline at crates/trusted-server-core/src/settings.rs:158
  • Rebuild // guard is bypassable and its test is vacuous: see inline at crates/trusted-server-core/src/proxy.rs:1931
  • EC cookie scope constraint is undocumented: see inline at docs/guide/configuration.md:418
  • tsjs_unified_script_tag is now dead and duplicated: see inline at crates/trusted-server-core/src/creative.rs:609

⛏ nitpick

  • Port 0 accepted: see inline at crates/trusted-server-core/src/settings.rs:3845

Cross-cutting / body-level findings

  • ♻️ Inline fallback ignores public_origin (crates/trusted-server-core/src/publisher.rs:6140-6144, outside the diff). build_bid_map_with_auction_id still falls back to format!("https://{}", settings.publisher.domain) when the request origin is empty. With public_origin = "https://ads.example.com:8443", domain = "example.com", and an empty request origin, the inline adm comes out as https://example.com/first-party/proxy?... (verified through build_bid_map(..., "", ...), the branch build_bid_map_rewrites_inline_adm_to_absolute_first_party_urls exercises). Use the configured origin:

    let base_origin = if request_origin.is_empty() {
        settings.publisher.effective_public_origin()
    } else {
        request_origin.to_owned()
    };
  • ⛏ The rollout caveat is repeated in many places. The "omission keeps old configs deserializable, not the serving page's scheme, host, or port" text, plus upgrade and rollback sequencing, appears in the CHANGELOG, the template, configuration.md (twice), auction/README.md, creative-processing.md, first-party-proxy.md, auction-orchestration.md, api-reference.md, and testing.md. Keep the full text in configuration.md#public_origin and link to it elsewhere, so a later change (for example an opt-in default) edits one place. The new testing.md section is guidance for this one feature's tests rather than the general testing guide, so it fits better in the spec or as test comments.

CI Status

  • .github/dependabot.yml: PASS
  • Analyze (actions): PASS
  • Analyze (javascript-typescript): PASS (both runs)
  • Analyze (python): PASS
  • Analyze (rust): PASS
  • browser integration tests: PASS
  • cargo check (cloudflare native + wasm32-unknown-unknown): PASS
  • cargo check/build/test (spin native + wasm32-wasip1): PASS
  • cargo fmt: PASS (required)
  • cargo test: PASS (required)
  • cargo test (axum native): PASS
  • cargo test (cross-adapter parity): PASS
  • cargo test (ts CLI, native) (macos-latest): PASS
  • cargo test (ts CLI, native) (ubuntu-latest): PASS
  • CI Node pin guard: PASS
  • CLAUDE.md symlink guard: PASS
  • CodeQL: PASS
  • docs-proposal scripts: PASS
  • format-docs: PASS (required)
  • format-typescript: PASS (required)
  • HTML EOF runtime (Fastly): PASS
  • integration tests: PASS
  • integration tests (Fastly EC lifecycle): PASS
  • prepare integration artifacts: PASS
  • ts dev lint domains (changed lines): PASS
  • vitest: PASS

Comment thread trusted-server.example.toml Resolved
Comment thread crates/trusted-server-core/src/proxy.rs Resolved
Comment thread docs/guide/configuration.md Resolved
Reject the public-origin template placeholder, warn on the retained domain fallback, preserve validation reasons, and reject port zero.\n\nRemove the redundant rebuild authority guard and unused script-tag helper. Retain script-origin precedence with explicit spoofing-limit coverage, use the configured inline fallback, and centralize rollout and EC-cookie guidance.\n\nRetarget template-based audit generators and test harnesses to their actual public endpoints.
@ChristianPavilonis

Copy link
Copy Markdown
Collaborator Author

Addressed the requested changes in a35fef4 and replied to/resolved all eight inline threads.

The two body-level findings are fixed too:

  • Inline bid-map rendering now uses the configured public origin when the request origin is empty. A nonempty request origin still wins, and inline creatives still do not inject the runtime.
  • The full fallback/upgrade/rollback warning now lives at configuration.md#public_origin, with links elsewhere. Feature-specific testing guidance moved into the spec, and configuration examples stay inside the EC cookie scope.

We explicitly reviewed and retained the always-absolute HTTPS/domain fallback and script-origin-first endpoint selection. The CLI now warns about an omitted public_origin. The runtime documentation/tests explicitly acknowledge pre-capture currentScript spoofing; this change does not claim to fix it. The unused script-tag helper was deleted after confirming there are no external consumers.

The placeholder guard also exposed template consumers that only replaced the previous three publisher fields. Audit-generated drafts, CLI test fixtures, EOF runtime fixtures, and the template-cache fixture generator now supply the appropriate public endpoint. EOF fixtures distinguish the runtime listener from the upstream origin.

Validation on the revised sources:

  • All five Rust adapter test aliases and cross-adapter parity passed.
  • All eight Clippy aliases, Rust formatting, native build-digest test/lint, integration lint, and Wasm clock lint passed.
  • Full CLI suite passed, including all 14 environment/config tests and real validate/push placeholder and warning regressions.
  • JS build/lint/format and all 52 Vitest files passed: 1,261 tests, no type errors.
  • Real Chromium focused suites passed all four before/after-spoofing and first-party/foreign-parent cases on both frameworks. Full Next.js: 25 passed, 1 framework skip. Full WordPress: 15 passed, 11 framework skips.
  • Native EOF harness tests/lint and actual Axum/Fastly EOF plus auction controls passed.
  • Docs lint/format/build, root Markdown formatting, documentation-proposal checks, CI Node-pin guard, and committed-state URL-host lint passed.
  • Template-cache inline/ESI fixture generation and real CLI validation passed; the full template-cache runtime harness was not rerun.

Fresh GitHub CI is running for the pushed commit. Requesting re-review from @aram356.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Emit absolute URLs when rewriting creative resources

2 participants