Repository navigation
Emit absolute creative rewrite URLs - #917
ChristianPavilonis wants to merge 8 commits into
Conversation
Retain main's sanitization controls, bounded CSS rewriting and opaque-origin click recovery while keeping configured absolute creative URLs. Keep the inline renderer's explicit origin override and both branches' regression coverage. Adapt root-relative test assumptions to absolute signing responses.
aram356
left a comment
aram356
left a comment
There was a problem hiding this comment.
Summary
Adds [publisher].public_origin and emits absolute proxy, click, runtime, sign, and rebuild URLs on it, with the creative runtime capturing its script origin for endpoints. The template's new active public_origin placeholder bypasses the deploy placeholder guard, so an operator following the guard's error message ships a config that points every rewritten creative at ads.example.com. The absent-key fallback and the new currentScript precedence also deserve a second look.
4 of the inline comments below carry a one-click GitHub
suggestion. Use Commit suggestion (or Add suggestion to batch for several at once) to apply them as commits on the PR branch. The remaining comments describe the fix in prose because the change touches multiple files or lines outside the diff and can't be auto-applied.
Blocking
🔧 wrench
- Template
public_originplaceholder escapes the deploy guard: see inline attrusted-server.example.toml:70
Non-blocking
🤔 thinking
- Absent-key fallback moves every existing deployment to
https://{apex}: see inline atcrates/trusted-server-core/src/settings.rs:275 currentScriptprecedence bypasses the pinned-stamp hardening: see inline atcrates/trusted-server-js/lib/src/integrations/creative/first_party.ts:8
♻️ refactor
public_originerrors drop the reason: see inline atcrates/trusted-server-core/src/settings.rs:158- Rebuild
//guard is bypassable and its test is vacuous: see inline atcrates/trusted-server-core/src/proxy.rs:1931 - EC cookie scope constraint is undocumented: see inline at
docs/guide/configuration.md:418 tsjs_unified_script_tagis now dead and duplicated: see inline atcrates/trusted-server-core/src/creative.rs:609
⛏ nitpick
- Port 0 accepted: see inline at
crates/trusted-server-core/src/settings.rs:3845
Cross-cutting / body-level findings
-
♻️ Inline fallback ignores
public_origin(crates/trusted-server-core/src/publisher.rs:6140-6144, outside the diff).build_bid_map_with_auction_idstill falls back toformat!("https://{}", settings.publisher.domain)when the request origin is empty. Withpublic_origin = "https://ads.example.com:8443",domain = "example.com", and an empty request origin, the inlineadmcomes out ashttps://example.com/first-party/proxy?...(verified throughbuild_bid_map(..., "", ...), the branchbuild_bid_map_rewrites_inline_adm_to_absolute_first_party_urlsexercises). Use the configured origin:let base_origin = if request_origin.is_empty() { settings.publisher.effective_public_origin() } else { request_origin.to_owned() };
-
⛏ The rollout caveat is repeated in many places. The "omission keeps old configs deserializable, not the serving page's scheme, host, or port" text, plus upgrade and rollback sequencing, appears in the CHANGELOG, the template,
configuration.md(twice),auction/README.md,creative-processing.md,first-party-proxy.md,auction-orchestration.md,api-reference.md, andtesting.md. Keep the full text inconfiguration.md#public_originand link to it elsewhere, so a later change (for example an opt-in default) edits one place. The newtesting.mdsection is guidance for this one feature's tests rather than the general testing guide, so it fits better in the spec or as test comments.
CI Status
- .github/dependabot.yml: PASS
- Analyze (actions): PASS
- Analyze (javascript-typescript): PASS (both runs)
- Analyze (python): PASS
- Analyze (rust): PASS
- browser integration tests: PASS
- cargo check (cloudflare native + wasm32-unknown-unknown): PASS
- cargo check/build/test (spin native + wasm32-wasip1): PASS
- cargo fmt: PASS (required)
- cargo test: PASS (required)
- cargo test (axum native): PASS
- cargo test (cross-adapter parity): PASS
- cargo test (ts CLI, native) (macos-latest): PASS
- cargo test (ts CLI, native) (ubuntu-latest): PASS
- CI Node pin guard: PASS
- CLAUDE.md symlink guard: PASS
- CodeQL: PASS
- docs-proposal scripts: PASS
- format-docs: PASS (required)
- format-typescript: PASS (required)
- HTML EOF runtime (Fastly): PASS
- integration tests: PASS
- integration tests (Fastly EC lifecycle): PASS
- prepare integration artifacts: PASS
- ts dev lint domains (changed lines): PASS
- vitest: PASS
Reject the public-origin template placeholder, warn on the retained domain fallback, preserve validation reasons, and reject port zero.\n\nRemove the redundant rebuild authority guard and unused script-tag helper. Retain script-origin precedence with explicit spoofing-limit coverage, use the configured inline fallback, and centralize rollout and EC-cookie guidance.\n\nRetarget template-based audit generators and test harnesses to their actual public endpoints.
|
Addressed the requested changes in a35fef4 and replied to/resolved all eight inline threads. The two body-level findings are fixed too:
We explicitly reviewed and retained the always-absolute HTTPS/domain fallback and script-origin-first endpoint selection. The CLI now warns about an omitted The placeholder guard also exposed template consumers that only replaced the previous three publisher fields. Audit-generated drafts, CLI test fixtures, EOF runtime fixtures, and the template-cache fixture generator now supply the appropriate public endpoint. EOF fixtures distinguish the runtime listener from the upstream origin. Validation on the revised sources:
Fresh GitHub CI is running for the pushed commit. Requesting re-review from @aram356. |
aram356
left a comment
aram356
left a comment
There was a problem hiding this comment.
Summary
All ten round-1 findings are resolved at a35fef44f. I re-ran the round-1 repros:
- Placeholder guard: with the real
tsbinary,ts config init, then editing onlydomain,cookie_domain, andorigin_url, now failsconfig pushwithInsecure default value for: publisher.public_originand publishes nothing. - Omitted-origin warning: the warning prints once per command.
- Mutation checks: removing each new guard (deploy and runtime placeholder checks, inline fallback, rebuild scheme and path checks, port 0, error reason) fails the new tests, so none of them is vacuous.
The PR can't merge yet: it conflicts with main, and CI has not run on this head.
Blocking
🔧 wrench
- Merge
mainand get CI to run on the head: see the body-level section below
Non-blocking
🌱 seedling
- Warn when
public_originis outside the EC cookie scope: see inline atcrates/trusted-server-core/src/config.rs:126
⛏ nitpick
{#public_origin}anchor doesn't resolve on GitHub: see inline atdocs/guide/configuration.md:414
Cross-cutting / body-level findings
-
🔧 Merge
mainand get CI to run on the head. GitHub reports the PR asdirty(mergeable: false), so thepull_requestworkflows did not run ona35fef44f. Only the two CodeQLAnalyzechecks exist. A trial merge with currentmain(6459e9470) conflicts in:CHANGELOG.mdcrates/trusted-server-integration-tests/src/bin/generate-viceroy-config.rscrates/trusted-server-integration-tests/tests/common/config.rscrates/trusted-server-integration-tests/tests/environments/axum.rscrates/trusted-server-integration-tests/tests/environments/cloudflare.rs
The integration-test conflicts change behavior, not just text. They come from #1107, which:
- added
--bidder-origin-urltogenerate-viceroy-config; - added a
traceparameter andintegration_app_config_envelope_with_trace, used by the newtests/common/trace_boundary.rs, which merges cleanly; - moved port selection into
AxumDevServer::spawn_with_app_config(&envelope)andCloudflareWorkers::spawn_with_readiness(ready).
Those spawn helpers now receive a finished envelope and pick the listener port themselves. The listener
public_origintherefore has to be applied there, after the port is chosen, by retargeting the envelope the wayspawn_with_config_jsonnow does withcloudflare_config_with_public_origin(or a sharedenvelope_with_public_origin). Otherwise the trace runtimes fall back tohttps://localhost. Keep both CLI flags, and both fields inArgs, in the generator. Then re-run the full CI: the parity, integration, and browser jobs cover exactly these files.
CI Status
- Analyze (javascript-typescript): PASS
- Analyze (python): PASS
- All other checks (cargo fmt, cargo test, format-docs, format-typescript, vitest, integration, browser, parity, lint-domains, and the rest): not run. GitHub does not trigger
pull_requestworkflows while the PR has merge conflicts. - Run locally on
a35fef44f:cargo fmt --check, JS prettier, docs prettier, and Markdown prettier clean;vitest1261/1261 pass.
| if self.settings.publisher.public_origin.is_none() { | ||
| log::warn!( | ||
| "publisher.public_origin is unset; generated first-party URLs use {}, not the serving page's scheme, host or port. Verify this endpoint before deployment, or configure publisher.public_origin.", | ||
| self.settings.publisher.effective_public_origin() | ||
| ); | ||
| } |
There was a problem hiding this comment.
🌱 seedling: The docs now say public_origin must be publisher.domain or one of its subdomains, or proxy and click requests lose the EC cookie (Domain=.{publisher.domain}). Nothing checks this. Since validate/push already warn about the omitted field here, the same block could warn about an out-of-scope host:
let publisher = &self.settings.publisher;
if let Some(host) = publisher
.public_origin
.as_deref()
.and_then(|origin| url::Url::parse(origin).ok())
.and_then(|url| url.host_str().map(str::to_ascii_lowercase))
{
let domain = publisher.domain.to_ascii_lowercase();
if host != domain && !host.ends_with(&publisher.ec_cookie_domain().to_ascii_lowercase()) {
log::warn!(
"publisher.public_origin host {host} is outside the EC cookie scope {}; proxy and click requests to it cannot forward the EC ID.",
publisher.ec_cookie_domain()
);
}
}A typo'd or copied example host then shows up before deploy instead of as missing EC IDs in reporting. Note that the integration fixture pairs domain = "localhost" with public_origin = "http://127.0.0.1:{port}", so it would log this warning too. Not required for this PR.
|
|
||
| **Port Handling**: Includes port if non-standard (not 80/443). | ||
|
|
||
| #### `public_origin` {#public_origin} |
There was a problem hiding this comment.
⛏ nitpick: {#public_origin} is VitePress syntax that GitHub doesn't support. On GitHub this heading renders as public_origin {#public_origin} (checked with GitHub's Markdown API), and GitHub generates the anchor #public_origin-public_origin. The docs-site links (/guide/configuration#public_origin) work. Three pointers read on GitHub land at the top of this 2,000-line page instead:
- the Breaking entry in
CHANGELOG.md crates/trusted-server-core/src/auction/README.md- the comment above
public_originintrusted-server.example.toml
## Publisher Configuration gets the same #publisher-configuration anchor on both renderers. Point those three at it and name the field in the text, for example:
Review the [`public_origin` warning under Publisher configuration](docs/guide/configuration.md#publisher-configuration) before deployment.
prk-Jr
left a comment
prk-Jr
left a comment
There was a problem hiding this comment.
Summary
Adds publisher.public_origin and emits absolute proxy, click, sign, rebuild, and runtime URLs on it. The code changes look correct, and the round-1 findings are resolved. The PR can't merge yet: it conflicts with main, so CI has not run on this head. Merging main also brings in a second "public origin" concept from #1251, and the merge should settle how the two relate.
Blocking
🔧 wrench
- Merge
mainand get CI to run on the head: see the body-level section below
Non-blocking
🤔 thinking
- Two "public origin" sources after merging #1251: see the body-level section below
Cross-cutting / body-level findings
-
🔧 Merge
mainand get CI to run on the head. This is the same open item as the previous review ona35fef44f. A trial merge with currentmain(d1bb3fa19) still conflicts inCHANGELOG.md,generate-viceroy-config.rs,tests/common/config.rs,tests/environments/axum.rs, andtests/environments/cloudflare.rs. Only the two CodeQLAnalyzechecks exist on this head. As a data point, the core library merges cleanly: with the integration-test conflicts taken frommain's side,cargo test -p trusted-server-core --libpasses (3351 passed). That doesn't exercise the merged integration-test logic, so the parity, integration, and browser jobs still need to run on the merged head. -
🤔 Two "public origin" sources after merging #1251. Since this branch's base,
mainaddedforwarder.rs. When an authenticated[trusted_forwarder]is configured, it derives a per-request public origin, andconfiguration.mdsays that value "supplies URL rewriting". Onmain, the inlinebuild_bid_mappath already takes itsrequest_originfrom it, throughRequestInfo::from_request. This PR adds a staticpublisher.public_originfor the other creative paths and documents "Do not derive it from request headers."With a forwarder configured and
publisher.public_originunset, one deployment then emits two authorities. Say the browser is onhttps://www.publisher.example: inline SSAT creatives usehttps://www.publisher.example, while/auctioncreatives, sign and rebuild responses, and the injected runtime usehttps://publisher.example.configuration.mdwould also use "public origin" for two different things.When resolving the merge, please state the precedence between the two in the
public_originand[trusted_forwarder]docs, or rename one term. Falling back to the forwarder origin would mean threading the request into the rewrite paths. That is probably out of scope here, so documenting the precedence is the minimal fix.
CI Status
- Analyze (javascript-typescript): PASS
- Analyze (python): PASS
- All other checks: not run. GitHub doesn't trigger
pull_requestworkflows while the PR has merge conflicts.
dhruv8sh
left a comment
dhruv8sh
left a comment
There was a problem hiding this comment.
Summary
Adds publisher.public_origin and emits absolute proxy, click, sign, rebuild and runtime URLs on it. The core and JS changes look correct, and the round-1 fixes hold up. Two things block the merge. First, the template's new placeholder breaks the documented local quickstart and the smoke scripts, which only overlay the three older publisher placeholders. Second, the PR still conflicts with main, so CI has not run on this head.
None of the inline comments carry a one-click
suggestion. The quickstart fix touches.env.dev,docs/guide/getting-started.mdandscripts/smoke-common.sh, which are all outside this diff, so it has to be applied by hand.
Blocking
🔧 wrench
- Template
public_originplaceholder breaks the.env.devquickstart and the smoke scripts: see inline attrusted-server.example.toml:68 - Merge
mainand get CI to run on the head: see the body-level section below
Non-blocking
👍 praise
- Rebuild rejection tests now carry valid tokens: see inline at
crates/trusted-server-core/src/proxy.rs:3281
Cross-cutting / body-level findings
- 🔧 Merge
mainand get CI to run on the head. This is still open from the two earlier reviews ona35fef44f.git merge-treeagainst currentmain(d1bb3fa19) still reports conflicts inCHANGELOG.md,generate-viceroy-config.rs,tests/common/config.rs,tests/environments/axum.rsandtests/environments/cloudflare.rs. GitHub reports the PR asCONFLICTING/DIRTY, and the only checks on this head are the two CodeQLAnalyzeruns. The PR description's "All GitHub checks passed on99d2109c2" refers to the previous head. Please update it once CI has run on the merged head.
CI Status
- Analyze (javascript-typescript): PASS
- Analyze (python): PASS
- All other checks (cargo fmt, cargo test, clippy, vitest, format-*, integration, browser, parity, lint-domains and the rest): not run. GitHub does not trigger
pull_requestworkflows while the PR has merge conflicts. No checks are marked required on this branch. - Run locally on
a35fef44f:cargo fmt --all -- --check: cleancargo test -p trusted-server-core --lib(native host): 3178 passedcargo test -p trusted-server-cli --test config_env_overlay --test config_store_defaults: 14 + 3 passedvitestwith the pinned Node 24.20.0: 52 files, 1261/1261 passed
| # Replace this placeholder before deployment. Keep this leaf for env overlays. | ||
| # Review the fallback and coordinated upgrade/rollback warning before deployment: | ||
| # docs/guide/configuration.md#public_origin | ||
| public_origin = "https://ads.example.com" |
There was a problem hiding this comment.
🔧 wrench: Making this placeholder an active leaf, and rejecting it at deploy and runtime, breaks the two flows that start from the template and fix only the three older publisher placeholders through environment overlays:
- Getting-started quickstart (
docs/guide/getting-started.md, Options A and B):cp trusted-server.example.toml trusted-server.toml,source .env.dev,ts config push --local..env.devsetsDOMAIN,COOKIE_DOMAINandORIGIN_URLonly. - Smoke scripts (
scripts/smoke-common.shsmoke_initialize_config, used bysmoke-{axum,fastly,cloudflare,spin}.sh):ts config init, the same three overlays, thents config validate --strict.
Reproduced with this head's ts binary:
$ set -a && source .env.dev && set +a
$ ts config push --adapter axum --local --yes
[ts] typed app-config failed validation: trusted_server: Insecure default value for: publisher.public_origin
exit=2 (no .edgezero/ output)
$ ts config init … && TRUSTED_SERVER__PUBLISHER__{DOMAIN,COOKIE_DOMAIN,ORIGIN_URL}=… ts config validate --strict
[ts] typed app-config failed validation: trusted_server: Insecure default value for: publisher.public_origin
exit=2
Adding TRUSTED_SERVER__PUBLISHER__PUBLIC_ORIGIN=http://localhost:8787 makes the same push succeed. crates/trusted-server-cli/tests/config_store_defaults.rs needed exactly this fourth overlay in this PR to keep passing. No CI job runs the quickstart or the smoke scripts, so CI won't catch this once it runs.
Proposed fix (apply manually: .env.dev, docs/guide/getting-started.md and scripts/smoke-common.sh are all outside this diff):
# .env.dev, under [publisher]
# Browser-facing endpoint for rewritten creative URLs (Viceroy quickstart port).
TRUSTED_SERVER__PUBLISHER__PUBLIC_ORIGIN=http://localhost:7676# docs/guide/getting-started.md, Option B, after sourcing .env.dev
# The Axum dev server listens on 8787, not Viceroy's 7676.
export TRUSTED_SERVER__PUBLISHER__PUBLIC_ORIGIN=http://localhost:8787# scripts/smoke-common.sh, smoke_initialize_config
export TRUSTED_SERVER__PUBLISHER__PUBLIC_ORIGIN="http://127.0.0.1"The smoke assertions don't follow creative URLs, so any non-placeholder value works there. Use the listener port instead if a smoke case later exercises rewritten creatives.
| } | ||
|
|
||
| #[test] | ||
| fn proxy_rebuild_rejects_invalid_click_schemes_and_paths_with_valid_tokens() { |
There was a problem hiding this comment.
👍 praise: Signing the rejection cases with valid tokens and asserting invalid tsclick path means a signature failure can no longer make this test pass vacuously. The companion test also pins every accepted authority shape (//, \\, leading whitespace, https:/) to the configured origin. That is the property that matters now that the input authority is discarded.
Resolve base-branch merge conflicts while preserving absolute creative URLs, trace fixtures, and controlled bidder backends.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Summary
srcset, creative-runtime, sign-response and rebuild-response URLs using validated, normalizedpublisher.public_origin.main, including its separate click switch and asset-host allowlist. Fix runtime-specific integration origins, foreign-parent browser coverage and the browser CI generator dependency.Closes #915
Spec
Absolute creative rewrite URLs, with two independent source-backed review rounds recorded in its review log.
Changes
publisher.public_origin; normalization at Settings/app-config loading and construction; TOML/JSON/blob and present/absent-leaf CLI overlay coverage.GET rebuild routing and absolute-click input support already exist on
main; they are not new routes in this PR. Useful compatibility/route tests remain. The inline SSAT/page-bids path keeps its request-origin override and no runtime injection. TSJS source selection and hashing are unchanged. No CORS expansion or signing-format migration.Compatibility and rollout
An omitted
public_originstays omitted in serialized configuration, but generated general-purpose URLs now usehttps://{publisher.domain}rather than following the visitor's scheme, host and port. Verify the actual browser-facing endpoint, including local HTTP ports. Old binaries reject the new key, so coordinate binary/config activation and rollback when the fallback cannot serve traffic. CLI environment overrides only replace an existing TOML scalar leaf.Test plan
Validation uses Rust 1.98.1, Node 24.20.0 and standalone Viceroy 0.21.0. Local artifacts and logs are retained outside the worktree.
cargo test-fastly && cargo test-fastly-reuse && cargo test-axum && cargo test-cloudflare && cargo test-spincargo fmt --all -- --checkts dev lint domains --changed-vs origin/main.fastly compute servesession. Automated Viceroy/Chromium checks cover the runtime path instead.All local gates above are complete. Native core rustdoc succeeds with 31 link warnings on unchanged
maindocumentation; it is not warning-clean. All GitHub checks passed on99d2109c2, including the fresh-runner browser config-generator build, both browser framework suites, integration matrix and Fastly EC lifecycle tests.Review
Four independent final lanes covered correctness, architecture, code smells and style. The missing browser-CI generator and changed-line example-host issues were fixed and re-reviewed. No blocking production findings remain. One nonblocking maintenance suggestion, sharing the two integration-fixture retargeting helpers, is deferred to avoid merging their distinct upstream/timestamp/error contracts in this fix.
Checklist
AGENTS.mdconventions.unwrap()calls or plaintext production credentials.