Skip to content

Presize dense tag stores and seed span prototypes in bulk - #12926

Draft
dougqh wants to merge 16 commits into
dougqh/tagmap-split-storesfrom
dougqh/prototype-fast-path
Draft

dougqh wants to merge 16 commits into
dougqh/tagmap-split-storesfrom
dougqh/prototype-fast-path

Conversation

@dougqh

@dougqh dougqh commented Oct 18, 2026

Copy link
Copy Markdown
Contributor

What Does This Do

Optimizations on top of the store split (#12918), mostly for the dense tag store (trace.experimental.dense.tags.enabled, off by default). With the flag off, spans allocate exactly what #12918's do.

  • Presize each span's dense store from a per-operation sizing hint. A port of Store known span tags densely, behind an experimental flag (phase 3b - dense tag storage) #12856's SizingHint / SizingHintTable: a bounded, racy table of hints, keyed by operation name, with separate lanes for entry spans and child spans.
    • Where the hint lives: on the span's DenseIdStore, not on DDSpanContext, so flag-off spans carry no extra field.

    • When spans record their size:

      • a published span, when it's serialized, with every tag present;
      • a span in a dropped trace, in CoreTracer.write, after the early post-processors and the root's sampling decision have added their tags.

      Recording at span finish would miss those late tags, and every later span of the operation would grow its store to twice the hint.

  • One reference store per dense write path. store and getAndPut resolve the slot index first, then store the value once. On JDK 17 and earlier, C2 expands each reference-store site's G1 write barriers inline, at about 250 B apiece.
  • Seed a SpanPrototype's known tags in bulk. The new method IdStore.seedAbsent appends a frozen, parentless, dense source with two array copies, provided the source's occupancy words share no bit with the target's live, tombstoned or inherited words, checked per level. If they share none, no prototype tag can already be present, so "absent only" holds without a per-tag check. If they share any, it copies nothing and the caller falls back to seeding tag by tag.
    • The plan: whether to use the bulk path is decided once per prototype and cached on it. It records whether the store supports bulk seeding, whether the interceptor needs any known tag other than span.kind, and span.kind's ordinal, which is set directly.
    • Where it applies: both at span construction and in applyOverwriting, the path BaseDecorator.afterStart takes today.
    • Custom prototype tags always go one at a time.
  • Sizing hints cached on the prototype. A span that uses its prototype's own operation name takes the hint cached on the prototype and skips the table lookup. A span given any other operation name gets that name's hint from the table, so the cache never shares a hint the table wouldn't.
  • singleSpanBuilder(SpanPrototype, …), so the prototype entry points mirror the plain ones (startSpan, buildSpan, singleSpanBuilder).
  • Public forEach on HashStore and IdStore, so the prototype paths can walk a prototype's custom store directly.

Motivation

Without sizing, every fresh span grew its dense store on its first tag. C2 then inlined grow() and its two allocations into DenseIdStore.store, which compiled to 2816 B, over InlineSmallCode, so every setTag called store out of line. With the hints converged, grow() is cold:

Fresh spans, dense on, JDK 17 before after
DenseIdStore.store 2816 B 768 B, inlined again
DDSpanContext.setTag(long, int) 3648 B 1792 B
DDSpanContext.setTag(long, CharSequence) 3328 B 1408 B

The id-keyed setters now compile under the 2500 B limit with the store inlined, which C2 needs before it can coarsen the locks in a decorator's run of setTag calls.

Measurements

SpanCreationBenchmark on an M1 Max with Zulu 17, -Xms2g -Xmx2g, 4 forks, 5 × 5 s iterations, -prof gc. Experiment jars carried a LongAdder fix for SpanMetricsImpl's counters, which isn't part of this PR; without it, counter contention dominates at 8 threads.

Bytes per span, dense on: jdbc-shaped spans drop by −130 to −196 B (about 12%) at 1 to 8 threads. Web-shaped spans barely change, because they already fit their first allocation. Flag off, bytes are identical to #12918.

Throughput, dense on, vs. #12918: flat. Single-threaded it's within ±1%, except bare spans, which are about 4% slower from the per-span hint lookup. With the LongAdder fix, this branch is ahead in 13 of 16 cells at 1 to 8 threads (+0.5% to +4.5%), but only one cell clears the error bars. Scaling to 8 threads is limited by per-trace costs outside the tag store, so the byte saving doesn't show up as throughput yet.

Prototype entry points (jdbc, dense on, with the LongAdder fix, ops/µs):

entry point 1 thread 8 threads
direct setTags 2.16 ± 0.04 7.43 ± 0.34
startSpan + applyOverwriting (decorators today) 2.27 ± 0.02 7.72 ± 0.14
buildSpan(prototype) 2.36 ± 0.02 7.72 ± 0.20
singleSpanBuilder(prototype) 2.29 ± 0.02 7.21 ± 0.24
startSpan(prototype) 2.36 ± 0.02 7.73 ± 0.16
  • Construction vs. after-start stamping: building the span from the prototype beats stamping it after start by about 4% single-threaded. That's two lock acquisitions fewer and no hint lookup. At 8 threads the two are even.
  • The bulk seed itself is within noise, because prototypes carry only two or three known tags today. It pays off as prototypes take on more constants.
  • singleSpanBuilder(prototype) measures slower than buildSpan(prototype), whose builder escape analysis already removes. It's kept for API symmetry.

Additional Notes

  • Stacked on Split TagMap into id and name stores, owned directly by the span #12918. Review this PR's own commits.
  • Reviews: /techdebt and /perf-review ran over these commits, and their findings are addressed. One was a real default-path regression: an extra lock acquisition in applyOverwriting with the flag off, which now has a test guarding it. One remains open: the hint only ever grows, so one unusually large span presizes every later span of its operation. That needs a benchmark with mixed span sizes before deciding on decay.
  • Tests:
    • SizingHintTable{,HashOnly}ForkedTest;
    • seedAbsent cases in TagStoresTest;
    • prototype construction, overwrite and custom-tag cases on both stores: SpanPrototypeConstructionDenseForkedTest reruns the suite with the dense store on and asserts that it is.
  • Follow-ups, kept separate on purpose:
    • SpanMetricsImpl counters to LongAdder: +28% at 8 threads in these runs, a separate small PR off master;
    • removing read-through;
    • cheap span snapshots, and moving post-processing off the application thread.

🤖 Generated with Claude Code

dougqh and others added 9 commits October 17, 2026 16:54
store and getAndPut wrote values[i] on the overwrite path and again in
appendSlot on the append path. Under G1 on JDK 17 and earlier C2 expands
each reference-store site's write barriers inline, roughly 250 bytes
apiece, and these paths inline into every setTag. Both now resolve the
index first (appendSlot reserves the slot and returns it) and store the
value once; the tombstone path uses reserveSlot the same way.

On fresh spans this alone trims appendSlot (2560 -> 2240 bytes of
compiled code) and the by-name setters by about 64-128 bytes; most of
the remaining size comes from grow() running on every span's first tag,
which the next commit addresses with sizing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ports #12856's SizingHint / SizingHintTable onto the store split. The
tracer looks up a hint by operation name (entry and child lanes, bounded,
racy by design) and the span's DenseIdStore allocates its arrays at the
hint's size when it is created. Spans record their size back when their
trace is written -- after the early post-processors and the root's
sampling decision have added their tags -- and when serialized.

Without it every fresh span grew its store on its first tag, so C2
inlined grow() and its two allocations into DenseIdStore.store (2816 B
of compiled code), over InlineSmallCode, and every setTag called store
out of line ("already compiled into a big method"). With the hints
converged grow() is cold, store compiles to 768 B and inlines again:
on fresh spans, dense on, JDK 17, setTag(long, int) is 1792 B and
setTag(long, CharSequence) 1408 B, under the 2500 B limit.

Recording at trace write rather than at span finish matters: a hint
taken at finish misses the sampling tags, so every later span grew its
store from the hint to twice that.

Dense on, single-threaded SpanCreationBenchmark: realistic spans
allocate 24-196 B less per span (jdbc -196, web -96); throughput is
unchanged except bare spans (~4% slower, the per-span hint lookup).
With the dense store off the hint is null and the lookup folds away.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the dense store, a prototype's known tags are now copied into a
span at once instead of one at a time through isPresent, the
interceptor and routePut:

- IdStore.seedAbsent appends a frozen, parentless, tombstone-free dense
  source with two array copies when its occupancy words share no bit
  with the target's live, tombstoned or inherited words, per level:
  then none of its tags can already be present, so "absent only" holds
  without a per-tag check. Otherwise it copies nothing and the caller
  seeds tag by tag. The hash store always declines.
- TagInterceptor.canSeedInBulk: no known tag needs the interceptor
  except span.kind, whose one effect -- the cached kind ordinal -- the
  seed sets directly. A split-by-tags span.kind counts as a hazard.
- The plan (bulk or not, the kind ordinal, and the interceptor it was
  checked against) is cached on the SpanPrototype on first use, racy
  and immutable. Custom prototype tags are always seeded one at a time.

SpanPrototypeConstructionDenseForkedTest reruns the prototype tests
with the dense store on, plus the fallback for a span that already has
one of the prototype's tags. The TagStoresTest cases for seedAbsent
also caught a bug from the single-store commit (now fixed there): the
tombstone path indexed this.values before reserveSlot could replace it.

Measured gain is small today: prototypes carry two or three known tags,
so single-threaded prototype span creation is within about +-1%. It
grows with prototypes that carry more constants.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The span held its SizingHint in a new DDSpanContext field, so with the
dense store off -- the default -- every span carried an always-null
reference: DDSpanContext grew from 160 to 168 bytes (+8 B per span in
the 8-thread SpanCreationBenchmark, hash store).

The hint now lives on the DenseIdStore it presized, and recordSize()
takes no argument: the span asks its store, the hash store does
nothing. DDSpanContext is back to 160 bytes; DenseIdStore grows from 72
to 80, paid only with the dense store on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A span built from a SpanPrototype that uses the prototype's own
operation name now takes its sizing hint from the prototype, which
resolves it once per lane (entry, child) and caches it, so those spans
skip the SizingHintTable lookup. A span given a different operation
name still gets that name's hint from the table: hints stay keyed by
operation name, so the cache never shares a hint the table wouldn't.

SpanPrototypeConstructionDenseForkedTest now turns the dense store on
with @WithConfig. Its static System.setProperty was cleared by the test
extensions before Config read it, so the class had been running on the
hash store (where the prototype tests pass too); theDenseStoreIsOn now
guards that. Adds a test for the cached hint.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The prototype entry points now mirror the plain ones: startSpan
(builder-free), buildSpan (a builder for any number of spans) and
singleSpanBuilder, which hands out the thread's reusable builder
carrying the prototype, so it doesn't allocate. The builder drops the
prototype on reset, so it can't leak into the thread's next span.
AgentTracer's default seeds identity only, like buildSpan's.

Adds jdbcClientSpanViaPrototypeSingleSpanBuilder to
SpanCreationBenchmark, next to the buildSpan and startSpan variants.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
applyOverwriting -- what BaseDecorator.afterStart does to every span
today -- stamped the prototype's tags one at a time through the
interceptor. Overwriting a tag that isn't there is just appending it,
so when the masks prove none of the prototype's known tags is on the
span (the usual case right after startSpan) they are now copied in at
once with IdStore.seedAbsent, and span.kind's ordinal is set directly,
like the construction-time seed. A conflict falls back to overwriting
tag by tag; custom tags always go through the interceptor.

Adds tests for stamping after start and for overwriting a tag the span
already has (both stores), and webServerSpanRetrofit /
jdbcClientSpanRetrofit to SpanCreationBenchmark: startSpan by name,
then applyOverwriting, as decorators do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Flag off, every decorated span took the tag lock twice: applyOverwriting
  tried the bulk path (the plan said bulk), whose seedAbsent always fails on
  the hash store, then locked again for its fallback. The seed plan is now
  bulk only when the store can seed in bulk (IdStore.supportsBulkSeed, true
  on DenseIdStore), so the default path skips the attempt and its lock.
  Guarded by a test on both stores.
- Published spans record their size when serialized, with every tag
  present; CoreTracer.write records only the traces it drops, so kept
  traces no longer walk their spans for it on the application thread.
- seedPrototypeTags and overwriteInBulk share seedKnownInBulk; the
  intercept-or-put step is putIntercepted, also used by setAllTags.
- IdStoreFactory.create(parent) and DenseIdStore(parent) delegate to their
  hinted forms; IdStore.create(TagMap) likewise; grow() delegates to growTo.
- The hint lookup is one call: NONE has no operation name, so
  SpanPrototype.sizingHint already falls through to the table.
- Docs: the orphaned needsInterceptCustomTag Javadoc is reattached, apply's
  stale "will land here" paragraph points to seedPrototypeTags, the
  hint-less DDSpanContext constructor is marked for tests and benchmarks,
  and a comment in SizingHintTable is rewrapped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HashStore's and IdStore's forEach overloads are public, so the
prototype paths in DDSpanContext walk a prototype's custom store
directly instead of iterating its whole map and skipping known tags.
Adds a test for a prototype carrying a custom tag, seeded at
construction and overwritten after start (both stores).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dougqh dougqh added type: feature Enhancements and improvements comp: core Tracer core tag: performance Performance related changes tag: no release notes Changes to exclude from release notes tag: ai generated Largely based on code generated by an AI or LLM labels Oct 18, 2026
@datadog-official

This comment has been minimized.

@dd-octo-sts

dd-octo-sts Bot commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.75 s 14.59 s [+0.3%; +1.9%] (maybe worse)
startup:insecure-bank:tracing:Agent 13.62 s 13.64 s [-1.0%; +0.7%] (no difference)
startup:petclinic:appsec:Agent 17.67 s 17.53 s [-0.1%; +1.7%] (no difference)
startup:petclinic:iast:Agent 17.43 s 17.43 s [-0.8%; +0.9%] (no difference)
startup:petclinic:profiling:Agent 17.24 s 17.10 s [-0.0%; +1.6%] (no difference)
startup:petclinic:sca:Agent 17.62 s 17.52 s [-0.3%; +1.4%] (no difference)
startup:petclinic:tracing:Agent 16.49 s 16.64 s [-1.8%; +0.1%] (no difference)

Commit: 0c628d32 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

dougqh and others added 7 commits October 19, 2026 12:52
…ype-fast-path

Clean merge; picks up #12918's reconciliation with the squashed #12882.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Clean merge; the IdStore contract test and DDSpanContextStoresTest pass
here too, with this branch's sizing and bulk seed in place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	internal-api/src/main/java/datadog/trace/api/DenseIdStore.java
With per-operation sizing hints presizing each span's store, the dense
store is the default here, so CI and benchmarks exercise it. Set
trace.experimental.dense.tags.enabled=false to use the hash store.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: core Tracer core tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes tag: performance Performance related changes type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant