Skip to content

[Subscription] Resume realtime delivery after WAL gap recovery - #18912

Merged
jt2594838 merged 1 commit into
apache:masterfrom
Caideyipi:fix/subscription-realtime-recovery
Oct 19, 2026
Merged

jt2594838 merged 1 commit into
apache:masterfrom
Caideyipi:fix/subscription-realtime-recovery

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Description

After a transient realtime admission failure, subscription WAL gap recovery can replay requests that have already been admitted into the pending queue. For example, rejecting index 1 and then admitting indexes 2–8 causes all eight requests to be delivered through WAL, while their pending copies are discarded as duplicates. This prevents the recovered realtime path from supplying those requests.

Stop gap replay at the next pending index and resume direct pending delivery. If a sparse WAL jumps to or past that boundary, retain its buffered request and report only the missing prefix. Replicated requests without local search indexes before the boundary are still replayed. Existing scan budgets, memory limits, ordering, and commit handling continue to apply.

Validation

  • Confirmed both admission-recovery regressions fail before the fix: WAL accepts 8 and 2 requests respectively instead of the single missing request.
  • Added real-WAL coverage for consensus-memory rejection, ACK-driven recovery under materialization memory pressure, a missing WAL prefix, and replicated writer entries before the pending boundary. Assertions cover delivery order, source counters, committed progress, and released memory.
  • Made the existing mixed pending/WAL redelivery fixture independent of cold-start scan deadlines; confirmed its original single-batch assertion also fails on unmodified master when the scan yields.
  • Full English reactor: mvn -nsu test -Dtest=ConsensusPrefetchingQueue*Test,ProgressWALIteratorTest -Dsurefire.failIfNoSpecifiedTests=false -Ddevelocity.off=true succeeds across all 53 modules; 111 tests pass and one opt-in performance test is skipped.
  • Full Chinese reactor: mvn -nsu test-compile -Pwith-zh-locale -DskipTests -Ddevelocity.off=true succeeds across all 53 modules.
  • mvn spotless:apply -pl iotdb-core/datanode and git diff --check pass.

The original pressure workload has not been rerun in a cluster.

This PR has:

  • been self-reviewed.
  • added comments explaining the replay boundary and sparse-WAL handling.
  • added unit tests covering the changed behavior.

@jt2594838
jt2594838 merged commit 4acf90c into apache:master Oct 19, 2026
38 of 39 checks passed
@jt2594838
jt2594838 deleted the fix/subscription-realtime-recovery branch October 19, 2026 05:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants