Repository navigation
feat(oracles): require every oracle test to reach a declared part - #2278
Merged
Merged
Conversation
Co-authored-by: Isaac <no-reply@databricks.com>
Declare the retained-state check so the retention histories and the partial-update lanes count as oracle coverage. The 28 tests that compare production with hand-written values or check generator coverage move unchanged to collection-state-retention.test.ts. Helpers shared with the oracle move to collection-state-retention-fixture.ts. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Declare the expected-map check so the randomized and fixed histories count as oracle coverage. The ordinary SortedMap behavior tests move unchanged to SortedMap.test.ts. Co-authored-by: Isaac <no-reply@databricks.com>
Declare the history runner, its agreement checks, and the Map model helpers. The inline Duration expectation moves unchanged into a named check, expectDurationFieldPublication, so the witness reaches a declared part. The grammar-coverage test moves to change-event-history.test.ts and its grammar to change-event-history-fixture.ts. Co-authored-by: Isaac <no-reply@databricks.com>
Declare the signature-class check. The group-label comparison that was inline in the first test moves unchanged into a named check, expectSignatureClasses. The two metadata-isolation unit tests move to collection-index-value.test.ts, with their shared source helper in collection-index-value-fixture.ts. Co-authored-by: Isaac <no-reply@databricks.com>
Name the per-caller outcome and uncaught-report comparisons, and the reported-value model, as declared functions. The inline assertions keep their logic and order; the matrix test now reaches declared parts. Co-authored-by: Isaac <no-reply@databricks.com>
Name the expected-status model and the publication and reentrant status comparisons as declared functions. Assertions keep their logic and order. Co-authored-by: Isaac <no-reply@databricks.com>
Declare the field-level equality model and route each exact key-set comparison through a named check, expectExactKeys. Assertions and their labels are unchanged. Co-authored-by: Isaac <no-reply@databricks.com>
Declare the size-suggestion and ordered-diagnostic models and checks. The inline suggestion and warning comparisons move unchanged into named checks. The five hand-written message examples move to index-suggestion.test.ts, keeping their bounded orderBy describe. Co-authored-by: Isaac <no-reply@databricks.com>
Declare the history runner and reference rows for the authority grammar. The interleaving describe's inline reference and its accepted-reader and publication comparisons move unchanged to named top-level functions, so the grammar reaches declared parts. Co-authored-by: Isaac <no-reply@databricks.com>
The file has no declared check or model. Its tests compare production with literal scenario rows, or compare two production outputs, so none is oracle coverage. The file moves to db-client-preload-identity.test.ts with its 17 tests unchanged. docs/contributing/oracle-coverage.md still names the old path and needs updating by the maintainer. Co-authored-by: Isaac <no-reply@databricks.com>
Declare the direct-versus-confirmed comparison as a named check, and move the unit witnesses to local-only-direct-write.test.ts: the single-publish and fallback-visibility tests and the schema-rejection test. The shared order helpers and the confirmed handlers move to local-only-direct-write-fixture.ts. Counts: 6 oracle + 22 ordinary = 28. Co-authored-by: Isaac <no-reply@databricks.com>
Declare the read checker and its order and read models. Two describe blocks of hand-written reversal examples (legacy reversal and a range-only index) move unchanged to reverse-index.test.ts. Counts: 4 oracle + 5 ordinary = 9. Co-authored-by: Isaac <no-reply@databricks.com>
Declare the read and history checks and the captured-rows model. Every test reaches a declared part; the count is unchanged at 6. Co-authored-by: Isaac <no-reply@databricks.com>
Extract the inline delivery and task-error comparison into a named check, expectQueueState, with explicit inputs and unchanged logic. Declare it with the step model. All 6 oracle tests reach declared parts. Co-authored-by: Isaac <no-reply@databricks.com>
Declare the imported expectHistoryOutcome in this file, so the observer calibrations reach a check; the companion module is unchanged. The cleanup-failure harness self-test, which exercises withHistoryCleanup and reaches no check, moves to optimistic-history-outcomes.test.ts. Counts: 23 oracle + 2 ordinary = 25. Co-authored-by: Isaac <no-reply@databricks.com>
Extract the bodies of the eight golden-record and differential tests into named checks with unchanged logic, and declare them. The sample-coverage pin moves to production-error-messages.test.ts; the exported class list it and the checks share moves to production-error-messages-fixture.ts. Counts: 18 oracle + 1 ordinary = 19. Co-authored-by: Isaac <no-reply@databricks.com>
…ests Extract the expected-owner fold and the visible-row comparison of the generated matrix into a named model and check, and declare them. Seven hand-written ownership scenarios move to optimistic-composition.test.ts, with shared fixtures in optimistic-composition-fixture.ts. Counts: 72 oracle + 12 ordinary = 84. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
observeArray and observeTyped record a call on a native or a proxied row, so they observe either side and predict nothing. The model is native behavior, which the checks compare with. Co-authored-by: Isaac <no-reply@databricks.com>
Declare the cursor denotation, local-order, and repeated-construction checks with the reference comparison and its model. The one test that compares buildCursor with hand-written values moves unchanged to cursor.test.ts. Co-authored-by: Isaac <no-reply@databricks.com>
Its tests compare index and live-query order with literal expectations and declare no check or model, so the file is not an oracle. The rename keeps every test name and body unchanged. Co-authored-by: Isaac <no-reply@databricks.com>
Its type assertions compare inferred types with hand-written type literals and declare no check or model, so the file is not an oracle. The rename keeps every test name and body unchanged. Co-authored-by: Isaac <no-reply@databricks.com>
Declare the native-visit comparison and the member checker. The native comparisons in the array and caller-owned insertion tests call the named check. The draft iteration, identity boundary, and literal nested-iteration tests move unchanged to proxy-iteration-contract.test.ts. Shared Item, protocols, and visit helpers move to proxy-iteration-fixture.ts. Co-authored-by: Isaac <no-reply@databricks.com>
Declare the cycle-shape, own-property, JSON, and binary-clone checks. The JSON and binary comparisons against native structuredClone now call named checks. Literal detachment regressions move unchanged to proxy-detachment-contract.test.ts. Shared fixtures and checks move to proxy-detachment-fixture.ts. Co-authored-by: Isaac <no-reply@databricks.com>
Declare the history driver and the change-set model. The fallback, detached-assignment, and descriptor-literal tests move unchanged to flat-change-tracking.test.ts. Shared row fixtures move to flat-change-tracking-fixture.ts. Co-authored-by: Isaac <no-reply@databricks.com>
Extract the snapshot-reentry expectations into the named check expectSnapshotReentryOutcome and declare it. The scenario regressions with literal expectations move unchanged to collection-subscription-reentrancy.test.ts. The shared predicate helper moves to collection-subscription-reentrancy-fixture.ts. Co-authored-by: Isaac <no-reply@databricks.com>
…mmar Declare the last-write-wins model and its mismatch comparison. The two inline lane and nested expectations move into named model functions, and the grammar calibration tests reach the declared history grammar. Every test in the file reaches a declared part, so no test moves. Co-authored-by: Isaac <no-reply@databricks.com>
expectMembers compares with whatever the caller passes, so it is not a declared check. The persistence test now declares the native write model that produces its expected members. The member-literal tests and the expectMembers calibration move unchanged to proxy-iteration-contract.test.ts. Co-authored-by: Isaac <no-reply@databricks.com>
The per-election leadership-need cells and two LocalStorage order histories moved out of their oracle files. Name the fixed-test files that now hold them and record that they have no oracle owner yet (#2264). Co-authored-by: Isaac <no-reply@databricks.com>
…movals check-oracle-run-counts now attributes the budgets in the test modules that an oracle file imports to that oracle file, so a campaign that moves into a support module keeps its tracking. A budget that moves to another oracle file is compared there. A key that is in no tracked oracle file at head is reported as removed (maintainer decision, 2026-10-27). The script reads base files above 1 MiB, treats only a missing file as missing, recognizes JavaScript oracle files and modules, counts untracked files as changed, and no longer runs its report when it is imported without argv[1]. The run-budget oracle draws where each campaign is at base and at head (oracle file, support module, ordinary file, second oracle file, removed), a base oracle file above 1 MiB, .ts and .js files, an untracked second oracle file, and a fixed seed shared with an unrelated campaign. Its driver fails on a crashed script, each campaign has its own literal fixed seed, and runs use private temporary directories. - Law: L5, every lowered or removed budget that an oracle property uses is reported (AGENTS.md "Lint feedback while you work"; removal rule from the maintainer decision of 2026-10-27). - Reach: all 16 declared states; fixed-seed campaigns alone reach each, for example shared-seed-kept 2, moved-to-oracle 4, large-base-file 27. - RED: the extended oracle against the script at 6f55107 fails 8/8 campaigns. - GREEN: 8/8 campaigns pass. - Mutants (copies via RUN_BUDGET_SCRIPT), all rejected: startup crash (driver failure), default maxBuffer (driver failure, ENOBUFS), no support-module attribution, no removal report, removal judged per file, untracked files dropped, .js excluded (assertion failures). - Limits: a fixed seed is not unique, so a key that another oracle file had at base is neither compared nor reported as removed. Co-authored-by: Isaac <no-reply@databricks.com>
…an edit agent-lint-feedback selects oracle files from every file in the working tree, tracked or untracked, and by the same TypeScript-or-JavaScript name rule as the run-count script. Its linter applies the oracle rules to every file that it passes, so a .js oracle file is linted. The lint-scope oracle's model is now a fixed-point closure over the drawn import graph, not a copy of the script's forward search. Its grammar adds oracle-to-oracle imports, untracked oracle files, .js oracle files, and an edit-hook target drawn over every module kind and the rule's source. The three scopes run on one repository; a crashed CLI fails the driver; runs use private temporary directories; histories yield to the test worker between runs; and the budget is 24 runs, so the fixed-seed campaign alone reaches every declared state at least twice. - Law: L4, every oracle file whose lint result can change after an edit is linted again (AGENTS.md "Lint feedback while you work" and "Name oracle files for discovery"). - Reach: all 15 declared states, minimum 2; for example oracle-dependent 8, untracked-dependent 15, js-oracle 19, hook-driver 7. - RED: against the script at 6f55107, both campaigns fail (a .js oracle file is not selected). - GREEN: both campaigns pass, on two random seeds. - Mutants (copies via LINT_SCOPE_SCRIPT): startup crash, untracked files dropped, .js excluded, one-hop search, oracle files not followed, rule edit ignored, plugin globs restored. - Limits: support modules and drivers are .ts only, because the rule resolves only TypeScript test modules; file names are passed in their real-path spelling. Co-authored-by: Isaac <no-reply@databricks.com>
An owner outside packages/ replayed from scripts/eslint-plugin-oracle, so the run-budget and lint-scope replays in scripts/ matched no test file and reported only that their target never executed. Such an owner now replays from its own directory, with `--config oracle-scripts.vitest.config.mjs` where the directory's config has no default name. The replay test checks every manifest owner: from its cwd, each file that the command names exists, the last one is the owner, and a Vitest config applies. On the old manifest it fails for exactly scripts.run-budget and scripts.lint-scope; it passes for every owner now. A guarded replay of scripts.lint-scope (seed 42, path 0) records one execution and exits 0. The coverage map gains an Oracle tooling section for the run-budget and lint-scope owners and their limits. Co-authored-by: Isaac <no-reply@databricks.com>
…out in no-unit-test The second review found that oracle/no-unit-test misjudged real oracle shapes: - B1: a caller-passed body was taken from the call's last argument. - B2: an imported predicate counted under fc.property but not under .prop. - B5: helpers reached through namespace or default imports, and helpers in oracle- machinery modules, escaped the imported-registration check. - B9: reach was lost through namespace and default imports, let bindings that a hook assigns, and for...of loop variables. - B10: a helper that wraps its caller's body counted as its own test. - B12: a spread in declareOracle crashed the rule. - B13: a named fc.constant counted as a generated arbitrary. - .js companion modules did not resolve. Law: every registration in an oracle file reaches a declared part or a generated property, or is reported (rule header contract; rubric "Name oracle files consistently"; ECMAScript: export * carries no default). Reach: 20 new declared states in the part-reach oracle, each entered by generated histories (counts in the PR evidence), plus fixed inputs for each reviewer repro through all four receiving paths. RED: on 6f55107 the extended oracle failed 24 tests; a probe confirmed B2 and B13 in both directions; the JavaScript-companion campaign failed on the rule before .js resolution. GREEN: pnpm test:lint-rules rule suites, 291 tests pass. Mutants: 14 mutants on plugin copies (last-argument body, imported predicate true/false, machinery exemption, namespace callee, let ignored, for...of with some instead of every, namespace member ignored, wrapper as own body, wrapper part ignored, spread ignored, raw constant check, default through star, .js unresolved) all fail the oracle by assertion. Limits: a let assigned more than once, a for...of over a computed array, a predicate or arbitrary from a package, and a .js companion's type syntax are not generated. Scratch directories of the CLI paths are now gitignored. Co-authored-by: Isaac <no-reply@databricks.com>
#2168 splits Collection status from sync errors, adds reload() and cleanup admission, and adds the health tree. Its edits to tests that this branch had moved out of oracle files are carried into the ordinary files that now hold those tests, so no moved test keeps main's old behavior. Resolution notes: - Content that earlier merges on this branch dropped by mistake is restored from main where it still applied (the React descriptor-binding comment, the render sync-start handoff comment, and the deferred-acquisition section prose). - Tests that #2168 added are placed by the oracle/no-unit-test law: a test that reaches a declared part or runs a generated property stays in the oracle file; one that compares with written values goes to the ordinary sibling. - Main's new oracle files now declare their parts: source-health, live-query-observer-hydration, persisted-reload-replacement, and persisted-sync-problem. source-health keeps 504 tests and moves 97 fixed tests to source-health.test.ts; the 601 titles equal main's. - The coverage map keeps main's new text and this branch's fixed-test notes. Co-authored-by: Isaac <no-reply@databricks.com>
…on-enforcement tests The oracle plugin now applies to .js, .mjs, and .jsx files whose names contain oracle, as the lint selection and run-count script already do. That surfaced the action-enforcement example's lint-rule suite, which declared no oracle part. Its expected diagnostic counts come from a written table chosen by the method under test, which the 2026-10-27 maintainer decision classifies as fixed tests, not a model. The file is renamed to no-direct-collection-mutations.test.js with its 32 tests unchanged, and the coverage map records it as fixed tests with no oracle owner (#2264). Co-authored-by: Isaac <no-reply@databricks.com>
…acle The fixed and random persistence.generation-notice campaigns moved to persisted-reset-startup.test.ts because oracle/no-unit-test did not count a .prop registration with an imported predicate (review finding B2). The rule now reads an imported predicate at its definition, so the campaigns return to the oracle file under their original describe and titles, with their check and grammar declared. The run-count script no longer reports their 24-run budget as removed. Co-authored-by: Isaac <no-reply@databricks.com>
…detachment oracle "copies ordinary fields without descriptor writes" is a generated property: for any number of ordinary fields, one write changes only that field and defines no property descriptors. Its predicate reads the generated count, so it states its law inline, which the oracle/no-unit-test contract accepts. The migration had moved it to the ordinary file with the literal regressions, which also took its fixed seed 20260914 out of run-budget tracking. It returns to proxy-detachment-contract-oracle.test.ts with its title, seed, and budget unchanged. Co-authored-by: Isaac <no-reply@databricks.com>
…cking limit The op-sqlite alias campaigns build their budget in registerAliasOracle at run time, so the run-count script cannot see it, and the unit test whose literal calls it used to count now lives in an ordinary file. Record the limit, its owner, and the witness it needs, and update the lint-scope note now that the rule resolves .js test modules. Co-authored-by: Isaac <no-reply@databricks.com>
…owner The generation-notice campaigns returned to persisted-oracle.test.ts, so the replay manifest's owner for persistence.generation-notice moves with them. oracle-replay.test.ts checks every owner against its source file. Co-authored-by: Isaac <no-reply@databricks.com>
From the code-simplifier pass, each verified on the current head: - Share the helpers that this branch's oracle/ordinary splits duplicated: gate and turn in the offline-transactions harness (removing the now empty transaction-settlement-support.ts), nodeSqliteDriver and toSqliteBinding in the electric persistence fixture, createDeferred and createOrderByInfo from ordered-source-loader-support, and storageId from browser-coordinator-support. - In oracle/no-unit-test, read declared parts from the resolved-reference map, drop an export loop that getDeclaredVariables already covers, and inline a one-caller helper. Simplify isFunctionBinding's last return, remove a stray JSDoc line in no-disabled, and a repeated condition in the run-count script. Verification: lint-rule suites 231 tests; oracle lint clean; db ordered-loader 145, electric 83, browser coordinator 82, offline-transactions 122 tests pass. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
|
Contributor
|
Important Review skippedToo many files! This PR contains 513 files, which is 213 over the limit of 300. To get a review, reduce the PR to 300 files or fewer by splitting it into smaller PRs or changing its base branch. Usage-priced reviews support at most 300 files.
 |
#2230 adds Convex collections with their oracles. Their oracle files now meet oracle/no-unit-test: the companion model modules declare their models and checks, two inline comparisons become declared parts (stepModel in the auth app oracle, expectMirrorsAllTasks in the backend oracle), and tests that compare with written values move unchanged to ordinary siblings (pagination.test.ts, lazy-paging.test.ts, e2e/backend.test.ts, and e2e/backend-startup.test.ts). The 88 test registrations equal main's. The coverage map marks the fixed tests as having no oracle owner (#2264). The transaction-serializer oracle keeps this branch's declaration and main's new campaign helpers and FakeStorageAdapter import. The db test:oracles script keeps main's new oracle and drops this branch's demoted files. Co-authored-by: Isaac <no-reply@databricks.com>
Contributor
Incremental update benchmarkComparing Overall median write time vs base: 0.97× · cold hydrate time: 0.96× (geometric mean of per-case ratios; lower is faster). Writes: 1 regression(s), 3 improvement(s) (threshold: ±20% and >0.05ms). Cold hydrate: 1 regression(s), 2 improvement(s) (threshold: ±50% and >5ms). Per-case flags are noisy on shared runners. Read the geometric means first. Writes
Cold hydrate
Each row aggregates the 14.4 scale/index/write-mode configurations of that query; per-configuration tables below. 100 rows/collection | source indexes: none | synced writes — geomean 0.97×, cold 0.98×
100 rows/collection | source indexes: none | optimistic writes — geomean 1.05×, cold 0.91×
100 rows/collection | source indexes: manual | synced writes — geomean 1.00×, cold 1.06×
100 rows/collection | source indexes: manual | optimistic writes — geomean 1.07×, cold 0.85×
1,000 rows/collection | source indexes: none | synced writes — geomean 0.96×, cold 1.17×, 1 change(s)
1,000 rows/collection | source indexes: none | optimistic writes — geomean 0.84×, cold 1.01×, 1 change(s)
1,000 rows/collection | source indexes: manual | synced writes — geomean 0.97×, cold 0.97×
1,000 rows/collection | source indexes: manual | optimistic writes — geomean 0.99×, cold 0.80×, 1 change(s)
10,000 rows/collection | source indexes: none | synced writes — geomean 0.90×, cold 0.95×, 1 change(s)
10,000 rows/collection | source indexes: none | optimistic writes — geomean 0.98×, cold 1.04×
10,000 rows/collection | source indexes: manual | synced writes — geomean 0.94×, cold 1.07×, 1 change(s)
10,000 rows/collection | source indexes: manual | optimistic writes — geomean 1.05×, cold 0.75×, 2 change(s)
Runner: node v24.8.0, linux 6.17.0-1022-azure, Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz. Timings on shared CI runners are noisy; treat small deltas as indicative only. |
Contributor
|
Size Change: 0 B Total Size: 236 kB ℹ️ View Unchanged
|
More templates
@tanstack/angular-db
@tanstack/browser-db-sqlite-persistence
@tanstack/capacitor-db-sqlite-persistence
@tanstack/cloudflare-durable-objects-db-sqlite-persistence
@tanstack/convex-db-collection
@tanstack/db
@tanstack/db-ivm
@tanstack/db-sqlite-persistence-core
@tanstack/electric-db-collection
@tanstack/electron-db-sqlite-persistence
@tanstack/expo-db-sqlite-persistence
@tanstack/node-db-sqlite-persistence
@tanstack/offline-transactions
@tanstack/powersync-db-collection
@tanstack/preact-db
@tanstack/query-db-collection
@tanstack/react-db
@tanstack/react-native-db-sqlite-persistence
@tanstack/react-router-with-db
@tanstack/rxdb-db-collection
@tanstack/solid-db
@tanstack/svelte-db
@tanstack/tauri-db-sqlite-persistence
@tanstack/trailbase-db-collection
@tanstack/vue-db
commit: |
A test that moved from the lifecycle oracle during the #2168 merge uses the ListResponse type, and the ordinary file did not import it. The Vitest type check reported it as an unhandled error, which failed the collections CI group. Co-authored-by: Isaac <no-reply@databricks.com>
KyleAMathews
pushed a commit
that referenced
this pull request
Oct 28, 2026
…rogressive-network-first #2278 requires every oracle test to reach a part declared with declareOracle. - The demand coverage model, fixed oracle, and campaign declare their check, model, and grammar. The release sweep's law moves into a declared `checkRelease`. - The withCleanup harness test moves to the supplemental test file. - persisted-readiness.test.ts (split out by #2278) drops main's case that expects on-demand network-first to throw, which this PR's law replaces. Co-authored-by: Isaac <no-reply@databricks.com>
KyleAMathews
pushed a commit
that referenced
this pull request
Oct 28, 2026
Brings in #2278, which splits the persisted and Electric descriptor test files and requires every oracle test to reach a declared part. - The persisted-reads grammar, which this branch turned into an oracle with an independent model, stays in persisted-oracle.test.ts and declares its check, model, and grammar. Main's unchanged copy of the older grammar leaves persisted-reload-metadata.test.ts. - The fail-stop coordinator harness's storage-ID emit moves to persisted-support.ts. - The failed-swap give-up witness moves to the ordinary descriptor tests, and the harness exposes the SDK's give-up callback. - The real-adapter and Browser multi-tab oracles name their drivers, model, and grammar with declareOracle. Co-authored-by: Isaac <no-reply@databricks.com>
KyleAMathews
pushed a commit
that referenced
this pull request
Oct 28, 2026
#2278 moves every test that reaches no declared oracle part out of the oracle files. Conflicts were blocks this branch had edited and #2278 moved: - the subscription reentrancy and replay tests now live in ordinary test files; their truncate-replay publication controls get this branch's required `siblingReplays`; - the ordered source loader state oracle takes #2278's support module and keeps this branch's `tieValue` helper beside its use; - import lines. Evidence: @tanstack/db 372 files, 15,530 tests; query-db 56 files, 1,492 tests (2 skipped); tsc, pnpm lint:changed, and the mangle check pass. Co-authored-by: Isaac <no-reply@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🎯 Changes
An oracle file now holds only oracle tests, and a lint rule enforces it. Each oracle file names its check, model, and grammar with
declareOracle. The neworacle/no-unit-testrule reports every test in an oracle file that reaches none of them. Such a test compares production with a value that a person wrote down. It is a unit test, so it belongs in an ordinary test file. This is the next step of #2234, after #2237.Before this PR, a file name was the only sign of oracle coverage. An agent could add pinned examples to an oracle file, and a reader would count them as oracle coverage. The rule makes that shortcut visible at the edit, in the pre-commit hook, and in CI.
The law
Every test in an oracle file must use a declared part, directly or through a driver. As an alternative, it can run a generated property that states its law inline:
The rule follows bindings through the file and through relative imports of other test modules. It does not follow imports of production
src/. A generated property counts as its own check when one arbitrary generates values and its predicate reads its input. A type-level oracle names its model types intypeModel. The rule also reports a call to an imported helper that registers tests of its own, because those tests then appear under the oracle file.Two maintainer decisions (2026-10-27) set the boundary:
The migration
Every oracle file now declares its parts. 184 files contain a declaration. Each test that reaches no part moved unchanged to an ordinary sibling file, with the same title and body. The migration compared test titles per package before and after each move, and no title was lost. A file that had no check or model at all took an ordinary name. #2264 tracks each law that now has only fixed tests. The coverage map marks each of them as fixed tests with no oracle owner.
The branch merges origin/main after #2168. The tests that #2168 changed in oracle files carry those changes into the ordinary files that now hold them. The new oracle files from #2168 declare their parts too. For example,
source-health-oracle.test.tskeeps 504 tests and moves 97 fixed tests tosource-health.test.ts. The 601 titles equal the titles on main.The branch also merges #2230, which adds Convex collections and their oracles. The Convex companion model modules now declare their models and checks. Tests that compare with written values move to
pagination.test.ts,lazy-paging.test.ts,e2e/backend.test.ts, ande2e/backend-startup.test.ts. The 88 test registrations equal the registrations on main.The tripwires
The rule is one of four checks that make an oracle shortcut visible:
scripts/check-oracle-run-counts.mjsreports each lowered or removed budget. It now follows imported and re-exported constants and campaigns, so a budget that moves into a support module stays attributed to its oracle file. It reads base files of any size, and it fails when it cannot read a file.pnpm lint:changed, and the pre-commit hook now lint each oracle file that imports an edited test module. They follow imports transitively and include untracked files..js,.mjs, and.jsxoracle files.oracle-replay.test.tschecks that each manifest owner's command finds its file from its working directory.🔬 Oracle Coverage
The tooling has its own generated oracles. Each one follows the oracle-first order in
AGENTS.md.scripts/eslint-plugin-oracle/part-reach-oracle.property.test.js(lint.part-reach)scripts/run-budget-oracle.property.test.js(scripts.run-budget)scripts/lint-scope-oracle.property.test.js(scripts.lint-scope)scripts/eslint-plugin-oracle/rules/no-unit-test.js. Also the oracle rubric section "Name oracle files consistently", AGENTS.md "How oracle work is scored", and the two decisions above.for-of-mixed-elements(5),member-callee-registration(10), andcaller-body-wrapper-reaches-part(12). The run-budget oracle declares 16 states and the lint-scope oracle declares 15. Generated histories entered each of them.66794a918, and 24 tests at6f5510727. The run-budget oracle failed 8 of 8 campaigns, and the lint-scope oracle failed both campaigns. The replay check failed for exactly the twoscripts/owners.pnpm test:lint-rulespasses 231 rule tests and 10 script-oracle tests.pnpm lint:oraclesreports nothing.somein place ofeveryfor afor...ofloop, a default export throughexport *, and an unresolved.jscompanion. The script mutants include a script that crashes at startup, the default 1 MiB buffer, and no removal report. Both script oracles reject each of them.letwith two assignments, afor...ofover a computed array, and a package predicate or arbitrary.registerAliasOracle. On main the script counted literaloracleRandomParameters(50, …)calls in a unit test, and that unit test now lives inop-sqlite-driver.test.ts. So the run-count check reportsop-sqlite.write-marker-aliases,op-sqlite.columnar-sql-aliases, and seed 271828 as removed, although both campaigns still run in the oracle file. The coverage map records this limit in its "Oracle tooling" section, with an owner and the witness it needs.✅ Checklist
pnpm test.A full run on
15a725e8cpassed the build,lint:check(warnings only),typecheck:tests, and the tests of every package except db and TrailBase. Two db tests failed there. The replay manifest named the wrong owner for a moved campaign, which668194543fixes. The aggregate select reach miss in the limits above was the other failure. In TrailBase, all 149 tests passed, but the Vitest type check found a missingListResponseimport in a moved test. CI found it too, and93a698a8efixes it.The final head
2d448f75cpasses the build,typecheck:tests,test:lint-rules,lint:oracles, and the tests of the packages that later commits changed. That is db (15,472 tests), electric (1,132), browser persistence (552), and offline transactions (301). The OPFS Playwright suite passes 41 tests, the IndexedDB Playwright suite passes 247 tests, and the browser end-to-end suite passes 169 tests.After the #2230 merge,
209242262passes the build,typecheck:tests, andlint:oracles. The Convex unit tests (112), the React Convex oracle (4), the transaction-serializer tests (51), and the db replay and campaign tests (46) pass. The Convex end-to-end oracles need a Convex backend, so CI runs them.🚀 Release Impact
The only
src/changes are in the privatedb-collection-e2epackage. The package manifests change only theirtest:oraclesscripts.Refs #2234. Follow-up: #2264.
This pull request and its description were written by Isaac.