feat(golang)!: generate one query method per operator, with Verify and TargetLimit on a matches query by coderdan · Pull Request #1216 · cipherstash/stack

coderdan · 2026-10-15T17:22:29Z

Summary

Stacked on #1215. This is the Go SDK half of ADR-0009 (appendix B): the generated code names an operator, not an index. PR 4 of 4, the top of the chain.

One query method per operator the engine answers. Each sealed field's Fields entry has Eq, Lt, Lte, Gt, Gte, Between or Matches, as the engine says the field answers, each taking the field's own Go type (Fields.Age.Lt(ctx, cipher, uint32(40)), Fields.Age.Between(ctx, cipher, lo, hi)). A field has no method for an operator it does not answer, so the query does not compile. Query and the per-index Equality, Match, Ore, Ope methods are gone.
The operators come from the engine. The catalog lists operators only for EQL types, and a field declared with indexes has no type to look up, so there is a new engine function and guest export: dynamic::field_operators(&plan, field, &resolver) and se_operators(plan, field), read in Go by Checker.Operators. They return the same {operator, fidelity, arity, index} objects as the catalog, from the same table that query_term reads, so the generator is told what a query on the field finds. stashgen asks for every sealed field, encrypt_into fields included, and holds no copy of the rules. enginetest.Static is still a stand-in. A test now generates the same declarations through it and through the guest and checks that the files match.
What each method returns. Eq returns an encrypt.EqualityTerm, or the EQL type's query value (eql.TextEqQuery) on an encrypt_into field. The orderings return an encrypt.OreTerm, or an encrypt.OpeTerm when the field declares no ORE. Between returns an encrypt.Range[T] with the two bounds' terms. Matches returns an encrypt.MatchQuery[T]. Every one implements driver.Valuer; a Range binds as two parameters, each bound a Valuer.
A probabilistic query carries Verify and TargetLimit, and an exact one carries neither. MatchQuery.Verify(ctx, values []T) checks a page of decrypted values in one guest call. MatchQuery.TargetLimit(ctx, want, stats) sizes the fetch. Both use no key. The generator only writes MatchQuery for an operator the engine reports as probabilistic. If the engine ever reports a combination the SDK has no Go type for (a probabilistic operator on an EQL type, for example), stashgen refuses the field and names it, rather than leaving a method out.
A MatchQuery holds its text. Verify cuts the query's tokens again in the engine, so the query keeps the text it was derived from. Go cannot wipe a string, so the text stays in memory until the garbage collector reclaims it. The README and the package docs say so. A MatchQuery never prints the text: Format, String, GoString and LogValue show only the field and the size of the term, and a test checks every fmt verb and both slog handlers.
se_query is removed. On a target field, eq gives the twin. Cipher.Query, the export, its guest ops and its tests are gone.
se_term stays. Generated code no longer uses it, and Cipher.Derive is now the unexported derive. One caller still needs it: when a record call fails on a term, the error does not say which row and field caused it, and locateTermFailure asks se_term again, one term at a time, to find them. se_query_term cannot do this job. It reaches only the index that answers an operator (ORE, when a field declares ORE and OPE), and it refuses text with no token, which storage accepts. The ADR-0007 amendment records this.
Generated code and goldens regenerated. This covers encrypt/internal/testusers (plus a new Profile type whose declaration matches the Rust query fixture), testpolicy, example, the stashgen goldens and the stub SDK. The order property tests now read the ORE and OPE terms that Encrypt stores, because a query method reaches only one index, and only on integers.

The width rules hold because each method takes the field's Go type: Lt on a uint32 field takes a uint32, and Matches on type Email string returns a MatchQuery[Email] whose Verify takes []Email.

Behaviour to note

An ORE or OPE index on text, bytes or a float answers no operator. This is the engine's rule (OpSpec::answered_by orders integers only, as the typed op::Ordered does). Such a field still stores its term, but no longer gets an Ore or Ope method.
When a field declares both ORE and OPE, its orderings use ORE, and ORE terms do not order in SQL. A field that should compare in SQL should declare OPE alone, or use an EQL type later.
A field declared with tags has no expected_tokens prior. Without a measured histogram, TargetLimit is unbounded and Limit.FetchOr(page) pages. The README says so.
A declared match column is bytes. The README shows a small match_positions(bytea) → int[] SQL function and @> for the containment test. This is what the PostgreSQL test runs.

What waits

Matches, or Between, on an EQL type needs the EQL v4 types (EQL bindings: generate the v4 Rust types, and derive Stack Encrypt's traits on v4 only #1179). Today the engine produces only TextEq, which answers eq. stashgen is ready for more operators: when the catalog lists them, an exact one gets a method automatically, while a probabilistic one is refused until an EQL query type carries Verify.
se_term goes once the record exports name the field behind a term failure themselves.
Go SDK: no test stores a generated type through database/sql, pgx, sqlc or GORM — add a PostgreSQL job #1114 asks for the full database matrix: sqlc, GORM, and pgx's native interface. This PR adds database/sql with the pgx driver only, enough to run the Matches query.
Block ORE is out of scope for the whole chain.

Tests

Rust (dynamic::query): every operator a field is said to answer derives a term through that index, and every other operator is refused as operator_not_answered, for equality, match, ORE+OPE and text-with-ORE fields. Also covered: a missing field, the wire form of an operator, and a target field answering what its type lists.
Guest native tests: ops::operators for each field of the record plan, with every listed operator validating and every other refused; a missing field and a malformed plan; in the eql build, a TextEq field answers eq. The in-crate TextEq round trip now queries through query_term with eq.
Go:
- Checker.Operators for each field of the fixture plan.
- Profile's generated Eq, Lt, Lte, Gt, Gte, Between and Matches against the Rust query fixture's terms.
- MatchQuery.Verify against the fixture's verified page, and MatchQuery.TargetLimit against all six fixture limits.
- The zero MatchQuery refuses both methods.
- The redaction test.
- QueryTerm refusals before state.
- The existing suites, moved onto the new methods.
stashgen: goldens and the policy golden are regenerated, and the stub SDK is updated (the agreement tests pass). The fake-versus-guest test now also compares the generated files for two new declarations: every index on an integer, and orderings on text, bytes and a float.
PostgreSQL (encrypt/postgres_test.go): the test writes Profile rows through database/sql and pgx, finds one row by Eq, then runs Matches with TargetLimit and @>, decrypts the rows, and checks that Verify keeps exactly the fixture's verified names and that the server returned every true match. It skips unless DATABASE_URL is set. The Linux Go job now starts postgres:17 and sets it.

Test plan

Every checkbox of #1189 is done:

One method per operator, from the engine
No copy of the rules
Verify and TargetLimit on probabilistic queries only, each query keeping driver.Valuer
One guest call per page
The width rules
README and doc.go
The hermetic tests and the PostgreSQL Matches test, which runs locally and in the Linux Go job once CI runs

Closes #1189

coderabbitai · 2026-10-15T17:22:38Z

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

changeset-bot · 2026-10-15T17:22:42Z

⚠️ No Changeset found

Latest commit: e289c7d

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

github-actions · 2026-10-15T17:26:40Z

Mutation testing (cargo-mutants, `--in-diff`, stack-auth + stack-encrypt)

caught	missed	unviable	timeout
4	0	6	0

Every mutant in the changed lines was caught by a test.

…ot the stored term Records the query-side model for the Rust Stack Encrypt library: the caller names an operator type, the domain type picks the best index that answers it, each (index, operator) has its own query term, and the trait a term implements (Term, Verifiable, Ranked) is its fidelity. Verification runs on the client after decrypt through `verify`, with `target_limit` giving the Bloom false-positive bound; the query loop stays a documented sqlx pattern. Supersedes ADR-0004 decision 5, whose premise that a probe and a stored term are the same bytes holds only for HMAC equality. ADR-0004 gets a note. Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB

Addresses the first review of #1177. - "probe" becomes "query term" throughout, except where ADR-0004 is quoted. - The central claim is index-specific: HMAC's query term is the stored term; OPE's is only when its order encoding is lossless, and cllw-ore collates text before encrypting it. ADR-0004's note says the same. - `Verifiable` is renamed `Probabilistic`; `fetch_for` becomes `target_limit` on `Ranked` too, with one meaning on both traits. `verify` is infallible, and the ADR says why. `Term::Plaintext` is the decrypted value the term is defined over, which for a similarity index is the embedding or the sketch. - The Bloom bound uses the mean of per-row `f_i^b`, not the mean fill raised to `b`, with the mixed-length example showing the gap. The statistics are a set-bit histogram from one sampled aggregate, or the declared `expected_tokens` prior, which is decided rather than deferred. The 200-token row of the worked table is corrected (λ 7,600, limit want + ~7,800; the `alice` row is want + 3). - Completeness is conditional on the server returning fewer candidates than the limit; the one-shot example checks it, and the ADR says what 99% means. - The cost of verification is λ wasted rows in a batched key retrieval, not λ ZeroKMS calls. - "Keyset paging" is "cursor paging", defined, with a paragraph on LIMIT, OFFSET and cursors. `Between` is spelled with SQL's BETWEEN and two parameters; the planner claim was wrong and is withdrawn. - The range-as-two-terms option says Gte and Lte stay. ADR-0007 is flagged for amendment. A tuning example follows the worked table. - Appendix A: each operator as the Rust call, the sqlx statement with EQL v4 casts, and what the caller does after decrypt. - Appendix B: what crosses the Go and TypeScript bindings under this model, what the generated Go and the TypeScript types look like, and the same query in sqlc, pgx, GORM, Drizzle, Prisma and Kysely. Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB

The TypeScript section described the existing EQL v3 SDK. It now sketches a binding and an SDK built on Stack Encrypt and EQL v4, which do not exist yet: the schema as the plan, operators as the methods a column's type answers, fidelity as the presence of `verify`, terms bound through pg, postgres.js and Kysely with the v4 casts, and the ORM helpers returning the term beside the SQL. Decision 5 also notes that a page sample can select nothing on a small table. Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB

…tack The TypeScript sketch imports from `@cipherstash/stack`, which is where it will ship, rather than a placeholder package name. Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB

Between is one term and one operand: `eql_v4.between(column, range)` with a range query domain per ordered family, in place of the native BETWEEN with the term's halves as two parameters. The function must inline so a btree over the ORE term still serves it as one range scan; that requirement and the domain are EQL-side work the consequences now ask for. Appendix A and B lose `bounds()`, `Lo()`/`Hi()`, and the two-parameter statements. Order terms are exact. `Lt` on text leaves the `Probabilistic` class, which is for Bloom filters only: an order term compares order keys, and two values with one key tie by definition, so there is nothing to verify. The order key is `orderable-bytes`' encoding after the column's declared transform (ADR-0002); the ADR no longer names cllw-ore's `orderize_string`, which the library does not use. The ORE row drops "right only, in future": PostgreSQL stores the left ciphertext too. Whether `verify` for `Matches` tests n-gram containment or a substring is the implementation's choice, and the ADR says so instead of fixing it. Measured statistics come from the server and the declared prior from the index declaration; the sentence saying both always came from the server is gone. The examples open rows with `cipher.open(rows).using(&plan)`; there is no `decrypt_all` on the cipher. Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB

- Decisions 1 to 4 carry the signatures from the implementation in #1190: operators passed by value and used by path (op::Eq), the query awaited, Supports<Op, S> with check and query (behind a QueryToken), Answers<Op, S> with a Default index, DefaultIndex for plans that declare their indexes, and QueryTerm as the base trait that Probabilistic and Ranked refine. Case is CaseInsensitive / CaseSensitive. Every example uses the same call shape. - The Bloom tuning paragraph said k = 3 also gives 268. It keeps b = 6; with k = 3 the needle sets 3 bits, and lambda is about 16,400, twice the k = 6 figure. A smaller k makes short needles worse. - ADR-0002 is the system ADR under docs/adr, linked, not this directory's ADR-0002.

Answers the follow-ups from the second review of ADR-0009 and the one it left for the ordering operators. eql_v4.between must not be STRICT: PostgreSQL inlines a STRICT SQL function only when its body is strict, and a >= x AND b <= y is not, so a STRICT declaration passes a test that calls the function and is then not inlined. Decision 7 says so beside the inlining requirement. The range query domain is per ordered query domain, OPE as well as ORE, one per (family, term) as decision 8 has it, so the OPE ordering that the operators reach through .via(Ope) has a Between operand too. Text ordering stays a catalog question. The ordering operators answer the integer types only, and an ordering on a text plan does not compile, so the exact-term list, the consequences and AtPath<Lt> on a text leaf say "if the catalog keeps text ordering" rather than stating it as settled.

`cipher.query(v).using(&plan)` now finishes with `.op(op::Eq)` or `.op(op::Matches::new())`, and the plan finds the index that answers the operator (ADR-0009, decisions 1 to 4). - `stack_encrypt::op` holds the operators `Eq` and `Matches<C>`, the case markers, and the traits `Operator`, `QueryTerm`, `Supports<Op, S>` (on an index), `Answers<Op, S>` (on a target type, by hand) and `DefaultIndex<S>` (the index a plan that declares indexes answers an operator with). - `.op_via(op, index)` names another index, such as a match index with its own configuration, or one of a target's several terms. - A match query is its own type, `sem::MatchQuery`, the needle's filter, so the query term is not typed as the stored `MatchTerms`. - A case-sensitive `Matches` against a case-folding index, an operator a target does not answer, and a text operator on an integer are compile errors (tests/ui). BREAKING CHANGE: `QueryUsing::equality()` is removed; write `.op(op::Eq)`. `QueryUsing::index(..)` stays: it is the only way to query ORE and OPE until the ordering operators exist (#1186). The ADR's code now uses the spelling that ships, and its base trait is `QueryTerm`, since `sem::Term` already names the derivation trait. Closes #1182 Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB

- A match configuration whose `MatchCase` disagrees with its options is refused when the query is awaited, with the new `PlanError::MatchCase`, instead of answering in a case its index does not fold. `Supports` gains `check`. - `Supports::query` takes an `op::QueryToken` only this crate can make, so no caller can turn a stored term into a query term by hand. - `MatchTerms::matches(&MatchQuery)` tests a stored filter against a query; `MatchQuery::filter` is crate-only. - Docs say what is a compile error and what is refused when awaited: on a field of a fields plan, which is data, an undeclared index, other match options and a mislabelled case are runtime refusals. A typed field of a fields plan answers through the operator's default index, not its type's `Answers`; the ADR, `DefaultIndex` and the CHANGELOG say so. - `QueryTerm`'s doc no longer calls every term that refines nothing exact before `Probabilistic` exists. A broken ADR sentence is fixed. - `Operator::NAME` is gone (nothing read it); `Matches` is `Hash`. - Tests: the case mismatch, a refused query loading no named keyset (kills the `QueryOp::check` mutant), a query under a named keyset, and `MatchQuery`'s `Debug`. The three `via` mutants and `Matches::clone`'s are equivalent (unit structs) and are excluded in .cargo/mutants.toml. Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB

… ends on the plan `cipher.query(op::Eq, "bob@example.com").on(&plan)` replaces `cipher.query("bob@example.com").using(&plan).op(op::Eq)`. The operator comes first, by value, and `on` names the plan, which chooses the index that answers the operator (`plan::Chooses`) and resolves the label the write used (`plan::Resolves`). An index the plan declares beside the default is named before the plan: `.via(index).on(&plan)`. An index's own term, for ORE and OPE until the ordering operators arrive, is `cipher.query_index(Ore, &40).on(&plan)`. `op::Matches` is now a value as well as a type, as `PhantomData` is, so `op::Matches::new()` is `op::Matches`; `op::Matches.case_sensitive()` is the other case. The case types are `CaseInsensitive` and `CaseSensitive`: `Sensitive` on its own read as data sensitivity. `Match` is a value the same way, so `Match::default()` is `Match`, and `Default` is implemented for every `Match<O>`. `Answers::via()` and `DefaultIndex::via()` are gone: an index is a unit type, so `Via: Default` says the same thing without a method per impl, and the three equivalent-mutant exclusions for those bodies go with them. "Needle" is gone from the prose: a match query is a filter over the query's value, where the stored term is a filter over the stored value. Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB

A Bloom filter can hold every bit of a query without the value holding the needle, so a `Matches` query returns false positives. ADR-0009 decision 4 makes such a term `Probabilistic`, with `verify` as the operator's own meaning run on a decrypted value. - `op::Probabilistic { type Plaintext; fn verify(&self, &Plaintext) -> bool }`. Infallible: every outcome other than a match is `false`. - `MatchQuery` keeps the needle's tokens as the index cut them, through a new `Supports::carry` taken before the value is consumed. `verify` cuts the decrypted value the same way and checks every needle token is among its tokens. The tokens are zeroized on drop and never shown by `Debug`. - `Supports::Query` is bounded by `op::TermFor<Op>`: any query term answers `Eq`, only a probabilistic one answers `Matches`, so an index cannot claim a text match without saying how its answer is checked (tests/ui/op_inexact_operator_needs_probabilistic.rs). - `verify` on an exact term does not compile (tests/ui/op_verify_exact_term.rs). The tests plant a real Bloom false positive under a 32-bit filter and check `verify` drops it. Closes #1183 Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB

- `Supports::query` takes the sealed `QueryToken` (from the #1190 fix), so no caller can pair a needle with another value's filter and have `verify` drop the real rows. `Needle`'s doc says how one is joined. - `TermFor`'s doc and ADR-0009 say the "only a probabilistic term answers `Matches`" rule binds this crate's terms; a term in another crate can opt in by hand. - The tokenizer zeroizes its own copies of the text: the folded string, its characters, and each n-gram, built at its final size so it does not reallocate. `verify` runs it on every decrypted row. - `MatchQuery`'s `PartialEq` says two needles can share a filter and still verify differently. - ADR-0009 decision 4's code shows the trait as this PR ships it; `target_limit` joins it with #1184. - Tests: `verify` is a token test, not a substring test ("abcd" verifies "abc bcd"), and the standard tokenizer verifies whole words in any order. Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB

ADR-0009 decision 5: a caller who wants `want` real matches from a probabilistic query fetches `want + Poisson_quantile(λ = p·N, confidence)` rows, where `p` is the mean over stored filters of `f^b`, `f` a filter's fill and `b` the bits the query sets. - `Probabilistic::target_limit(&self, want, &Stats) -> Limit`. - `op::Stats::declared(rows)` uses a new `MatchConfig::expected_tokens()` prior, a default method returning `None`, so existing configurations compile unchanged; it is not part of the index's identity. - `op::Stats::measured(rows, histogram)` takes `(set bits, rows)` buckets from one sampled aggregate over the column, and takes the mean of the power, never the power of the mean fill. An empty histogram falls back to the prior. - `op::Limit` is `Rows(n)` or `Unbounded`, the latter when nothing is known of the stored filters or the confidence is not in (0, 1). `fetch_or(page)` reads either as a fetch size. - The quantile sums the Poisson mass exactly in log space from `λ − 10√λ`, so it costs about `12√λ` terms; past `λ = 1e7` it is the conservative `λ + 8√λ`. Unit tests reproduce the ADR's worked table and its mixed-length column, and pin the quantile to reference values computed with `lgamma`. Closes #1184 Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB

- The Poisson quantile is a bounded, normalised sum over `[λ − 10√λ, λ + 12√λ + 50]`. Each term is a log relative to the window's first, so a large `λ` no longer loses the starting term to cancellation, and the sums run from the top, where the terms are smallest, so a confidence near 1 is compared against a tail rounding did not swallow. It matches an exact reference to within one up to 1 − 1e-12 (was off by thousands at λ ≈ 4.75e6), and no mutation of it can loop forever. `ln_factorial` is gone. - A `Limit` never exceeds `op::MAX_ROWS` (`i64::MAX`), the largest SQL `LIMIT`; `reltuples` of −1 read as a `u64` no longer overflows a bind. The ADR's statements use `greatest(reltuples, 0)`. - Histogram counts are summed as floats, so huge counts cannot overflow. - The independence assumption is stated in ADR-0009 decision 5 and on `target_limit`: a row sharing tokens with the needle passes more often, so the confidence is nominal on natural text; `verify` still decides. - Docs: the stats SQL filters NULL values; `target_limit` says a fetch that returned the limit is a page only when fewer than `want` verified; `expected_tokens` counts distinct tokens; decision 4's code shows `target_limit` on `Probabilistic`. - Tests: exact references at high confidence and at λ = 1e7 (which kills the threshold mutant), the `MAX_ROWS` cap, float-summed counts, and the measured-column test bounds the fetch against the histogram's own λ. Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB

CI's mutants gate found one survivor and three timeouts in `poisson_quantile`: - Shifting each log term by the window's peak never mattered: relative to the window's first term no term exceeds about e^100, so `-` → `+` there was equivalent. The shift is gone. - A mutated threshold or window bound sent the `MAX_ROWS` test's λ of about 9e18 into a window of billions of terms. The window is now `22√λ + 50` wide from its start, capped at 2^20 terms whatever the input, so no mutation can hang a test. - The window's end is the answer when the confidence allows no mass (above 1), now pinned by a test. A focused `cargo mutants` run over `op/stats.rs` leaves no survivor and no timeout. Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB

CI's mutants gate found `start..=start + width` → `start * width` surviving: the loop's range sat outside the cap on the window's width, so the mutation ran for billions of terms instead of failing. The loop now runs over `0..=width`, the capped width itself, and derives each `k` from it. A focused `cargo mutants` run over `op/stats.rs`: 91 mutants, 87 caught, 4 unviable, none missed, no timeouts. Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB

Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB

…er, refuse only the query A match value that cuts no token under the index's tokenizer (empty or separator-only text, or text shorter than the n-gram length) used to fail term derivation on both the stored and the query side, so one two-letter name failed a whole row. ADR-0009 splits the two: the stored term is the empty filter, which no query matches, and only a query of such text is refused, because its filter would match every row. `match_term` now derives the empty filter for such text, with no PRF call, so `MatchTerms` as a target, `KeysetCipher::match_terms` and the `dynamic` record path all store it, and the value seals and opens as any other. The refusal moves to the query paths: `op::Supports::check` takes the query's value, the `Match` index refuses token-less text there with `TermError::EmptyTermText` when the query is awaited, before any term is derived or keyset loaded, and `dynamic::term` refuses it for a match kind. The error keeps its code and is reworded for the query. The Go module builds the stack-encrypt guest by path, so its round-trip test of the old refusal is rewritten to the new behaviour: a short `TextMatch` value seals, opens and matches no query, and a short match query is still `ErrTerm`. The package docs and the term-failure message follow. Two tests that used token-less text as a leaf that fails before any I/O now use a match configuration with invalid options, which still fails during the synchronous build. Closes #1185 Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB

match_terms is no longer a query. In 0.2.0 the same call served write time and query time and refused text that cuts no token; it now returns the empty filter for that text, and EQL treats an empty filter as contained by every stored filter. A caller who still sends its output as a query gets every row and no error, so the CHANGELOG entry moves to Breaking and says so plainly, naming cipher.query(op::Matches, ..) as the query. The query side validated the match options only when it derived the term, after the new no-token check had tokenised the text, so an n-gram length of zero panicked in the tokenizer instead of returning InvalidOptions. Both query prechecks, op::Supports::check and dynamic::term, now validate first, as the stored path always has. Go's term-failure locator asks the query path for each term, and the query path refuses short match text that a record stores as the empty filter. It could stop at an earlier row with a short email and name the wrong row. It now skips that one refusal by its code, and a test pins it with a short email in row 0. ADR-0009 and the Go package doc say that a stored empty filter tells a reader of the column the value is empty, separator-only or shorter than the n-gram length. The tokenizer's doc no longer says match_terms rejects that text.

… no query_index ADR-0009 decisions 1 and 7. `op::Lt`, `Lte`, `Gt` and `Gte` take one value and `op::Between` a pair of bounds, inclusive at both ends. The ORE and OPE indexes answer them over the integer types (`op::Ordered`), ORE by default and OPE through `.via(Ope)`; an ordering on text does not compile, since text has no order term this crate produces. A comparison's query term is the index's own term for the value: this crate's ORE is CLLW, whose ciphertext has no left half to send alone, so what is sent is what is stored, as for an HMAC, and `Supports` is where the left-only term of block ORE (#1140) goes when it lands. A range's query term is one `sem::RangeQuery` carrying both bounds' terms, derived under the one label, read with `lo()` and `hi()`. A range whose low bound is above its high bound is refused when the query is awaited, before either term is derived and before any keyset is loaded (`TermError::InvertedRange`). An operator now says what a query takes through `op::Arity`: `One` plaintext or a `Pair` of them. `op::Input<A>` relates the value a query is given to the plaintext the plan is checked against, and derives one term per plaintext. `Supports`, `Answers` and `DefaultIndex` are keyed by that input, so `Supports<Eq, S>` reads as before and `Between` is `Supports<Between, (S, S)>`; `Supports::query` takes the input's terms. `StackCipher::query_index` goes, with `QueryIndexBuilder` and `QueryIndex`: every index a plan can declare is reachable through an operator now, and the chain holds its own input, index and label. `Resolves::resolve` takes the index and the call's context and returns the index and the label. ADR-0009's decision 2 shows the shape that ships, and its mentions of `query_index` are past tense. Closes #1186 Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB

Ordered was public and not sealed, so a downstream crate could implement it for its own type, such as a newtype over folded text that also implements the CLLW traits. Between refuses an inverted range with the type's PartialOrd while the server compares ORE terms, so such a type could refuse a valid range or accept an empty one, and it would open ordering on text, which ADR-0009 leaves to the catalog. Ordered now requires op::sealed::Sealed, which the macro implements for the integer types. A compile_fail doctest pins it with a type that meets every other bound, so the seal is the only reason it fails; without the seal the same snippet compiles. A trybuild test would pin rustc's list of sample implementors, which changes whenever a later layer seals another type.

… and AtPath ADR-0009 decision 1 names three JSON operators. `op::Contains` and `op::ContainedBy` are the containments (`@>` and `<@`), exact, and take one value, the document. `op::AtPath<Op>`, written `op::AtPath(op::Eq)`, applies another operator at a JSON path and inherits that operator's fidelity through `TermFor`: an exact term answers `AtPath<Eq>` and `AtPath<Lt>`, and only a probabilistic one answers `AtPath<Matches>`. No index answers them. This crate has no JSON index, and EQL v4 has no JSON type Stack Encrypt produces, so a containment query or a query at a path does not compile on any plan, which the compile-fail tests pin. `AtPath`'s arity is a new `op::Pathed`, a path and the leaf's input, and no `op::Input` of it exists until the JSON index arrives, since the path becomes a selector only that index can derive. The value-selector and selector-plus-leaf term shapes the issue lists wait for the same index. Closes #1187 Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB

…scribed once and every leaf records its type Pins the root workspace's vitaminc crates to the branch that implements vitaminc ADR 0001 (cipherstash/vitaminc #439 to #445) with a temporary `[patch.crates-io]`, and moves stack-encrypt and the guest ABI onto it. - A value describes itself with vitaminc's `Plaintext` and reads back through `Decrypt`. `PendingStackCipherText::plan` walks it into vitaminc's pending tree with no I/O; the target layer's `Pending` still fetches every key of an assembly in one batched call, and the leaves are then sealed and opened through the bytes-only `Cipher`: `MintedKeys` seals each leaf under the next generated key, `BoundKeys` opens a leaf under the key bound to it, and vitaminc derives every leaf's AAD and checks the markers and container rules on both sides. - Every leaf is tagged (wire version 2), so a Rust `u32` field and a data plan's `uint32` field seal the same leaf and each opens the other's. The record fixture is regenerated; its terms are unchanged. - `None` is a null leaf, a passthrough is a `PassthroughFrame`, and `StackCipherText` is `Node<SealedValue>`. - `dynamic::FfiValue` and `ValueKind` move into this crate, unchanged in shape, with `Plaintext` and a `Decrypt` sink, and cross a binding's boundary as transport frames. The deeper rework of `dynamic::Value` and `Scalar` is #1140. - stack-guest-abi's outputs and last error are a `HostValue` written as a transport frame. Closes #1118

…with no absent marker to fuzz The fuzz crate is its own workspace and compiles stack-encrypt by path, so it carries the same temporary vitaminc pin. check_record's model loses the None node, which ADR 0001 retires, and builds its passthroughs as frames.

…guests Both WASI guests build against the same temporary vitaminc pin as the root workspace, and every value crosses as a transport frame. - The encrypt guest decodes inputs with `FfiValue::from_frame`, writes outputs through vitaminc's `FrameWriter` into a buffer still sized before the first byte, and reads and writes ciphertext trees with `vitaminc_aead_frame`'s codec, a passthrough as its frame. - The credential guest's outputs are `HostValue`s. - The Go module moves vcffi and vcvalue to the same vitaminc commit, whose container tags and leaf kinds match the frame. The absent leaf kind is gone from the binding's leaf set. vcffi sends an 8- or 16-bit integer at its own width, which the engine reads as the 32-bit integer of its sign, so a generated field seals as it did.

…and drop the unpinnable capacity hint cargo-mutants found the unused half of MintedKeys and BoundKeys, and the UINT8 arm of FfiValue's sink, untested: no Rust source walks a u8 as an int, so the test reads a host's frame of one. The sink's capacity hint changed nothing a test can see, so it is gone.

… 0001 so the EQL jobs and the EQL guest stay green eql-bindings' `stack-encrypt` feature and tests/encryption compile stack-encrypt by path, so they move with it. - The EQL workspace carries the same temporary vitaminc pin as the root, the fuzz crate and both guests: every workspace that compiles stack-encrypt now resolves one vitaminc. - `encryption::targets` and the generated `v3::targets` take `FfiValue` and `ValueKind` from `stack_encrypt::dynamic` (the feature turns on stack-encrypt's `dynamic`); the generator emits the new import. `vitaminc-aead-value` is gone from both manifests. - The `Ciphertext` target takes `Plaintext` in place of `Encrypt` and `Decrypt + Send + 'static` in place of `Decrypt<'static>`. - The encrypt guest's lockfile drops `vitaminc-aead-value`, which only its `eql` build still pulled in. No other EQL code names a vitaminc API.

…c-normalize so text compares in Unicode 16 NFC An equality term is now the PRF over the value's vitaminc-normalize encoding under vitaminc's domain for its kind, read through that crate's Terms sink: one term per number whatever its width, text in Unicode 16 NFC, and no term for a boolean, a null or a container. Match text is tokenized in the same normal form. The equality and match domains move to /v2 and the pins and the record fixture are regenerated; ORE and OPE do not move. vitaminc-normalize is not on crates.io, so it joins the vitaminc patch blocks of every workspace that builds stack-encrypt. Part of #1140

…Plaintext and give a boolean no order term dynamic::Value and dynamic::Scalar restated vitaminc's vocabulary: one to make an FfiValue Clone, the other to pick a Rust type per variant. FfiValue is now Clone itself and is a plan field's plaintext, and dynamic::term takes any Plaintext, so a guest walks the frame it was handed in place and the term derives from the scalar vitaminc hands the sink. IndexSpec::defines replaces supports, and admits reads the same table over kinds: equality is vitaminc's (floats now have a domain, booleans none), match takes text, and ORE and OPE refuse a boolean, whose order term would tell anyone holding the column which rows are true. Part of #1140

…refuse order terms on a boolean se_term parses the value's frame and hands it to dynamic::term without decoding it into a value tree. The stashgen fake engine and the Go tests follow the engine's new table: a float takes an equality term, a boolean takes no equality or order term. Part of #1140

…pt's normalized equality term Hmac256 encrypts from any Plaintext, eql-bindings no longer depends on vitaminc-prf, the EQL workspace patches vitaminc-normalize, and the TextEqQuery fixture the Go SDK reads is regenerated. Part of #1140

…d verify and size a probabilistic query with no key A binding names a query by field, operator and input, as ADR-0009 has a Rust caller name it by operator type: dynamic::query_term reads the input as the field's declared type, finds the index that answers by the typed path's default table (OpSpec::answered_by), and derives the bytes the typed query chain derives. An operator the field does not answer is refused as a mistyped value is, naming the field. dynamic::verify and dynamic::target_limit run the typed MatchQuery's code under the field's own match options, and the catalog entry of each EQL type lists the operators the engine answers for it with their fidelity. tests/fixtures/query_operators.json records each term, a verified page and fetch limits, checked against the typed chain, for the Go SDK's tests. Part of #1188

…the Go SDK can query by operator without a tokenizer of its own The encrypt guest derives a query term from a plan, a field, an operator name and its input, verifies a page of decrypted values in one call, and sizes a probabilistic query's fetch from its term and the table's statistics; the last two use no key and run with no cipher. se_targets entries carry the operators each EQL type answers, and the Go reader requires them. Cipher.QueryTerm, Cipher.Verify and Cipher.TargetLimit wrap the exports for generated code, and the Go tests hold them to the Rust query fixture through the deterministic guest. Part of #1188

…oposed until ADR-0009 is accepted The data grammar gains the operator and its input, the term comes back with its query domain, verify and target_limit become engine exports, and the catalog reports each EQL type's operators and their fidelity. The amendment rests on ADR-0009, which is proposed, and takes effect when it is accepted. Part of #1188

…nswers, so a generator holds no copy of the table dynamic::field_operators lists the operators a field of a data plan answers, each with the index that answers it: for a field declared with indexes, what OpSpec::answered_by finds under its kind; for a field that names an EQL type, what the type's catalog entry lists. query_term reads the same table, so what a generator is told a field answers is what a query on it finds. operator_to_value writes one entry in the catalog's wire form, which TargetDescriptor::to_value now shares. Part of #1189

…ers, with Verify and TargetLimit on a matches query stashgen asks the engine which operators each sealed field answers, through a new guest export, se_operators, and writes one method for each: Eq, Lt, Lte, Gt, Gte, Between and Matches, each taking the field's own Go type. A field has no method for an operator it does not answer, so such a query does not compile. The generator holds no copy of which index answers which operator; the static test engine is held to the guest by a test that generates the same declarations through both and compares the files. Eq returns an EqualityTerm (or, on an encrypt_into field, the EQL type's query value), an ordering an OreTerm or OpeTerm, and Between an encrypt.Range of the two bounds' terms. Matches returns an encrypt.MatchQuery, whose answer can hold false positives, so it carries Verify, one guest call for a page of decrypted values, and TargetLimit. Every query type implements driver.Valuer. MatchQuery keeps the text it was derived from, because Verify cuts its tokens again in the engine; it never prints it. se_query, Cipher.Query and gensupport's per-index term methods are gone: eq on a target field is its twin, and a query names an operator. se_term stays, now behind the unexported Cipher.derive, because a record call's term failure does not say which field it came from and the host asks se_term again to name it. The order tests read the terms Encrypt stores, since a query method reaches only the index that answers its operator. Part of #1189

…SQL, through database/sql and pgx The test stores generated rows in separate columns, finds one by its equality term, and runs a matches query whose filter containment PostgreSQL tests with @>, then decrypts the rows it returned and verifies them: the verified rows are the Rust query fixture's. It uses the deterministic guest's keys and skips unless DATABASE_URL is set; the Linux Go job now starts PostgreSQL 17 and sets it. pgx is a test-only import. Part of #1189

…decrypt-and-verify step, and say se_query is gone The README and the package documentation describe one method per operator, what each returns, why a matches query carries Verify and TargetLimit and an exact one neither, and that a MatchQuery holds its text because Go cannot wipe a string. The ADR-0007 amendment records se_operators, the removal of se_query, and the one reason se_term stays. Part of #1189

coderdan added 6 commits October 16, 2026 21:48

docs(adr): ADR-0009 appendix B names the next major of @cipherstash/s…

22e8a4c

…tack The TypeScript sketch imports from `@cipherstash/stack`, which is where it will ship, rather than a placeholder package name. Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB

coderdan mentioned this pull request Oct 16, 2026

chore: release candidate for EQL v4 and Stack Encrypt Rust and Go support #1229

Draft

coderdan force-pushed the feat/stack-encrypt-operator-exports branch from 139bd15 to fd7e4e5 Compare October 16, 2026 11:16

coderdan force-pushed the feat/golang-operator-queries branch from d0869e7 to 1f2bffc Compare October 16, 2026 11:16

coderdan mentioned this pull request Oct 16, 2026

feat(stack-encrypt)!: a query names an operator, not an index #1190

Open

6 tasks done

coderdan force-pushed the feat/golang-operator-queries branch from 1f2bffc to e289c7d Compare October 16, 2026 11:17

coderdan force-pushed the feat/stack-encrypt-operator-exports branch from fd7e4e5 to c0d5dda Compare October 16, 2026 11:17

coderdan mentioned this pull request Oct 16, 2026

feat(stack-encrypt): ordering operators Lt, Lte, Gt, Gte and Between, and remove query_index #1208

Open

3 of 4 tasks

coderdan added 14 commits October 16, 2026 22:32

docs(stack-encrypt): the query, not the needle, in target_limit's prose

8aed9a6

Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB

coderdan added 18 commits October 16, 2026 23:00

coderdan force-pushed the feat/golang-operator-queries branch from e289c7d to 8bb6585 Compare October 16, 2026 12:06

coderdan force-pushed the feat/stack-encrypt-operator-exports branch 6 times, most recently from 744d5ff to 4213d5c Compare October 16, 2026 15:09

Navigation Menu

Repository navigation

feat(golang)!: generate one query method per operator, with Verify and TargetLimit on a matches query - #1216

feat(golang)!: generate one query method per operator, with Verify and TargetLimit on a matches query#1216
coderdan wants to merge 38 commits into
feat/stack-encrypt-operator-exportsfrom
feat/golang-operator-queries

coderdan commented Oct 15, 2026

Uh oh!

coderabbitai Bot commented Oct 15, 2026

Draft PR not reviewed

Uh oh!

changeset-bot Bot commented Oct 15, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 15, 2026 •

edited

Loading

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

coderdan commented Oct 15, 2026

Summary

Behaviour to note

What waits

Tests

Test plan

Uh oh!

coderabbitai Bot commented Oct 15, 2026

Draft PR not reviewed

Uh oh!

changeset-bot Bot commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

github-actions Bot commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Mutation testing (cargo-mutants, --in-diff, stack-auth + stack-encrypt)

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

changeset-bot Bot commented Oct 15, 2026 •

edited

Loading

github-actions Bot commented Oct 15, 2026 •

edited

Loading

Mutation testing (cargo-mutants, `--in-diff`, stack-auth + stack-encrypt)