LibRed: ACE page allocation and byte parity, stored UNION/crosstab/IN queries, ACE numeric typing, and a cached catalog by ChrisJollyAU · Pull Request #304 · CirrusRedOrg/EntityFrameworkCore.Jet

ChrisJollyAU · 2026-10-22T16:16:59Z

This branch takes LibRed's writes the rest of the way to ACE's bytes, page allocation included, widens the Access SQL it stores and runs (UNION, crosstab and IN externaldatabase queries, ANY/ALL, CHECK constraints held to ACE), types numbers and conversions as ACE works them out, and restores the speed alpha.4 lost when it stopped caching the catalog. Nearly every behaviour change was measured against ACE first, and the file-format changes are pinned by byte comparisons against ACE's own output: the step-by-step corpus now leaves a file byte-identical to ACE's after every one of its 479 statements, and a continuous run over it (views and procedures aside) and a large-file run end with ACE's file.

Packaging and build

.NET 11 RC 2 and EF Core 11 RC 2 (11.0.0-rc.2.26504.105).
Central package management: Directory.Packages.props replaces Dependencies.targets and the Directory.Build.targets that imported it. Resolved packages are unchanged; NuGet.Config maps every package to both sources, as CPM requires.
System.Security.Cryptography.Pkcs is lifted to the patched build in the projects that reference Microsoft.Build.Tasks.Core, whose dependency GHSA-2fmv-rv2c-59x7 flags.
Microsoft.DotNet.XUnitV3Extensions and the dotnet-eng feed are dropped: EF's specification tests no longer bring it, and its attributes only added a trait nothing filters on.
A Release build of EFCore.Jet.Data no longer fails to pack on an unused System.Threading (IDE0005).
JetDecimalConverter (shared by the Jet provider and LibRed) converts a Decimal to a Double or Single through OLE Automation's VarR8FromDec, never the .NET 11 cast.

LibRed EF Core provider

The OLE epoch reads back as default for DateTime and DateTimeOffset, through CustomizeDataReaderExpression, so EF's queries and SaveChanges read-back get it while LibRedDataReader keeps returning the stored date.
Two output columns aliased alike are accepted. A TPT/TPC projection's IsNull flags, one per navigation, were refused by the parser.

LibRed SQL engine

Numbers are typed and worked out as ACE does.
- A literal with a decimal point or exponent is what ACE makes of it: a Long where it is a whole number a Long holds, a Decimal where one holds it exactly, else a Double. INSERT … VALUES (-123456789.0123456789) stores every digit in a DECIMAL column, where it was rounded to fifteen.
- ACE works a value out two ways, and the evaluator now does both: comparisons, logical operators and function arguments in OLE Automation arithmetic, a Decimal beating a Double; result values the same but for a written number beside which the result is a Double.
- A Decimal result carries ACE's precision as well as its places: DECIMAL(18,4) * DECIMAL(10,2) is DECIMAL(18,4), where any two different scales made a Double. A make-table writes a DECIMAL of that precision (it wrote every Decimal as Currency) and a BIGINT as BIGINT (it wrote Text).
Conversions follow VBA, as ACE's do, not .NET's invariant parsing: text into text and date columns as CStr/CDate in the regional settings; text into number and Yes/No columns as VBA reads a number; whole-number columns round half to even; a Byte keeps the low byte; a date stored into a number column is its OLE serial, and a number into a date column the date at that serial.
Every seek path asks the index what a value is to it (TryGetSeekKey): another kind scans, a value a whole-number column cannot hold finds nothing for = and is ACE's error for a range bound. An indexed TEXT range used to answer differently from an unindexed one. BETWEEN bounds are ordered when the seek runs, so a parameter bound can seek.
New query forms:
- x op ANY | SOME | ALL (subquery), with ACE's three-valued result. IN is its = ANY case and keeps its semi-join, and Null IN an empty subquery is now False, as ACE has it. ANY and SOME are reserved.
- IN externaldatabase: a FROM ending in IN reads that FROM's tables from another Access file, seeking through their own indexes; INTO t IN makes a table there; INSERT INTO t IN … appends to one, with @@IDENTITY the other file's AutoNumber. Only Access files are opened.
- TRANSFORM … PIVOT runs, with ACE's column names, order, fixed IN headings, row totals and refusals.
- SELECT * beside other items, and output columns named as ACE names them (Expr1000 and up, a reference written twice, names told apart by table).
Stored queries:
- UNION queries are stored, read back and run as ACE stores them: two segment rows around the operation row, the ORDER BY and parameters as a SELECT's, and an option row for UNION, UNION ALL and OWNERACCESS.
- Crosstabs, an IN on a FROM or a make-table, DISTINCTROW and TABLE t are stored as ACE stores them.
- A stored append or make-table keeps its GROUP BY, HAVING and ORDER BY; a stored TOP 1 … ORDER BY Id DESC came back choosing another row. INSERT … VALUES with no field list is stored, as ACE stores it.
- From a sweep of real Access files: DELETE t.column FROM t as the designer writes it, a ; before UNION, and an UPDATE target an older engine stored quoted all read back.
- A pass-through query's connect string and server SQL are read, kept apart from executable SQL.
- A stored query is expanded inside an INSERT, UPDATE or DELETE too, as a SELECT expands it, and a WHERE on a single-table saved query is moved into it, where its table's index serves it.
Grouping, subqueries and joins:
- A grouped query reading a column outside an aggregate is refused with ACE's message, and so is t.* in one; a scalar subquery returning a second row is refused with "At most one record can be returned by this subquery."
- Single = Double compares as ACE does, in the type of ACE's inner table, and hash joins, hashed IN sets, semi-joins and GROUP BY keys hash a Single the same way, so none falls back to a per-row loop.
- DISTINCTROW tells records apart by their row, so identical records stay two.
- A correlated subquery whose body has an OFFSET is never decorrelated. EF's ElementAt(n) != null emits exactly that, and the semi-join skipped n rows of the whole body instead of n per key.
DDL held to ACE:
- A table constraint may stand anywhere among the columns, resolved where it is written; a column-level CHECK is refused.
- A CHECK is validated when created, stored as ACE stores it (the table's own qualifier dropped, or written U+FFFE inside a subquery that does not read the table), and enforced once each row is written, so a subquery sees the statement's writes. A deleted row is checked as a row of Nulls, and a refusal is in ACE's words.
- A DEFAULT may start with =, and is stored verbatim; applying one reads past only a first-character =, a bare word is text, and a Text default is cut to the column's width.
- CREATE TABLE, ADD COLUMN and ALTER COLUMN raise the file's format version for a BIGINT, DATETIME2 or calculated column instead of refusing, on the SQL path and the Core API alike.
- A read-only database refuses each write with ACE's message, in ACE's order of checks; a missing table, query or design target is refused with ACE's messages; CREATE VIEW refuses ORDER BY and INTO.
- NATIONAL TEXT is TEXT, as ACE has it.
Index names are ordered and matched in the database's collation, as ACE stores them, and names equal in it are refused as duplicates.
A digraph does not contract where a lowercase letter meets an uppercase one (cS, lJ, Hungarian dzS), as ACE keys them. In a Hungarian file, the catalog seek could not find the query ~sq_cSzallitoLista, and EXECUTE of it failed.
The VBA function list is complete against the library: CVDate, IsEmpty, IsMissing, IsObject and IMEStatus were missing.

LibRed performance

alpha.4 read the catalog afresh for every request, which made a primary-key SELECT about four times slower and DML two to three times. This branch caches it again and goes past the earlier figures. Measured with the benchmark harness against the last recorded run before that change (µs):

	before	now
Primary-key point query, end to end	27.8	23.6
Northwind text-key lookup	25.1	22.4
SQL `INSERT`, autocommit / in a transaction	99 / 68	95 / 68
Storage insert, autocommit / in a transaction	69 / 40	60 / 33
`UPDATE` by primary key / of an indexed column	50 / 81	45 / 73
`DELETE` by primary key	160	92
Set-based `UPDATE`	356	358

A table's DDL is cached across requests; its counters are not. Columns, indexes, relationships and properties are kept, revalidated against the catalog row and definition page; row counts, index counts and AutoNumber values are read from the definition page every time. Real workloads rarely change a table's design, so the cache survives ordinary writes, ACE's included. The decision is recorded in docs/design/catalog-cache.md.
Catalog lookups seek. Relationships are found by child, parent or name, and complex columns by owner, through the system tables' indexes, where each lookup read the whole table.
Index pages are searched by binary search on descent, seek, insert and delete, where each leaf was walked entry by entry.
A row decoder works out its layout lazily and walks only the columns asked for; it was quadratic in the column count to open.
Per-query work is done once: column pruning once per plan, a function's name and arity once per call site, a projection's schema once.
An insert reads its table's definition page once, drawing its ids and advancing its counters in that one copy.
Allocations fell with it: a DELETE by primary key went 277 → 99 KB.

LibRed file format — page allocation, as ACE allocates

Pages come from ACE's shared reservations and owner groups, not the lowest free page: a session's four-group reservation, an owner's current group once it has eight pages, an index across two groups as ACE places its leaves and nodes, and CREATE INDEX in order from the index's own group. alpha.4 recorded this as deliberately unmatched; the ordered bulk insert now matches ACE byte for byte.
A page a reservation or owner group holds past the end of the file is never handed out. A wide table's bulk insert handed one to the index, the commit returned it as unused, and the next split took it again, overwriting a live leaf. PageOwnershipTests now check, through every workload, that no page has two owners and no owned page is free.
Usage maps:
- An empty map is re-based on the first page it marks, rounded down to 8.
- A table's map converts to reference form where growing it would leave its holder fewer than 4 bytes free, and where its window cannot take a page below it.
- A non-empty free-pages map grows in place.
- A new data page is marked free only if its first row leaves room for a second as large.
- A reference map's new bitmap page is placed where ACE places it.
Freeing follows ACE's timing. A chained long value's pages are freed at once, a rewritten definition's old continuations at statement end, everything else at close. When an UPDATE replaces a chained value, ACE takes and drops a long-value page it never writes, and LibRed does the same.
Deletes:
- A row-by-row delete keeps an emptied page only where the free-pages map marks nothing below it.
- A long-value column with no free map keeps its emptied page owned.
- A DELETE with no WHERE empties a table in one go, a complex-column table included, leaving its flat rows as DAO and ACE do.
A rollback leaves what ACE leaves: drawn AutoNumber and complex ids are never given back, a failed statement keeps the counts its accepted rows left, and the session's reservation survives.

LibRed file format — writes that now match ACE

Rows:
- A row's variable trailer is decided from its stored column count, in one rule every reader shares.
- INSERT writes the live fixed region, while UPDATE and the ALTER COLUMN re-lay write over the old record, through the one persistent buffer ACE never clears.
- A retype to Yes/No adds no fixed slot.
Indexes:
- An index identical to an existing one shares its data block, and dropping one name keeps the block for the other. Every named index over a shared block is now listed: scaffolding and the schema rowsets saw one.
- A leaf's prefix is measured over the row pointers too.
- A new maximum splits a leaf rather than store it at a shorter prefix, and a deleted-from leaf keeps its prefix.
- A parent's incoming relationship blocks are named past fifteen as ACE names them (.rAB).
Catalog:
- A CHECK is recorded as ACE's type-9 MSysObjects object.
- MSysQueries rows are written in ACE's clause order.
- Catalog rows are deleted in index order.
- A new table's catalog row is written before its index roots.
- MSysComplexColumns and the complex-type templates are written as user definitions (0x4E).
Complex columns are created in Core: ColumnSpec.Attachment and ColumnSpec.MultiValue, through CreateTable or AddColumn, in a new table or one with rows, as DAO creates them. Attachment types the engine itself refuses are refused under enforceAccessLimits.
Validation held to ACE's:
- Page 0's minor version byte is checked as ACE checks it.
- A record past the 4,060-byte cap refuses every value read from it, while counting and truncating still work.
- Extended flag 0x40 alone makes a calculated column, with 0xC0 written; bit 0x01 of descriptor byte 0x11 makes a column write-once.
- The 0x659/0x783 markers are rewritten on every design change.
- DefaultValue and Required count only with the property's DDL flag.
- AllowZeroLength is enforced.
Index keys are built from the value the row stores, so a Boolean in a numeric column keys as the row holds it, and a Core-API insert is all-or-nothing.

Tests and CI

The whole-file parity test asserts both ways: step by step over 479 statements, and as one continuous run over the corpus without views and procedures. A large-file run matches ACE byte for byte.
CI's ACE 2016 redistributable is guarded where it differs from the Microsoft 365 runtime: the BIGINT/DATETIME2 cross-checks skip, the backtick-quoted CHECK cases skip (it refuses every row against one), and the parity test masks a stored query's LvExtra outside a parameter's facets, which that engine leaves uninitialised.
New suites pin the CHECK, DEFAULT, retype, rollback, read-only, stored-query, UNION and allocation behaviour against ACE, and the page-ownership invariant.
StoredTextConversionTests run under plain en-AU and en-US rather than a culture they built.

Docs

Format spec (src/LibRed/docs/format): updated with everything above that was measured, and tidied twice down to what the format is. The second pass resolved seven contradictions against the code, among them the column marker's width, the calculated envelope's status field and a relationship's swapped MSysACEs masks.
docs/verification-backlog.md lists every statement the docs still call unverified, with the probe that would settle it; docs/design/page-allocation.md and docs/design/catalog-cache.md record the allocation measurements and the catalog cache.
The LibRed README lists ACE defects LibRed does not reproduce, such as an equality lookup that loses matches.
CLAUDE.md/AGENTS.md: a failing test is evidence against a change, never a test to update, with hooks that enforce it; and the AccessTests suites' current times.

🤖 Generated with Claude Code

- A table's logical index blocks are sorted by name in the database's collation, as ACE stores them, not ordinal ignoring case: `_` and punctuation before letters, accented letters beside their base letter, each locale's own letters where it places them. Applies to CREATE TABLE, CREATE INDEX and the hidden .rX block a referenced table gains. - Index names equal in that collation are refused as duplicates, as ACE refuses `ss` beside `ß`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

- ColumnSpec.Attachment and ColumnSpec.MultiValue create a complex column through CreateTable or AddColumn, in a new table or on one with rows, as DAO does: the owner descriptor with its Int32 ComplexID at 0x0B-0x0E, the f_<GUID>_<column> flat table with its value columns, bookkeeping columns and three indexes, the MSysComplexColumns row, the owner's unique index, and the flat table's Required property. Existing rows are numbered 1..n and 0x1C becomes the larger of its old value and n. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ACE writes them with TDEF type 0x4E despite their system flag; only the four core catalog tables are 0x53. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Another pass of the cleanup first done in 5695194: the spec had again accumulated the record of how each fact was found alongside the fact itself - fixture file names, corpus counts, "pages touched"/whole-file-diffing methodology, test and probe names, and research framings ("what the probes settled", "verified by diffing before/after a CREATE TABLE"). That material describes the research, not the bytes, so it is gone. Verified/inferred status stays. The pass also resolved two contradictions it surfaced: - The long-value in-row descriptor's final four bytes are the chain stamp, not "4 reserved" - system-catalog restated to match long-values.md and the code (LongValueDescriptorChainStampOffset = 8). - A row's colCount and null-bitmap width is the TDEF 0x29 high-water, not max(live id) + 1 - page-02a had it backwards and contradicted page-01 and RowCodec; the retype path's newMaxId only coincides because the burned id is live there. Also fixed a garbled clause in system-catalog ("the page page 0's"). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Add a new tidy-format-spec SKILL.md that documents a standardized process for trimming src/LibRed/docs/format/*.md down to authoritative on-disk format facts.

…findings Fixes from the LibRed.Core/Engine audit, each measured against ACE: - Index keys are built from the value the row stores (JetTypeCodec.ToStoredValue), so a Boolean in a numeric column and a short value in a fixed BINARY(n) key as the row holds them. A Core-API insert is all-or-nothing. - CREATE TABLE numbers its indexes, and lays out their usage-map rows, in declaration order whatever their kind (DeclaredAfter). - A relationship over existing orphans is refused in Core; a self-reference needs a unique or primary parent key. - A calculated column added to a populated table is computed for the existing rows; an overflowing calculated result is VBA error 6. - Now() and Time() read the clock in whole seconds. Conversions that used .NET's invariant parsing now follow VBA, as ACE does: - Text into text and date columns is written/read as CStr/CDate in the regional settings (VbaText, VbaDateText moved into Core). - Text into number and Yes/No columns is read as VBA reads a number (the evaluator's reader, moved into VbaText); whole-number columns round half to even; a Byte keeps the low byte; a Yes/No column stores whether the number, rounded half to even, is not 0. - Decimal to Double/Single goes through OLE Automation's VarR8FromDec (JetDecimalConverter.FromDecimal/ToDouble/ToSingle), never the .NET 11 cast. - Calculated columns' Str and text reading match the evaluator's. Seeks: every seek path (equality, join, range, relationship check) asks TryGetSeekKey what a value is to the index. Another kind scans; a value a whole-number column cannot hold finds nothing for =, and is ACE's error for a range bound (Overflow past the range, Data type mismatch for a fraction). The range path had no kind check, so an indexed TEXT range answered differently from an unindexed one. BETWEEN bounds are ordered when the seek runs, so a parameter or expression bound can seek. Also: the "never change a test because a change makes it fail" rule in CLAUDE.md/AGENTS.md, with hooks that re-read the house rules on every prompt and stop the turn after a test edit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

A Yes/No column is fixed but holds its value in the null bitmap, so an ALTER COLUMN to YESNO leaves the target's fixed offset at 0 and appends no fixed slot to the re-laid rows. The row re-lay now takes whether the target adds a slot from the one fixed-length computation the record-size check already made, and encodes the value only where it is stored. Verified byte for byte against ACE from a LONG and from an indexed TEXT holding numeric text and NULLs; the text-to-number retype test gains the YESNO row ACE produced. Spec §3.8 records the Yes/No target. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

StoredTextConversionTests cloned en-AU and overwrote its date and time patterns to reproduce one machine's Windows regional settings, so it tested a culture of its own making rather than following the current one. Each case now runs under plain en-AU or en-US and expects that culture's own text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Rows: - A row's variable trailer is decided from its stored column count and the descriptors' 0x07, as ACE decides it, in one rule every reader shares. - INSERT writes the live fixed region; UPDATE and the ALTER COLUMN re-lay write over the old record, keeping its region, dead slots, dropped chunks and dead bits. The old per-row fixed-length inference is gone. - ADD COLUMN gives a Yes/No fixed offset 0, as CREATE and ALTER do. Read-only databases: - Each writer refuses with ACE's message, in ACE's order of checks. - BEGIN TRANSACTION is allowed. - A row written through the table API gets DAO's recordset message. Format raise: - SchemaEditor.RaiseFormatFor raises the file for CREATE TABLE, ADD COLUMN and ALTER COLUMN on the SQL path and the Core API alike, to the higher of the type's and a calculated column's need, instead of refusing. - The duplicate checks and SQL-side mappings are removed. - A BIGINT or DATETIME2 procedure parameter is stored without a raise, and DATETIME2 reads back as DATETIME2. Also: GenUniqueID() defaults are checked in one place, and JetDatabase.DefinitionPage follows the format version. Tests compare each behaviour with ACE; the spec, CLAUDE.md and AGENTS.md record the measurements. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

- A DEFAULT is read past a leading '=', a bare word is text unless the grammar reads it as a constant or the niladic Now, and a Text default is cut to the column's width. An AutoNumber defaulting to =GenUniqueID() is random, as one defaulting to GenUniqueID() is. - A date stored into a number or Yes/No column is its OLE Automation serial, and a number stored into a date column the date at that serial. - Missing objects are refused with ACE's messages: an input table or query, an INSERT's output table, DROP TABLE, and every design statement. CREATE VIEW refuses ORDER BY and INTO. - EXEC and ExecuteStoredActionQuery share one stored-query lookup, and an unbuilt stored query throws its reason through StoredQuery.RequireSql. - One niladic-function check, one CLR-number check and one design-table lookup replace the copies that had grown up beside them. - Tests: DefaultValueParityAccessTests, date/number retypes in ColumnRetypeParityAccessTests, and FixedWidthOverflowAccessTests opening the file writable (DAO's AddNew on a read-only file is refused as read-only first). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Rows and pages: - DELETE with no WHERE over one table, with no enforced child relationship and no complex column, empties it as ACE does: data and long-value pages released, the first page too; index roots reset to an empty leaf and the other index pages released; the row and index counts zeroed. - A delete holds an index's unique count to no more than its total; a unique index counts a new key only, and a repeated NULL key is not new. - A page found too full has its free-map bit cleared. - UPDATE carries the null-mask bits past the last column from the old record. The slack between a record's variable data and its offset table is kept: from the defaults on INSERT, from the old record on UPDATE, measured on the encoded chunks. Catalog: - A CHECK constraint gets the type-9 MSysObjects object ACE records it with (ObjectType.SingleRecord is now CheckConstraint): Connect, Database and ForeignName as ACE writes them. What it reads is now required, and a CHECK that does not parse is refused when created, as ACE refuses it. - MSysQueries rows are written in ACE's clause order; LvExtra's carry is held per query. - Catalog rows are deleted in index order. DROP TABLE of a child changes only the TDEF's type byte, deletes its CHECK objects before its own row and detaches its relationships after. - Cascaded deletes take the children in the order found. - ColumnSpec.Ordinal replaces IsEngineColumn; make-table columns use it. - The CHECK reads scan shares ColumnPruning.Walk. Docs: the format spec tidied, the MSysQueries row-order section restated as the format has it, and the CHECK and relationship naming rules as measured. Tests: WholeFileParityProbeTest gains many statements and a before-image line. CheckConstraintEnforcementTests expects the unparseable CHECK refused at CREATE, as ACE refuses that exact statement. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… to ACE CREATE TABLE: - A table constraint may stand anywhere among the columns. Each is resolved where it is written: one naming a column declared after it, or not at all, is refused with ACE's message. A self-reference resolves its key before its own columns, and only a key declared before it serves. - A column-level CHECK is refused, as ACE refuses it. CHECK constraints: - Validated when created, in CREATE TABLE and ALTER TABLE alike, by the walk that already works out what a check reads: text that does not parse, a name no scope has, an unqualified row column inside a subquery, a subquery over a missing table, an aggregate. - Stored as ACE stores them: the table's own qualifier dropped outside a subquery and written U+FFFE inside one, unless that subquery reads the table; the whole expression re-formed (CheckTextWriter) when a qualifier was dropped; the qualifier spellings ACE refuses refused. Backtick quoting is kept, as ACE keeps and enforces it. - Enforced once each row is written, through a runner of its own, so a subquery sees the statement's writes; a deleted row is checked as a row of Nulls; a cascade's row and an evaluation error are refused in ACE's words. A DELETE with no WHERE goes row by row on a table with a CHECK. - One refusal (CheckConstraint.Refusal), one delimiter strip (Undelimited), OptionalIdentifier for constraint names. Lexer: ALike reads as LIKE, U+FFFE as a name, and a stored expression is read without comments. An integer literal past a Long is a Decimal, then a Double. Docs: the CHECK text rules and the check object's Connect as measured; a CHECK table deleted row by row; the CHECK backtick claim corrected. Tests: CheckConstraintCreation/Enforcement/Text and CreateTableConstraintOrder parity classes; WholeFileParityProbeTest gains a CHECK-table delete and two qualified checks. CheckConstraintTests, CheckConstraintEnforcementTests and StandaloneExpressionParsingTests now expect what ACE does with those exact statements: column-level and trailing-token CHECKs refused at CREATE, and a trailing comment refused in a stored expression. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…bRed's EF layer - SDK, EF Core and the Microsoft.Extensions/System.Data packages move to 11.0.0-rc.2.26504.105. - System.Security.Cryptography.Pkcs is lifted to the patched build in the projects that reference Microsoft.Build.Tasks.Core, whose 10.0.10 dependency GHSA-2fmv-rv2c-59x7 flags. - Microsoft.DotNet.XUnitV3Extensions is dropped with the dotnet-eng feed: EF's specification tests no longer bring it, and the ConditionalClass/SkipOnCI attributes it supplied only added a trait nothing filters on. - LibRedDateTimeTypeMapping and LibRedDateTimeOffsetTypeMapping read the OLE epoch, where default is stored, back as default via CustomizeDataReaderExpression, so EF's queries and SaveChanges read-back get it while LibRedDataReader keeps returning the stored date. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

- A DEFAULT may start with =, in CREATE TABLE, ADD COLUMN, ALTER COLUMN … DEFAULT and SET DEFAULT: one defaultValue rule, = optional, as ACE's DDL takes it. LibRed takes the spaced forms ACE's DDL refuses (= 1, =1 + 1, =UCase('x')); ACE's expression service applies every one of them. - A default's text is stored verbatim, = and spacing included, as ACE stores it, where GetText dropped the whitespace between tokens. - Applying a default reads past only a first-character =, as ACE does: " =1" and "==1" are refused on insert by both. - A default holding a subquery is refused when parsed; it had only been refused because the stripped text no longer parsed. - The scaffolder reads a default's value past the = through the same PropertyBlob.DefaultExpression. Docs: page-02c records the DDL forms ACE accepts and refuses after =, and what its expression service reads past. Tests: DefaultValueParityAccessTests gains LibRed-written defaults applied by ACE and the misplaced-= refusal; LvPropByteParityAccessTests the = and spaced shapes; ScaffoldingTests the = default's value. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…backticked CHECK - Directory.Packages.props replaces Dependencies.targets and the Directory.Build.targets that imported it; EFCoreVersion and MSLibVersion stay as properties there. The unused DotNetVersion and DocumentationAnalyzers entries are dropped. Resolved packages are unchanged. - NuGet.Config maps every package to both sources, which CPM requires (NU1507). - Spec: ACE stores a backtick-quoted CHECK on every build measured, but only the current Microsoft 365 runtime enforces it; ACE 2010 and the 2016 redistributable fail every insert. - .mcp.json adds the Microsoft Learn, NuGet and GitHub MCP servers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

- CREATE PROCEDURE takes a UNION body and stores it as ACE does: a type-128 query object, two segment rows (the text before the last set operator, and the term after it, named X7YZ_____1 and X7YZ_____2) with the operation row (Flag 9) between them, the ORDER BY and parameters as a SELECT's, and an option row of 3 for UNION, 1 for UNION ALL, 4 more for OWNERACCESS. INTERSECT, EXCEPT and paging are refused; CREATE VIEW still refuses a UNION, as ACE does. - A stored UNION reads back as its two segments joined by the recorded operator, then its ORDER BY, and runs as a query returning rows — through EXECUTE and as a source alike. - TABLE t is a query term, as SELECT * FROM t, so a segment ACE stored that way parses. - A WITH OWNERACCESS OPTION ending an earlier SELECT of a UNION stays in that segment's text; only one ending the last SELECT or the query is the query's. Docs: system-catalog records the UNION rows, corrects the 0x01/0x02 option bits, and records what a query's MSysObjects.Lv is: written by ACE on every writable run, not the same from run to run, dropped by compacting, and never relied on — so a file holding one cannot be byte-identical. Tests: StoredActionQueryWriteAccessTests compares 19 UNION shapes and a parameterised one with ACE, rows and results; CreateViewTests round-trips three. StoredQueryKindAccessTests: the DAO-authored UNION it used as a kind LibRed refuses is one ACE stores as a UNION and runs (94 rows, measured), so it now reads back as one, and the refusal is pinned on a crosstab instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

- An index over the same columns, directions and flags as a data block the table has takes that block, and only a logical block of its own: no statistics, root or usage-map row. CREATE INDEX, ALTER TABLE ADD CONSTRAINT and CREATE TABLE all share, whatever the two indexes are — plain, unique, primary key, relationship. - Dropping one name of a shared block keeps the block for the other: RemoveIndexBlocks keeps a data index a surviving logical block names and says whether it went, and DROP INDEX and DROP CONSTRAINT free its B-tree only then. DROP INDEX CustomerID ON Northwind's Orders used to be refused. - An index is found by its logical name (TableDefinition.FindIndex), not by the one name the real index carries: DROP CONSTRAINT on a relationship sharing an index's block found nothing, yet deleted the relationship's rows and left its blocks; DROP TABLE left the parent's incoming block; CREATE INDEX let a name the table already had through. Docs: page-02d records when ACE shares a data block, and what a drop and a CREATE TABLE do with one. Tests: WholeFileParityProbeTest gains shared indexes created, filled and dropped through CREATE INDEX, ADD CONSTRAINT, CREATE TABLE, DROP INDEX, DROP CONSTRAINT and DROP TABLE, all byte-identical to ACE, and two UNION procedures, which differ only in ACE's MSysObjects.Lv. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…d document a query's compiled plan LibRed's allocator follows ACE's measured policy instead of always taking the lowest free page. A request is served from the session's shared reservation: the free bits of a four-group window from the selected page's 8-page boundary, clipped to the selected global bitmap's coverage. A data or B-tree owner that already has eight pages, or a reference owned map, reuses its current group first, then takes the lowest fully free aligned group. Long values always use the shared reservation. Unused group members return at statement commit, and the reservation state is part of transaction and savepoint rollback. The ordered bulk insert now matches ACE byte for byte, so the whole-file probe drops its known-differences list. A query's MSysObjects.Lv is ACE's compiled plan: a packed memory image with heap and code addresses in it, rewritten on every run. system-catalog.md now records its container, its packing, the structures every image shares and the ones each feature adds, and that nothing stored tells a CREATE VIEW from a CREATE PROCEDURE. The whole-file step probe copies ACE's plan into LibRed's file through LibRed's own row update before comparing, so the value's descriptor, LVAL page and maps are still compared while its unmatched content is not. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ACE builds every re-laid row in one buffer it never clears: the old record is copied in and the new one written over it, so a byte the new record does not write holds whatever record last reached it, not necessarily the one just before. LibRed kept only the previous record. It now keeps the whole buffer, and a null-bitmap byte a row did not have before takes its padding bits from it: ACE's 0x79 and 0x65 there are an earlier row's text under one live bit. A stored query's WHERE row can carry an uninitialised LvExtra: the same within one ACE process, different in the next. The whole-file probe masks only that field, keeping parameter rows' facets, the other rows' zeros and every row's null-bitmap bit compared. Two explicit probes are added: how ACE types a literal (every decimal-point literal is a Decimal over OLE DB), and one statement run through ACE three times to tell its noise from its rules. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

JetCommand and JetDataReader use it only for their DEBUG-only active-object counters, so a Release build flagged the using as unnecessary (IDE0005, an error in src) and the Jet packages failed to pack. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…two ways A number written with a decimal point or an exponent is no longer a Double carrying its written value on the side. It is what ACE makes of it: a whole number a Long holds is a Long (3.0, 1E5); a number a Decimal holds exactly is that Decimal (1.50, 15E-1, 1E-28, 3E9); anything else is a Double (1.5E-28, 1E29, 3000000000.0). So INSERT INTO t VALUES (-123456789.0123456789) stores every digit in a DECIMAL column, where it was rounded to fifteen. ACE works a value out in one of two ways. A comparison, a logical operator and a function's arguments go to its expression service: OLE Automation arithmetic, a Decimal beating a Double, so WHERE F * 1.11 = 0.111 holds for a Double 0.1. A result value - a column, a sort or group key, an aggregate's argument, a value written - is the same but for a written number beside which the result is a Double, which is a Double first: F * 1.11 is 0.11100000000000002 there, where L * 1.11 stays an exact Decimal. The evaluator now does both. A Currency beside a Double, or divided, is worked out as a Double, and a Currency plus a Double stays a Currency; a date times a Decimal is a Decimal. A Double compares with a Decimal as a Decimal, and the hash join hashes them alike. A result column takes ACE's type where a Decimal is involved: DOUBLE * 1.5, 1.5 * 1.25 and DECIMAL * 1.11 are Doubles, L * 1.5 and 0.1 + 0.2 Decimals padded to their places. Round, Abs, Int and Fix of a Decimal, and a choice beside one, stay Decimals as .NET and EF expect, where ACE makes them Doubles. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…UPDATE as ACE does A Decimal result has a precision as well as places. Beside a Decimal an Integer or a Boolean counts five whole digits, a Byte, a Long and any whole result ten, a decimal literal, text and a date 28; a Decimal with a whole number keeps its places at the most whole digits either side counts, up to 28. Two Decimals are one of the larger places and the more whole digits where that fits 28 digits, and a Double only where it does not: DECIMAL(18,4) * DECIMAL(10,2) is DECIMAL(18,4), where LibRed made every pair of different places a Double. A Sum is 28 digits; Round, Abs, Int and Fix keep their operand's type, and a choice its Decimal arms', as their declared types already said. A result column reports that precision. A make-table query made every Decimal a Currency column, its CLR type the only thing it looked at. It now takes the result as described: a DECIMAL of the result's precision and scale, its length the bytes the largest number of that precision takes and a sign byte (6 for DECIMAL(12,2), where a declared one has 17); a BigInt a BIGINT, which was Text. After a Decimal, every later column but a text one carries that Decimal's precision and scale in the descriptor bytes a collation takes, as ACE leaves them. An UPDATE builds its rows in one buffer per table, zeroed when the statement starts, which only the rows it updates pass through: a null-bitmap byte a row gains, and a fixed slot past its old record, hold the last updated record that reached that far. LibRed cleared them. The whole-file parity probe now leaves all 355 statements' files identical to ACE's. The behaviour of this and the previous commit is pinned by focused tests measured against ACE. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

An equality between an expression over the first table in the FROM clause - a Double holding a decimal literal, such as a.F * 1.5 - and a bare column of a later table matches nothing in ACE, 0 * 1.5 = 0 included, in ON and WHERE alike. Swapping the tables, making the column side an expression, or comparing with <, > or <> gives the right rows, so it is ACE's equality lookup, not a rule. LibRed matches the same rows whatever the FROM order. The LibRed README gains a list of ACE defects LibRed does not reproduce, so a parity probe that shows one reads as ACE's fault rather than a gap, and a test pins LibRed's answer in both FROM orders. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… of the file A shared reservation and an owner's group take their pages from the global free map before they are written, so a clear bit past the end of the file is a held page, not an unused one. Two places did not allow for that. When nothing eligible was free and the map already covered the end of the file, growth appended the next page and handed it out as "already represented as used". In a wide table's bulk insert that page was the data group's unwritten 959: the index took it for a leaf, the commit returned it to the free map as the group's unused member, and the next split took it again, overwriting a live leaf. Growth now always extends the map past every page it covers and allocates from the new bits, as ACE skips a group's unwritten pages for the next free one. The growth loop also no longer marks a covered page past the end free. Materializing the pages up to a selected one refused a gap page that was neither free, released nor this session's own, so a second connection allocating past another's reservation failed. Such a page is a reservation; it is written out like any other gap. That a second handle meets one this way is reasoned from the single-session measurements, not measured against two ACE handles, and the spec says so. A backtick-quoted CHECK's test now expects what ACE stores for that statement, `Qty` > 0, as LibRed has stored it since CHECK constraints were held to ACE; this machine's ACE enforces it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The reference-form growth path had the same shortcut the inline one did: when the frontier's range already had a bitmap page, it appended the next page and handed it out as "represented, bit already clear", though a clear bit past the end of the file is a reservation's or an owner group's. It now moves to the first range with no bitmap and places the new bitmap on that range's first page, as ACE places it, writing out the gap before it. Promoting an inline map to reference form placed the bitmap pages at the end of the file, which could be a held page, and rebuilt every bit from the file's length - marking held pages past the end free and dropping genuinely free ones below it. The bitmap pages now go past every page the inline map covered, and the new bitmaps carry its bits. The hand-built reference-map test marked nothing free past the end of its file, which now reads as held pages; its bitmaps carry the growth region free, as every map does, and it finishes the session's reservation before checking that a freed page is reused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… map back over a released page A page handed out twice surfaced only as a corrupt index two splits later, because the page is reused legitimately afterwards and the evidence goes with it. AssertPageOwnership reads every table's definition pages, data map, real indexes' maps and long-value maps through Core's own readers, as DROP TABLE reads them, and fails when a page has two owners or an owned page is free in the global free map. PageOwnershipTests runs it through a wide table's fill, long-value churn, indexes built over existing rows and two connections writing in turn, checking as each workload goes; with the old growth shortcut put back, the fill fails naming page 959. The churn found a second fault: an UPDATE that frees a packed long value puts its page back into the column's free-pages map, and when that map's window had moved above the page it refused with "this map's window cannot move". The other two writers of that map slide or widen the window; this one now does too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ot an unchanged file The memo-update case of ReleaseAtCloseTests assumed the second update would land on the chain the first one freed and leave the file its length. Measured through ACE on the same sequence from the same fixture, the second update's chain starts on the old chain's first page, 354, and continues through the session's shared reservation, growing the file from 375 to 384 pages - which LibRed does page for page. The test now asserts that: the freed page reused in the same session is what shows the chain was freed at once rather than held to close. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…say which bytes ACE alone changed The insert into Order Details grows the file from 353 to 361 pages, and the report read each changed page's original bytes out of the original file - which has no page 353 - so it threw before comparing the pages past the end. Such a page now compares against a zero page. A page only ACE changed has its differing byte ranges reported: here page 0's single byte 0xE02, user 1's slot in the commit-byte table, which LibRed keeps no lock file to count. With that, every page both engines write for the insert is byte-identical, the eight past the end included. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

A correlated EXISTS, IN or scalar aggregate runs per outer row until the decorrelation gate has spent 25 ms on it, then switches to a hash semi-join mid-scan. The semi-join's key query kept the body's OFFSET, so it skipped n rows of the whole uncorrelated body instead of n per key. EF's ElementAt(n) != null emits exactly that, and under a long test run the gate tripped part-way through and admitted customers with too few orders. A key set cannot express "more than n rows per key", so the rewrites now decline an OFFSET and the body keeps running per row. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

A full run of LibRed.Core.AccessTests took 11 m 37 s and LibRed.Engine.AccessTests 4 m 12 s, about 12 and 4.5 minutes with the build. CLAUDE.md, AGENTS.md and the hook that refuses either suite whole in the foreground still said 7 and 1.5. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

A table's inline maps - owned and free, for its data, each index and each long-value column - start at page 0 with nothing marked. Asked to mark a page outside its window while it marks none, ACE moves the map's start to that page rounded down to 8: a table whose first page is 605, 1151 or 1535 starts its maps at 600, 1144 or 1528. LibRed grew the owned map out from 0 and slid a free map to the 512-page boundary below the page, so a table created past page 512 laid its maps out differently from ACE's. The free map's window "slide" is the same rule: the page an insert finds full leaves the map before the next tail is marked, so the map is empty when it moves, and it lands on the tail's byte, not on a 512 boundary (600 -> 1112, a long-value free map 1152 -> 1672). page-05 said floor(page / 512) * 512; it now gives the rule, and marks what a non-empty map does past its window as not measured. A table created past page 512, with an index and a memo, grown until its free maps move, is added to the whole-file parity corpus, byte-identical with ACE. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ACE converts a table's inline map to a reference map as soon as growing it would leave its holder page fewer than 4 bytes free - the reserve LibRed already kept for the global map. A key-less table of one-row pages converts once it needs page 32,000 (inline at 4005 bytes; 4009 would leave 0), and a keyed table only has a smaller budget, by its index map: with a 1,785-byte index map it converts at page 17,696 (2221 bytes would leave 1). LibRed grew the record until it no longer fitted at all, so it converted later in both. Measured to the row against ACE with and without a key; LibRed now converts at the same row. page-05's budget table gave 32,032 and left the keyed case unreconciled; it now gives both, with the reserve. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ACE puts a newly allocated data page in its table's free-pages map only when the row it was taken for leaves room for a second as large: a first row of 2019 bytes, 2061 left, puts the page in the map; one of 2059 bytes, 2021 left, does not, and a table of rows over half a page keeps an empty free map. LibRed marked every new page free and cleared it on the next insert that found it full, so the last page stayed marked, and on a reference-form free map a page in a new range cost a bitmap page ACE never allocates - which then moved every later page. WideTableUsageMapTests asserted the tail of its 2,000 one-page rows was in the free map. ACE, filling that exact table, leaves the free map empty with 2,010 bytes free on the tail, so the test now asserts that. The whole-file parity corpus gains a table either side of the edge, and PageGroupProbeTest a case that opens a third global bitmap range through a session's shared reservation, where this was found - ACE and LibRed now place every page there alike. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

page-05 left unmeasured where ACE puts a reference-form global map's new bitmap when pages an owner still holds lie between the end of the file and the range. It is the range's own first page, the same as with the file ending at the range, and what LibRed already does: with 65465-65471 held by a table's own group, a long value in the same statement opened range 2 with its bitmap at 65472, and LibRed placed the bitmap, the long value, the row, the owned map's new bitmap and the file's end exactly as ACE did. No code change. PageGroupProbeTest gains the two cases: a long value in the row that opens the table's group, and one in the row after it, which is the one that holds pages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

An index past eight pages keeps a current group, whose free remainder it holds, and an other group, the one it left, whose remainder is back in the free map. A new leaf takes the lowest page of the two not in use when the table's data has moved to another 8-page group since the index's previous page; otherwise it takes the other group's lowest free page, or a fresh group when the other has none. A node page takes the current group's held pages, then the other's, then a fresh group. Leaving a group hands its held pages back and makes it the other group. These follow the shared reservation, except that when the data has taken a new group of its own, a leaf counts reserved pages too and takes its other group's ahead of the reservation, the group it leaves handing its held pages to the reservation where its window covers them. Measured on a controlled ascending append at four row sizes and on statement 118's bulk insert from twelve shifted starts. LibRed now matches ACE page for page on all of them, every explicit continuous-session prefix of statement 118 leaves a byte-identical file, the step-by-step corpus stays at 479 of 479, and the whole continuous run rises from 263 to 285 identical pages. page-05 now carries the allocation rules alone; the measurement record that was growing in it moves to docs/design/page-allocation.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…fied Since an empty map is re-based on the first page it marks, a table's owned map can start above a page the table is later given - a freed page below its first one, taken when an UPDATE relocates a row. An owned map cannot slide, and LibRed threw. ACE converts it to a reference map, as it does a free-pages map whose window cannot take the page: a table whose owned map was re-based on 600, then given 42, 44 and 45, holds them in a reference map. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Two indexes over the same columns, directions and flags share one index-data block, as ACE writes them: two CREATE INDEX statements on one column leave one real index and two logical ones naming it. The table definition kept one IndexDef per data block, named after only one of the indexes on it, so the other vanished from everything that reads the table's indexes - scaffolding, INFORMATION_SCHEMA and the schema rowsets among them. Each named index now has an entry of its own over its block, the primary key first; a block only relationships name keeps the first relationship's name. DROP TABLE retires each block's index map once. Measured against ACE: Northwind's Order Details, Orders and Products list five, nine and six indexes through ACE's OLE DB schema, the names LibRed now reads, where it read three, seven and four. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The parser refused a SELECT whose outputs share an alias, which broke EF Core queries that project the same alias more than once (a TPT/TPC projection's IsNull flags, one per navigation). Two equal aliases now both keep the name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Attachments: the engine itself refuses a long list of file types through DAO, not just the Access UI - each fails with 0x800C000E whatever its case, a list inferred to come from Windows' dangerous file types. AddAttachment now refuses the measured set under enforceAccessLimits; dll, psm1, vss and a name with no extension still pass. Property flags: ACE recognises a column's DefaultValue and Required only with the entry's DDL flag set. LibRed's reader applied them whatever the flag; it now ignores them without it, as ACE does. Type names: NATIONAL TEXT is TEXT to ACE - bare a Memo, sized a variable Text - and LibRed refused it. The other NATIONAL and N spellings already mapped as ACE maps them, and the names ACE refuses LibRed refuses too. Pass-through queries keep their connect string on the operation row (Name1), the SQL in Expression, with no 0x04 row - the format doc and two code comments had the connect string in a 0x04 row. Extended column flag 0x20 marks a table's hidden version-history column and nothing else; the flags enum now names it. The backlog moves these to a settled section. Create-time DEFAULT validation stays open with ACE's messages recorded: it needs the evaluator to say whether a function is known without running it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…history by its flag A pass-through query keeps its connection string in the operation row's Name1 and the server's SQL in its Expression. LibRed recognised the kind and read neither - it took the SQL for an IN externaldatabase clause and dropped it. StoredQuery now carries both, as Connect and PassThroughSql, apart from Sql since the dialect is not Jet's; the query is still not run. Only an append or a make-table reads an IN clause off its operation row. Extended column flag 0x20 marks a table's version-history column. ColumnDef reads it as IsVersionHistory, and dropping an append-only memo finds the table's version history by it rather than by its template's name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…, and settle two more backlog items - verified Deleting the last single-page value from a long-value column that has an owned map and no free map leaves the page released (0x09) but still in the owned map and still allocated in the global one, and the next value takes a fresh page. LibRed cleared the owned bit and freed the page. It now keeps it, as ACE does; a chained value's pages were already freed as ACE frees them. A calculated column's error envelope is 39 bytes, the declared length of every value-type result, zero after the status - not 38. A Text or Memo result never caches an error: ACE refuses the insert, as LibRed does. Type codes 0x0D and 0x0E are not ACE types: ACE opens a table with such a column but refuses its value, predicates over it and writes to it, and DAO refuses the table. LibRed matches on writes and reads the raw bytes by decision. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…DEF markers, and add the missing JES functions - verified Extended column flag 0x40 makes a column calculated; 0x80 lets its expression be evaluated, and ACE writes the two together. LibRed took either bit for calculated. It now reads 0x40 as calculated and writes 0xC0, and over 0x40 alone does what ACE does: every read is refused as an invalid expression, a value result is still computed on write, and a Text or Memo one is refused. An update or delete no longer reads a calculated result, so a row whose result cannot be read - a cached error included - can still be rewritten or deleted, as in ACE. Bit 0x01 of a column descriptor's 0x11 byte makes the column write-once: an INSERT may set it, an UPDATE is refused. LibRed reads it as IsWriteOnce and refuses the update. On every design change ACE keeps only that bit of the four-byte field and zeroes the rest, and writes the 0x659/0x783 record markers afresh whatever they held; LibRed carried all of them over and now rewrites them as ACE does. Jet 4.0 bounds the markers at its own build, 9801, and also checks the index ones. The JES function list is now complete against the VBA library. CVDate (CDate), IsEmpty, IsMissing and IsObject (always False) and IMEStatus (0) were missing. ACE's CDec takes two arguments and returns 0, so LibRed's stays an extension. The format docs record the reserved fields ACE rejects, ignores or regenerates, and the backlog moves the settled items out: property-flag protection, the calculated-index trap, Irish 1084, changing a saved expression, the long-value chain boundary and page 0 0x6A. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…and stop checking bitmap-page bytes 2..3 - verified Page 0's minor version byte at 0x15 is signed, and ACE validates it on open: a Jet 4 or ACE 12 file must carry 0x00 ("Unrecognized database format"), an ACE 14 file at most 0x02 ("requires a newer version"), and ACE 16 and 17 take any value. LibRed never read it; Detect now refuses what ACE refuses. A record past the 4060-byte cap fits its page but the engine reads no value from it, its key included: a scan, a seek, an UPDATE and a DELETE through a WHERE that reads a column all fail, while COUNT(*), a DELETE with no WHERE and the table's other rows are unaffected. Jet 4.0 behaves the same on a .mdb. 4060 is a fixed constant, neither page space nor lock bookkeeping. LibRed capped writes but read such a record; RowCodec.Decode now refuses any value read from one, so counting and truncating still work. Bytes 2..3 of a usage-map bitmap page are written zero when the page is created and never read or rewritten by ACE. LibRed refused a bitmap page whose bytes 2..3 were not zero; it now checks only the page type. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…nd pin whole-run file parity without stored queries - verified ACE frees a chained long value's pages at once, whether an UPDATE replaces the value or a DELETE removes its row. A rewritten definition's old continuation pages become free when the statement ends. Everything else stays held until close, including a packed long value's page that an UPDATE or a DELETE empties. LibRed now frees on that same timing: Release takes untilCommit, and ReturnGroups frees those pages at statement commit. When an UPDATE replaces a chained value with NULL, an inline value or a packed value, ACE first takes the next long-value page and later frees it without ever writing it. A page past the end of the file is never written out. Allocate takes materialize: false for this page. The whole-file parity test gets a continuous run over the corpus without views and procedures, and it asserts that both files match. The masks now cover the run's clock values wherever they sit, dead bytes included: MSysObjects dates, including one cut short by the next record, and chained long-value tick stamps. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…p pages where ACE does, and enforce AllowZeroLength - verified A rollback or a failed statement is not a byte-for-byte restore in ACE, and LibRed now leaves what ACE leaves. A drawn AutoNumber or complex id is never given back, and with two connections the next id follows the ids both drew. A failed statement keeps each table's row and index counts as its accepted rows left them, plus the keys a refused row gave the indexes ahead of the one that refused it. A row is written before ACE checks its parent or its unique keys, so the refused row takes its page. ROLLBACK takes an ordinary table's counts back, but not MSysObjects' or MSysACEs'. The session's shared reservation survives the rollback; the pages the undone writes took from it go to the free map. A reservation page another handle has committed meanwhile is let go, which fixes the commit-conflict retry. In a user transaction, a retired usage-map record keeps its bytes until commit. A new first entry on a leaf of one duplicated key keeps on the left only as many entries as fit there whole. A usage map's new bitmap page comes from the session's reservation if it has one. Otherwise it takes the lowest page not in use, free or held by an owner's group, from the start of the range it covers, searching no earlier than the selected global bitmap. A free page opens the reservation round it, and the next rows take that reservation's pages. A page a DELETE empties gets its free-pages bit before its release clears it, so that map turns reference form as ACE's does. The large-file parity run now matches ACE through 30,103 pages. DAO writes AllowZeroLength 0 on the text fields it creates. LibRed reads the property and refuses '' with ACE's message. Four byte-for-byte rollback tests now compare against ACE's file after the same statements. ConcurrentAllocationTests expects retry id 3, as ACE gives for that sequence. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…n order, and split a leaf rather than lower its prefix - verified A non-empty free-pages map asked for a page past its end grows in place from its start page, as an owned map does. A DELETE over a one-row-a-page table grows it to 77, 89 and 477 bytes. LibRed slid the window or turned the map reference form instead; the slide is gone. CREATE INDEX takes its pages in order from the index's own group. The two groups an insert's B-tree works against the table's data apply only to inserts; using them during a build gave alternating leaf pages once the reservation ran out. A new maximum that would store a compressed leaf at a shorter prefix splits the leaf, both on insert and in a build. The page keeps its entries at its prefix and the new key starts the next page. An owner whose last page lies before the selected bitmap of a reference-form global map no longer continues in that page's group, as no search goes back there. A bitmap page that takes a page an owner's group holds gives the group back first and takes the page as a free one, so the reservation opens round it. The large-file run now matches ACE through 34,666 pages. Still open, and recorded in the spec: ACE also keeps some pages a DELETE empties besides the table's lowest page, by a rule the probes have not settled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…w it, and keep a deleted-from leaf's prefix - verified A row-by-row DELETE keeps an emptied page when no lower page is in the table's free-pages map, and releases it otherwise. This replaces "the table's first data page is kept", which it contains. Over one-row pages, deleting Ids 10-19 keeps the first page it empties; a later DELETE of 20-24 keeps none, and one of 0-4 keeps its first again, below it. Rows that fill a kept page take it out of the map, so the next DELETE keeps its first emptied page once more. A leaf a delete takes entries from keeps its stored prefix, down to one entry. A leaf it empties is written empty, still at that prefix, before it leaves the tree; one that stays in the tree empty carries no prefix. The large-file parity run now matches ACE byte for byte, and the new DELETE cases are regular tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ACE writes a new table's MSysObjects row first, then the index roots, then the definition's continuation pages. When the row needs a new MSysObjects page, that page now comes before the primary key's root, as it does in ACE. This makes Jet 4 statement CirrusRedOrg#231 byte-identical to ACE. The definition is still built before anything is written, so a declaration ACE refuses leaves no catalog row behind. The build runs again once the roots are allocated. The corpus runs reach a full catalog page only by chance. A new parity test adds filler tables through ACE until a given CREATE TABLE is the statement that takes MSysObjects a new page, then compares that statement on Jet 4 and ACE 12. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…hose types CI installs the 2016 Access Database Engine redistributable, which knows neither type. These cases now guard themselves with AceTestDatabase.SupportsColumnType, as the other BIGINT/DATETIME2 cross-checks already do: - CalculatedColumnAccessTests: the BIGINT row of the page-0 raise theory. - AceCoreApiShapeProbeTests: the bigint-in-ace12 and datetime2-in-ace12 shapes. - StoredParameterNameAccessTests: the BIGINT and DATETIME2 parameter cases. They skip on ACE 2010 and run and pass on the Microsoft 365 runtime. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…n Single = Double, keep DISTINCTROW records apart, and push a WHERE into a saved query All measured against ACE. - A grouped query that reads a column outside an aggregate, neither a GROUP BY expression nor inside one, is refused with ACE's message; so is t.* in a grouped query. A scalar subquery that returns a second row is refused with "At most one record can be returned by this subquery." CorrelatedExistsSemiJoinTests' HAVING case now expects ACE's refusal of that exact query. - Single = Double compares as ACE does: a join in the type of ACE's inner table (an outer join's optional side; else the indexed side, else the larger, else the first), widening the Single with CDbl where that is the Double. Hash joins, the hashed IN set, the correlated EXISTS/IN/aggregate semijoins and GROUP BY keys hash and group a Single beside another number as a Single, or as a Double where ACE seeks a Double index, so none of them falls back to a per-row loop. - DISTINCTROW tells records apart by their RowId, carried by the scans under it, so identical records stay two. - A WHERE on a single-table saved query is moved into it, where its table's index serves it. - DerivedColumn replaces ProjectionExprFor/FindProject, which followed a derived table into the wrong scope. - README: nested joins and an ON not referencing both sides are deliberate differences from ACE. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

An insert read the table's definition page twice: once to draw its AutoNumber and complex ids, and again to advance the row, AutoNumber and index counters before writing it. Nothing between the two writes that page, so the insert now reads it once, draws its ids from that copy, advances the counters in it and writes it back. The bytes written and their order are unchanged. This makes each insert one read and one write of the page, the span a page lock will cover once LibRed takes ACE's locks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Another pass of the cleanup done in 5695194, 33b74a7 and 5bd6686: the spec had again picked up the record of how each fact was found alongside the fact - fixture and table names, sample values and timings, how a field was patched and read back, which provider a statement went through, test names. That is gone. Verified/measured/inferred status stays, and so do the boundary values that define a rule. page-05, left alone last time while the usage-map work was open, is covered this time. It also takes out LibRed implementation notes that describe the code rather than the bytes: the catalog decode cache, row-decoder memoisation, the page repacker's slices, binary search in the B-tree and the insert path's parsed-page reuse. The catalog cache decision now has its own note in docs/design/catalog-cache.md: table DDL is cached across requests, counters are read live from the definition page every time, and the cache is dropped on any schema change. The pass resolved the contradictions it surfaced, each against the code: - page-02a called the column descriptor's minimum-build marker 2 bytes; it is 32 bits, as page-02b, page-02a's own build table and TableDefinition.WriteParts have it. - page-02d said the index-block markers are not read; that holds for ACE only, and Jet 4.0 refuses a value above its build once a write reaches the index, as page-02a's table says. - page-02e gave the calculated envelope 16 reserved bytes; it is a 4-byte status then 12 reserved (CalculatedValue.StatusSize), as page-02b has it. - page-01's row diagram and page-02b 3.8 gave colCount as max column id + 1; it is the TDEF's 0x29 id high-water (RowCodec.MaxColumnId). - page-03-04 10.4d said an emptied leaf is unlinked without being rewritten; 10.3 and IndexTree.RemoveEntry write it once more, holding no entry, first. - page-09 said a row-by-row delete keeps the table's first page; it keeps the lowest page the free-pages map marks (RowInserter.IsLowestFreePage), and a long-value column with no free-pages map keeps its emptied page owned. - system-catalog had a relationship's MSysACEs masks swapped; admin takes 0xF00FE and Users 0xFFFFF, as section 11 and RelationshipObjectTests have it. page-02b's extended-flags table also gains 0x20 (version-history column), which the note below it and the appendix already listed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…CHECKs ACE cannot evaluate CI installs the 2016 Access Database Engine redistributable, which differs from the Microsoft 365 runtime in two ways these cross-checks met: - WholeFileParityProbeTest: after a declared parameter, the 2016 engine leaves LvExtra uninitialised in a stored query's FROM row and carries an earlier row's value into the others (0x14 after a TEXT(20) parameter, 0x01, 0x05), where the Microsoft 365 runtime writes 0. The mask now covers every row's LvExtra except a text or decimal parameter's facets, which StoredQuery.UnpackParameterFacets decides; it covered the WHERE row alone. - The backtick-quoted CHECK cases: the 2016 engine stores the check as written and then refuses every row with "Could not find field '`Qty`'", so there is no enforcement to compare against. AceTestDatabase.EnforcesBacktickCheck asks the installed ACE once, through the probe SupportsColumnType now shares, and those cases skip where it cannot. Both run and pass on the Microsoft 365 runtime. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ChrisJollyAU and others added 30 commits October 9, 2026 02:05

List BigBinary (0x11) in the appendix's column type table

d426888

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Write MSysComplexColumns and the complex-type templates as user TDEFs

687a2ea

ACE writes them with TDEF type 0x4E despite their system flag; only the four core catalog tables are 0x53. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Add tidy-format-spec skill; remove audit report

79ca377

Add a new tidy-format-spec SKILL.md that documents a standardized process for trimming src/LibRed/docs/format/*.md down to authoritative on-disk format facts.

ChrisJollyAU and others added 28 commits October 18, 2026 01:15

Restore LibRed benchmark performance and retain DDL catalog metadata

ad6098f

Record full LibRed benchmark run for ad6098f

d24d7e0

Restore LibRed benchmark parity across query and write paths

f8abb1a

Record full LibRed benchmark run for f8abb1a

2219c80

ChrisJollyAU merged commit c25c423 into CirrusRedOrg:master Oct 22, 2026
17 of 25 checks passed

ChrisJollyAU mentioned this pull request Oct 25, 2026

Bump the version to 11.0.0-alpha.5 #305

Merged

Navigation Menu

Repository navigation

LibRed: ACE page allocation and byte parity, stored UNION/crosstab/IN queries, ACE numeric typing, and a cached catalog - #304

LibRed: ACE page allocation and byte parity, stored UNION/crosstab/IN queries, ACE numeric typing, and a cached catalog#304
ChrisJollyAU merged 71 commits into
CirrusRedOrg:masterfrom
ChrisJollyAU:alpha5

ChrisJollyAU commented Oct 22, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

ChrisJollyAU commented Oct 22, 2026

Packaging and build

LibRed EF Core provider

LibRed SQL engine

LibRed performance

LibRed file format — page allocation, as ACE allocates

LibRed file format — writes that now match ACE

Tests and CI

Docs

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant