Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/design/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,11 @@ Two shapes, needing different instruments. CROSS-SESSION distillation is what th

**Reviewing a docs/design change: the axes.** Run these before merging anything that touches rules/, decisions/, mechanisms.md — or THIS FILE, which carries the conventions those three rest on and has the same failure modes. Axes 3, 5, 6 and 12 are rules-specific (they need rule statements, examples and the rules a rule names); the other eight apply to any of the four — axis 11 included, since an invariant over two inputs is prompted by what a rule implies but is RUN without reading one, which is the whole of its value. Say which axes you ran and which you skipped as inapplicable — a checklist that asks for something meaningless teaches the reader to skim it. Axes 1–9 were each earned by a defect that survived every other check on #386, where the review passes surfaced ten issues in total — nine of them by these axes, one by the spec self-review above — and unaided prose reading surfaced none. Axes 10 and 11 were earned the same way on other arcs — #291's, which PREDATES this list, and #461/#466's, which follows it. Axis 12 was earned by no defect at all, which is its point: it covers the growth every other axis passes. The parenthetical is the case that earned the axis.

Skip what the suite already enforces. `tests/v2/test_rules_doc.py`, `tests/v2/test_doc_citations.py` and `tests/v2/test_design_docs.py` machine-check example lines and their `deviates:` today-values, boundaries, the one-of `implemented:`/`tracked:` rule, `implemented:` against the modules that actually cite the rule, the rule IDs a rule names anywhere in its prose, cross-doc anchor IDs, citation excerpts quoted verbatim, legacy citation forms (the pre-#713 file-anchor form among them), that every rule and decisions file is indexed once, under a title matching its heading and file name, each decisions file's `## Standing` block against its live entries and against the roster of files exempt from one, the README's copy of rules/D2 against the file and its split-candidate grep against the rules' Signal lines, that the NFD-typed examples stay decomposed, a denylist of spellings that shipped wrong (`tests/v2/test_doc_spellings.py`, over nameparser/ and docs/), each rule's `Signal:` line (its presence, spelling and place) and the README's list of the tags those lines use, a hub's rules against the bullets under its Then, both ways, a comment naming an Except or Precedence bullet against that bullet's bold lead, the skeleton grammar in `tests/v2/test_rules_skeleton.py` (labels, field order, fences, the footer, one paragraph each for When, Tolerated and Then, the two sentence limits — Then one sentence, Because three at most — and an example under every Except bullet), the refusal of a rule in the retired block format, and — in `tests/v2/test_rules_budget.py` — each rule's recorded size. Assume they pass; a finding one of them would catch is a false positive. Read those files rather than trusting this sentence — it named five of the nine guards when written, and goes stale as guards are added.
Skip what the suite already enforces. `tests/v2/test_rules_doc.py`, `tests/v2/test_doc_citations.py` and `tests/v2/test_design_docs.py` machine-check example lines and their `deviates:` today-values, boundaries, the one-of `implemented:`/`tracked:` rule, `implemented:` against the modules that actually cite the rule, the rule IDs a rule names anywhere in its prose, cross-doc anchor IDs, citation excerpts quoted verbatim, legacy citation forms (the pre-#713 file-anchor form among them), that every rule and decisions file is indexed once, under a title matching its heading and file name, each decisions file's `## Standing` block against its live entries and against the roster of files exempt from one, a rule file reusing an ID a decisions file records `Retired:` and a misspelled `Retired:` line, the README's copy of rules/D2 against the file and its split-candidate grep against the rules' Signal lines, that the NFD-typed examples stay decomposed, a denylist of spellings that shipped wrong (`tests/v2/test_doc_spellings.py`, over nameparser/ and docs/), each rule's `Signal:` line (its presence, spelling and place) and the README's list of the tags those lines use, a hub's rules against the bullets under its Then, both ways, a comment naming an Except or Precedence bullet against that bullet's bold lead, the skeleton grammar in `tests/v2/test_rules_skeleton.py` (labels, field order, fences, the footer, one paragraph each for When, Tolerated and Then, the two sentence limits — Then one sentence, Because three at most — and an example under every Except bullet), the refusal of a rule in the retired block format, and — in `tests/v2/test_rules_budget.py` — each rule's recorded size. Assume they pass; a finding one of them would catch is a false positive. Read those files rather than trusting this sentence — it named five of the nine guards when written, and goes stale as guards are added.

1. **Recompute every number.** Any count, ratio or "N names" in the diff gets recomputed by a script written now, never read. (A filter described as returning one name returned three; a rule's stated blast radius listed two names its own scope excluded.)
2. **Your detector is a second, unreviewed implementation.** It must read the same vocabulary and boundaries the rule reads. (A particle-run detector that walked only the never-given half — the rule chains through ANY particle — reported 50 false movers where the true count was 1, printing cleanly both times.)
3. **Rule vs. its own examples.** (P1 said the fold takes "the particle and the one name word"; its example `de la Vega` is two particles onto one word.)
3. **Rule vs. its own examples.** (P1 said the fold takes "the particle and the one name word"; its example `de la Vega` is two particles onto one word.) An Except bullet's fence is examples of that bullet, not of the rule: a row there must be one where Then does NOT hold, so read each row in an Except fence against its own bullet's bold lead and move a row Then still decides out of it — to the main fence, or to the Accepted bullet that records it as a cost. Nothing checks this — the runner asserts a row's output, never which fence it sits in — and the #761 migrations, which sorted every rule's examples into fences by hand, put rows where Then held into an Except fence twice that review caught: H1's given-name rows (PR #779, back to the listing-record Accepted bullet) and S3's switched-off `Msc.Ed.` row (PR #783, to the main fence).
4. **Any claim about a document vs. what that document says about itself.** Two directions, both bitten. Inward: follow every `history:` and read the entry (P6's rationale asserted what decisions/vocabulary-collisions denies, and the rule's own guard exists because the assertion is false). Outward: when prose anywhere — this file, a code comment, another design doc — describes what one of these files contains or does, read that file's preamble or entry conventions first (an AGENTS.md bullet claimed the rules name the pipeline stages; their preamble forbids exactly that — "Statements are implementation-free: no stage names, no function names, no regexes"). Those declarations sit at the top of files people cite without reading. Mind their scope before calling a violation: the rules' ban binds STATEMENTS, which `_statements()` in test_doc_citations.py defines — a rule's fields, When, Tolerated, Then and its bullets, Because, and the Except, Precedence and Accepted bullets, without fences or footer — so Backgrounds and Not-in-scope may name functions and do.
5. **Does the change move another rule's OUTPUT?** A marker lands on the rule whose STATEMENT changed; a rule can move another's output without touching its statement, and the runner cannot see it. Walk the rules it names: the IDs in its When, Then, Except and Precedence. (family_base moved initials(); R3 had no marker.)
6. **Contested inputs: is precedence stated?** Naming a rule pins nothing: an ID in Then or Except does not say which rule wins. A rule states precedence in a `**Precedence:**` bullet — the input, the winning rule, why. Walk the same IDs as axis 5 and ask, for each, whether an input both rules claim has its winner written down. (P6 and S2 both claimed `Berg, Jan vd`.)
Expand Down
1 change: 1 addition & 0 deletions docs/design/decisions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ Entry conventions:
3. **Compaction, on a stated trigger.** A superseded entry is cut to a tombstone: its date, issue, verdict line and `SUPERSEDED by`, plus whatever it recorded that the superseding entry did not replace — a gap it found standing before its change, or an option it declined that the reversal does not answer, stays beneath the tombstone (decisions/T4's #731); a superseded passage inside a live entry is cut to its decision and what replaced it. A measurement paragraph (what moves, cost, blast radius) is cut to one line naming its population, counts, baseline and PR once its change has shipped in a release. The full text stays in git and in the PR the entry cites. #728 is the worked compaction. A harvested entry (below) keeps its framing line through compaction, a tombstone included, and its measured-vs-remembered markings wherever its measurements survive. A trap #759's review found: a one-line replacement carries each count with the classification the old text gave that count and no other (decisions/S2's #614 line had given a whole grid the classification measured on one slice of it).
4. **A per-word fact that moves into the vocabulary's TOML is REPLACED by a query that finds it** (`uv run python tools/vocab/query.py --issue 661`, `--excluded generation`), not left beside a dated "moved" note, which would keep two copies to drift. Only the per-word sentences go; the decision, its criterion and its measurement stay as written. Sanctioned since 2026-10-18 (decisions/vocabulary-model).
- **`## Standing`**, at the head of a file: the one part edited freely. One line per decision in force, in the present tense, ending with the entry that decided it by its exact date and the first issue its head names (`— 2026-10-06 #613`), or by its date alone where the head names none and no other entry shares the date, plus a `Not standing:` line for entries whose plan was carried out or whose schedule events overtook — a decision reversed by a later DECISION is marked under 1 instead and an `Open:` line for each open question. It indexes the log and is never a decision itself: a Standing line with no entry behind it, or only a superseded one, fails `tests/v2/test_design_docs.py::test_every_standing_line_points_at_a_live_entry`, and changing a decision is still a new entry, which updates Standing in the same diff. The entries follow under `## Log`. Every decisions file created after C1's block landed (2026-10-19) opens with one, the cheap moment to write it being the file's first entry; the older files are listed in `tests/v2/decisions_without_standing.json`, decisions/C12 and decisions/vocabulary-model among them though they too were created that day, before the rule. `test_a_decisions_file_off_the_list_opens_with_standing` fails on a file off that list with no block (the heading on a line of its own; the words quoted in prose do not count), on a listed file that has gained one, on a listed key no file carries, and on a listed key the list did not hold when the rule landed (a fixed set in `tests/v2/design_docs.py`, its size asserted), so the list never holds a key it did not start with: a new file can join it neither alone nor in exchange for one that leaves. The block stands ahead of every `## Log` heading; a `## Log` before it is a finding, and so is a heading that misspells `## Standing` (a trailing space, a different level), which would otherwise switch the pointer checks off. A decision of another rule's that this log is the record of stays a bullet naming its rule (decisions/C13's line for rules/C2, decisions/P9's for rules/S5); one recorded under another key is an `Elsewhere:` line pointing at that entry, not a bullet. Whether the older files get one is #762. Five files split out on 2026-10-19 (C13, C14, P8, P9, T5) showed the rule was needed: none had a block, though C1's had landed the same day.
- **`Retired:`**, for a rule taken out of rules/: a line of its own in the rule's decisions file, opening `Retired:` and naming the date, the issue and what replaced the rule (`Retired: 2026-11-02 #800 — folded into C1.`), in the Standing block where the file has one, beside the dated entry that decided it. The ID is then never reused: `tests/v2/test_design_docs.py::test_no_rule_reuses_a_retired_id` refuses a rule file carrying it, a `Retired:` line in a file keyed by no rule ID, and a line that misspells `Retired:` (bold, a list marker or number, lowercase), which would switch the refusal off for its ID. No rule had been retired when the check landed (2026-10-20), so it guarded nothing on the tree until the first one is; its synthetic case is what shows it bites.
- **Harvested entries** (content landed from a session's report) keep their provenance: a "measurements are that session's, spot-checked at landing" framing line, the contributor's measured-vs-remembered markings where they matter, and — the hard-won one — a flagged uncertainty is resolved by reading the source artifact, never by inference from the neighboring argument.
- **`Declined:`** — proposals rejected WITH the evidence that killed them. Resolved-as-no is a decision; without a home for it, the next person re-derives the rejected proposal and its measurement.
- **`Excluded:`** — standing prohibitions with indefinite lifetime, keyed by vocabulary set: entries that must stay OUT of a wordlist, each with its reason. Distinct from Declined because the failure mode differs — nobody re-derives a rejected proposal, but someone sweeping a wordlist ships the excluded entry as a bug. **A word-level refusal is better as data**: an `excluded` key on the word's TOML entry makes adding the refused reading a build error that quotes the reason, which prose cannot do. So these blocks narrow to CLASS-level prohibitions (a criterion that keeps a whole kind of word out) as their words move, under the exception above; a block whose words have not moved yet stays as it is.
Expand Down
2 changes: 1 addition & 1 deletion docs/design/rules/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ This document is NORMATIVE, not descriptive: the rules state how names are writt

Authority, scoped: `tests/v2/cases.py` pins CURRENT behavior; this document states INTENDED behavior. A mismatch between them must be classified, never defaulted: either the rule is wrong (fix it here) or the parser is wrong (the example takes a `deviates:` marker and an issue). Where this document is silent, the behavior is pinned-but-undocumented — an extraction gap to close, not a specification, and not license to change the behavior.

Rule IDs are stable forever: never renumbered, never reused. A retired rule's decisions entry records the retirement. Cross-references use the form `decisions/P2` / `mechanisms.md#SPANS`; a bare ID is never a citation. The citation-integrity test checks that every rule ID written in a rule's quotable prose (every field, below) exists — not that the two rules really meet, and not which of them wins, which a Precedence bullet states. `implemented:` and `tracked:` are checked too: every rule carries exactly one. `implemented:` names the modules honoring the rule, checked against the modules that cite it; `tracked:` names the issues that would ship a rule nothing implements yet, so a wholly-aspirational rule cannot sit here untracked, and a shipped rule cannot keep a stale tracking pointer.
Rule IDs are stable forever: never renumbered, never reused. A retired rule's file leaves this folder, and its decisions file records the retirement on a line of its own opening `Retired:` (decisions/README.md); `tests/v2/test_design_docs.py::test_no_rule_reuses_a_retired_id` refuses a rule file carrying an ID a decisions file records that way. Cross-references use the form `decisions/P2` / `mechanisms.md#SPANS`; a bare ID is never a citation. The citation-integrity test checks that every rule ID written in a rule's quotable prose (every field, below) exists — not that the two rules really meet, and not which of them wins, which a Precedence bullet states. `implemented:` and `tracked:` are checked too: every rule carries exactly one. `implemented:` names the modules honoring the rule, checked against the modules that cite it; `tracked:` names the issues that would ship a rule nothing implements yet, so a wholly-aspirational rule cannot sit here untracked, and a shipped rule cannot keep a stale tracking pointer.

## The skeleton

Expand Down
2 changes: 1 addition & 1 deletion nameparser/_parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ class Parser:
vocabulary DECLINES, so a locale pack's surnames always win where
they match; returning None declines in turn and the token stays
whole. Two promises narrow when one is supplied (the first is
rules/A1's Accepted clause):
rules/A1's Accepted bullet "The one exception to totality"):
parse-totality gains its one exception -- an exception raised by
the segmenter propagates, because a user-supplied callable's own
error is a user-code error, not a content error -- and this Parser
Expand Down
12 changes: 7 additions & 5 deletions nameparser/_pipeline/_comma.py
Original file line number Diff line number Diff line change
Expand Up @@ -483,11 +483,13 @@ def decide(state: ParseState) -> ParseState:
tokens[i] = copy_with(tokens[i], role=role, tags=tags)
# rules/C11: "A decision either way at the comma C1 reads is reported;
# for a run of words the decision is the flip to the credential
# run, reported once over the whole part" -- "A flip in which no
# listed word of the ambiguous credential class (C4) takes part is
# the exception and is made in silence" -- "unless what said so is
# nothing but other paired initials". This implements its Then,
# the bullet on runs S7 reads, and its Except bullet
# run, reported once over the whole part" -- its Then, which this
# implements, with rules/C11's Then bullet "A run the count leaves
# in the listing form reports as S7 reads the words in it" and
# rules/C11's Except bullet "A flip in which no listed word of the
# ambiguous credential class (C4) takes part is the exception and
# is made in silence" -- "unless what said so is nothing but other
# paired initials", the clause that bullet ends on
# (decisions/C11 (#544)). A run whose every class word is listed and settled by
# its capitals is read on that evidence, not flipped by the count.
# A caps word bound above carries the class tags now, so membership
Expand Down
7 changes: 4 additions & 3 deletions nameparser/_pipeline/_extract.py
Original file line number Diff line number Diff line change
Expand Up @@ -70,9 +70,10 @@ def _maiden_marked(content: str, lexicon: Lexicon) -> bool:
clause's word (it may be the surname Nee). A PHRASE marker is one
marker, so the word after is the word after the whole run --
'(z domu Nowak)' has one, '(z domu)' has none. The word after is
not tested for anything -- M3's Accepted line, and the reason a
bracketed '(née V)' reads maiden 'V' where the bare 'née V' gives
M2 a suffix.
not tested for anything, which is
rules/M3's Accepted bullet "The word taken after the marker is not tested"
(decisions/M3) and the reason a bracketed '(née PhD)' reads maiden
'PhD' where the bare 'née PhD' gives M2 a suffix.

This stage runs before tokenize, so it calls maiden_marker_run
itself rather than reading the tags classify will set. The two
Expand Down
Loading
Loading