Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,4 @@
- 2026-09-08 #518 review round (the entry above ran into a second day; its own measurements are dated 2026-09-08 under a 2026-09-07 header) — a title standing after a family comma now silences the report. `John V, Dr.` is the shape: a comma with no name word after it hands segment 0 back to the positional read, the leading-title peel counts only what stands in segment 0, so `n` is 0 and the report named `given` for a word H1 then wrote to `family` — the caller was told the convention chose a field the output does not show. `_assign_main` takes a keyword-only `titled`, which the family-comma call fills from the gate's own reading (a False in it is a title, not a suffix), and the field-deciding emitters require `not titled`. Measured on this branch: `John V, Dr.`, `Smith V, Prince` and `Smith V, Dr.` keep every role and lose the `given-or-family` report, keeping the roman numeral's; `John V, Sir` does too, and it is the one worth naming — a given-name title leaves the word in `given`, which is the field the report would have named, so the report and the reading agreeing is what a silenced clause looks like when the rule that decided it happens to agree. Zero corpus names move: no corpus name reaches the site under a comma at all, which is also why the `len(state.segments) == 1` clause above measures inert.
Consolidated the same day, 2026-09-08, and BEHAVIOR IDENTICAL. Segment 1 is now assigned BEFORE segment 0's positional read, so the title standing after the comma already carries a `Role.TITLE` when that read reaches its emitters; the keyword is gone and `n == 0`, `not titled`, the maiden scan and `not resolved.by_script` are one `field_undecided` predicate that both field-deciding emitters read. The token scan subsumes `n == 0` because nothing before assign sets `Role.TITLE` and assign sets it in exactly two places, the leading-title peel and the post-comma reading. Measured against a `git archive` of the previous commit: every distinct name in the deduped `corpus*.jsonl` glob plus the shapes named in this entry and in the #491 entries — 1136 names, 5680 parses — reads the same seven roles and reports the same kinds, details and token texts under the default order, FAMILY_FIRST and FAMILY_FIRST_GIVEN_LAST. One shape constraint came out of the measurement rather than the reading: the two token scans sit UNDER the peel count rather than beside it, because run on every parse they cost 10 calls/name and put the facade 457 over the 456 ceiling of its band (decisions/parse-cost).

- 2026-10-20 #761 — the family-comma silencer stays a Then bullet with no example, by decision: the only one-word input of that shape, `, Andrew`, is not a name anyone types, so it stays out of the corpus names (Derek, PR #781).
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,4 @@
- 2026-09-06 #511 — a suffix value handed to `revise()` derives its entries from its own commas, by the rule a whole name uses. `Parser.revise` sub-parsed each value and forced every harvested token to the named role AFTER the sub-parse had run R1's entry pass, which keys on Role.SUFFIX; a bare 'MD PhD' reads there as a title and a family name, so the pass joined nothing and the field rendered 'MD, PhD'. The fix runs the same pass again: `revise` now sub-parses to a ParseState, forces the role on every non-dropped token, and calls `suffix_entries` — the pass, lifted unedited out of post_rules' tail into a function the two callers share (mechanisms.md#ONE-PREDICATE-PER-QUESTION) — over the forced state, then assembles and harvests as before. So a comma in the value parts two credentials and a space joins them. TWO SPELLINGS of the lifted pass, and the reason is the call budget: a first draft had post_rules call the state-in/state-out wrapper, and the second ParseState build cost three more calls per parse against the band tests/v2/test_benchmark.py holds (py3.11, 2026-09-06, tools/perf/call_count.py: 450 calls/name before the move, 451 with the in-place worker `_mark_suffix_entries` that post_rules now calls, 454 with the draft; the facade band tops at 455.9), so the worker writes in place as every other post rule does and the wrapper exists for the one caller with no token list of its own. NO `_run` HELPER for the same reason: a draft routed `parse()` through a private state builder shared with `revise`, and that cost one frame per parse on the hot path (py3.11, 2026-09-06: 415 calls/name against 414 without it, in a 402-418 band), so the four-line construction is spelled twice and `parse()` is untouched. MEASURED 2026-09-06 over the 1117 distinct corpus names, revising each of the 368 with a non-empty suffix by its own suffix: 38 suffixes moved before and 1 after, and the differential gate was byte-identical at all four baselines (recipe in the #511 issue body). THE ONE LEFT is '김민준씨, J.씨', and it is not entry structure: the whole-name parse keeps 'J.씨' one glued suffix token, suffix '씨, J.씨', while the sub-parse of the bare value '씨, J.씨' peels the honorific off the initial, so the revised field renders '씨, J. 씨' — right entries, the spurious comma of before ('씨, J., 씨') gone, one word read differently by the value's own parse than by the whole name's. That is the "classified ON ITS OWN" limit `revise`'s docstring has always recorded, and CJK honorific peeling is a W-rule question this change does not move; pinned in `test_revise_reads_a_glued_honorific_on_its_own`. SUPERSEDES the phd-merge acceptance of 'Ph., D.' on this path (its bullet says how). STALE TAGS, tried and backed out: a draft cleared the sub-parse's own "joined" with the forcing and re-derived it, because the pass only ADDS the tag and `revise(n, family="Jones MD PhD")` carries the sub-parse's between-piece suffix mark on 'PhD' onto a FAMILY token. Measured 2026-09-06 against `330ee55`, the clear also destroyed every WITHIN-piece mark on a non-suffix value — 'D.' of `revise(n, family="John Ph. D. Smith")` lost the merge mark the #436 bullet's DECLINED list calls role-blind and correct for every role — while on a ParsedName only the suffix string view reads "joined" (`_text_for`'s suffix_join gate) — the facade's `_list_for` heals it for every role, but no path puts a revised name into a HumanName, the v1 setters going through `replace()`, so wiring those setters onto `revise` is the change that would show the stale mark, as `last_list == ['Jones', 'MD PhD']` — `initials()` and every field string being identical at both trees for every shape measured. So the tags are kept minus FOLDED_TAG as before; the between-piece mark on a forced non-suffix role is a tag-only oddity that predates this change and stays, and for a suffix value nothing depends on the sub-parse's marks, every pair it joined sharing a bucket with no parting token and the pass setting it again; pinned in `test_revise_keeps_the_sub_parses_within_piece_mark`. Dropped tokens keep their role and tags through the forcing, because the pass filters them by index and assemble omits them. ONE MORE LIMIT, pinned in `test_revise_leaves_a_policy_delimiter_unparted_without_a_tail_segment`, and it is the "classified ON ITS OWN" limit again rather than a rule of revise's: a delimiter the policy names through `extra_suffix_delimiters` is dropped, and so parts entries, only on a segment after a comma that the reading of the words makes a tail, and a value with no comma of its own has none — under `Policy(extra_suffix_delimiters=frozenset({" - "}))` the whole name 'Doe, John, MD PhD - FACS' renders 'MD PhD, FACS' while `revise(n, suffix="MD PhD - FACS")` renders 'MD PhD - FACS', the dash surviving as a token and the forced role making it a suffix word; measured 2026-09-06, a value whose own words read with a tail segment does part ('John Doe, MD - FACS' revises to 'John Doe, MD, FACS') and one after a suffix comma does not ('MD, PhD - FACS' stays), which is how the value's words read as a name deciding it. The round-trip is unaffected, the whole-name view having already rendered that boundary as a comma. DECLINED: a list-valued `revise(p, suffix=["MD PhD", "FACS"])`, which widens the API and leaves the string path where it was; documenting the limit with pins alone, the docstring already recording it and #511's measurement showing it reachable on every space-joined run #436 produced; and a text-only read of the delimiter cores inside `revise`, a second rule for a case no round-trip reaches. Out of scope and left as is: `ParsedName.replace(suffix="MD, PhD")` renders 'MD,, PhD', `replace` whitespace-splitting by contract and the facade setters riding on it for v1 parity.
- 2026-10-18 #728 — MOVED FROM decisions/C1. These entries decide that one comma segment is one suffix entry, which is R1's entry pass rather than the comma decision, and were filed under decisions/C1 until #728; they moved unchanged but for the pointers the move broke, the measurement paragraphs, each cut to one line naming its population, counts, baseline and PR, and the superseded passages, each cut to its decision and what replaced it.
- 2026-10-19 #760 — MOVED FROM the entry pass's code comment (`_post_rules._mark_suffix_entries`), the facts the entries above did not hold. ORDER: the pass runs after assign because it keys on Role.SUFFIX; the same span rule run role-blind would join the A and B of `John A B Smith` into one middle_list element, pinned by `test_the_pass_runs_after_roles_are_settled` in tests/v2/pipeline/test_post_rules.py. Within post_rules it runs last by convention, not by necessity: every retag there targets a name role (`_retag` is called with FAMILY, GIVEN, MIDDLE and `_name_positions`' return, which is those three, and the `role=Role.FAMILY` replaces of P6's no-comma attachment and O3's fold are FAMILY too), so no rule moves a token into or out of SUFFIX or {TITLE, NICKNAME, MAIDEN} and the predicate reads the same answer wherever in the stage it stands (measured 2026-09-06 by reading the stage's retag targets). THE UNGATED CORE READ: group drops a core only on a `tail` segment, while the pass reads `delimiter_cores` whole and asks by TEXT alone, so a dropped token whose text the policy names parts the run whatever dropped it. The gate is not needed: a core group dropped was on a tail segment by construction, and the one case the ungated read adds is a maiden marker the policy also lists — under `Policy(extra_suffix_delimiters=frozenset({" nee "}))`, `John Doe, MD nee Jones PhD` renders 'MD, PhD' where the default policy renders 'MD PhD' (measured 2026-09-06), the writer's configuration having named that text a separator. That example no longer reaches the case: re-measured 2026-10-19 while moving it here, `John Doe, MD nee Jones PhD` reads title 'MD', given 'nee', middle 'Jones', family 'John Doe' and suffix 'PhD' under both policies, the comma now read as a family comma, so the marker is never dropped and the pass sees one post-nominal; `Smith, MD nee Jones PhD`, the bare-marker example of the #436/#437 entry above, reads the same way. The witness today is the bracketed clause: `Smith, MD (nee Jones) PhD` renders 'MD PhD' under the default policy and 'MD, PhD' under that one, extract having dropped the marker as Role.MAIDEN and group never dropping it, so the ungated text read is what parts the run (re-measured 2026-10-19 in #760's review). The same input is the filter's example below. THE DROPPED-INDEX FILTER is belt-and-braces: cores leave `pieces` before assign and get no role, and the one dropped token that arrives roled is the MAIDEN one tokenize roles from an extracted clause (`Smith, MD (nee Jones) PhD` drops index 2, the marker, as Role.MAIDEN); the filter keeps the suffix list and the parting scan agreeing about what a dropped index is.
- 2026-10-20 #761 — MOVED FROM rules/R1's revise() Accepted clause when R1 moved to the skeleton format, the measurement it carried: the #511 bullet counted 368 of 1117 distinct corpus names with a suffix on 2026-09-06; re-measured 2026-09-10 with that bundle's rows in the corpora, 372 of 1156 distinct names, and the same single failure, '김민준씨, J.씨'.
24 changes: 14 additions & 10 deletions docs/design/rules/A2-empty-name.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,17 @@
# A2 — the empty name

A2. Rationale: an input with no name content names nobody, and
saying so beats inventing fields from punctuation.
A name with no name content parses to the empty name — every
field empty, false as a boolean — while ambiguities born from
its punctuation survive on the empty result.
".," → family=""
"(" → ambiguities=("unbalanced-delimiter",)
"John . Smith" → family="Smith" · boundary
Signal: a count: no name content, no letter or digit, in the input
history: decisions/A2 · implemented: nameparser/_pipeline/_assemble.py
**When:** an input holds no name content, no letter or digit, as in ".,".

**Then:** a name with no name content parses to the empty name — every field empty, false as a boolean — while ambiguities born from its punctuation survive on the empty result.

**Because:** an input with no name content names nobody, and saying so beats inventing fields from punctuation.

```
".," → family=""
"(" → ambiguities=("unbalanced-delimiter",)
"John . Smith" → family="Smith" · boundary
```

---
**Signal:** count: no name content, no letter or digit, in the input
**history:** decisions/A2 · **implemented:** nameparser/_pipeline/_assemble.py
26 changes: 13 additions & 13 deletions docs/design/rules/D1-segmenterless-activation-warning.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
# D1 — the segmenterless-activation warning

D1. Rationale: a parser whose activated scripts nothing can divide
behaves like a working parser minus a feature, silently — the
one misconfiguration a caller cannot see in output.
Constructing a parser that activates division for scripts with
no covering surnames and no segmenter warns at construction,
naming the dead scripts and each way out.
[segmenterless-ja] → warns="deactivate with Policy(segment_scripts=frozenset())"
no-boundary: any covering surname vocabulary, configured
segmenter, or deactivation silences it — the default parser and
the zh pack never warn, which every other example in this
document exercises.
Signal: none in the name: the configuration's activated scripts against its surnames and segmenter
history: decisions/D1 · implemented: nameparser/_parser.py
**When:** a parser is built that activates division for a script no surname vocabulary it holds covers and no segmenter divides.

**Then:** constructing a parser that activates division for scripts with no covering surnames and no segmenter warns at construction, naming the dead scripts and each way out.

**Because:** a parser whose activated scripts nothing can divide behaves like a working parser minus a feature, silently — the one misconfiguration a caller cannot see in output.

```
[segmenterless-ja] → warns="deactivate with Policy(segment_scripts=frozenset())"
```

---
no-boundary: any covering surname vocabulary, configured segmenter, or deactivation silences it — the default parser and the zh pack never warn, which every other rule's examples exercise.
**Signal:** none in the name: the configuration's activated scripts against its surnames and segmenter
**history:** decisions/D1 · **implemented:** nameparser/_parser.py
55 changes: 23 additions & 32 deletions docs/design/rules/O1-east-slavic-rotation.md
Original file line number Diff line number Diff line change
@@ -1,35 +1,26 @@
# O1 — East Slavic rotation

O1. Rationale: an East Slavic name written family-first still shows
its patronymic — the distinctive ending identifies which word is
the patronymic, and the patronymic sits next to the given name.
With East Slavic patronymic handling active and no comma in the
name, a name of exactly three name words — titles, suffixes and
nicknames aside — whose last name word carries a patronymic
ending and whose middle name word does not reads as family-first:
the words are restored to given, patronymic, family. A middle
word that also carries the ending blocks the reading, because
the surname itself may be patronymic-derived.
The restoration is the default order's work: it recovers the
given-first reading a family-first listing hides, so where the
caller has DECLARED a family-first order there is nothing left
to restore and position decides.
"Сидоров Иван Петрович" [ru] → family="Сидоров"
"Sidorov Ivan Petrovich Jr." [ru] → family="Sidorov"
"Иван Петрович Абрамович" [ru] → family="Абрамович" · boundary
Accepted: under a declared family-first order the readings part
on natural-order input, and the declaration wins. A family-first
listing reads the same either way under FAMILY_FIRST — the first
example above is that same parse with FAMILY_FIRST declared,
and under FAMILY_FIRST_GIVEN_LAST the listing's given and middle
swap, so only the family is invariant — while with East Slavic
handling active and FAMILY_FIRST declared the natural-order
Иван Петрович Сидоров reads family Иван, given Петрович, middle
Сидоров. That is the caller's declaration being honored on input
they said was written family-first, not a defect (#384). No
registered example annotation combines a pack with an order, so
the parse is pinned in tests/v2/test_locales.py instead, and
decisions/O1 records why options 2 and 3 were declined.
Signal: a shape: a patronymic ending on the last of exactly three name words, and not on the middle one
history: decisions/O1 · implemented: nameparser/_pipeline/_post_rules.py
**When:** East Slavic patronymic handling is active, the default given-first order stands, and the last name word of a name with no comma carries a patronymic ending, as in "Сидоров Иван Петрович".

**Then:** a name of exactly three name words — titles, suffixes and nicknames aside — whose last name word carries a patronymic ending and whose middle name word does not reads as family-first: the words are restored to given, patronymic, family.

- **A middle word that also carries the ending blocks the reading**, because the surname itself may be patronymic-derived.
```
"Иван Петрович Абрамович" [ru] → family="Абрамович" · boundary
```
- **The restoration is the default order's work**: it recovers the given-first reading a family-first listing hides, so where the caller has DECLARED a family-first order there is nothing left to restore and position decides.

**Because:** an East Slavic name written family-first still shows its patronymic — the distinctive ending identifies which word is the patronymic, and the patronymic sits next to the given name.

```
"Сидоров Иван Петрович" [ru] → family="Сидоров"
"Sidorov Ivan Petrovich Jr." [ru] → family="Sidorov"
```

**Accepted:**

- Under a declared family-first order the readings part on natural-order input, and the declaration wins. A family-first listing reads the same either way under FAMILY_FIRST — the first example in the fence is that same parse with FAMILY_FIRST declared, and under FAMILY_FIRST_GIVEN_LAST the listing's given and middle swap, so only the family is invariant — while with East Slavic handling active and FAMILY_FIRST declared the natural-order Иван Петрович Сидоров reads family Иван, given Петрович, middle Сидоров. That is the caller's declaration being honored on input they said was written family-first, not a defect (#384). No registered example annotation combines a pack with an order, so the parse is pinned in tests/v2/test_locales.py instead, and decisions/O1 records why options 2 and 3 were declined.

---
**Signal:** shape: a patronymic ending on the last of exactly three name words, and not on the middle one
**history:** decisions/O1 · **implemented:** nameparser/_pipeline/_post_rules.py
Loading
Loading