Skip to content

Migrate the T rules and W1, W2, W4 to the skeleton format (#761, part 7) - #782

Merged
derek73 merged 12 commits into
masterfrom
claude/wizardly-varahamihira-cc9119
Oct 20, 2026
Merged

derek73 merged 12 commits into
masterfrom
claude/wizardly-varahamihira-cc9119

Conversation

@derek73

@derek73 derek73 commented Oct 20, 2026 •

Copy link
Copy Markdown
Owner

Part of #761.

Moves T1–T5 (tokens, initials and punctuation) and W1, W2, W4 (scripts) from the block format to the skeleton format, one commit per rule. Four more commits follow: review fixes, and one carrying Derek's decisions (below). W3 was already a skeleton rule. Its full-stop split is Derek's call and is not touched here.

No rule changes meaning. Every rule's example lines are the same multiset as on master (checked per commit), corpus_rules.jsonl regenerates with no diff, and no code-comment quote needed retargeting: each quoted clause kept its verbatim home in Then or a bullet. tests/v2/rules_budget.json is re-recorded, one row per rule commit, each row changing format.

Per rule

  • T1 (separators, not joiners). The opt-out (keep-emoji) is an Except. It's an input inside When where Then stops holding.
  • T2 (the katakana middle dot divides like a space). A straight move; keeps its no-boundary: waiver.
  • T3 (the interpunct's flank guard). A straight move. The Gal·la boundary stays in the main fence, because Then's "anywhere else" clause still holds for it.
  • T4 (the bare letter as an initial). The one-case statement is Then's head, carrying both halves (one case: an initial wherever its capital is; mixed case: no initial).
    • The four paragraph-scoped sentences are Then bullets, each with the scope its old paragraph gave it written into its lead.
    • The P8 hand-off is a Precedence bullet.
    • The maiden-numeral cost is Accepted.
  • T5 (the written shape of an initial). The non-Latin bare capital is a Then bullet, since Then still holds there. The interacts list became the bullet that names the rules reading this shape. The digit-with-period cost is Accepted.
  • W1 (unspaced CJK division). The default/opt-in split and the segmenter's undivided-only condition are Then bullets. The interpunct stand-down is an Except (decided, below).
  • W2 (the glued-honorific peel). The post-nominal step-past is a Then bullet with the 박사님 and 선생님 rows. Both Accepted clauses are kept verbatim; 田中部長 moved into the first one's fence.
  • W4 (script-scoped order). The katakana-middle-dot clause moved from Accepted to a Then bullet: the input is inside When and Then holds for it. The interpunct stand-down is an Except (decided, below).

Strict mode in T4 and W2

  • T4 named C14's strict mode in a general clause: "a suffix test that vetoes an initial vetoes it — C14's strict mode, and the scan-back of W2's peel". That clause is kept word for word in the one-case Then bullet.
    • A first draft moved it to a Precedence bullet. Review found that bullet widened the claim, and no input is contested: T4 and C14 compose, and C14's own Then bullet carries the other side.
    • Measured under strict: john smith, v gives given v, the same reading as JOHN SMITH, V. Mixed-case John Smith, v keeps suffix v.
  • W2 restates no part of strict mode. Its scan-back always uses the strict token test, its own and not C14's switch. Measured: 田中さん v and 田中さん V read alike under both settings. The switch reaches the peel only across a comma, and that is W3's existing Precedence bullet. So W2 needed no C14 bullet.

Old interacts: IDs

Rule ID Fate
T4 M2, M4, O5, P3, S10, W2 kept, Then bullets (M2 also in Accepted)
T4 P8 kept, Then bullet and Precedence
T4 T5 kept, Then bullet
T4 C14 kept, Then bullet
T4 P2 dropped: the statement never named it, and no example reads through P2's chain
T4 R3 dropped: the statement never named it; R3's initials read T4's tag but state nothing of it
T4 R4 dropped from the rule: the one R4 interaction (case repair of a connective letter in a one-case maiden clause) is a snapshot-invisible move recorded in decisions/T4, which keeps it
T5 T4, M4, O5, P7, C9, S10, H2 kept, all named in Then
W1 W3 kept as the decisions/W3 pointer the statement already carried; W3 names W1 from its side
W2 W3 dropped: W3's Then bullet and Precedence state the comma crossing and name W2
W2 T4 dropped: T4's Then bullet names W2's scan-back
W4 T3 kept, Except (T2 is now named too, in a Then bullet)

Decided (Derek, 2026-10-20)

  • README signal-tag list. T1 and T2's footer signal reads character:, translating the block format's "a character:". README's tag list was already missing separator (C10). The list gets fixed in the final Give rule blocks a skeleton: Rule, Except, Precedence #761 PR, the one that retires the block format. Not here.
  • W1's and W4's interpunct stand-downs moved from Accepted to Except. Each is an input inside When where Then stops holding. Words and examples are verbatim, and each Except bullet carries its example. Dated bullets record the move in decisions/W1 and decisions/W4. Three references that called these clauses "W1's Accepted" or "W4's Accepted pair" now point at the Except: in tests/v2/cases.py, in tests/v2/pipeline/test_script_segment.py and in decisions/T3.
  • T1's strip_bidi=False half stays without an example. A corpus name made of invisible direction marks is not worth a new policy annotation. A dated bullet in decisions/T1 records this.

Review

Two docs-review rounds.

  • Round 1 found four T4 scope losses: two bullets had lost the one-case or own-words scope of their paragraph (axis 13's trap), the C14 Precedence widened the claim, and the general veto clause had narrowed. It also found that T1's Except added promises. All were fixed in d8a117d.
  • Round 2 reviewed that commit and found nothing.
  • Round 3 reviewed the decisions commit (7957411). It found that tests/v2/cases.py's 间隔号 note had cited W1 for an order claim, which belongs to W4's Except and is about declared order, not source order. The note was wrong before this PR too. It also found a test_lexicon.py comment that cited rules/W1 for the NFD degradation that decisions/W1 records, and that decisions/T1's new bullet misdescribed the example it declines. All were fixed in e51b1e9.
  • Round 4 reviewed e51b1e9. The same NFD pointer still stood at the surname site in _script_segment.py, and "the 2026-07-29 ja-amendment bullet" matched two bullets. Both were fixed in e47e0e3, a two-line comment change that names decisions/W1's NFD bullet (ja amendment 1a). That commit itself has not had a review round.

The suite (22,157 passed), mypy and ruff are clean.

🤖 Generated with Claude Code

derek73 and others added 9 commits October 20, 2026 11:45
Part of #761.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eton format

Part of #761.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Part of #761.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Part of #761.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Strict mode now stands as a Precedence bullet citing C14.

Part of #761.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Part of #761.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Part of #761.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Part of #761.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
T4: the bullet listing the rules that read a lower-case letter as its
capital, and the case-question bullet, regain the scope their paragraph
gave them; C14's strict mode goes back into the general suffix-veto
clause, since T4 and C14 contest no input. T1: the opt-out bullet says
only what the old statement did.

Part of #761.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73 derek73 added docs Documentation fixes and updates design labels Oct 20, 2026
@derek73 derek73 self-assigned this Oct 20, 2026
@codecov

codecov Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.05%. Comparing base (3ecbc9e) to head (e47e0e3).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #782   +/-   ##
=======================================
  Coverage   99.05%   99.05%           
=======================================
  Files          47       47           
  Lines        4642     4642           
=======================================
  Hits         4598     4598           
  Misses         44       44           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

derek73 and others added 3 commits October 20, 2026 13:21
…ions

A name the interpunct divides is inside each rule's When, and Then
stops holding there, so it is an exception by the skeleton's own
definition; the words and examples are unchanged. Dated decisions
bullets record that move (W1, W4) and the decision to leave T1's
strip_bidi=False half without an example. Three references to
"W1's Accepted" / "W4's Accepted pair" are retargeted.

Part of #761.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cases.py's 间隔号 note cited W1 for an order claim W4's Except makes,
and said source order where the rule says declared order. A
test_lexicon.py comment cited rules/W1 for an NFD degradation that
decisions/W1's ja-amendment bullet records. decisions/T1's new bullet
now describes the example it declines.

Part of #761.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The surname-site comment that test_lexicon.py's pin supports cited
rules/W1, which says nothing about NFD; both now name decisions/W1's
2026-07-29 NFD bullet (ja amendment 1a), the one of its two
2026-07-29 bullets that records the trade.

Part of #761.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73
derek73 merged commit 713d2b7 into master Oct 20, 2026
12 checks passed
@derek73
derek73 deleted the claude/wizardly-varahamihira-cc9119 branch October 20, 2026 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

design docs Documentation fixes and updates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant