Repository navigation
Conversation
Muguett-DBY
left a comment
There was a problem hiding this comment.
I tried two uppercase Turkish month names and both silently parse as January:
pendulum.from_format("15 MAYIS 2024", "DD MMMM YYYY", locale="tr")
# 2024-01-15 (expected 2024-05-15)
pendulum.from_format("15 NİSAN 2024", "DD MMMM YYYY", locale="tr")
# 2024-01-15 (expected 2024-04-15)The case-insensitive regex accepts these names, but match_translation() compares the results of .lower(), which differ for Turkish dotted and dotless I. The lookup returns None; _get_parsed_locale_value() then checks value rather than the lookup result, so the missing month defaults to January. On the base branch both inputs raise a format mismatch.
Could we add a Turkish uppercase test and make sure an unmatched translation cannot silently become January?
match_translation compared lowercased text, and "MAYIS".lower() does not equal "Mayıs".lower(). The name regex accepted the text, the lookup returned None, and from_format silently used January. The lookup now uses the same re.IGNORECASE rule as the regex, and a name that still cannot be mapped raises the parser's ValueError instead of defaulting. The parse also checks a full match and reads its groups directly. Before, re.sub stopped at the first alternative, so "Cumartesi" parsed as "Cuma". Agent: claude
Month and weekday names were joined into the parse regex unescaped. In fr, the name "lun." put its dot in the regex as "any character", so "LUNX" matched and the lookup then found no name for it. The five name tokens (MMMM, MMM, dddd, ddd, dd) now escape each locale name before joining. Other tokens keep their patterns. The locale sweep test now checks every shipped locale's month and weekday names in exact, upper and lower case. Each must parse to that exact name. A ValueError is allowed only where the token regex rejects the text. Agent: claude
|
Thanks for the repro. The new head is 05c1368. Name matching now uses the same case-insensitive rule as the regex, so MAYIS and NİSAN map to May and April. Locale names are now escaped in the regex, and Cumartesi no longer parses as Cuma. A name the lookup cannot map raises ValueError. A new test covers every shipped locale's names in exact, upper and lower case. AI disclosure: written and checked with Claude Code (tests and live run agent-run); I take responsibility for it. |
Pull Request Check List
Previously,
from_formatmatched the locale's month and weekday names case-sensitively, so14nov22withDDMMMYYraisedString does not match formatand TurkishMAYISforMayısraised the same error.Cumartesiwithddddintrparsed as Friday (Cuma), because the parse read the first alternative that matched, not the full match.This wraps the name tokens (
MMMM,MMM,dddd,ddd,dd) in a scoped(?i:...)group inFormatter._replace_tokensand escapes each locale name, so the.inlun.is literal.Locale.match_translationlooks a value up by exact match first, then by an escapedre.IGNORECASEfull match, the same rule the regex uses, soMAYISfinds May.from_formatchecks for a full match before it reads the groups, soCumartesiis Saturday. Other tokens (T,Z,A) keep their case rules.match_translationis only called fromfrom_formatparsing.Callers checked:
src/pendulum/__init__.py(from_formatis the only caller ofFormatter.parse, and the full test suite passes);rust/src/parsing.rs,rust/src/python/parsing.rs,src/pendulum/parser.pyandsrc/pendulum/parsing/__init__.pyhave their own parsers and do not call it.Fixes #668.
Ran live on
05c1368:pendulum.from_format('15 MAYIS 2024','DD MMMM YYYY', locale='tr')->2024-05-15 00:00:00+00:00(ValueErrorbefore);pendulum.from_format('LUNX','ddd', locale='fr')->ValueError;pendulum.from_format('14nov22','DDMMMYY')->2022-11-14 00:00:00+00:00(ValueErrorbefore).