fix(stats): preserve Unicode line separators in filenames

Port PR #2301's regression for U+0085, U+2028 and U+2029 in filenames
with `core.quotePath=false`. Both backends raised `ValueError` because
`Commit.stats` and `Stats._list_from_string()` treated those characters as
record separators in Git's output.

Split only at LF and ignore empty records, preserving filenames and empty
commit results. Keep the existing Gix filename-quoting CLI fallback; this
change adapts Git's output and does not implement native diff behavior.
Clarify that fallback in `doc/gix-backend.md` and record the user-visible fix.

The scripted fixture creates root, modified and empty commits through Git.
Direct NUL-delimited `git diff-tree --numstat` and `--name-status` queries
validate the expected names, counts and change types before the GitPython
assertions. The regression fails before the fix with both backends; all 42
commit/statistics tests and 21 subtests pass afterward on each backend.
Ruff lint/format, changed-source mypy, ShellCheck and `git diff --check` pass.

Author: codex

doc/gix-backend.md

@@ -375,7 +375,7 @@ repository-format validation gap.
 | Committer identity, `var` | Native process/config identity from `Repository.committer()` | Per-command identity overrides, other variables and normalization/identity errors; GIX-21 |
 | Selected `set_object` / reference deletion, `update_ref` | Native ref edits with GitPython's existing validation | Strict create/CAS, branch-target validation, symbolic aliases other than HEAD, symbolic detachment, active-branch HEAD log, unchanged targets and log-message cleanup; GIX-9/10/21/24 |
 | Tree/commit `Diffable.diff` | Raw change records and unambiguous exact renames | Patches, paths, index/worktree/root diff, inexact/ambiguous renames; GIX-11 |
-| `Commit.stats` | Native Myers line counts, binary counts, first-parent tree comparison | Diff attributes, other algorithms, gitlinks, quoted filenames, large blobs; GIX-2/18 |
+| `Commit.stats` | Native Myers line counts, binary counts, first-parent tree comparison | Diff attributes, other algorithms, gitlinks, non-ASCII or quoted filenames, large blobs; GIX-2/18 |
 | `Repo.is_dirty` | Index/tree/worktree status, configured submodule checks and pathspecs | Custom/sparse indexes and non-root command directories |
 | `Repo.untracked_files` | Native directory walk | Extra status options and non-root command directories |
 | `Repo.ignored` | Native excludes with tracked-file suppression | Symlink/submodule traversal, unsupported path normalization, and unavailable path metadata |

doc/source/changes.rst

@@ -51,6 +51,10 @@ Git executable and safety
 API changes
 -----------
 
+* ``Commit.stats`` preserves Unicode line separators in filenames when
+  ``core.quotePath=false``. Git's LF-delimited output remains intact for root,
+  subsequent and empty commits, including the Gix backend's existing filename
+  quoting fallback.
 * ``GitConfigParser[section] = values`` again replaces a nonempty section,
   restoring Kas's Git identity cleanup. Git performs each mutation and locks
   it independently; replacement is not one atomic transaction. Empty sections

git/objects/commit.py

@@ -512,6 +512,8 @@ def stats(self) -> Stats:
         def process_lines(lines: List[str]) -> str:
             text = ""
             for file_info, line in zip(lines, lines[len(lines) // 2 :]):
+                if not line:
+                    continue
                 change_type = file_info.split("\t")[0][-1]
                 (insertions, deletions, filename) = line.split("\t")
                 text += "%s\t%s\t%s\t%s\n" % (change_type, insertions, deletions, filename)
@@ -520,12 +522,12 @@ def process_lines(lines: List[str]) -> str:
         if not self.parents:
             lines = self.repo.git._call_process_safe(
                 "diff_tree", self.hexsha, "--", numstat=True, no_renames=True, root=True, raw=True
-            ).splitlines()[1:]
+            ).split("\n")[1:]
             text = process_lines(lines)
         else:
             lines = self.repo.git._call_process_safe(
                 "diff", self.parents[0].hexsha, self.hexsha, "--", numstat=True, no_renames=True, raw=True
-            ).splitlines()
+            ).split("\n")
             text = process_lines(lines)
         return Stats._list_from_string(self.repo, text)
 

git/util.py

@@ -1051,7 +1051,9 @@ def _list_from_string(cls, repo: "Repo", text: str) -> "Stats":
             "total": {"insertions": 0, "deletions": 0, "lines": 0, "files": 0},
             "files": {},
         }
-        for line in text.splitlines():
+        for line in text.split("\n"):
+            if not line:
+                continue
             (change_type, raw_insertions, raw_deletions, filename) = line.split("\t")
             insertions = raw_insertions != "-" and int(raw_insertions) or 0
             deletions = raw_deletions != "-" and int(raw_deletions) or 0

test/test_commit.py

@@ -191,6 +191,40 @@ def test_stats_preserve_filename_spaces(self):
             }
             assert removed_stats.total == {"insertions": 0, "deletions": 30, "lines": 30, "files": 5}
 
+    def test_stats_preserve_unicode_line_separators(self):
+        paths = [f"before{separator}after.txt" for separator in ("\x85", "\u2028", "\u2029")]
+        with fixture_repo("commit.sh", "unicode-separators") as repo:
+            for revision, change_type in (("update-1", "A"), ("update-2", "M")):
+                assert repo.git.diff_tree(
+                    "--root",
+                    "--no-commit-id",
+                    "--no-renames",
+                    "--numstat",
+                    "-z",
+                    revision,
+                    "--",
+                    stdout_as_string=False,
+                ) == b"".join(f"1\t0\t{path}\0".encode("utf-8") for path in paths)
+                assert repo.git.diff_tree(
+                    "--root",
+                    "--no-commit-id",
+                    "--no-renames",
+                    "--name-status",
+                    "-z",
+                    revision,
+                    "--",
+                    stdout_as_string=False,
+                ) == b"".join(f"{change_type}\0{path}\0".encode("utf-8") for path in paths)
+                stats = repo.commit(revision).stats
+                assert stats.total == {"insertions": 3, "deletions": 0, "lines": 3, "files": 3}
+                assert stats.files == {
+                    path: {"insertions": 1, "deletions": 0, "lines": 1, "change_type": change_type} for path in paths
+                }
+            assert repo.git.diff_tree("--no-commit-id", "--numstat", "-z", "empty", "--") == ""
+            empty_stats = repo.commit("empty").stats
+            assert empty_stats.files == {}
+            assert empty_stats.total == {"insertions": 0, "deletions": 0, "lines": 0, "files": 0}
+
     def test_renames(self):
         commit = self.rorepo.commit("185d847ec7647fd2642a82d9205fb3d07ea71715")
         files = commit.stats.files

tests/fixtures/commit.sh

@@ -3,13 +3,18 @@ set -eu
 git init --initial-branch=master repo
 cd repo
 case "$1" in
-    spaces)
+    spaces|unicode-separators)
         # Construct trees directly: Windows rejects trailing spaces in index paths.
+        paths=(name ' name' 'name ' ' name ' ' ')
+        if [ "$1" = unicode-separators ]; then
+            git config core.quotePath false
+            paths=($'before\302\205after.txt' $'before\342\200\250after.txt' $'before\342\200\251after.txt')
+        fi
         for multiplier in 1 2; do
             tree=$(
                 count=0
-                for path in name ' name' 'name ' ' name ' ' '; do
-                    count=$((count + 1))
+                for path in "${paths[@]}"; do
+                    if [ "$1" = spaces ]; then count=$((count + 1)); else count=1; fi
                     oid=$(for ((line=0; line<count*multiplier; line++)); do printf 'line\n'; done | git hash-object -w --stdin)
                     printf '100644 blob %s\t%s\0' "$oid" "$path"
                 done | git mktree -z