Repository navigation
fix(preprocessor): handle pandas categorical missing values - #74
Conversation
Preserve fit-time output-format resolution and move pandas missing-value counting into the shared density helper. Add automatic-output regression coverage and clarify missing-marker behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Welcome to Codecov 🎉Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests. ℹ️ You can also turn on project coverage checks and project coverage reporting on Pull Request comment Thanks for integrating Codecov - We've got you covered ☂️ |
mkumar73
left a comment
There was a problem hiding this comment.
@Boulea7, thank you for your contribution!
The updated changes address the categorical missing-value inconsistencies reported in #64.
-
Default categorical imputation now treats None, pd.NA and NaN consistently.
-
Ordinal encoding correctly maps missing values to zero, and automatic one-hot encoding handles nullable categorical inputs.
-
Explicit imputer missing markers, category definitions and drop settings retain their existing semantics.
-
Raw categorical passthrough preserves the original missing markers without crashing output reporting.
-
The merge-conflict resolution preserves #73’s fit-time output-format selection. Missing-value-safe density counting is shared between automatic format selection and output reporting, rather than duplicated.
-
The regression tests cover nullable inputs, transform-time missing values, missing indicators, cloning, serialization and raw passthrough. Additional tests verify that later low-density batches retain the output format selected during fit.
-
Validation is satisfactory,
-
CI and Docs workflows also pass, including the supported Python/OS matrix and minimum-dependency checks.
No further blocking code issues were identified in the reviewed changes.
Approved.
Fixes #64.
Treat
Noneandpd.NAconsistently withNaNduring default categorical imputation and automatic one-hot encoding. Ordinal encoding maps missing values to zero, while raw passthrough preserves them without crashing the output report. Explicit imputer, category and drop settings retain their existing semantics.Validation: 1,479 core tests passed with locked dependencies and with minimum NumPy/pandas/scikit-learn/SciPy versions; locked branch coverage is 92.66%. Ruff and Pyright pass.