# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).


## [0.22.0] 15.08.2026

### Fixes
- Multi-dimensional splits now evaluate every feature combination once per
  repetition, keeping all features aligned to the same count/group split
  instead of combining candidates from different repetitions.
- Preserve the median selection for single-feature and combined candidates
  while retaining the per-repetition data needed for correctly aligned joint
  count tables.

### Changes
- Document the repetition behavior across `n_dims`, its interaction with
  `calcs_per_dim`, and the associated training-time trade-offs.
- Add regression coverage for multi-dimensional combinations with multiple
  repetitions.

## [0.21.4] 13.08.2026

### Fixes
- `CatsMixin.get_sympy_from_bits`: the "all but one bit" optimization now only
  emits `Not(<bin>)` when the excluded bin is the missing bin (which renders
  as `IS NOT NULL`) or when no missing label is configured. Negating a plain
  bin is unsound under SQL three-valued logic: `NOT(bin)` evaluates to `NULL`
  (not `TRUE`) for `NULL` rows, so a cube slice whose intent includes the
  missing bin silently dropped rows with missing values, which then matched no
  spore and scored exactly 0


## [0.21.3] 12.08.2026

### Fixes
- `cat_cater_impl`: the missing sentinel now gets a dedicated label (`__missing__<bin>`) when it shares a quantile bin with regular categories, so the `is null` filter no longer silently drops the other categories in that bin
- `CatCats.get_bits`/`get_single_filter`: bit positions and single-bin filters are resolved by the label's position within `all_labels` instead of its numeric value, decoupling them from the (non-deterministic) `set` ordering
- `CatCats`/`ContCats.all_labels`: the missing label is only appended when it does not already occur among the regular bin labels, preventing duplicate labels
- `ContCats.cut`: the binned series is cast to plain strings before assigning the missing label, so missing values are always mapped even when `pl.cut` returns an `Enum` series (polars >= 1.41) where `fill_null` would silently do nothing
- `CategorizedFeatureMixin.get_left_right_filter`: the missing label is `fill_null`-ed when joining the target/non-target count tables, so the missing bin is scored normally instead of falling into the neutral branch


## [0.21.2] 09.08.2026

### Fixes
- `TrainDataframes.score`: a node with a single row (`sum == 1`) now uses a correction factor of `0.5` instead of the log formula, which would otherwise evaluate to the maximum factor `1.0` (`log(1) == 0`); single-row leaves are now weakened the most


## [0.21.1] 09.08.2026

### Fixes
- Revert the deterministic tie-breaking in the categorical binning sort (`cat_cater_impl`): categories are sorted again only by weight (descending) and by target rate (ascending). The two-key sorts added in 0.21.0 are removed because training is inherently non-deterministic (random data selection), so the extra sort keys only changed binning order without improving reproducibility


## [0.21.0] 09.08.2026

### Changes
- Categorical features (`cat_cater_impl`) treat null rows as a regular category in the binning via an internal null sentinel (`MISSING_SENTINEL`) instead of excluding them from the group-by; the missing bin can now be merged with other categories by the quantile binning
- Mapping keys of `CatCats` are always strings: the feature column is cast to `Utf8` in `CatCats.cut` before applying the mapping, so numeric categorical features (e.g. MNIST pixels) stay consistent
- Deterministic tie-breaking when sorting categories by weight / target weight
- `ContCats.get_single_filter`: a feature whose group set is entirely null (no cuts learned) now renders its value bin as `is not null` instead of raising an `IndexError`; combined with the existing `is null` missing bin this yields a clean "has value vs missing" two-way split

### Fixes
- The binning (and therefore `calc_diff`) is now invariant to whether missing rows are represented as `NULL` or imputed with `missing_value`: identical rows produce identical category groupings and scores
- `CatCats.cut` no longer fails on null-dtype columns; the `Utf8` cast resolves the invalid `i64 -> null` conversion for all-null count sets
- `dataframes.set_diff_df`: use `how="full"` instead of the deprecated `how="outer"` join argument (polars >= 0.20.29)


## [0.20.2] 08.08.2026

### Fixes
- `darkwing.get_df_from_files`: read multiple CSV/parquet files with `pl.concat` instead of `pl.read_csv(",".join(files))` (polars does not accept a comma-joined string)
- `train.cont_cater_impl`: normalize cumulative weights by the non-null weight sum so quantile cuts work for features where null rows carry significant weight (previously `IndexError: index 0 is out of bounds`)
- `dataframes.get_left_right_filter`: treat bins with `diff == 0` as neutral instead of non-target when `neutral_faktor` is 0 (matches the documented three-way split semantics)
- `train.counter`: return `(None, None, None)` when no feature survives categorization instead of raising `IndexError`, so the node becomes a leaf
- `pilz.get_where_sql`: render spores without cuts as `WHEN TRUE` so a root leaf with an empty cut list produces valid SQL


## [0.20.1] 08.08.2026

### Changes
- Add docstrings to the public modules (cli, datacard, settings, dataframes, filter, pilz, darkwing, datacardhelper, eval, thresholds, train, mermaid_fence)
- Split the monolithic `tests/test_models.py` into per-module test files with shared fixtures, and add tests for the CLI, training, eval, darkwing, thresholds and datacardhelper


## [0.20.0] 08.08.2026

### Added
- Missing values in features without `missing_value` are now treated as a separate "missing" category during categorization (detected per subsample), producing `IS NULL`/`IS NOT NULL` conditions in the trained spores
- `IS NULL`/`IS NOT NULL` SQL support in the filter layer

### Changes
- Features with `missing_value` set are still imputed at load time; features without it keep nulls and learn an explicit missing category instead


## [0.19.0] 06.08.2026

### Changes
- Move `train_files`/`test_files` from the DataCard into the settings: `train_files` (and optional `eval_files`) in `TrainSettings`, `test_files` in `EvalSettings`
- Thresholds during training are computed on `eval_files` (falls back to `train_files` if empty)


## [0.18.0] 04.08.2026

### Added
- Compute threshold files (youden and f1) at the end of the training step on the train data

### Changes
- Eval only reads the threshold files from the training folder (selected via different_target_pilz_comb_method), thresholds are not computed in eval anymore
- Add max back as different_target_pilz_comb_method (highest score, no threshold file)

## [0.17.4] 04.08.2026

### Added
- Combine different target pilze with max and youden as optional argument

## [0.17.3] 02.08.2026

### Added
- Combine same target pilze with max as optional argument

### Fixes
- Bug for eval with colum names conatining - 

## [0.17.2] 30.07.2026

### Changes
- Change weakening factor to log scale

## [0.17.1] 30.07.2026

### Added
- Add weakening factor for score for low popopulated spores

## [0.17.0] 29.07.2026

### Added
- Add n_rep which repeats the cut calulation to avoid statisccal fluctations


## [0.16.2] 07.06.2026

### Updates
- rm use_neutral_state cli arg
- disable simplfy usage
- Calc Youden index for multiclass result

## [0.16.1] 12.05.2026

### Updates
- Add missing files

## [0.16.0] 12.05.2026

### Updates
- Docs now in a good shape


## [0.15.3] 10.05.2026

### Updates
- update docs

## [0.15.2] 27.04.2026

### Added
- update docs

## [0.15.1] 26.04.2026

### Added
- KI generated documentation
- Fix links

## [0.15.0] 22.04.2026

### Added
- add churn example
- neutral state calulations


## [0.14.0] 12.03.2026

### Added
- implement infer (same as evel except no label is given in data)
- implemnet output file for eval/infer 
- added keep extra columns for evel/infer
- no hist plots (they need more work)


## [0.13.1] 09.03.2026

### Fixed
- Fix Homepage Path

## [0.13.0] 04.03.2026

- Rename to Pilz


## [0.12.0] 25.01.2026

- cli for creating data card
- add support for parquete files
- handle missing values 


## [0.11.0] 05.01.2026

### Added

- add neutral state for low populated states (optionla feature)
- add espresso minmizer for smaller queries
- introduce batched eval, since larger where clauses are not good for duckdb


## [0.10.0] 10.12.2025

### Changed

- use Sympy to handle binary logic (very slow but works, could also simplify numerical expressions but then even more slow)
- Bank Term Deposit example (many cat features)

### Added
- Works with Categorical features


## [0.9.1] 07.10.2025

### Added
- First version of CICD pipeline

### Changed

- make eval folde to list

## [0.9.0] 08.09.2025

### Added

- Initial version of project
- Works with continus features only
- Example version runs with mnist 
- No docmention so far
