Metadata-Version: 2.4
Name: probcal
Version: 0.1.1
Summary: Universal post-hoc probability calibration for binary classifiers: methods, metrics, diagnostics, and auditable offsetting — numpy-only.
Project-URL: Homepage, https://github.com/wlazlod/probcal
Project-URL: Documentation, https://wlazlod.github.io/probcal/
Project-URL: Changelog, https://github.com/wlazlod/probcal/blob/main/CHANGELOG.md
Project-URL: Issues, https://github.com/wlazlod/probcal/issues
Author: Daniel Wlazło
License-Expression: MIT
License-File: LICENSE
Keywords: binary-classification,calibration,credit-risk,isotonic-regression,probability,reliability
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Requires-Dist: numpy>=1.26
Provides-Extra: dev
Requires-Dist: black>=24; extra == 'dev'
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Requires-Dist: scikit-learn>=1.4; extra == 'dev'
Requires-Dist: scipy>=1.11; extra == 'dev'
Requires-Dist: statsmodels>=0.14; extra == 'dev'
Provides-Extra: docs
Requires-Dist: ipykernel>=6; extra == 'docs'
Requires-Dist: mkdocs-jupyter>=0.24; extra == 'docs'
Requires-Dist: mkdocs-material>=9.5; extra == 'docs'
Requires-Dist: mkdocstrings[python]>=0.24; extra == 'docs'
Provides-Extra: viz
Requires-Dist: matplotlib>=3.8; extra == 'viz'
Description-Content-Type: text/markdown

# probcal

[![PyPI](https://img.shields.io/pypi/v/probcal.svg)](https://pypi.org/project/probcal/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![Python](https://img.shields.io/badge/python-3.11%2B-blue.svg)](pyproject.toml)

Universal post-hoc probability calibration for binary classifiers: methods, metrics,
diagnostics, and auditable offsetting — **numpy-only**.

`probcal` unifies the binary calibration literature (Platt, temperature, beta, isotonic,
centered isotonic, histogram binning, scaling-binning, BBQ, ENIR, Venn–Abers, spline
calibration), an extensive catalog of calibration evaluation metrics and statistical tests,
calibration visualization on both probability and logit scales, an auditable logit-offset
(central tendency) adjustment, automatic method selection under nested validation, and two
data flows (prefit and cross-validation). Primary application domain: credit-risk PD models;
the package is fully general.

**Status:** released on PyPI, beta. The API is stable enough to build on; breaking changes
bump the minor version until 1.0.

## Installation

```bash
pip install probcal            # runtime: numpy only
pip install "probcal[viz]"     # + matplotlib for probcal.plots
```

Development setup (tests, lint, type-check):

```bash
git clone https://github.com/wlazlod/probcal && cd probcal
uv sync --extra dev
```

## Quickstart

```python
from probcal import BetaCalibrator, make_pd_portfolio
from probcal.metrics import calibration_guardrails

port = make_pd_portfolio(n=8000, random_state=42)   # synthetic 3% PD portfolio

g_before = calibration_guardrails(port.y, port.scores)
print(f"before: slope={g_before.slope:.3f}  intercept={g_before.intercept:+.3f}  ok={g_before.all_ok}")

cal = BetaCalibrator().fit(port.scores, port.y)
p = cal.predict_proba(port.scores)

g_after = calibration_guardrails(port.y, p)
print(f"after:  slope={g_after.slope:.3f}  intercept={g_after.intercept:+.3f}  ok={g_after.all_ok}")
print()
print(cal.interpret())
```

Output:

```text
before: slope=0.968  intercept=-0.765  ok=False
after:  slope=1.000  intercept=+0.000  ok=True

Interpretation[BetaCalibrator]
parameter  value
---------  --------
a          0.875054
b          1.58922
c          -1.15227
- a = 0.875: sensitivity near s -> 0; a < 1 raises the smallest probabilities (model was overconfident in the low tail), a > 1 deepens them
- b = 1.589: sensitivity near s -> 1; the mirrored reading for the high tail
- c = -1.152: base-rate shift of -1.152 log-odds, odds factor 0.316
- identity map corresponds to (a, b, c) = (1, 1, 0)
- a != b (gap -0.714): asymmetric tail distortion that no symmetric (Platt/temperature) map could express
```

Automatic selection, model wrapping, offsetting, and threshold translation:

```python
from probcal import CalibratedModel, CalibratorSelector, PlattCalibrator

sel = CalibratorSelector().fit(s_cal, y_cal)             # nested CV, log-loss criterion
wrapped = CalibratedModel(model, PlattCalibrator(), flow="prefit").fit(X_cal, y_cal)
wrapped.offset_to(target_mean=0.031)                     # auditable central-tendency stage
lo_z, hi_z = wrapped.interval_inverse(0.0, 0.02, space="logit")   # "PD <= 2%" in raw margins
```

## Why probcal

| Capability | probcal | scikit-learn | netcal | probcal (R)² | single-method packages¹ |
|---|---|---|---|---|---|
| Calibration methods | 11 | 2 | many | 5 binary³ | 1 each |
| Runtime dependencies | numpy | scipy stack | torch stack | native R | varies |
| Logit-scale diagnostics (low-PD readable) | yes | — | — | — | — |
| First-class auditable offset (central tendency) | yes | — | — | — | — |
| Automatic selection under nested validation | yes | — | — | — | — |
| Venn–Abers intervals | yes | — | — | — | venn-abers |
| Metric catalog with selection-suitability guidance | yes | partial | partial | partial | — |
| Per-grade regulatory backtests (binomial, Jeffreys) | yes | — | — | — | — |
| Kernel calibration error and test (SKCE, Widmann et al.) | yes | — | — | **yes** | — |
| Calibrated→raw threshold translation (`interval_inverse`) | yes | — | — | — | — |
| SHAP additivity repair on the calibrated scale | yes | — | — | — | — |
| Parameter interpretation (`interpret()`) on every method | yes | — | — | partial | — |

¹ betacal, venn-abers, ml-insights.
² prdm0/probcal (P. R. Diniz Marinho), unaffiliated — see the FAQ. Verified against v0.2.0, 2026-08-08.
³ Platt, temperature, beta, isotonic, histogram binning; its multiclass methods (Dirichlet, vector scaling, one-vs-rest) are out of probcal's binary scope.

Performance note: the PAVA family and special functions are hand-rolled numpy/stdlib;
profiling on real workloads has not shown them to be a bottleneck. Rust acceleration is
deliberately out of scope unless benchmarks on production-sized portfolios say otherwise.

## Documentation

Built with mkdocs-material; run locally with `uv run mkdocs serve`. Start with
*Getting started*, then the *Concepts* chapters — the package's theoretical foundation —
and the executed *PD calibration walkthrough* notebook. The *Visualization* chapter is a
gallery of every plot, regenerated deterministically by `docs/scripts/generate_figures.py`.

## License

MIT. See [LICENSE](LICENSE) and `docs/LICENSING.md` for the conceptual-reference policy on
GPL-licensed R packages.
