Metadata-Version: 2.5
Name: quantconformal
Version: 0.3.0
Summary: Exact finite-sample conformal prediction: correct rank arithmetic, explicit eligibility, no silent saturation
Author: Jules Verdez
License-Expression: MIT
License-File: LICENSE
Keywords: conformal-prediction,finance,prediction-intervals,uncertainty-quantification
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Mathematics
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: numpy>=1.24
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff>=0.6; extra == 'dev'
Provides-Extra: differential
Requires-Dist: crepes==0.9.1; extra == 'differential'
Requires-Dist: mapie; extra == 'differential'
Requires-Dist: scikit-learn; extra == 'differential'
Description-Content-Type: text/markdown

# quantconformal

Exact finite-sample conformal prediction. Correct rank arithmetic, explicit
eligibility, no silent saturation.

Split conformal prediction turns on a single integer: the rank of the
calibration score used as the threshold.

```
k = ceil((n + 1) * (1 - alpha))
```

Every library knows that formula. Three of the four audited while writing this
package still select the wrong order statistic somewhere in an ordinary
parameter range — each by a different arithmetic route, and none of the failures
visible without an implementation-independent reference to compare against.

## Install

```bash
pip install quantconformal
```

## Use

```python
import numpy as np
from quantconformal import calibrate

result = calibrate(scores, alpha=0.10)
result.threshold   # the selected order statistic
result.rank        # 9 — the exact one-based rank
result.eligible    # True — a finite guarantee exists
```

When the calibration set is too small for the coverage you asked for, the
default is to refuse:

```python
>>> calibrate(np.arange(10.0), alpha=0.05)
InsufficientCalibration: 10 calibration scores cannot deliver 0.95 coverage:
the required rank is 11, which exceeds 10. Supply at least 10 scores, or raise
alpha to at least 0.0909091.
```

Adaptive methods that revisit their window each step can opt into continuing
instead:

```python
result = calibrate(scores, alpha=0.05, unavailable="infinite", warn=True)
if not result.eligible:
    ...  # widen, defer, or collect more data
```

Build an interval from a calibrated threshold:

```python
from quantconformal import absolute_residual_score, calibrate, split_conformal_interval

cal = calibrate(absolute_residual_score(y_cal, model.predict(X_cal)), alpha=0.10)
interval = split_conformal_interval(model.predict(X_test), cal)

interval.lower, interval.upper
interval.eligible                      # is the guarantee real?
interval.empirical_coverage(y_test)    # descriptive
```

Or conformalize a predicted quantile band (CQR):

```python
from quantconformal import cqr_score, cqr_interval

cal = calibrate(cqr_score(y_cal, lo_cal, hi_cal), alpha=0.10)
interval = cqr_interval(lo_test, hi_test, cal)
```

Negative corrections are kept, not clipped — that is how CQR *tightens* an
over-wide band. And if the quantile model returns a lower bound above its upper
bound, that is a misfit, so by default it refuses:

```python
>>> cqr_interval([1.0], [0.0], cal)
QuantileCrossing: predicted quantiles cross on 1 of 1 rows (worst inversion 1).
The quantile model is misfitted; conformalising a crossed band cannot repair
it. Refit with non-crossing quantiles, or pass crossing='repair' to collapse
crossed rows to their midpoint with a warning.
```

Check before you commit:

```python
>>> from quantconformal import is_eligible, minimum_calibration_size
>>> is_eligible(10, 0.05)
False
>>> minimum_calibration_size(0.05)
19
```

## What goes wrong elsewhere

Each of these was observed by executing the library in an isolated, hash-pinned
environment against an implementation-independent oracle. Each is pinned as a
regression test in `tests/test_known_library_defects.py`.

### Double rounding through a quantile routine

Convert the rank to a probability, hand the probability to `np.quantile`, and
it rounds a second time. The two roundings do not compose.

MAPIE 1.4.1 and 1.5.0 form `q = (1 - alpha)(1 + 1/n)` and call
`np.quantile(scores, q, method="higher")`. In six cells of a 34-cell grid this
selects the *following* order statistic — at `n=137, alpha=0.50` it takes rank
70 where the exact rank is 69.

### Float round-tripping through a confidence level

Accept `confidence`, recover `alpha = 1 - confidence`, and discover the
recovered value is not the one the caller meant. `1 - (1 - 0.1)` is
`0.09999999999999998`.

crepes 0.9.1 computes `int((1 - confidence) * (n + 1)) - 1`. At `n=9,
alpha=0.10` — exactly on the eligibility boundary — the truncation drops to
`-1` and crepes refuses a cell that is eligible with rank 9. At `n=9,
alpha=0.20` it returns rank 9 where the exact rank is 8.

The same algebra at `n=19, alpha=0.05` is *fine*, because `1 - 0.95` lands just
above the integer rather than just below. Conditional on the last bit, which is
what makes this class of defect hard to notice.

### Saturating to the largest observed score

When the required rank exceeds `n`, no observed score implements the requested
guarantee. Returning `max(scores)` reports a finite threshold for a guarantee
nothing backs — a silent claim rather than a loud refusal. This one was in
*this* project's own history before it was found and repaired.

### Silently repairing a crossed quantile band

If a quantile model returns a lower quantile *above* its upper quantile, the
model is misfitted and no amount of conformalising fixes it.

PUNCC 0.9.3, given raw quantiles of `1.0` and `0.0`, returned an interval of
`[0.5, 0.5]` — zero width, still labelled at the nominal 80% coverage, with no
exception and no warning. A degenerate interval reported as a calibrated one.

Here it raises by default, at calibration *and* at prediction, because a model
can behave on calibration rows and cross on test rows. `crossing="repair"`
collapses crossed rows to their midpoint and warns, if that is what you want.

### Absorbing corrupted calibration scores

A single `NaN` or infinity among the calibration scores is a corrupted input.
Audited libraries variously returned a `NaN` threshold, or silently absorbed an
infinite score and returned an ordinary-looking finite one. The second is worse:
nothing in the output distinguishes it from a valid result.

## What this package does instead

- **The rank is computed once, in exact rational arithmetic**, from the value
  the caller actually passed. No probability is reconstructed, so there is no
  second rounding to disagree with the first.
- **The threshold is selected, never interpolated** — `np.partition` at the
  exact index, which is `O(n)` and does not mutate the input.
- **`alpha` may be stated exactly.** `calibrate(scores, "1/10")` is exactly one
  tenth; the float `0.1` is the binary double nearest to it. They differ, and at
  the eligibility boundary that difference can move the rank by one. Both are
  handled correctly, as written.
- **The ineligible domain is an explicit state**, surfaced as an exception, a
  warning, or `ConformalCalibration.eligible`. Gate on that flag rather than
  testing a bound for infinity — a caller that tests the bound is coupled to how
  an absent guarantee happens to be represented.
- **Non-finite calibration scores are rejected**, with a count of how many.
- **Crossed quantile bands are refused**, at calibration and at prediction.

## Scope

This package is deliberately small. It does the part that the audited libraries
get wrong, and it does not reimplement the parts they get right.

It does **not** provide regressors, classifiers, conformal predictive systems,
Mondrian binning, or cross-conformal methods. If you need those, MAPIE, PUNCC
and crepes are all reasonable, and PUNCC 0.9.3 was the one implementation in the
audit whose exact-rank arithmetic was correct in every cell tested.

Use this where the rank has to be right.

## Verify the claims yourself

Every divergence above ships as a test. The default suite reproduces the other
libraries' published arithmetic inline, so it runs without pulling in four
dependencies:

```bash
pytest                       # 262 tests, no third-party libraries needed
```

To execute the real packages and confirm the divergence on your own machine:

```bash
pip install "quantconformal[differential]"
pytest tests/test_live_differential.py -v
```

That suite installs crepes and MAPIE, runs them over the disputed cells, and
asserts both that they diverge and that `quantconformal` agrees with an
independently written pure-Python oracle. If an upstream repairs its arithmetic,
the corresponding test starts failing — which is the correct signal, and the
test should then be retired with a note rather than adjusted.

## Provenance

The audit that motivated this package ran each library in an isolated,
network-free environment against a pure-Python oracle that sorts explicitly and
indexes `k - 1`, over a frozen grid of calibration sizes and miscoverage rates,
with two byte-identical runs required per result.

Versions audited: MAPIE 1.4.1 (1.5.0 inspected), PUNCC 0.9.3, crepes 0.9.1.

## Licence

MIT.
