Metadata-Version: 2.5
Name: reliax-core
Version: 0.2.1
Summary: Method components of the Reliax reliability envelope: split and Mondrian conformal sets, Venn-Abers brackets, a conformal test martingale, the credibility p-value, the fast-loop evaluator with route trace and certificate reasons, kNN distance, PSI, an error auditor, subjective-logic fusion and calibration opinions. Pure functions, no I/O.
Project-URL: Homepage, https://reliax.io
Project-URL: Source, https://github.com/reliax-io/reliax-core
Project-URL: Issues, https://github.com/reliax-io/reliax-core/issues
Project-URL: Changelog, https://github.com/reliax-io/reliax-core/blob/main/CHANGELOG.md
Project-URL: Evaluation, https://github.com/reliax-io/reliax-evaluation
Author: Reliax
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: calibration,conformal-prediction,credit-risk,drift-detection,model-risk,test-martingale,uncertainty-quantification,venn-abers
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Financial and Insurance Industry
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: numpy>=1.26
Requires-Dist: scikit-learn>=1.4
Provides-Extra: test
Requires-Dist: pytest>=8; extra == 'test'
Description-Content-Type: text/markdown

# reliax-core

Method components of the Reliax reliability envelope, as a plain Python
package. Pure, stateless functions and small classes: no I/O, no
configuration, no network. Apache-2.0.

This is the code behind every number in the Reliax whitepaper. The evaluation
that produces those numbers, with the datasets, the runners, the
pre-registration and the negative results, is
[reliax-evaluation](https://github.com/reliax-io/reliax-evaluation), which
pins this package.

## Install

```
pip install reliax-core
```

Or from a tagged release on GitHub:
`pip install "reliax-core @ git+https://github.com/reliax-io/reliax-core.git@v0.2.0"`.

The import name is `reliax_core`. Python 3.10 or later; numpy and
scikit-learn are the only dependencies.

## What is in it

| Module | What it computes | Class of output |
|---|---|---|
| `conformal.py` | `ConformalCalibrator`: split conformal prediction set from a calibration split. `P(Y in C(X)) >= 1 - alpha` marginally on exchangeable data (Vovk, Gammerman and Shafer, 2005). | guarantee |
| `fairness.py` | `MondrianConformal`: the same coverage per declared segment, with the segment attribute used for calibration only, never shown to the model. `coverage_audit` compares marginal and per-segment coverage. | guarantee |
| `venn_abers.py` | `VennAbersCalibrator`: inductive Venn-Abers bracket `[p0, p1]` around the base model's stated probability (Vovk and Petej, 2014). One of the two isotonic predictors is calibrated; the width of the bracket is itself a signal. | guarantee |
| `martingale.py` | `ConformalMartingale`: conformal test martingale on a label-free nonconformity score, a mixture over betting exponents; anytime-valid, WATCH at 20, ALARM at 100 (Vovk, Nouretdinov and Gammerman, 2003). | guarantee |
| `credibility.py` | `CredibilityReference`, `credibility_p_value`: the label-free conformal p-value of an input's kNN distance against the calibration distances, `(#{a_i >= a} + 1) / (n + 1)`. Valid under exchangeability; answers "does the guarantee cover this input?". Deterministic, so it replays. | guarantee |
| `evaluator.py` | `Policy`, `Envelope`, `evaluate`: the fast-loop evaluator, six rows in a fixed order, first match wins, on certified quantities only; returns the route, the row, the route trace, the reason codes and the certificate reasons in words. | exact |
| `ood.py` | `KNNOODDetector`: distance to the k nearest calibration rows in standardised feature space, as a percentile of the calibration set's own leave-one-out distances. | signal |
| `auditor.py` | `ErrorAuditor`: a second model, trained on the calibration split, that predicts when the base model is wrong. | signal |
| `drift.py` | `PSIMonitor`: population stability index per feature and on the score, over a rolling window. | signal |
| `scoring.py` | `reliability_score`: the 0 to 100 reliability composite. The product shows it as the criticality score, 100 minus this value, so higher means a person should look sooner. | signal |
| `sl_fusion.py` | `fuse_signals`, `averaging_fusion`: subjective-logic fusion of the signals into (belief, disbelief, uncertainty). | signal |
| `fairness.py` | `ImpactMonitor`: rolling ALLOW rate per segment and the four-fifths ratio. | signal |
| `calibration_trust.py` | `CalibrationTrust`: calibration opinion per (segment x score-bin) cell and fused per segment. A closed form of the binned calibration error and the sample size, with a Hoeffding envelope on the disbelief. Feeds the slow loop; never a route. | signal |

A guarantee is a theorem that holds on exchangeable data at the stated level,
with no assumption on the model or on the data distribution. An exact output
is recomputed bit for bit from its inputs, so a verifier can replay it from
the record. A signal is a measurement without such a proof. Routing rules should read guarantees;
signals order the review queue and inform the slow loop. The theory behind
the calibration opinion, with its finite-sample propositions, is in
[CALIBRATION_TRUST.md](https://github.com/reliax-io/reliax-evaluation/blob/main/CALIBRATION_TRUST.md)
in the evaluation repository.

## What the guarantees say, and what they do not

- Coverage is a property of the procedure over exchangeable data, not a
  probability about any one prediction. Distribution-free per-instance
  conditional coverage is not attainable (Barber, Candès, Ramdas and
  Tibshirani, 2021), and this package does not claim it.
- Per-segment coverage holds for each declared segment that has its own
  calibration rows.
- The martingale's false-alarm bound holds when calibration and production
  rows are exchangeable and the nonconformity score is computed the same way
  on both. The evaluation measured it breaking on sparse one-hot inputs with
  a small calibration set (TableShift hospital readmission), so as shipped the
  tripwire is valid on dense numeric inputs and on sparse inputs with a large
  calibration set. A distance that is exchangeable by construction on sparse
  inputs is pre-registered work, not in this release.
- Nothing here is a probability that a given decision is right, and no output
  of this package should be shown as a percentage next to a decision.

## Determinism

Every component is deterministic given its inputs and a seed.
`ConformalMartingale(seed=...)` draws its smoothing uniforms from a seeded
generator and `ErrorAuditor(seed=...)` seeds its gradient-boosting fit. The
credibility p-value and the evaluator use no randomness at all.
Nothing reads the clock, the environment or a file, so a result can be
recomputed later from the same inputs.

## Evidence

Every number in the whitepaper and the deck is produced by a runner in
[reliax-evaluation](https://github.com/reliax-io/reliax-evaluation) that
imports this package, and every results file is committed there, including
the negative ones: under a severe distribution shift the fused score ranks
worse than referring cases at random, and the earlier 2x bar was withdrawn
once that was measured.

## The routing rule

`evaluate(policy, envelope)` reads six rows in this order and stops at the
first that fires: the guarantee is not active on this segment (ALARM, model or
cohort mismatch) → the policy's `invalid_action`, BLOCK by default; the
credibility p-value is below the policy floor → `ood_action`, REVIEW by
default, or below the extreme floor → BLOCK; the prediction set is not a
singleton → REVIEW; the bracket is on, the prediction is on the approve side
and its upper end is above the policy ceiling → REVIEW; WATCH on the stream →
REVIEW if the policy says so; otherwise ALLOW. The result carries the row,
a trace of every row read, the reason codes and the certificate reasons in
words. BLOCK never means decline: a person decides. The criticality score
orders the REVIEW queue and is never a condition.

## Not in this package

The Reliax platform, which holds all state: the calibration builder, the
reliability engine (drift state per segment, outcome verdicts, recalibration
tickets, signed policy versions), review queues, the audit service, the
readable view and the dashboard. The certificate format, the hash chain and
the verifier are [`reliax-certificate`](https://github.com/reliax-io/reliax-certificate);
the client is [`reliax-sdk`](https://github.com/reliax-io/reliax-python).
Not yet written, and said so: differentially private noise on the sorted
calibration scores (coverage is proven for sets; the p-values are roadmap), and
the surrogate scorer for replayable reason codes.

## Where this code comes from

Until 25 September 2026 this code was the `reliax_core/` directory of
reliax-evaluation, and its history is kept here. The pre-registration in that
repository names commits of that repository as the frozen method code; those
commits remain there. Version 0.1.0 is the code as it stood on 25 September
2026, with no numerical change since the freeze of 13 September 2026. The one
change since is wording in the certificate line written by
`CalibrationTrust.certificate_line` (24 September 2026). Version 0.2.0 adds
the credibility p-value and the evaluator and changes nothing in the frozen
components.

## Development

```
git clone https://github.com/reliax-io/reliax-core.git && cd reliax-core
python -m venv .venv && .venv/bin/pip install -e ".[test]"
.venv/bin/python -m pytest
```

A change to a threshold, a score or a guarantee goes through an issue first:
see [CONTRIBUTING.md](https://github.com/reliax-io/.github/blob/main/CONTRIBUTING.md).
A CLA is required before a first merge.

## Licence

Apache-2.0, with its patent grant. See [LICENSE](LICENSE).

## About this documentation

The documentation in this repository was written with the help of AI and
reviewed by the Reliax team.
