Metadata-Version: 2.4
Name: numeric-rule-finder
Version: 0.2.0
Summary: Discover the conservation/balance rules a set of numbers must satisfy, find where they break, and honest-stop when there is no such structure.
Author-email: Edward Chalk <edward.chalk@sapientronic.ai>
License: numeric-rule-finder
        
        Copyright (c) 2026 Edward Chalk (sapientronic.ai)
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to use,
        copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
        the Software, and to permit persons to whom the Software is furnished to do
        so, subject to the following conditions:
        
        1. The above copyright notice and this permission notice shall be included
           in all copies or substantial portions of the Software.
        
        2. Attribution. Any publication, presentation, derivative work, or product
           that uses or builds on this Software must include visible attribution to
           Edward Chalk and sapientronic.ai. The phrase "Built with numeric-rule-finder by
           Edward Chalk (sapientronic.ai)" or equivalent is acceptable.
        
        3. The Software is provided "AS IS", without warranty of any kind, express
           or implied, including but not limited to the warranties of merchantability,
           fitness for a particular purpose, and noninfringement. In no event shall
           the authors or copyright holders be liable for any claim, damages, or
           other liability, whether in an action of contract, tort, or otherwise,
           arising from, out of, or in connection with the Software or the use or
           other dealings in the Software.
        
        This license is modeled on the MIT License with an explicit attribution
        clause (clause 2).
        
Project-URL: Homepage, https://github.com/pcoz/numeric-rule-finder
Project-URL: Repository, https://github.com/pcoz/numeric-rule-finder
Project-URL: Documentation, https://github.com/pcoz/numeric-rule-finder/blob/main/ebook/index.md
Project-URL: Issues, https://github.com/pcoz/numeric-rule-finder/issues
Keywords: reconciliation,conservation-laws,anomaly-detection,invariants,ledger,balance,audit,data-quality
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Financial and Insurance Industry
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Office/Business :: Financial :: Accounting
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: petri
Requires-Dist: petra-nn; extra == "petri"
Provides-Extra: ml
Requires-Dist: scikit-learn; extra == "ml"
Requires-Dist: numpy; extra == "ml"
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: scikit-learn; extra == "dev"
Requires-Dist: numpy; extra == "dev"
Dynamic: license-file

# Numeric Rule Finder

An exact, dependency-light library that **discovers** the conservation laws latent
in structured movement/transaction data, instead of making you declare one.

You declare nothing. It recovers the **complete lattice of independent
conservation laws the data actually obeys** — including laws nobody wrote down —
finds where they break, types each break (a re-attributable slip vs. a genuine
hole), and **honest-stops** when there is no structure to exploit.

> 📖 **There's a book.** [**Read the ebook →**](https://github.com/pcoz/numeric-rule-finder/blob/main/ebook/index.md) — what it
> is, how it works, runnable worked examples (with real output), three
> machine-learning head-to-heads, a raw-XML showcase, a tour across number
> systems, and a mathematics appendix.

## Two front doors

**Just want answers? (no maths).**

* **See it in action** — the headline walkthrough, *Closing the month at Northwind
  Retail*, in
  [`BUSINESS_GUIDE.md`](https://github.com/pcoz/numeric-rule-finder/blob/main/BUSINESS_GUIDE.md#headline-walkthrough--closing-the-month-at-northwind-retail):
  six plain-English checks, each ending in an action (run it:
  [`examples/northwind_close/`](https://github.com/pcoz/numeric-rule-finder/tree/main/examples/northwind_close/)).
* **On your own data** — `python -m numeric_rule_finder.cli` (or
  `from numeric_rule_finder import Reconciler`); hand it a CSV and two column names.
* **What you get** — what balances, where it breaks and probably why, and any
  hidden *separate sub-systems* — found exactly in one pass (`independent_groups`,
  or the `groups` command).
* **It's intelligent** — when ordinary balancing finds nothing, it silently
  escalates to the deeper maths (e.g. modular/parity structure) and reports it in
  plain words.
* **Across domains** — accounting, energy, supply chain, ETL, clinical trials,
  elections, …: `python examples/gamut/gamut_demo.py`.

**Want the mathematics?** — read the ebook
**[Appendix — The mathematics](https://github.com/pcoz/numeric-rule-finder/blob/main/ebook/chapters/A_appendix_mathematics.md)**. The
engine lives in the `numeric_rule_finder/` package.

## How it works (in brief)

Numeric Rule Finder treats your data as signed **movements** — a *group*, a
*bucket*, an *amount* — and finds the weighted combinations of buckets that every movement leaves
unchanged (the conservation laws), then measures exactly where they break. It
climbs only as far as it needs:

* **balance & structure** — which groups don't net out; which buckets form hidden
  *separate books* (the exact independent groups, in one linear pass);
* **modular laws** — patterns invisible to ordinary arithmetic ("moves only in
  cases of 12", parity), via Smith Normal Form;
* **typed residuals** — is a break re-attributable (a *coboundary*) or a genuine
  hole (an *obstruction* that names the violated law)?
* **multi-source consistency** — whether independent reconciliations can all be
  true at once (an `H¹` obstruction);
* **substrate generality** — the same engine over ℤ, ℚ, 𝔽ₚ, and ℚ[t]
  (parametric, rate-dependent laws).

Everything is exact (integer/rational arithmetic, never floating point) — and it
scales: a modular-rank fast path certifies the *honest stop* (no conservation
structure) in machine-integer arithmetic, skipping the exact rational solve where
there is nothing to find. **The actual mathematics** — definitions, theorems, the
`coker(S)` / `H¹` residual typing, Smith Normal Form, and a map of the code — is
in the ebook
**[Appendix — The mathematics](https://github.com/pcoz/numeric-rule-finder/blob/main/ebook/chapters/A_appendix_mathematics.md)**.

## Real data carries more than one law

Point it at a dataset and it reports how many *independent* conservation laws
hold. Real data routinely has several — separate books, conserved moieties,
per-SKU stock — not just the one obvious balance:

| the data | laws found | what that means |
|---|---|---|
| a clean double-entry ledger | **1** | every transaction nets to zero |
| two ledgers that never share a transaction | **2** | they are really **separate books** — nobody had declared that |
| a stock network with two products | **2** | each product's stock is conserved on its own |
| an enzyme reaction network | **2** | two conserved quantities (the enzyme, and total substrate) |
| a shared-resource / mutex process | **3** | each client's work-item **plus** the resource invariant |
| data with no balancing structure | **0** | **honest stop** — it refuses to invent a reconciliation |

## With and against machine learning

Where the signal is an exact law this beats statistical anomaly detection; where
the job is partly fuzzy, it makes the model's job easier; and where the answer is
an exact structure, it replaces a slow unsupervised job outright. Three
reproducible head-to-heads (all walked through in Chapter 7 of the [ebook](https://github.com/pcoz/numeric-rule-finder/blob/main/ebook/index.md)):

* it beats an Isolation Forest outright on skim fraud —
  [`examples/fraud_vs_ml/`](https://github.com/pcoz/numeric-rule-finder/tree/main/examples/fraud_vs_ml/);
* it *feeds* a Random Forest as an exact pre-filter + feature, lifting its score —
  [`examples/ml_assist/`](https://github.com/pcoz/numeric-rule-finder/tree/main/examples/ml_assist/);
* it finds the exact groups in one pass — `Reconciler.independent_groups` — where a
  clustering `k`-sweep is thousands of times slower *and* wrong —
  [`examples/grouping_vs_ml/`](https://github.com/pcoz/numeric-rule-finder/tree/main/examples/grouping_vs_ml/).

## Measurements, not just ledgers — uncertainty-aware testing

A ledger balance either closes to the penny or it does not, and a single scalar
`tolerance` is the right test for it. **Measurements are different**, and until
`0.2.0` this library had no way to say so:

* a residual of 5 is a catastrophe if sigma is 0.1 and noise if sigma is 50;
* a law with coefficients `(17, -17)` should not face the same absolute cut as one
  with `(1, -1)` — both its residual and its sigma scale by 17;
* and "residual consistent with zero" hides two different situations that must not
  be conflated: the law was **tested and holds**, or the data **could never have
  tested it**.

`check_conservation_noisy` judges each law against **its own propagated sigma**:

```python
from numeric_rule_finder import discover_invariants, check_conservation_noisy

d = discover_invariants(rows, entity_key="account", event_key="txn", qty_key="delta")

report = check_conservation_noisy(
    measurements, d.laws,
    entity_key="account", event_key="txn", qty_key="value",
    sigma_key="value_err",      # per-record uncertainty
    z_break=3.0,                # |z| above which a law is judged broken
    resolution=0.5,             # the residual size that would have mattered
)
print(report.summary())
print(report.chi2_per_constraint)
```

**Three verdicts, not two.** `holds`, `BREAKS`, and — the one that matters —
`untestable`: the law exists, but sigma is larger than the residual size you
declared would matter, so no clean bill of health is issued.
`report.statistically_mute` says *"the structure is there and this data cannot
speak to it"*, which is a different finding from `Discovery.honest_stop`
(*"there is no structure"*) and worth saying out loud rather than hiding behind a
passing tolerance check.

**A basis-independent joint statistic.** `chi2 = r.T C^-1 r` with
`C = Y Sigma Y.T`. A naive sum of per-law `z^2` depends on which basis of the
conservation space happened to be returned; this does not. `chi2_per_constraint`
is then the natural "how badly does the structure fail, per independent
constraint" number.

**Correlated uncertainties** are supported via `cov=`, because correlation is the
normal case whenever a shared quantity was estimated from the same data the laws
are tested on.

**Discovery stays exact.** Which laws exist depends on the incidence pattern, not
on measurement precision, so `discover_invariants` keeps its exact
Fraction/integer arithmetic and gains no noise model. The statistical layer sits
strictly on top. Supply no uncertainties and you get the previous exact behaviour
back unchanged.

### Worked example: loop closure across overlapping instruments

Conservation laws are not only ledger balances. Given several vantages that
measure the *same* quantity with unknown per-vantage offsets, and overlapping
coverage, the offsets must sum to zero around every cycle in the overlap graph —
a **loop-closure** condition, testable with no reference truth. Map it by taking
**entities = overlaps** and **events = vantages**, and the discovered laws *are*
the independent loops:

```python
recs = []
for (a, b), (delta, sigma) in pairwise_measurements.items():
    recs.append(dict(edge=f"{a}|{b}", node=a, inc=1))
    recs.append(dict(edge=f"{a}|{b}", node=b, inc=-1))

d = discover_invariants(recs, entity_key="edge", event_key="node", qty_key="inc")
# d.n_laws == the number of independent loops == the closure degrees of freedom
```

Feeding the measured pairwise differences and their sigmas to
`check_conservation_noisy` then reproduces the weighted-least-squares closure
chi-squared exactly, and additionally **names which loops break**. Validated
against eight overlapping astronomical instruments (JWST NIRISS/NIRCam/NIRSpec,
HST/STIS+WFC3, VLT/FORS2, Spitzer/IRAC): 5 independent laws recovered, joint
chi-squared agreeing with a hand-rolled solve to 1e-9, and the two breaking loops
both localised to the same triangle of instruments.

## Run

`numeric_rule_finder` has **no third-party dependencies**; `petra_adapter` uses
`petra-nn` *only* to read Petri nets, and the ML examples use `scikit-learn`.

```bash
pip install -e .                                      # optional (pure-stdlib core)
python -m numeric_rule_finder.cli check  data.csv --group txn_id --amount amount
python -m numeric_rule_finder.cli groups data.csv --group txn_id --account account  # exact independent groups
python examples/northwind_close/close_the_books.py    # a worked example
python examples/gamut/gamut_demo.py                   # the same engine across domains
python -m pytest tests examples -q                    # the suite
```

## License

MIT-with-attribution — see [`LICENSE`](https://github.com/pcoz/numeric-rule-finder/blob/main/LICENSE).
