Metadata-Version: 2.4
Name: dataproof
Version: 0.1.0
Summary: Find out what your dataframe code actually did — silent row loss, join fan-out, misleading statistics
Author: oluwagbemilesi
License: MIT
Keywords: pandas,data-quality,verification,observability,AI,agents
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas<4.0.0,>=2.2.0
Provides-Extra: dev
Requires-Dist: pytest>=8.0.0; extra == "dev"
Dynamic: license-file

# dataproof

Find out what your dataframe code actually did.

```python
import pandas as pd, dataproof

with dataproof.observe() as report:
    orders = pd.read_csv("orders.csv")
    clean  = orders.dropna()
    joined = clean.merge(tiers, on="customer_id")
    gold   = (joined.tier == "gold").sum()

print(f"gold customers: {gold}")
print(report)
```

```
gold customers: 0
dataproof: 1 finding(s)
  - dropna() removed ALL 122 rows — a row was dropped if any value in it was
    null. Pass how='all', a subset=[...] of columns that matter, or thresh=N
    to keep partly-populated rows.  [analysis.py:5]
```

The answer was zero. The real answer was 33. One column in that file was
entirely empty, so every row contained a null, so `dropna()` removed all of
them — and pandas said nothing, because nothing went wrong as far as pandas is
concerned.

That is the problem this solves. Not crashes: crashes are easy. **Plausible
answers that are quietly wrong.**

## What it watches for

| | |
|---|---|
| **Silent row loss** | `dropna()` that empties a frame, deduplication that removes most of it |
| **Join fan-out** | a repeated key multiplying rows instead of pairing them |
| **Joins that matched nothing** | usually a type mismatch — `1` and `"1"` do not join |
| **Unmatched rows in a left join** | row count unchanged, but a third of the result is null on one side |
| **Numbers stored as text** | statistics silently computed over nothing |
| **Derived correlations** | `r = 1.000` because one column was computed from the other |
| **Sparse columns** | a confident mean resting on 4 of 60 values |

## What it does not do

**It does not change your results.** Every wrapped call returns exactly what
pandas returned. The code inside the block is ordinary pandas.

**It does not ask you to write differently.** No new API to learn, no wrapper
types, no rewriting a pipeline to use someone else's functions. That is
deliberate — a verification tool you have to remember to call is one you will
not call.

**It does not cry wolf.** Every threshold here exists because a noisier version
was tried and rejected. A group-by is not a 96% row loss. A few points beyond
the IQR fence are not outliers worth reporting unless they actually move the
mean. **A finding that fires on correct code teaches you to skip all of them**,
which is worse than having none.

## Install

```bash
pip install dataproof
```

Pandas is the only dependency.

## Also useful

Check any frame on demand, without a block:

```python
dataproof.Report().check(df)
```

Use it as an assertion in a pipeline or a notebook:

```python
with dataproof.observe() as report:
    ...
assert report.ok, str(report)
```

## Why this exists

It came out of building a data library for AI agents, and measuring what agents
actually do. Given six purpose-built tools and a free choice, an agent used none
of them and wrote pandas — correctly. The tools were not the problem. The
problem was that neither the agent nor the person reading its answer could tell
when an operation had quietly destroyed the data.

Agents now write a great deal of dataframe code, at a speed no one reviews line
by line. They are good at it. But `dropna()` behaves the same for everyone, and
the failure is silent for everyone.

MIT licensed.
