Metadata-Version: 2.4
Name: sparpartner
Version: 1.5.6
Summary: Deterministic, benchmark-driven stratified sampler for train/test prep.
Author: Henry
Author-email: Henry <osas2henry@gmail.com>
License: All Rights Reserved
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas>=1.3
Dynamic: author
Dynamic: license-file
Dynamic: requires-python

# sparpartner

A deterministic stratified sampler for train/test prep. It doesn't
split your data randomly, it deliberately finds the rows that most
resemble a benchmark case, so you can hold those out as a genuine
test set and train on everything else.

## Why this exists

A random train/test split assumes the test set should look
statistically like the train set. That's the wrong question if
what you actually want to know is: **does the model generalize past
one specific profile, or did it just memorize the neighborhood
around it?**

`sparpartner` answers that by ranking every row in your data by how
closely it resembles a benchmark ("bench_marks") you define, then
sorting closest-to-benchmark first. You then slice the sorted frame
yourself:

- **Lookalikes as test, the rest as train**: the rows most like the
  benchmark are always at the top of the sorted result, so `head()`
  gives you the test set and `tail()` (or everything past your cut)
  gives you the train set. This is the harder, more honest check: it
  tells you whether the model actually learned something general, or
  only performs well near cases it's already seen a lot of.
- **Just the lookalikes, nothing else**: pass `sparring_n` and skip
  the manual slice entirely. `sample()` hands you back only the top
  `sparring_n` rows, already ranked, plus a report on exactly that
  set (see the sparring report section below).

`sparpartner` only produces the ranking (and, optionally, the
top-N slice). The actual train/test cut beyond that is a plain
slice on your side (see the usage examples below).

## Where the idea comes from

A fighter in camp doesn't spar with whoever's free in the gym, they
specifically look for a sparring partner who moves, reaches, and
hits like the opponent they're about to face. Training against a
random partner tells you nothing about how you'll actually do;
training against someone who resembles the real threat does.
`sparpartner` applies that same logic to a model: instead of a
random holdout, it finds the rows that resemble the toughest, most
relevant "opponent" profile and holds those back as the real test,
so what's left to train on is everything *unlike* that opponent,
and the test genuinely checks whether the model can handle the
match it's actually walking into.

## How the scoring works

You give it:

- `df`: your data. Any column names are fine, including ones
  starting with `spar`; `sample()` never touches or overwrites
  your own columns (see [Validation](#validation)).
- `bench_marks`: a dict of `{column_name: benchmark_value}`, one
  entry per signal you care about
- `custom_weights`: a **dict** of `{column_name: weight}`, saying
  which columns matter and how much. Insertion order is preserved
  and drives both the `show_progress` readout order and, combined
  with descending weight, the tie-break cascade order (see below).

For each weighted column, `sparpartner` auto-detects the column's
type and scores every row's distance to the benchmark on a 0–1
scale (1.0 = exact match, 0.0 = as far as possible):

| Detected type | How distance is measured |
|---|---|
| **numeric** | `abs(value - bench)`, capped by the column's own max observed distance from bench |
| **date** | both sides converted to "age in days" relative to the benchmark date, capped by the column's own max observed age |
| **string** | exact match = 1, anything else = 0 |

Date detection is automatic. A column is only treated as a date if
its values look date-shaped (contain a separator like `-`, `/`, `.`
or a recognizable month name) **and** parse successfully at least
98% of the time. Bare numeric-looking strings (e.g. `"12345"`) never
even reach the date-parsing attempt, and object-dtype columns of
digit strings fall through to the exact-match string path instead of
being mistaken for numbers. Only a real numeric dtype gets the
numeric path.

Each column's 0–1 score is multiplied by its weight and summed into
one raw score per row. That raw sum is divided by the total weight
to get each row's overall match score, **but only if the total
weight is > 0**. In that normal case the match score always lands in
the 0–1 range, however many signals or weights you used. If the
weights sum to `<= 0`, normalization is skipped entirely and the raw,
unnormalized weighted sum is used instead (not guaranteed to fall in
0–1).

All of this (the per-signal 0–1 scores and the per-row overall
match score) is working state `sample()` uses internally to sort,
tie-break, and slice. **None of it is added as columns to the `df`
you get back.** The df you receive always contains only your
original columns, reordered (and sliced, if `sparring_n` is set).
Score info comes back separately, as aggregates in `sparring_report`
, see [The sparring report](#the-sparring-report).

The result is always sorted by match score descending, the row
closest to the benchmark is always first, unless you pass
`best_first=True`, see [Row order (`best_first`)](#row-order-best_first)
below.

### Tie-breaking

Rows that land on the exact same match score aren't left to random or
arbitrary order. Ties are broken by the per-signal score of the
**highest-weight** signal first (higher wins), then the next-highest,
cascading down the weight-sorted signal list until the tie resolves.
Signals that share the same weight are compared in the order they
appear in `custom_weights` (dict insertion order). Only if every
signal is exhausted and rows are still tied does it fall back to
pandas' stable sort (original row order).

## Row order (`best_first`)

By default, the returned `df` is sorted with the closest match to
the benchmark first, same as always. Pass `best_first=True` and, as
the very last step before returning, `sample()` flips that same set
of rows so the worst-of-selection is first and the best-of-selection
is last.

This only changes **presentation order**. It never changes which
rows get selected (e.g. via `sparring_n`), never touches scoring or
tie-breaking, and `sparring_report` is identical either way, since
it's an order-independent aggregate.

It exists to replace a manual post-hoc flip like:

```python
result = result.sort_index(ascending=False).reset_index(drop=True)
```

which is fragile against how `sample()`'s own indexing/reset works.
Use `best_first=True` instead:

```python
result, report = sample(
    df,
    bench_marks=bench_marks,
    custom_weights=custom_weights,
    best_first=True,
)
```

## The sparring report

`sample()` doesn't just return the ranked frame, it returns a
`(df, sparring_report)` tuple. `sparring_report` is a flat dict
summarizing match quality, with one `spar_<name>` key per signal
plus `spar_min_score` / `spar_max_score` / `spar_mean_score`, e.g.:

```python
{
    "spar_age": 55.0, "spar_income": 57.5,   # one key per signal, avg score
    "spar_min_score": 0.0,
    "spar_max_score": 91.93,
    "spar_mean_score": 56.45,
}
```

Every value is on a 0–100 scale (100 = perfect match to benchmark)
and rounded to 2 decimal places. This is the *only* place score
information comes back to you: individual per-row scores are never
returned, only these aggregates.

A signal with `weight=0` is excluded from ranking/sorting entirely
(it can never move the match score or break a tie), but it still
gets its own `spar_<name>` entry in `sparring_report`, so you can
track how close rows are on a signal without letting that signal
influence which rows are considered "closest".

By default (`sparring_n=None`) both the returned `df` and the report
cover **all** rows. Pass an int and `sample()` slices the sorted
result down to just the top `sparring_n` rows (the ones closest to
the benchmark) **before** anything else happens. That slice is what
you get back as `df`, and it's also exactly what `sparring_report`
and the score distribution are computed on. There's no separate
"full set" kept around once `sparring_n` is set; if you need the
rest of the rows too (e.g. to build the train set), take them from
your original `df` yourself, or call `sample()` again with
`sparring_n=None`.

### Progress readout (`show_progress`)

When `show_progress=True`, the sparring report section of the
printed readout marks each signal's average score, and the min /
max / mean of the score distribution, with a traffic-light emoji:

- 🔴 avg score in the bottom third (0–33.3)
- 🟡 avg score in the middle third (33.3–66.7)
- 🟢 avg score in the top third (66.7–100)

```
    age                  avg score= 64.33  🟡
    signup_date          avg score= 38.00  🟡
    country              avg score= 50.00  🟡

  SCORE DISTRIBUTION
  ------------------
    spar_min_score       =   6.67  🔴
    spar_max_score       =  82.00  🟢
    spar_mean_score      =  50.78  🟡
```

This is purely a print-time visual, it doesn't change anything about
`sparring_report`'s actual values.

## Usage

### Sample usage

`df` is the only argument you can pass positionally. Every other
argument, including `bench_marks`, `custom_weights`, and
`best_first`, must be passed by keyword (see
[Keyword-only arguments](#keyword-only-arguments) below).

```python
import pandas as pd
from sparpartner import sample

df = pd.DataFrame({
    "id": [1, 2, 3, 4, 5],
    "age": [25, 30, 47, 52, 33],
    "signup_date": ["2023-01-15", "2023-03-02", "2022-11-20", "2023-01-10", "2023-06-01"],
    "country": ["US", "US", "CA", "US", "MX"],
})

bench_marks = {
    "age": 30,
    "signup_date": "2023-01-01",
    "country": "US",
}

custom_weights = {
    "age": 2,
    "signup_date": 1,
    "country": 1,
}

result, sparring_report = sample(
    df,
    bench_marks=bench_marks,
    custom_weights=custom_weights,
    best_first=False,     # required, no default, False = best match first; True = worst-of-selection first
    sparring_n=None,      # None = every row scored, sorted, and returned
    drop_nan=True,
    show_progress=True,   # prints the full scoring breakdown
)

print(result)
print(sparring_report)
```

`age=30`, `signup_date="2023-01-01"`, and `country="US"` closely
match row `id=1` (age 25, close date, US), so that row lands at or
near the top of the sorted output (or the bottom, if
`best_first=True`).

```python
# --- post-sample: turn the ranking into an actual train/test split ---
# Result is always sorted closest-to-benchmark first (default best_first=False).
ranked, _ = sample(
    df,
    bench_marks=bench_marks,
    custom_weights=custom_weights,
    best_first=False,
)

# Slice however large you want the test set to be, e.g. the top 30%:
cut = int(len(ranked) * 0.3)
test = ranked.iloc[:cut]    # lookalikes, the harder, honest test set
train = ranked.iloc[cut:]   # everything unlike the benchmark

# Or skip the manual slice and get the test set directly:
test_only, report = sample(
    df,
    bench_marks=bench_marks,
    custom_weights=custom_weights,
    best_first=False,
    sparring_n=cut,
)
```

### Parameters

| Name | Type | Default | What it does |
|---|---|---|---|
| `df` | DataFrame | required, positional | Must contain a column for every name (key) in `custom_weights`. No restriction on your own column names, `spar`-prefixed columns are fine |
| `bench_marks` | dict | `None`, but required (raises if left `None`) | `{column_name: benchmark_value}`. Keyword-only |
| `custom_weights` | dict of `{name: weight}` | `None`, but required (raises if left `None`) | Which columns to score and how much each contributes. Keyword-only |
| `best_first` | bool | `None`, but required (raises if left `None`) | Applied last, after everything else (including `sparring_n` slicing). `False` = best match first. `True` = flips that same set of rows to worst-of-selection first, best last. Never changes which rows are selected; see [Row order](#row-order-best_first). Keyword-only |
| `sparring_n` | int or `None` | `None` | `None` = every row scored, sorted, and returned, report covers all of them. An int slices the sorted result down to the top `sparring_n` rows **before** anything else. That slice is what's returned as `df`, and what the report/distribution are computed on. Keyword-only |
| `drop_nan` | bool | `True` | If `True`, drops any row with a NaN in a per-signal working score **or** in the overall match score, after printing (if `show_progress`) a sanity check of what was dropped and why. Keyword-only |
| `show_progress` | bool | `False` | Prints a full readout, in run order: header (including a `signals used` count), per-column type/cap/sample scores, drop_nan check (if `drop_nan=True`), normalize check, sort-apply readout, sparring_n slice readout (if `sparring_n` is set), top N ranked rows with weighted contributions (`N` in the header always matches the number of rows actually shown), and the sparring report itself, with 🔴/🟡/🟢 markers next to each score, flagging any signal contributing zero separation. Keyword-only |

### Returns

`sample()` returns a `(df, sparring_report)` tuple, not just a
DataFrame. The `df` always contains only your original columns
(reordered/sliced), no score columns are ever attached to it. See
[The sparring report](#the-sparring-report) for how score
information comes back to you instead.

### Validation

Input validation runs upfront, before any scoring starts, in two
passes: general parameter checks, then `custom_weights` checks.

Raises `TypeError` if:
- `df` isn't a pandas DataFrame
- `bench_marks` isn't a dict
- `custom_weights` isn't a dict
- `drop_nan`, `show_progress`, or `best_first` isn't a bool
- `sparring_n` isn't an int or `None` (bools are rejected too)

Raises `ValueError` if:
- `bench_marks` is left as `None` (its default)
- `custom_weights` is left as `None` (its default)
- `best_first` is left as `None` (its default)
- `df` has no rows
- `custom_weights` is an empty dict
- `sparring_n` isn't a positive integer
- a `custom_weights` key isn't a string, or doesn't match a column
  in `df`
- a `custom_weights` key has no matching entry in `bench_marks`
- a `custom_weights` key is literally named `"score"`. `sample()`
  keeps its own overall-total working score internally, and a
  signal named `"score"` would generate the exact same internal name,
  corrupting that total instead of just shadowing a per-signal value.
  Rename that column in `df` (and its entries in
  `bench_marks`/`custom_weights`) before calling `sample()`. Names
  that merely *contain* "score", like `test_score` or `score_pct`,
  are unaffected, only an exact match on `"score"` collides
- a weight isn't numeric (bools are rejected too, a `bool` is
  technically an `int` in Python but was never meant as a weight)
- a weight is `NaN`, `inf`, or `-inf`

**Note on duplicate signal names:** since `custom_weights` is now a
dict, keys are inherently unique, so a repeated column name can no
longer be passed in the first place, Python itself resolves a
repeated key in a dict literal (keeping only the last value) before
`sample()` ever sees it. There's nothing left for validation to
catch here.

## A couple of things worth knowing

- **Your own column names are unrestricted**: `sample()` computes
  its working scores under internally-generated names that can't
  collide with anything you'd realistically name a column, and those
  working columns are always dropped before the df is returned. You
  can freely have your own columns named `spar_score`, `spar_age`,
  or anything else, `sample()` won't touch, rename, or overwrite
  them.
- **A signal literally named `"score"`**: still rejected, the one
  case where a *signal name itself* (not a pre-existing df column)
  would collide with `sample()`'s own internal total. See
  [Validation](#validation) above.
- **Object-dtype numeric strings**: a column of strings like
  `"100"`, `"200"` (object dtype, no separator) is scored as an
  exact-match string column, *not* auto-converted to numeric. Only
  genuine numeric dtypes (`int`, `float`) get the numeric distance
  path.
- **Keyword-only arguments**: `df` is the only argument `sample()`
  accepts positionally. Every other argument, `bench_marks`,
  `custom_weights`, `best_first`, `sparring_n`, `drop_nan`, and
  `show_progress`, must be passed by name. This is enforced by
  Python itself: a positional call like `sample(df, weights,
  benchmarks)` fails immediately with a `TypeError`, before any of
  `sample()`'s own code runs. It exists specifically to rule out
  accidentally swapping `bench_marks` and `custom_weights`, which are
  both dicts and can't be told apart by type alone. `bench_marks`,
  `custom_weights`, and `best_first` additionally default to `None`
  but are not actually optional, leaving any of them out (or passing
  `None` explicitly) raises a `ValueError` naming exactly which one
  is missing.
