Metadata-Version: 2.4
Name: mclovin
Version: 1.2.0
Summary: Trustworthy features from messy tabular data: schema resolution, provenance tiers, small-sample correction and leakage-safe panel features.
Author: SidharthJoly
License-Expression: MIT
Keywords: feature-engineering,data-quality,panel-data,time-series,empirical-bayes,shrinkage,provenance,schema-matching,sports-analytics,cryptocurrency
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas>=1.5
Requires-Dist: numpy>=1.23
Requires-Dist: scipy>=1.9
Provides-Extra: survival
Requires-Dist: lifelines>=0.27; extra == "survival"
Provides-Extra: models
Requires-Dist: scikit-learn>=1.0; extra == "models"
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0; extra == "dev"
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"
Requires-Dist: mypy>=1.0; extra == "dev"
Requires-Dist: pandas-stubs; extra == "dev"
Requires-Dist: scipy-stubs; python_version >= "3.10" and extra == "dev"
Requires-Dist: lifelines>=0.27; extra == "dev"
Requires-Dist: scikit-learn>=1.0; extra == "dev"
Provides-Extra: docs
Requires-Dist: sphinx>=7.0; extra == "docs"
Requires-Dist: furo; extra == "docs"
Requires-Dist: myst-parser>=2.0; extra == "docs"
Requires-Dist: sphinx-autodoc-typehints; extra == "docs"
Dynamic: license-file

# mclovin

[![codecov](https://codecov.io/gh/SidharthJoly/36120-26SP-group10-25664929-package/branch/main/graph/badge.svg?token=MPNVUWTTKY)](https://codecov.io/gh/SidharthJoly/36120-26SP-group10-25664929-package)
[![PyPI](https://img.shields.io/pypi/v/mclovin.svg?cacheSeconds=3600)](https://pypi.org/project/mclovin/)
[![TestPyPI](https://img.shields.io/badge/TestPyPI-mclovin-orange.svg)](https://test.pypi.org/project/mclovin/)
[![Downloads](https://img.shields.io/pypi/dm/mclovin.svg)](https://pypi.org/project/mclovin/)
[![Python versions](https://img.shields.io/pypi/pyversions/mclovin.svg?cacheSeconds=3600)](https://pypi.org/project/mclovin/)
[![License](https://img.shields.io/pypi/l/mclovin.svg)](https://opensource.org/license/mit)
[![Checked with mypy](https://www.mypy-lang.org/static/mypy_badge.svg)](https://mypy-lang.org/)

Trustworthy features from messy tabular data: schema resolution, provenance
tiers, small-sample correction and leakage-safe panel features.

It began as basketball code. What survived the move to a general-purpose
library is the part that was never really about basketball. The basketball
metrics are still here, in `mclovin.basketball`, as one domain pack among
others -- and the previous release is still on PyPI as
[moneyBBall 0.6.2](https://pypi.org/project/moneyBBall/).

---

## Design principles

**1. Schema-flexible.** Functions resolve canonical names (`PTS`, `STL_PCT`,
`TEMP_MAX`, …) against whatever your DataFrame actually calls them, using a
`Vocabulary` you point the resolver at. The resolver itself holds no domain
knowledge, which is what lets the same machinery work on a box score and a
weather archive without modification.

**2. Honest about provenance.** Many derived quantities are *defined* in terms
of context a given dataset may not carry -- team totals, station baselines,
cohort aggregates. Rather than quietly approximating, every metric declares
its inputs and reports a reliability tier
(`EXACT`, `APPROXIMATED`, `PASSTHROUGH`, `DERIVED`, `UNAVAILABLE`), so you
know which numbers can bear weight.

**3. Leakage-resistant by default.** Rolling windows require a full window,
horizon joins match on the key rather than on position, and splits are
chronological. The defaults are the careful choice, not the convenient one.

**4. Research-cited.** Domain formulas carry their sources: Oliver's
*Basketball on Paper*, Kubatko et al. (2007) *JQAS*, Myers' BPM 2.0,
Hollinger's PER, Sill's RAPM regularisation, Pelton's age adjustment, Vashro
on steal rate, and Cheng on free-throw percentage as a shooting-touch proxy.

---

## Modules

| Module | Purpose |
|---|---|
| `schema` | Canonical-name resolution, coverage auditing, context detection |
| `vocab` | Domain vocabularies: aliases, context markers, rate ceilings |
| `core` | Provenance system: `MetricSpec`, `ComputationReport`, `Reliability` |
| `panel` | Complete calendars, lags, deltas, rolling windows, horizon joins, temporal splits |
| `units` | Detect and harmonise mixed per-period / cumulative columns |
| `shrinkage` | Empirical-Bayes (flat and hierarchical) and James-Stein small-sample correction |
| `quality` | Rate validation, artifact detection, robust standardisation |
| `similarity` | PCA decomposition, GMM soft archetypes, Mahalanobis-distance neighbours |
| `survival` | Kaplan-Meier, Cox, censoring-aware label construction |
| `evaluation` | Shared regression and classification scoring helpers |
| `basketball` | Domain pack: Four Factors, possessions, BPM/PER, draft features, count reconstruction |
| `crypto` | Domain pack: CoinMarketCap and Kraken daily prices, return/trend/volatility/volume features, the Asset Performance Index and its 7-day label |

---

## Validation

BPM reconstruction, unit harmonisation, and rate-artifact detection have
been checked against a real NCAA box-score dataset and an independent
reference. See [VALIDATION.md](VALIDATION.md) for methodology, results,
and the data-quality issues these modules handle.

---

## Installation

Published to [PyPI](https://pypi.org/project/mclovin/) and
[TestPyPI](https://test.pypi.org/project/mclovin/).

```bash
# From PyPI (recommended)
pip install mclovin

# From TestPyPI (dependencies still resolve from PyPI)
pip install --index-url https://test.pypi.org/simple/ \
            --extra-index-url https://pypi.org/simple/ mclovin

# Local development
pip install -e ".[dev]"

# With survival analysis (Cox models)
pip install -e ".[survival]"

# With the scoring helpers (scikit-learn)
pip install -e ".[models]"
```

---

## Usage

### Quickstart

Resolve your schema once, then call any `add_*` function. Each one returns
your DataFrame with new columns appended.

```python
import pandas as pd
import mclovin.basketball as bb

df = pd.read_csv("players.csv")
resolver = bb.resolver(df.columns)          # basketball vocabulary supplied

df = bb.draft.add_prospect_features(df, resolver=resolver)
```

`add_prospect_features` skips any feature whose inputs it can't resolve
rather than raising, so it's safe to run on a dataset you haven't audited
yet. For real-world data, though, run it after the fuller pipeline below,
since features computed on mixed units or unreconstructed counts will be
wrong.

### Recommended pipeline order

Run these steps in order, since later ones depend on earlier ones: units
first, then reconstruction, then everything else.

```python
import pandas as pd
from mclovin import ComputationReport, units, quality
import mclovin.basketball as bb
from mclovin.basketball import box_metrics, draft, reconstruct

df = pd.read_csv("players.csv")
df = df.drop_duplicates(subset=["pid", "year"])

r = bb.resolver(df.columns)
report = ComputationReport()

# 1. Audit what this dataset supports
print(r.coverage().query("available"))
print("team context:", r.has_team_context())

# 2. Fix units FIRST. Nothing downstream is valid without this
detection = units.detect_stat_units(df, resolver=r)
print(detection.evidence)
df = units.harmonize_units(df, resolver=r, target="season_total",
                           detection=detection)
print(units.verify_scoring_identity(df, resolver=r))

# 3. Recover missing raw counts
r = SchemaResolver(df.columns)
df = reconstruct.reconstruct_counting_stats(df, resolver=r, report=report)

# 4. Clean small-sample artifacts
r = SchemaResolver(df.columns)
print(quality.find_impossible_rates(df, resolver=r))
df = quality.clip_rates(df, resolver=r)
df = quality.add_sample_size_flag(df, resolver=r)

# 5. Compute metrics and prospect features
df = box_metrics.add_box_plus_minus_linear(df, resolver=r, report=report)
df = draft.add_prospect_features(df, resolver=r, report=report)
df = draft.add_age_adjusted_production(df, production_col="bpm", report=report)

# 6. Audit what you can trust
print(report.to_frame())
print("trusted:", report.trusted())
```

### Small-sample correction

Three-point percentage needs roughly **750 attempts** to become reliable
(Blackport 2014). A college season provides a fraction of that, so nearly
every college 3P% is under-sampled.

```python
from mclovin import shrinkage

df = shrinkage.empirical_bayes_rate(df, made_col="TPM", attempted_col="TPA")
df = shrinkage.add_reliability_weight(df, "TPA", stat_key="TP_PCT")
```

A 4-for-7 shooter moves substantially toward the population mean; a
200-for-500 shooter barely moves.

### Lags and windows on gappy panel data

`add_lags` and `add_rollings` count rows. When a series has missing periods,
put each entity on a complete calendar first, so a lag of 20 is 20 periods
and a window over a gap comes out missing rather than quietly spanning it:

```python
from mclovin.panel import add_lags, align_to_frame, complete_calendar

daily = complete_calendar(prices, ["close"], by="token")   # NaN rows fill the gaps
daily = add_lags(daily, ["close"], [20], by="token")
prices["close_20d_ago"] = align_to_frame(daily, "close_lag20", prices, by="token")
```

### Cryptocurrency features

`mclovin.crypto` builds the features and target of the 36120 AT3
Cryptocurrency Financial Intelligence Platform on that machinery. The same
calls produce the training table in the experiment notebooks and the live
features in the deployed API:

```python
import mclovin.crypto as crypto

prices = crypto.load_coinmarketcap("data/raw/coinmarketcap")   # or crypto.kraken.parse_ohlc(...)
table = crypto.add_asset_performance_target(crypto.add_features(prices))
```

---

## Testing

```bash
pip install -e ".[dev]"
pytest --cov=mclovin
```

**351 tests pass**, including:

- BPM's steal coefficient is the largest positive weight (Myers)
- Oliver's weights match the published 40/25/20/15
- PER normalises to exactly 15.00
- Turnover recovery round-trips to `rtol=1e-9`
- Robust z-score resists a single 1072 artifact where classical z-score fails
- Empirical Bayes shrinks small samples more than large ones
- Censored careers become `NaN`, not false negatives
- A lag or rolling window on a complete calendar never spans a missing day
- The crypto label is the index exactly 7 calendar days later, never the next row

---

## Known limitations

- **RAPM is not computable** from box scores. It needs play-by-play lineup
  data. BPM is its box-score approximation, and is treated as such here.
- **Full BPM 2.0 coefficients are only partially published.** This implements
  Myers' simplified *linear* version; correlation with a full-model reference
  is 0.81, not 1.0.
- **Team-context metrics need team data.** Usage%, AST%, TRB%, ORtg, PER and
  Win Shares all require team/opponent totals absent from a player-only
  dataset. They raise a clear `KeyError` rather than silently approximating.
- **PER's `normalise` centres against whatever rows you pass**, which
  equals the true league average only if you pass the full league.
- **Pelton's 0.5/year age penalty is in WARP units.** Applied to another
  production scale it should be rescaled, e.g. by the ratio of that scale's
  standard deviation to WARP's. This package does not fit the penalty
  against outcome labels; doing so is a modelling decision left to the
  caller, consistent with `mclovin` computing statistics rather than
  training predictive models.
- **Rate ceilings in `quality` are judgement calls**, not published constants.
  Review them against your own data before relying on the clipping.

---

## Key references

- Oliver, D. (2004). *Basketball on Paper*. Potomac Books.
- Kubatko, J., Oliver, D., Pelton, K., & Rosenbaum, D. (2007). "A Starting
  Point for Analyzing Basketball Statistics." *JQAS* 3(3).
- Myers, D. (2020). "About Box Plus/Minus (BPM)." Basketball-Reference.
- NBA.com Advanced Stats glossary. "Performance Impact Estimator (PIE)."
- Sill, J. (2010). "Improved NBA Adjusted +/− Using Regularization and
  Out-of-Sample Testing." MIT Sloan Sports Analytics Conference.
- Rosenbaum, D. (2004). "Measuring How NBA Players Help Their Teams Win."
- Pelton, K. "Explaining Kevin Pelton's NBA draft projection system." ESPN.
- Vashro, L. (2014). "How Do We Assess 'Potential' Among NBA Draft
  Prospects?" Canis Hoopus.
- Cheng, C. (2020). "Scouting NBA Three-Point Shooting." Harvard Sports
  Analysis Collective.
- Blackport, D. (2014). "How Long Does It Take For Three Point Shooting To
  Stabilize?" Nylon Calculus.
- Vaci, N., Cocić, D., Gula, B., & Bilalić, M. (2019). "Large data and
  Bayesian modeling: aging curves of NBA players." *Behavior Research Methods*
  51(4).
- Cui, Y., et al. (2019). "Key Anthropometric and Physical Determinants…NBA
  Draft Combine." *Frontiers in Psychology*.
- Efron, B. & Morris, C. (1975). "Data Analysis Using Stein's Estimator."
  *JASA* 70(350).
- Casella, G. (1985). "An Introduction to Empirical Bayes Data Analysis."
  *The American Statistician* 39(2), 83-87.
- Brown, L. D. (2008). "In-season prediction of batting averages: A field
  test of empirical Bayes and Bayes methodologies." *Annals of Applied
  Statistics* 2(1).
- Gelman, A. & Hill, J. (2007). *Data Analysis Using Regression and
  Multilevel/Hierarchical Models*. Cambridge University Press.
- Morris, C. N. (1983). "Parametric Empirical Bayes Inference: Theory and
  Applications." *JASA* 78(381), 47-55.
- Searle, S.R., Casella, G., & McCulloch, C.E. (1992). *Variance
  Components*. Wiley.
- Jolliffe, I.T. (2002). *Principal Component Analysis* (2nd ed.). Springer
  Series in Statistics.
- Alagappan, M. (2012). "From 5 to 13: Redefining the Positions in
  Basketball." MIT Sloan Sports Analytics Conference.
- Dempster, A.P., Laird, N.M. & Rubin, D.B. (1977). "Maximum Likelihood
  from Incomplete Data via the EM Algorithm." *JRSS Series B* 39(1), 1-38.
- Mahalanobis, P.C. (1936). "On the Generalised Distance in Statistics."
  Proceedings of the National Institute of Sciences of India, 2(1), 49-55.
- Ledoit, O. & Wolf, M. (2004). "A well-conditioned estimator for
  large-dimensional covariance matrices." Journal of Multivariate
  Analysis, 88(2), 365-411.
</content>
