Metadata-Version: 2.4
Name: tlf-correlation-engine
Version: 0.1.0
Summary: Country-agnostic Pearson/Spearman/Kendall correlation analysis over pandas DataFrames — part of The Living Facts (TLF).
Author-email: Sanchita Karki <karkisanchu06@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/ctpl-git/TLF-Data-Analysis
Project-URL: Repository, https://github.com/ctpl-git/TLF-Data-Analysis
Project-URL: Issues, https://github.com/ctpl-git/TLF-Data-Analysis/issues
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas>=1.5
Requires-Dist: scipy>=1.9
Requires-Dist: openpyxl>=3.1
Requires-Dist: questionary>=2.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Dynamic: license-file

# tlf-correlation-engine

Country-agnostic correlation analysis — part of **TLF** ("The Living Facts").

Computes Pearson, Spearman, or Kendall correlations across any pandas
DataFrame's numeric columns, along with p-values and observation counts,
and produces a sorted, filterable long-form report of the strongest and
most statistically significant relationships in a dataset.

Unlike `tlf-census-stats`, this package has no dependency on a specific
country schema — it works on any DataFrame with 2+ numeric columns.

---

## Install

```bash
pip install tlf-correlation-engine
```

Or from source, inside the `TLF-Data-Analysis` monorepo:

```bash
cd tlf-correlation-engine
pip install -e ".[dev]"
```

---

## Usage

```python
import pandas as pd
from tlf_correlation_engine import CorrelationEngine

df = pd.read_csv("census_data.csv")

engine = CorrelationEngine(df, method="pearson")  # or "spearman" / "kendall"

# Coefficient matrix (like df.corr(), but validated numeric-only)
engine.matrix()

# P-value matrix, aligned with matrix()
engine.pvalue_matrix()

# One specific pair, with n and p-value
engine.pairwise("literacy_rate", "urban_population")

# Long-form report of all pairs, strongest relationship first
engine.report()

# Only strong (|r| >= 0.5) and statistically significant (p < 0.05) pairs
engine.report(threshold=0.5, significant_only=True)
```

### Methods

| Method | Use for |
|---|---|
| `pearson` | Linear relationships between continuous variables |
| `spearman` | Monotonic (rank-based) relationships, robust to outliers/non-linearity |
| `kendall` | Rank concordance, more robust on small samples or many tied ranks |

---

## CLI

```bash
tlf-correlation-engine --data census.csv --method spearman --output report --export csv --export-path out.csv
```

Run with no flags at all for a fully interactive walkthrough (file path → sheet selection → method → output type → report filters → export format):

```bash
tlf-correlation-engine
```

For unattended/scripted runs, `--yes` disables all prompting and fails loudly (rather than silently guessing) if something required — like `--data` — is missing:

```bash
tlf-correlation-engine --data census.csv --yes --method pearson --output matrix
```

### CLI flags

| Flag | Description |
|---|---|
| `--data` | Path to a CSV, Excel, or JSON file |
| `--sheet` | Excel sheet name (default: first sheet) |
| `--method` | `pearson` \| `spearman` \| `kendall` |
| `--output` | `report` \| `matrix` \| `pvalues` |
| `--threshold` | Report only: minimum \|coefficient\| to include |
| `--significant-only` | Report only: only include pairs with p < `--alpha` |
| `--alpha` | Significance threshold for `--significant-only` (default 0.05) |
| `--export` | `csv` \| `json` |
| `--export-path` | Export file path |
| `--yes` | Non-interactive mode: never prompt, error on missing required values |

---

## Errors

- `InvalidMethodError` — unsupported `method` value
- `InsufficientDataError` — fewer than 2 numeric columns in the DataFrame

---

## License

MIT — see `LICENSE`.
