Metadata-Version: 2.5
Name: forecastlens
Version: 0.1.0
Summary: Diagnostic layer for time-series forecasts: regime-aware calibration, leak-safe backtesting, and economic decision-value evaluation on top of darts/neuralforecast outputs.
Project-URL: Homepage, https://github.com/patrickresearch/forecastlens
Project-URL: Repository, https://github.com/patrickresearch/forecastlens
Project-URL: Changelog, https://github.com/patrickresearch/forecastlens/blob/main/CHANGELOG.md
Author: Patrick Petter
License: Apache-2.0
License-File: LICENSE
Keywords: backtesting,darts,evaluation,forecasting,neuralforecast,time-series
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering
Requires-Python: >=3.10
Requires-Dist: numpy>=1.24
Requires-Dist: pandas>=2.0
Requires-Dist: scipy>=1.10
Provides-Extra: all
Requires-Dist: darts>=0.27; extra == 'all'
Requires-Dist: matplotlib>=3.7; extra == 'all'
Requires-Dist: neuralforecast>=1.6; extra == 'all'
Provides-Extra: darts
Requires-Dist: darts>=0.27; extra == 'darts'
Provides-Extra: dev
Requires-Dist: mypy>=1.8; extra == 'dev'
Requires-Dist: pandas-stubs; extra == 'dev'
Requires-Dist: pre-commit>=3.6; extra == 'dev'
Requires-Dist: properscoring>=0.1; extra == 'dev'
Requires-Dist: pytest-cov>=4.1; extra == 'dev'
Requires-Dist: pytest>=7.4; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Requires-Dist: sktime>=0.24; extra == 'dev'
Provides-Extra: docs
Requires-Dist: mkdocs-material>=9.5; extra == 'docs'
Requires-Dist: mkdocstrings[python]>=0.25; extra == 'docs'
Provides-Extra: neuralforecast
Requires-Dist: neuralforecast>=1.6; extra == 'neuralforecast'
Provides-Extra: test
Requires-Dist: properscoring>=0.1; extra == 'test'
Requires-Dist: pytest-cov>=4.1; extra == 'test'
Requires-Dist: pytest>=7.4; extra == 'test'
Requires-Dist: sktime>=0.24; extra == 'test'
Provides-Extra: viz
Requires-Dist: matplotlib>=3.7; extra == 'viz'
Description-Content-Type: text/markdown

# ForecastLens

[![CI](https://github.com/patrickresearch/forecastlens/actions/workflows/ci.yml/badge.svg)](https://github.com/patrickresearch/forecastlens/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/forecastlens.svg)](https://pypi.org/project/forecastlens/)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)

A diagnostic layer on top of your existing time-series forecasting stack (`darts`, `neuralforecast`, or anything else you can put in a DataFrame). ForecastLens doesn't train models — it tells you things your forecasting framework doesn't:

- **Regime-aware evaluation.** A single averaged CRPS/MAE across a whole backtest period hides that it was earned across several different market regimes. ForecastLens breaks accuracy down *by regime*, not just by time.
- **Leak-safe backtesting.** A causal, walk-forward splitter and rolling decision buckets that are provably unable to see future data — with tests that assert it.
- **An economic decision layer.** Forecasts translated into concrete decisions ("buy now vs. wait") and scored in currency per unit, not just abstract error metrics.

**Status:** pre-alpha (`0.1.0`), API not yet stable.

## Installation

```bash
pip install forecastlens                  # core only (numpy, pandas, scipy)
pip install forecastlens[darts]           # + darts adapter
pip install forecastlens[neuralforecast]  # + neuralforecast adapter
pip install forecastlens[viz]             # + plotting
pip install forecastlens[all]             # everything
```

## Quickstart: economic value of waiting on a forecast

No API keys, no real data needed — `forecastlens.synthetic` ships regime-switching presets with known ground truth for exactly this kind of demo.

```python
import numpy as np
from forecastlens.core import ForecastResult
from forecastlens.decision import ProcurementTimingModel, ThresholdTimingRule
from forecastlens.synthetic import load_preset

series = load_preset("supply_shock_spike")
start, horizon = 30, 30  # the procurement window opens right at the price spike's peak
baseline_forecast = series.values[:20].mean()  # our model expects reversion to the pre-shock level
forecast = ForecastResult(
    timestamps=series.timestamps[start : start + horizon],
    freq="D",
    point=np.full(horizon, baseline_forecast),
)
realized_prices = series.values[start : start + horizon]

model = ProcurementTimingModel(
    decision_rule=ThresholdTimingRule(wait_if_forecast_below_current_by=0.03),
    deadline_horizon=horizon,
    units=1000,
)
report = model.evaluate(forecast, realized_prices)
print(f"Savings per unit: {report.savings_per_unit:.2f}")   # 21.94
print(f"Total savings:    {report.total_savings:.2f}")      # 21940.29
print(f"Bought on day:    {report.buy_day}")                # 11, not forced by the deadline
```

The model waited 11 days for the price to revert toward its forecast baseline instead of buying at the spike — a result expressed directly in currency, no statistics vocabulary required.

## Quickstart: regime-conditional accuracy

```python
import numpy as np
from forecastlens.core import ForecastResult
from forecastlens.evaluators import RegimeAwareEvaluator
from forecastlens.regime import CUSUMDetector
from forecastlens.synthetic import load_preset

series = load_preset("structural_break")
y_true = series.values

# A toy quantile forecast: one-step persistence with a fixed +-5 band.
median = np.roll(y_true, 1)
median[0] = y_true[0]
forecast = ForecastResult(
    timestamps=series.timestamps,
    freq="D",
    quantiles={0.1: median - 5.0, 0.5: median, 0.9: median + 5.0},
)

detector = CUSUMDetector(warmup=80, threshold_std=10.0, drift_std=1.5)
report = RegimeAwareEvaluator(detector).evaluate(forecast, y_true)

print(f"Overall CRPS: {report.overall_crps:.3f}")
for regime in report.per_regime:
    print(f"Regime {regime.regime_label}: n={regime.n_periods}, CRPS={regime.crps:.3f}")
```

The overall CRPS averages away exactly the thing a regime shift analysis needs: some regimes here score close to 0.94, others above 1.4 — a >50% spread invisible in the single averaged number.

## Using darts or neuralforecast output directly

```python
from forecastlens.adapters import DartsAdapter, NeuralForecastAdapter

# darts: model.predict(...) -> TimeSeries
result = DartsAdapter.from_timeseries(prediction, model_name="ExponentialSmoothing")

# neuralforecast: nf.predict() -> DataFrame
result = NeuralForecastAdapter.from_dataframe(predictions_df, model=nf.models[0])
```

Both normalize into the same `ForecastResult` that every metric, evaluator, and decision model in this package consumes — deterministic predictions become a point forecast, probabilistic ones become quantiles.

## What's in the box

| Module | Purpose |
|---|---|
| `core` | `ForecastResult` — the shared, immutable forecast representation |
| `adapters` | Normalize `darts`, `neuralforecast`, or any tabular output into `ForecastResult` |
| `metrics` | CRPS, MASE, WQL, pinball loss — each validated against a reference implementation |
| `backtesting` | `LeakSafeWalkForwardSplitter` — causal, provably leak-free walk-forward CV |
| `regime` | `VolatilityRegimeDetector`, `CUSUMDetector` — swappable regime/changepoint detection |
| `decision` | `DecisionRelevantBucketAccuracy`, `ProcurementTimingModel` — forecasts priced as decisions |
| `evaluators` | `DemandForecastEvaluator`, `RegimeAwareEvaluator` — use-case-specific reports |
| `synthetic` | 10 regime-switching price/demand presets with known ground truth, zero setup |

See `examples/` for full end-to-end tutorials, one per use case (demand planning, energy/commodity regime shifts).

## A note on `relative_value_score`

`decision.relative_value_score` is a **deliberately simplified MVP metric** — `(cost_naive - cost_model) / cost_naive` — not the academic Murphy value score (which normalizes against a perfect-foresight forecast on a 0-1 scale). Ours can go negative or exceed 1. See the docstring in [`decision/value_score.py`](src/forecastlens/decision/value_score.py) for the full rationale.

## License

Apache 2.0 — see [LICENSE](LICENSE).
