Metadata-Version: 2.5
Name: forgedge
Version: 0.2.0
Summary: FORGE — Feature-Oriented Rule Generation Engine for systematic alpha discovery.
Project-URL: Homepage, https://github.com/mattcond/forgedge
License: MIT
License-File: LICENSE
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.9
Requires-Dist: numpy>=1.23
Requires-Dist: pandas>=1.5
Provides-Extra: dev
Requires-Dist: pyarrow>=10.0; extra == 'dev'
Requires-Dist: pytest-xdist>=3.0; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Description-Content-Type: text/markdown

# FORGE — Feature-Oriented Rule Generation Engine

FORGE is a quantitative research system for the **systematic discovery of algorithmic trading rules** from historical market data. Starting from a KPI Table (OHLCV + technical indicators), FORGE identifies boolean events with stable temporal structure, measures their predictive power against an economic target derived from the data, and produces formal contracts ready for operational validation.

[🇮🇹 Versione italiana](README_it.md)

---

## What FORGE does, in plain terms

If you trade — or work with people who do — you've probably heard something like *"whenever X happens, the price tends to go up."* Spotting a pattern like that is the easy part. The hard part is knowing whether it's **real**, or whether it just happened to look good because enough ideas were tried that one of them was bound to, by chance alone. That's the trap FORGE is built to avoid, in four steps:

1. **Look for a repeatable pattern** in the price and indicator history — without ever looking at what happened *afterward*, so the pattern has no way to "cheat" by secretly knowing the future.
2. **Check whether that pattern actually predicts anything** — does the price reliably move in a given direction afterward, often enough and by enough to matter?
3. **Simulate trading it for real** — trading fees, realistic order fills, and tested only on data the pattern never saw while it was being found.
4. **Check whether it still holds up elsewhere** — on other assets, not just the one it was discovered on.

Every pattern comes out the other end with an honest verdict — *this works*, *this is borderline*, or *this doesn't hold up* — plus the specific reasons behind it, never a black-box score. No pattern is ever adjusted after the fact to look better; a rule that doesn't clear the bar is reported as such, not quietly reworked until it does.

One thing FORGE deliberately does **not** do: place trades, or talk to an exchange. It's a research tool that tells you which ideas are worth taking further and hands off the operational details (when to enter, when to exit, how strong the evidence is) — turning that into live trading is a separate, deliberate step for whatever system you actually execute through.

The rest of this document goes into how that works technically, for readers who want to run FORGE themselves.

---

## Why FORGE

Systematic edge research suffers from three recurring problems:

- **Look-ahead bias** — event thresholds calibrated while observing returns already "know" the future before discovery
- **In-sample optimisation** — thresholds and horizons tuned on the same window used for evaluation produce circular backtests
- **Missing operational separation** — statistical evidence of predictive power is not the same as profitability under real fees and order mechanics

FORGE addresses all three with a **strictly separated pipeline**: each module answers exactly one question and passes only a formal artefact to the next. No module can access the next module's data; no threshold can be recalibrated after discovery.

---

## Pipeline

```
KPI Table (OHLCV + technical indicators)
    │
    ▼
┌──────────────────────────────────────────────────────────────────┐
│  Module 0 — Market Context                                       │
│  Classifies every bar by market regime (5 levels).               │
│  Output: KPI Table + 'regime' and 'regime_stable' columns        │
└──────────────────────────────────────────────────────────────────┘
    │
    ▼
┌──────────────────────────────────────────────────────────────────┐
│  Module 1 — Event Discovery                                      │
│  Discovers boolean events from the temporal structure of         │
│  indicators. Never sees the forward return.                      │
│  Output: list[EventCandidate]                                    │
└──────────────────────────────────────────────────────────────────┘
    │
    ▼
┌──────────────────────────────────────────────────────────────────┐
│  Module 2 — Alpha Discovery                                      │
│  Derives target per event, measures IS predictive power,         │
│  confirms on the OOS tail. First exposure to forward return.     │
│  Output: list[AlphaContract]                                     │
└──────────────────────────────────────────────────────────────────┘
    │
    ▼
┌──────────────────────────────────────────────────────────────────┐
│  Module 3 — Rule Discovery                                       │
│  Realistic backtest with order mechanics, fees, walk-forward OOS.│
│  Output: EDGE / PARTIAL-EDGE / NON-EDGE + operational parameters │
└──────────────────────────────────────────────────────────────────┘
    │
    ▼
┌──────────────────────────────────────────────────────────────────┐
│  Module 4 — Rule Registry                                        │
│  Deduplication, cross-ticker backtest, genericity classification.│
│  Output: flat table + self-contained HTML report                 │
└──────────────────────────────────────────────────────────────────┘
```

---

## Key invariants

| Invariant | What it prevents |
|---|---|
| Module 1 never sees the forward return | Look-ahead bias in event selection |
| Thresholds are immutable after discovery | Threshold optimisation on the evaluation sample |
| Target horizon, direction, and take-profit are derived from data per event | Economic assumptions pre-baking the result |
| The tradeable verdict is gated by walk-forward OOS (Module 3); Alpha's OOS confirmation is recorded on every contract | Post-hoc confirmation of a foregone conclusion |

---

## Installation

FORGE depends solely on `numpy` and `pandas`. No `scipy`, `statsmodels`, or ML library required: all statistical primitives (Spearman, t-test, OU regression, Benjamini-Hochberg FDR, incomplete beta) are implemented in pure numpy.

```bash
pip install forgedge
```

---

## Quick start

```python
import pandas as pd
from forgedge import forge

# KPI Table with OHLCV + technical indicators ('close' column required)
kpi = pd.read_parquet("kpi_table.parquet")

# Full pipeline: from KPI Table to validated rules in a single call
result = forge(kpi, ticker="BTCUSDC", timeframe="1H")

print(result.summary())                         # one row per candidate + rule_verdict
for contract, response in result.edges():       # EDGE / PARTIAL-EDGE only
    print(contract.alpha_id, response.verdict)
print(result.registry.summary())                # Module 4 — catalogued rules
```

Before any of that runs, `forge()` resolves every unset config field against the session's timeframe/schema and checks the whole bundle for internal contradictions — by default (`strict=True`) it raises `ValueError` immediately if it finds one that would make every candidate fail for reasons that have nothing to do with the signal, rather than letting the run finish and produce a wall of unexplained rejections. `strict=False` downgrades that to a warning and runs anyway; `forgedge.config_report(...)` runs the same check standalone, before you commit to a full run.

On a daily-or-slower `timeframe` the default holding-period grid is
automatically replaced by a daily-calibrated one (the stock grid is calibrated
on ~hourly bars); passing your own `AlphaConfig` keeps full control.  For
frequency-consistent per-module configuration use
`forgedge.presets.forge_preset("balanced", timeframe="1D", asset=...)`.
Each event's grid is additionally **enriched** with horizons at 0.5×/1×/2× its
indicator's dominant window (`AlphaConfig.horizon_enrichment` — a union, never
a restriction, statistically capped so slow conditioning windows cannot demand
unmeasurable holding periods; the added hypotheses are counted by the ledger
and priced by the rotation null).

By default `forge()` also runs the **fast search-level rotation null** — the
exact null distribution of the best standardised excess over every circular
offset, computed via FFT in ~seconds — and a full `EDGE` verdict additionally
requires beating it (`rotation_p <= 0.05`): a rule that only won the
multiple-testing lottery of its own discovery session is capped at
`PARTIAL-EDGE`.  The session's hypothesis surface is recorded on
`ForgeResult.ledger`; disable with `fast_null=False`.

Temporal splits are **purged**: in-sample bars whose forward window crosses
the IS/OOS boundary are excluded from every Alpha Discovery measure, and the
walk-forward train windows end one worst-case trade span before their test
window (see `forgedge.timebudget.TimeBudget` — pass one to `forge()` to put
every module on a single shared axis, with an optional embargo).

Rule Discovery selects its operating point **inside the walk-forward train
windows only** (`RuleDiscoveryConfig.selection_mode="walk_forward"`, the
default): the published parameters come from the last train window (or the
consensus across windows), and every verdict-feeding metric is computed on
the selection span — the final test window is never read by any selection.
`selection_mode="full_sample"` restores the legacy whole-table behaviour.

Verdicts are **power-aware**: a would-be `EDGE`/`PARTIAL-EDGE` whose pooled
out-of-sample evidence cannot support it (too few pooled test trades, or a
minimum detectable expectancy above the claimed effect) is emitted as
`INSUFFICIENT-DATA` — not tradeable, but it keeps its `ValidatedRule` for
re-evaluation on more data.  The assessment reads only the concatenated
test-window ledger, never individual walk-forward windows (short by design).
Disable with `SelectionCriteria(power_gate=False)`.

Multi-ticker sessions with `forge_multi`:

```python
from forgedge import forge_multi

frames = {"BTCUSDC": btc_kpi, "ETHUSDC": eth_kpi, "ADAUSDC": ada_kpi}
results, registry = forge_multi(frames, timeframe="1H")

# GENERIC: rule generalises to ≥ 2/3 of tested tickers
df = registry.flat_table()
print(df[["rule_id", "classification", "pf", "cross_ticker_score"]])

# Self-contained HTML report (inline SVG, no CDN)
html = registry.html_report(timeframe="1H")
with open("report.html", "w") as f:
    f.write(html)
```

Monitoring published rules on fresh candles:

```python
from forgedge import RuleSpec, rule_performance_report

# One spec per tradeable rule of a forge() run — is_end / verdict / OOS
# expectancy are filled in automatically from the session's artefacts.
specs = RuleSpec.from_forge_result(result)

# Or hand it the ForgeResult directly — same effect.
html = rule_performance_report(result, fresh_candles)
with open("rules_report.html", "w") as f:
    f.write(html)
```

Each rule is replayed **deterministically** on the given candles (the same
`EventCandidate.apply()` path Rule Discovery itself uses) — the candles need
not be the discovery table, so this is the natural way to monitor published
rules on data collected after discovery. The report shows, per rule: equity
vs buy & hold on a dual axis (independent scales — compares shapes, not
magnitudes), a monthly activation trend split into in-sample/out-of-sample,
the gain/loss distribution, return-distribution KDEs — low/close/high, each
the forward return over the holding horizon (target_h bars ahead of the
signal bar, anchored on that bar's close, oriented by direction: the same
quantity Alpha Discovery derives the target from and Rule Discovery trades),
base vs event — a MAE→net scatter (intra-trade risk for these stop-less
rules), rolling expectancy (edge-decay detector), per-regime
performance, the most recent trades, and a "signal active now" badge.

---

## The three concepts

FORGE structures the discovery process around three formal concepts that each answer a distinct question and produce a distinct artefact.

### Event — observing the market without bias

An **event** is a boolean condition on historical bars discovered from the temporal structure of indicators — without ever computing a forward return. Thresholds are distributional (asset-specific percentiles) and immutable once fixed.

```python
c = candidates[0]
print(c.expression)            # "rsi_14 < 31.2 AND spread_ema_9_25 < -0.0118"
signal = c.apply(new_kpi)      # pd.Series bool — deterministic, no look-ahead
```

The ConsistencyGate filters out events with unstable temporal structure (too few activations, seasonal clustering, low monthly frequency) before any return is computed.

### Alpha — measuring predictive power

An **alpha** is the empirical answer to: *given that the event activated, what happens statistically in the next h bars?* Horizon, direction, and take-profit level (`sell_pct`) are all **derived from data** — never assumed — by scanning `|mean_advantage|/√h` across a horizon grid and taking the MFE quantile of active bars.

```python
c = promoted[0]
dt = c.derived_target
print(f"{dt.direction} at h={dt.holding_period_h}h  sell_pct={dt.sell_pct:.4f}")
print(f"Grade {c.alpha_score.grade}  |  OOS lift: {c.oos_validation.lift:.4f}")
```

The only hard rejection gate is an undetermined direction (no finite advantage across any horizon). All other statistical metrics (IC, Cohen's d, lift, FDR) contribute to the A–D grade without blocking promotion.

### Rule — trading realistically

A **rule** is the operational verdict on an alpha contract. Rule Discovery runs a realistic backtest — take-profit exit, horizon stop, per-side fees — evaluated first at a market entry (that verdict is authoritative), then optionally with a limit entry adopted only if it clears strict out-of-sample conditions — and validates the best parameter configuration on a rolling walk-forward OOS.

```python
resp = RuleDiscovery(ed.df, contract, cand).run()
print(resp.verdict)                               # "EDGE", "PARTIAL-EDGE", "NON-EDGE"
if resp.is_edge:
    p = resp.validated_rule.params
    print(f"Entry: limit -{p.buy_drop_pct:.2%}  TP: +{p.sell_pct:.2%}  h={p.target_h}")
    print(f"IS PF: {resp.in_sample_summary.profit_factor:.2f}"
          f"  WF consistency: {resp.walk_forward.consistency:.0%}")
```

---

## Module overview

| Module | Question answered | Key output |
|---|---|---|
| 0 — Market Context | Which regime is this bar in? | `regime` column (5 levels), `regime_stable` |
| 1 — Event Discovery | Is this indicator configuration stable and repeatable? | `EventCandidate` — immutable thresholds, `apply()` |
| 2 — Alpha Discovery | Does the event predict an oriented return? | `AlphaContract` — derived target, A–D grade |
| 3 — Rule Discovery | Is this alpha profitable under real order mechanics? | `RuleDiscoveryResponse` — EDGE verdict, `ValidatedRule` |
| 4 — Rule Registry | Does this rule generalise across tickers? | Flat table, HTML report — GENERIC / PARTIAL / SPECIFIC |

---

## Implementation status

| Module | Status |
|---|---|
| 0 — Market Context | ✅ Implemented |
| 1 — Event Discovery | ✅ Implemented |
| 2 — Alpha Discovery | ✅ Implemented |
| 3 — Rule Discovery | ✅ Implemented |
| 4 — Rule Registry | 🚧 WIP |

---

## Documentation

**Start with the manual** — [`docs/manual-en.md`](docs/manual-en.md) is a single, comprehensive, example-verified guide covering installation through production architecture, troubleshooting, best practices/anti-patterns, an FAQ and a glossary. Everything in it was checked against the current source and, where a number is quoted, against a real run. Everything below is narrower, complementary reference material.

| File | Contents |
|---|---|
| [`docs/manual-en.md`](docs/manual-en.md) | The comprehensive, example-verified manual — installation to production, every module's API, troubleshooting, FAQ |
| [`concepts_en.md`](src/forgedge/docs/specs/concepts_en.md) | Conceptual guide: event, alpha, and rule — from market to signal |
| [`how_to_use_en.md`](src/forgedge/docs/specs/how_to_use_en.md) | End-to-end production pipeline guide with full configuration |
| [`modulo_0_en.md`](src/forgedge/docs/specs/modulo_0_en.md) | Market Context: regime classification, EMAProxy, configuration |
| [`modulo_1_en.md`](src/forgedge/docs/specs/modulo_1_en.md) | Event Discovery: 5-step pipeline, ConsistencyGate, EventCandidate |
| [`modulo_2_en.md`](src/forgedge/docs/specs/modulo_2_en.md) | Alpha Discovery: derived target, IC, OOS, AlphaContract |
| [`modulo_3_en.md`](src/forgedge/docs/specs/modulo_3_en.md) | Rule Discovery: backtest, EDGE verdict, walk-forward, reports |
| [`modulo_4_en.md`](src/forgedge/docs/specs/modulo_4_en.md) | Rule Registry: deduplication, cross-ticker, genericity, export |

---

## License

MIT
