Metadata-Version: 2.4
Name: trade-study
Version: 0.3.0
Summary: Multi-objective trade-study orchestration: scoring, Pareto optimization, and Bayesian stacking
Author-email: "Joshua C. Macdonald" <jmacdo16@jh.edu>
License-Expression: MIT
Project-URL: Homepage, https://github.com/jcm-sci/trade-study
Project-URL: Documentation, https://jcm-sci.github.io/trade-study/
Project-URL: Repository, https://github.com/jcm-sci/trade-study
Project-URL: Changelog, https://github.com/jcm-sci/trade-study/blob/main/CHANGELOG.md
Project-URL: Issues, https://github.com/jcm-sci/trade-study/issues
Keywords: trade-study,model-selection,scoring-rules,pareto,bayesian-stacking,design-of-experiments,sensitivity-analysis,multi-objective-optimization
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.24
Provides-Extra: dataframe
Requires-Dist: pandas>=2.0; extra == "dataframe"
Provides-Extra: scoring
Requires-Dist: scoringrules>=0.8; extra == "scoring"
Provides-Extra: pareto
Requires-Dist: pymoo>=0.6; extra == "pareto"
Provides-Extra: stacking
Requires-Dist: arviz>=0.20; python_version < "3.12" and extra == "stacking"
Requires-Dist: arviz>=1.0; python_version >= "3.12" and extra == "stacking"
Requires-Dist: scipy>=1.10; extra == "stacking"
Provides-Extra: design
Requires-Dist: pyDOE3>=1.5; extra == "design"
Requires-Dist: SALib>=1.5; extra == "design"
Requires-Dist: scipy>=1.10; extra == "design"
Provides-Extra: adaptive
Requires-Dist: optuna>=4.0; extra == "adaptive"
Provides-Extra: parallel
Requires-Dist: joblib>=1.3; extra == "parallel"
Provides-Extra: viz
Requires-Dist: matplotlib>=3.6; extra == "viz"
Provides-Extra: surrogate
Requires-Dist: scikit-learn>=1.3; extra == "surrogate"
Provides-Extra: all
Requires-Dist: trade-study[adaptive,dataframe,design,parallel,pareto,scoring,stacking,surrogate,viz]; extra == "all"
Provides-Extra: examples
Requires-Dist: scikit-learn>=1.3; extra == "examples"
Requires-Dist: trade-study[all]; extra == "examples"
Provides-Extra: test
Requires-Dist: pytest>=8.1; extra == "test"
Requires-Dist: pytest-cov>=6.0; extra == "test"
Requires-Dist: trade-study[all]; extra == "test"
Provides-Extra: docs
Requires-Dist: mkdocs-material>=9.5; extra == "docs"
Requires-Dist: mkdocstrings[python]>=0.27; extra == "docs"
Provides-Extra: dev
Requires-Dist: mypy>=1.18; extra == "dev"
Requires-Dist: ruff>=0.13; extra == "dev"
Requires-Dist: trade-study[docs,test]; extra == "dev"
Dynamic: license-file

# trade-study

[![CI](https://github.com/jcm-sci/trade-study/actions/workflows/ci.yml/badge.svg)](https://github.com/jcm-sci/trade-study/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/trade-study)](https://pypi.org/project/trade-study/)
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.19599838.svg)](https://doi.org/10.5281/zenodo.19599838)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![Code style: ruff](https://img.shields.io/badge/code%20style-ruff-000000.svg)](https://github.com/astral-sh/ruff)
[![jcm-sci](https://img.shields.io/badge/jcm--sci-jcmacdonald.dev-blue)](https://jcmacdonald.dev/projects/)

Multi-objective trade-study orchestration: define factors, build
parameter grids, run hierarchical study phases, and extract Pareto
fronts — for any domain where you compare alternatives against
competing objectives.

## Statement of need

Comparing design alternatives against multiple objectives is a common
task across scientific simulation, engineering trade-offs, and ML
hyperparameter tuning. Researchers typically glue together separate
tools for grid construction, execution, scoring, and Pareto analysis,
writing ad-hoc scripts that are hard to reproduce or extend to
multi-phase studies (screening → refinement → benchmark).

`trade-study` provides a single orchestration layer that composes these
steps into a reproducible, protocol-driven workflow. Users supply a
`Simulator` (generates data) and a `Scorer` (evaluates it); the
framework handles grid construction, parallel execution, Pareto
extraction, and phase chaining. All components are modular and
optional — use only what you need.

This package targets researchers and practitioners who need:

- structured multi-phase experimental design (screen → refine → benchmark),
- multi-objective Pareto analysis across heterogeneous factors, and
- a reproducible Python API that separates domain logic from study orchestration.

## Why trade-study?

| Need                    | Without trade-study                                 | With trade-study                                                           |
| ----------------------- | --------------------------------------------------- | -------------------------------------------------------------------------- |
| Parameter grid          | Manual `itertools.product` or one-off scripts       | `build_grid(factors, method="sobol")` — full factorial, LHS, Sobol, Halton |
| Multi-objective ranking | Call pymoo directly, handle direction normalization | `extract_front(scores, directions)` — direction-aware                      |
| Phased studies          | Custom loop with manual filtering between stages    | `Study(phases=[Phase(..., filter_fn=top_k_pareto_filter(k=20)), ...])`     |
| Adaptive search         | Set up optuna study from scratch                    | `run_adaptive(world, scorer, factors, observables, n_trials=600)`          |
| Reproducibility         | Scattered scripts, no standard protocol             | `Simulator` / `Scorer` protocols + `save_results()` / `load_results()`     |

Existing tools solve pieces of this problem — [optuna](https://optuna.org/) for adaptive optimization, [pymoo](https://pymoo.org/) for multi-objective solvers, [SALib](https://salib.readthedocs.io/) for sensitivity analysis — but none provide the **hierarchical phase orchestration** that connects them into a single study.

## Quick start

This wiring sketch assumes domain-specific `MySimulator` and `MyScorer`
implementations like those shown under [Protocols](#protocols). See the worked
guides in the documentation for complete, executable examples.

```python
from trade_study import (
    Direction,
    Factor,
    FactorType,
    Observable,
    Phase,
    Study,
    build_grid,
    top_k_pareto_filter,
)

# 1. Define objectives
accuracy = Observable("accuracy", Direction.MAXIMIZE)
latency = Observable("latency_ms", Direction.MINIMIZE)
cost = Observable("cost_usd", Direction.MINIMIZE)

# 2. Define factors and build a design grid
factors = [
    Factor("learning_rate", FactorType.CONTINUOUS, bounds=(1e-4, 1e-1)),
    Factor("backend", FactorType.CATEGORICAL, levels=["A", "B", "C"]),
]
grid = build_grid(factors, method="lhs", n_samples=500)

# 3. Run a hierarchical study
study = Study(
    world=MySimulator(),  # implements Simulator protocol
    scorer=MyScorer(),  # implements Scorer protocol
    observables=[accuracy, latency, cost],
    phases=[
        Phase(
            "screening",
            grid=grid,
            filter_fn=top_k_pareto_filter(k=20),
        ),
        Phase("benchmark", grid="carry", filter_fn=None),
    ],
)
study.run(n_jobs=-1)

# 4. Inspect results
print(study.summary())
front = study.front("benchmark")  # non-dominated config indices
study.compare_phases()  # per-phase hypervolume and IGD+ between successive fronts
```

### Protocols

Users implement two protocols to plug in their domain:

```python
from trade_study import Scorer, Simulator


class MySimulator:
    """Implements the Simulator protocol."""

    def generate(self, config: dict) -> tuple:
        """Return (truth, observations) for a given config."""
        ...


class MyScorer:
    """Implements the Scorer protocol."""

    def score(self, truth, observations, config: dict) -> dict[str, float]:
        """Return {observable_name: value} for a single trial."""
        ...
```

## Installation

```bash
pip install trade-study[all]
```

Or install only the extras you need:

```bash
pip install trade-study[design,pareto]
```

| Extra      | Packages                                                                                                       | Purpose                                     |
| ---------- | -------------------------------------------------------------------------------------------------------------- | ------------------------------------------- |
| `design`   | [pyDOE3](https://github.com/relf/pyDOE3), [SALib](https://github.com/SALib/SALib), [scipy](https://scipy.org/) | Grid construction and sensitivity screening |
| `pareto`   | [pymoo](https://pymoo.org/)                                                                                    | Non-dominated sorting and indicators        |
| `scoring`  | [scoringrules](https://github.com/frazane/scoringrules)                                                        | Proper scoring rules (CRPS, WIS, etc.)      |
| `stacking` | [arviz](https://github.com/arviz-devs/arviz), scipy                                                            | Bayesian and score-based ensemble weights   |
| `adaptive` | [optuna](https://optuna.org/)                                                                                  | Adaptive multi-objective search (NSGA-II)   |
| `parallel` | joblib                                                                                                         | Parallel grid execution                     |
| `viz` | matplotlib | Plot scores and Pareto fronts |
| `surrogate` | scikit-learn | GP/RF score and regime surrogates |
| `dataframe` | pandas | ResultsTable export for analysis and CSV |
| `all`      | All of the above                                                                                               |                                             |

**Core dependency**: numpy only.

## API overview

### Design

```python
from trade_study import Factor, FactorType, build_grid, screen

factors = [
    Factor("x", FactorType.CONTINUOUS, bounds=(0.0, 1.0)),
    Factor("method", FactorType.CATEGORICAL, levels=["a", "b", "c"]),
]

grid = build_grid(factors, method="sobol", n_samples=1024)
si = screen(run_fn, factors, method="morris", n_trajectories=3000)
```

### Execution

```python
from trade_study import run_grid, run_adaptive

# Grid mode: evaluate all configs (optional parallelism)
results = run_grid(world, scorer, grid, observables, n_jobs=-1)

# Adaptive mode: evolutionary multi-objective optimization (Optuna NSGA-II)
results = run_adaptive(world, scorer, factors, observables, n_trials=600)
```

### Pareto analysis

```python
from trade_study import extract_front, hypervolume, pareto_rank

front_idx = extract_front(results.scores, directions)
ranks = pareto_rank(results.scores, directions)
hv = hypervolume(results.scores[front_idx], ref_point, directions)
```

### Stacking

```python
from trade_study import stack_scores, stack_bayesian, stack_proportional, ensemble_predict

weights = stack_scores(score_matrix)  # simplex-constrained optimization
weights = stack_proportional(score_matrix)  # preserve weight across near ties
weights = stack_bayesian(idata_dict)  # arviz stacking (Yao et al. 2018)
combined = ensemble_predict(predictions, weights)
```

### I/O

```python
from trade_study import save_results, load_results

save_results(results, "study_results")
results = load_results("study_results")
frame = results.to_dataframe()  # optional pandas extra; includes trial metadata
frame.to_csv("study_results.csv", index=False)
```

### Multi-fidelity search and surrogates

```python
from trade_study import run_successive_halving, run_hyperband, fit_surrogate, fit_regime_surrogate

# Cheap-first budget-constrained screening
results = run_successive_halving(world, scorer, factors, observables, budget_param="maxiters")
results = run_hyperband(world, scorer, factors, observables, budget_param="maxiters")

# Interpolate scores from a completed results table (GP or RF)
surrogate = fit_surrogate(results, method="gp")

# Recommend a config per regime, interpolating over regime descriptors
regime_surrogate = fit_regime_surrogate(results, regime_cols=["n", "p"])
```

Surrogate fits report cross-validated `cv_r2` and `cv_rmse`; check those
diagnostics before trusting recommendations or post-hoc sensitivity results.

`Constraint` / `feasibility_filter` express infeasible regions of the
design space; `FactorConstraint` couples factors during grid
construction (e.g. keep `x + y <= 1`); `screen()` supports both Morris
and Sobol sensitivity analysis. `run_grid(..., n_reps=N)` supports replicated
trials, and `sensitivity_from_table()` estimates post-hoc sensitivity from an
existing results table through a validated surrogate.

## Release compatibility

Version 0.3.0 adds replicated and paired studies, uncertainty diagnostics,
persistent adaptive sessions, checkpointing, and DataFrame export. Existing
phase-name-only checkpoints are refused; use a fresh checkpoint directory and
set `checkpoint_key` to track model/data revisions. Confidence constraints
require estimated standard errors rather than assuming unknown uncertainty is
zero. `stack_proportional()` now uses exponential score utilities in both
directions; set its temperature in score units to control concentration.

## Related packages

| Package                                                   | Description                                                                                     |
| --------------------------------------------------------- | ----------------------------------------------------------------------------------------------- |
| [TradeStudy.jl](https://github.com/jcm-sci/TradeStudy.jl) | Inactive design scaffold for a possible Julia port; it does not contain a usable implementation |

## Development

See [CONTRIBUTING.md](CONTRIBUTING.md) for setup, code standards, and the PR workflow.

```bash
uv sync --extra dev
just ci          # lint → mypy --strict → pytest with coverage
just format      # auto-format
just check       # auto-fix lint
```

## Citation

If you use this package in your research, please cite:

```bibtex
@software{trade_study2026,
  author = {Macdonald, Joshua C.},
  title = {{trade-study}: Multi-Objective Trade-Study Orchestration},
  year = {2026},
  url = {https://github.com/jcm-sci/trade-study},
  version = {0.3.0},
  doi = {10.5281/zenodo.19599838},
}
```

See [CITATION.cff](CITATION.cff) for machine-readable metadata.

## License

MIT
