Metadata-Version: 2.4
Name: fable-pyculator
Version: 0.1.0a5
Summary: FABLE Calculator-specific notebook controls for Modelwright-generated Python models.
Author: UBC FRESH Lab
License-Expression: MIT
Project-URL: Homepage, https://github.com/UBC-FRESH/fable-pyculator
Project-URL: Documentation, https://ubc-fresh.github.io/fable-pyculator/
Project-URL: Repository, https://github.com/UBC-FRESH/fable-pyculator
Project-URL: Issues, https://github.com/UBC-FRESH/fable-pyculator/issues
Project-URL: Changelog, https://github.com/UBC-FRESH/fable-pyculator/blob/main/CHANGE_LOG.md
Keywords: fable,spreadsheet-models,jupyter,modelwright,scenario-analysis
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: modelwright[notebook]>=0.1.0a9
Requires-Dist: openpyxl>=3.1
Requires-Dist: pandas>=2
Requires-Dist: PyYAML>=6
Provides-Extra: web
Requires-Dist: fastapi>=0.110; extra == "web"
Requires-Dist: uvicorn>=0.29; extra == "web"
Provides-Extra: browser
Requires-Dist: playwright>=1.45; extra == "browser"
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == "dev"
Requires-Dist: fastapi>=0.110; extra == "dev"
Requires-Dist: freshforge<0.2,>=0.1.0a5; extra == "dev"
Requires-Dist: httpx>=0.27; extra == "dev"
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: ruff<0.16,>=0.8; extra == "dev"
Requires-Dist: sphinx>=7; extra == "dev"
Requires-Dist: sphinx-rtd-theme>=2; extra == "dev"
Requires-Dist: twine>=5; extra == "dev"
Provides-Extra: docs
Requires-Dist: sphinx>=7; extra == "docs"
Requires-Dist: sphinx-rtd-theme>=2; extra == "docs"
Provides-Extra: freshforge
Requires-Dist: freshforge<0.2,>=0.1.0a5; extra == "freshforge"
Provides-Extra: notebook
Requires-Dist: ipykernel>=6; extra == "notebook"
Requires-Dist: ipywidgets>=8; extra == "notebook"
Requires-Dist: matplotlib>=3.8; extra == "notebook"
Provides-Extra: quality
Requires-Dist: ruff<0.16,>=0.8; extra == "quality"
Provides-Extra: release
Requires-Dist: build>=1.2; extra == "release"
Requires-Dist: twine>=5; extra == "release"
Provides-Extra: test
Requires-Dist: fastapi>=0.110; extra == "test"
Requires-Dist: freshforge<0.2,>=0.1.0a5; extra == "test"
Requires-Dist: httpx>=0.27; extra == "test"
Requires-Dist: pytest>=8; extra == "test"
Dynamic: license-file

# FABLE Pyculator

FABLE Pyculator is a FABLE Calculator-specific notebook layer built on top of
Modelwright-generated Python models.

The goal is to keep Modelwright generic while collecting FABLE-C conventions in one public package:

- discover likely FABLE scenario input controls from workbook structure;
- declare those controls as named scenario parameters;
- expose a Jupyter-friendly control surface for scenario changes;
- run a generated Modelwright model with the selected scenario inputs;
- render standard FABLE outputs as pandas tables and matplotlib figures.

This repository does not store original FABLE Calculator workbooks, decompressed generated Python
clones, or raw validation outputs. Keep those under ignored `tmp/` paths. The public 2021 generated
model is the one explicit exception: it is tracked as a compressed, validation-backed artifact under
`examples/fable_2021/`.

## Development Workflow

This project uses the same agent-assisted workflow as Modelwright:

- read `AGENTS.md`, `ROADMAP.md`, and `CHANGE_LOG.md` before project-shaping work;
- keep the current plan in `ROADMAP.md`;
- record completed deliverables in `CHANGE_LOG.md`;
- use `planning/` for focused investigations and contracts;
- keep source workbooks, generated models, extracts, logs, and validation reports under ignored
  `tmp/`;
- once the public GitHub repo and `gh` access exist, map roadmap phases to GitHub parent issues and
  roadmap tasks to child issues;
- close every phase through a PR back to `main`, with Sphinx docs rebuilt in CI and deployed to
  GitHub Pages after merge.

The local remote is expected to be:

```text
https://github.com/UBC-FRESH/fable-pyculator.git
```

## Install For Development

Recommended VSCode/Jupyter setup from the `fable-pyculator` repo root:

```bash
scripts/bootstrap_dev_env.sh
```

Then select this interpreter as the VSCode notebook kernel:

```text
.venv/bin/python
```

Manual equivalent:

```bash
python -m venv .venv
.venv/bin/python -m pip install --upgrade pip
.venv/bin/python -m pip install -e '.[dev,notebook,docs]'
```

## Optional Web Portal

The package includes an optional local portal over the same artifacts the notebook workflow uses.
It is a FastAPI service plus a dependency-free vanilla JavaScript SPA; there is no Node build and no
second modelling engine. Install the optional dependencies and start it from the repository root:

```bash
.venv/bin/python -m pip install -e '.[web]'
.venv/bin/fable-pyculator web --root . --port 8766
```

Open `http://127.0.0.1:8766`. The portal offers:

- **Overview** and **Artifacts**: real file readiness, compressed/generated model state, and tracked
  scenario bundles.
- **Workbench**: prepare a workbook version, choose one option per FABLE selection control, and run
  the matching generated model in a background job.
- **Results**: headline KPI cards, time-series charts, every rendered output table, and CSV export.
- **Compare**: align saved runs by year and inspect selection differences.
- **Validation**: compact validation evidence from
  `tmp/validation-evidence/fable-<version>/summary.json`.
- **Assistant**: optional local `opencode` bridge, enabled only with `--allow-agent`. A green
  **+ Ask** pill opens an overlay with markdown rendering, persistent chat sessions, rename/search/
  pin/archive controls, Markdown export, auto-titles, session continuation, tool-action chips, and
  `Enter`-to-send / `Shift+Enter`-for-newline behavior. New
  chats are grounded by a generated cheat sheet that points to the FABLE Pyculator and Modelwright
  docs, APIs, and served model instances.

Preparation needs the source workbook under the ignored
`tmp/private-workbooks/<version>_Open_FABLECalculator.xlsx` path and either the decompressed or
tracked compressed generated model. The portal decompresses the approved archive when needed,
verifies working models against the tracked checksum sidecars, and re-materializes a stale working
model before preparation. It caches workbook discovery and saved runs under ignored `runtime/portal/`
paths. Default headline
curation is version-tolerant: 2020/2021 expose FOOD, LAND, GHG, and WATER; the 2024 Canada workbook
exposes FOOD, LAND, and GHG because it has no `TotalResultsWF` water-footprint table. The published
2024 `LAND[TOTAL]` formula contains `#REF!`; the portal can display a clearly labelled component sum
that reproduces the workbook's valid `calc_land_cor[TotallandFin]` value. The tracked 2024 artifact has
been repaired and a fresh Modelwright `0.1.0a9` regeneration from the repaired workbook reproduces it
byte-for-byte. The GHG headline is derived from the declared `OUTPUT-8`
components (`CropCO2`, `LiveN2O`, `CropN2O`, `LiveCH4`, `CropCH4`, `DeforCO2`, `OtherLandCO2`,
land-sequence, and `BiofuelSavings`) because `TotalCO2e` is a `CALC` cell rather than a generated
output. Values are generated-model outputs, so they can differ from workbook-cached totals where an
individual component model differs. The 2024 series notes that `LandSeqAffor` is excluded because it
is not a declared `OUTPUT-8` cell in that artifact. To repair a local copy of the 2024 workbook itself, run:

```bash
.venv/bin/python scripts/repair_fable_2024_land_total.py
```

To reproduce the tracked 2024 generated model from the repaired workbook, install Modelwright
`0.1.0a9` or newer and run:

```bash
.venv/bin/python scripts/regenerate_fable_2024_model.py
```

The check translates all 410,299 formulas, closes all 10,274 declared outputs, generates the model
with zero diagnostics, and compares it byte-for-byte with the tracked archive. Real 2021 preparation
takes about two minutes on a warm local checkout and a scenario run takes about a minute; both run
as background jobs with progress reporting.

The portal is a presentation layer. It does not recompute a FABLE result in the browser, edit source
workbooks, or provide authentication. A non-loopback bind requires `--allow-remote`.

The Workbench can optionally extend a run to **2070** using a linear trend over 2035–2050. This is
a clearly labelled presentation-layer projection, not native FABLE output. Native results end at
2050. Multi-row-per-year tables are extended group-by-group when a text identity column is available; tables without a usable trend stay native with an explicit note.

Results pages show run-scoped **data-quality notes** for caveats such as a 2070 extension, the 2024
land component fallback, missing 2024 water data, GHG-vs-cached divergence, and tables that were not
extended.

Assistant grounding and browser checks:

```bash
# Fast structural grounding checks (safe for CI)
.venv/bin/python scripts/evaluate_assistant_grounding.py

# Opt-in live assistant questions with source/path assertions
.venv/bin/python scripts/evaluate_assistant_grounding.py --live

# Opt-in Chromium smoke for the overlay, Enter/Shift+Enter, markdown, and chat reload persistence
.venv/bin/python -m pip install -e '.[browser]'
.venv/bin/python -m playwright install chromium
.venv/bin/python scripts/browser_smoke.py --root . --server-python .venv/bin/python --live-agent
```

Deployment:

```bash
# Local
scripts/run_portal.sh

# Authenticated remote bind
FABLE_PORTAL_HOST=0.0.0.0 \
FABLE_PORTAL_ALLOW_REMOTE=1 \
FABLE_PORTAL_AUTH_USER=operator \
FABLE_PORTAL_AUTH_PASSWORD='a-long-random-password' \
scripts/run_portal.sh

# Jupyter/codeserver proxy prefix
FABLE_PORTAL_ROOT_PATH=/user/me/proxy/8768 \
FABLE_PORTAL_ALLOW_REMOTE=1 \
scripts/run_portal.sh
```

Example systemd and nginx files are under `deploy/`; see `docs/guides/web-deployment.rst`.



The current Modelwright benchmark metadata points to this public FABLE Calculator Dropbox folder:

```text
https://www.dropbox.com/scl/fo/ndgldfnq81v794mm8yebe/ADusMz23xtmYKDXoEkiNtJM?rlkey=d87qhjf5zd0pcowd5pfl5qdu7&st=qijm4tta&e=2&dl=0
```

In this checkout, the three public FABLE-C workbooks are stored under `tmp/private-workbooks/`.
They are ignored local benchmark artifacts. Verify them with:

```bash
sha256sum -c benchmarks/fable-calculator/checksums.sha256
```

The public 2020 FABLE Calculator documentation PDF is tracked under `reference/fable-calculator/`.

The core FABLE-C output data surfaces are the consecutive workbook sheets `FOOD`, `PRODUCTION`,
`TRADE`, `BIODIVERSITY`, `LAND`, `GHG`, and `WATER`.

## Early Notebook Shape

```python
from fable_pyculator import (
    FableCalculatorSpec,
    OutputIndicator,
    ScenarioControlSurface,
    ScenarioParameter,
    discover_output_tables,
    discover_scenario_definition_tables,
    discover_selection_controls,
    output_table_frame,
    run_scenario,
    scenario_definition_table_frame,
    scenario_definition_tables_for_location,
)

spec = FableCalculatorSpec(
    parameters=[
        ScenarioParameter(name="ambition", label="Scenario ambition", cell_ref="SCENARIOS selection!D20"),
    ],
    outputs=[
        OutputIndicator(name="ghg", label="GHG emissions", cell_ref="SCENARIOS selection!D22", unit="MtCO2e"),
    ],
)

controls = ScenarioControlSurface(spec)
controls

result = run_scenario(generated_model, spec, controls.values())
result.outputs
```

For real FABLE-C workbooks, discover the high-level selection controls, scenario-definition tables,
and output tables:

```python
selection_controls = discover_selection_controls("tmp/private-workbooks/2020_Open_FABLECalculator.xlsx")
definition_tables = discover_scenario_definition_tables("tmp/private-workbooks/2020_Open_FABLECalculator.xlsx")
output_tables = discover_output_tables("tmp/private-workbooks/2020_Open_FABLECalculator.xlsx")
spec = FableCalculatorSpec(
    selection_controls=selection_controls,
    scenario_definition_tables=definition_tables,
    output_tables=output_tables,
)

scenario_definition_table_frame(spec, "DietTarget")
scenario_definition_tables_for_location(spec, "S.3")

run = run_scenario(generated_model, spec, {"gdp_scen": "SSP1"})
output_table_frame(run, "ghg_resultsghg")
output_table_frame(run, "ghg_resultsghg", column_flavour_tags="OUTPUT-8")
output_table_frame(run, "ghg_resultsghg", column_flavour_tags="DATA")
output_table_frame(run, "ghg_resultsghg", column_flavour_tags="OUTPUT-*")
```

The first 2020 notebook loop helper uses ignored local artifacts by default:

```text
tmp/private-workbooks/2020_Open_FABLECalculator.xlsx
tmp/generated-models/fable-2020/generated_fable_2020_model.py
```

```python
from fable_pyculator import run_2020_notebook_loop

result = run_2020_notebook_loop({"gdp_scen": "SSP1"})
result.output_tables.keys()
result.output_tables["ghg_resultsghg"]
result.headline_frames["ghg_total_co2e"]
result.headline_figures["ghg_total_co2e"]
```

The 2021 helper uses separate ignored local artifacts and does not fall back to the 2020 generated
model:

```text
tmp/private-workbooks/2021_Open_FABLECalculator.xlsx
tmp/generated-models/fable-2021/generated_fable_2021_model.py
```

```python
from fable_pyculator import run_2021_notebook_loop

result = run_2021_notebook_loop({"gdp_scen": "SSP1"}, include_figures=False)
```

The 2021 example notebook can materialize
`examples/fable_2021/generated_fable_2021_model.py.xz` into the ignored generated-model path above.
Phase 8 validated that generated model against the public 2021 workbook with 281,922 comparable
outputs, 281,922 matches, and 0 mismatches.

To smoke-test the same Abdulateef-facing notebook loop path after restoring local artifacts, run:

```bash
.venv/bin/python scripts/smoke_2021_notebook_loop.py --json
```

The smoke check renders default `OUTPUT-*` output tables, reports any skipped non-matching tables,
and verifies that the `trade_resultstrade` context columns `PRODUCT` and `YEAR` are populated.
Those context values are display context from cached workbook table values, not additional
generated-model equivalence evidence. On restored 2021 artifacts this smoke can take a few minutes.

To rebuild a generated model from a local source workbook with the FreshForge/Modelwright workflow,
start with plan-only preparation. The generic command defaults to the public 2021 path convention:

```bash
.venv/bin/python scripts/build_fable_model.py
```

To compare output-ref boundaries before choosing a rebuild target, run:

```bash
.venv/bin/python scripts/compare_fable_output_ref_strategies.py --json
```

Use `--workbook-version 2020` or `--workbook-version 2021` to choose a workbook version by
convention. Add `--include-matrix` or `--matrix-plan` to write or inspect a FreshForge matrix across
the default strategy cases; use `--matrix-run` only after reviewing the generated matrix and local
artifacts. The older `scripts/build_fable_2021_model.py` command remains a 2021 shortcut. Use
`--run` only after reviewing the generated `tmp/generated-models/fable-YYYY/` workflow artifacts.

FABLE Pyculator discovers wrapper metadata and renders notebook surfaces. It does not currently
generate Modelwright `contract.json`, `expressions.json`, or `constants.json` files from a FABLE
workbook. See `docs/guides/generated-model-artifacts.rst` for the generated-model artifact boundary
and the 2020/2021 path contract.

By default the loop renders every discovered output table and every curated headline frame from the
single generated-model run. Pass `output_table_names` or `headline_series_names` only when you want a
smaller rendered subset.

To repeat that loop across named selection-control scenarios, use a JSON or YAML scenario bundle:

```bash
.venv/bin/python scripts/run_fable_scenario_bundle.py \
  --bundle examples/scenario-bundles/fable_2021_ssp_demo.yaml \
  --dry-run \
  --json
```

Remove `--dry-run` after the matching workbook and generated model are restored locally. Bundle
outputs are written under ignored `tmp/scenario-runs/fable-YYYY/<bundle-id>/` paths. See
`docs/guides/scenario-bundles.rst` for the bundle schema and artifact layout.

To validate conservative `SCENARIOS definition` table edits without mutating the source workbook,
use a scenario-definition patch:

```bash
.venv/bin/python scripts/validate_fable_scenario_definition_patch.py \
  --patch examples/scenario-definition-patches/fable_2021_diet_target_demo.yaml \
  --workbook-version 2021 \
  --json
```

Valid patches target non-formula `DIRECT` cells and become generated-model input overrides. Read-only
role tags such as `SCEN`, `DATA-*`, and `CALC` fail before model execution. See
`docs/guides/scenario-definition-editing.rst`.

When repeated bundle runs need explicit graph planning, namespace-isolated artifacts, and a compact
FreshForge run summary, use the FreshForge-backed path:

```bash
.venv/bin/python scripts/run_fable_scenario_bundle.py \
  --bundle examples/scenario-bundles/fable_2021_ssp_demo.yaml \
  --freshforge-plan \
  --json
```

Switch to `--freshforge-run --run-namespace scenario/demo` only after the plan and local artifacts
look right. See `docs/guides/scenario-bundle-freshforge-orchestration.rst`.

To treat each scenario as a FreshForge matrix case, use:

```bash
.venv/bin/python scripts/run_fable_scenario_bundle.py \
  --bundle examples/scenario-bundles/fable_2021_ssp_demo.yaml \
  --freshforge-matrix-plan \
  --json
```

Switch to `--freshforge-matrix-run` only after the matching workbook and generated model are restored.

To package compact validation evidence from existing local generated-model artifacts, use:

```bash
.venv/bin/python scripts/package_fable_validation_evidence.py --json
```

This writes sanitized summaries under `tmp/validation-evidence/fable-YYYY/`. Missing local artifacts
produce a skipped summary by default; use `--require-artifacts` when absence should fail. See
`docs/guides/validation-evidence-packaging.rst` for the evidence status and claim boundary.

For the opt-in benchmark workflow wrapper, which can package evidence, prepare a FreshForge plan, or
explicitly run the restored local benchmark workflow, use:

```bash
.venv/bin/python scripts/run_fable_benchmark_evidence.py --mode evidence-only --json
```

Switch to `--mode freshforge-plan` or `--mode freshforge-run` only when local artifacts are restored
and the run is intentional. The manual GitHub workflow uploads only compact summaries under
`tmp/validation-evidence/**`; it does not upload private workbooks, generated models, raw reports, or
raw generated values. See `docs/guides/benchmark-evidence-workflow.rst`.

To package compact evidence from an explicit FreshForge output-ref strategy matrix run, use:

```bash
.venv/bin/python scripts/package_fable_matrix_evidence.py --workbook-version 2021 --json
```

That command expects a matrix run summary such as
`tmp/strategy-comparisons/fable-2021/matrix-run-summary.json` and writes sanitized matrix summaries
under `tmp/validation-evidence/fable-2021/matrix/`. See
`docs/guides/fable-2021-benchmark-matrix-evidence-cookbook.rst`.

Tracked notebook example:

```text
examples/notebooks/fable-pyculator-2020-loop.ipynb
examples/notebooks/fable-pyculator-2021-loop.ipynb
examples/notebooks/fable-pyculator-2021-scenario-definition-patch.ipynb
examples/notebooks/fable-pyculator-2021-freshforge-build-plan.ipynb
examples/notebooks/fable-pyculator-2021-freshforge-run.ipynb
```

The 2020 notebook is intentionally committed after a successful 2020 benchmark run so GitHub can
render the example tables and figure directly in the browser. The 2021 notebook is a runnable
artifact-wiring template: it still requires the ignored local workbook, but it can restore the
validated generated model from the tracked compressed 2021 archive. The 2021 notebook prints a
rendered/skipped table summary and contains an explicit `trade_resultstrade` context-column proof for
alpha testers. The FreshForge notebooks show
how to validate a scenario-definition patch without mutating the workbook. The FreshForge notebooks
show how to rebuild the 2021 model from the source workbook: one notebook plans the graph, and the
run companion gates the full FreshForge/Modelwright build behind `RUN_FRESHFORGE = False`.

In VSCode, point the notebook kernel at the `.venv` created in the `fable-pyculator` repo root.
The notebook setup cell prints the active environment prefix and warns if the selected kernel does
not appear to be that repo-local `.venv`.

The Sphinx guide expands this into a full workflow under
`docs/guides/2020-notebook-workflow.rst`, with validation boundaries recorded in
`docs/guides/validation-scope.rst`.

`fable-pyculator` is pre-release. The current alpha line is `0.1.0a5`; alpha releases must not be
described as stable public API compatibility, full editable scenario-definition widgets, production
readiness, or arbitrary country-calculator support. Generated-model equivalence claims are limited
to the exact 2020, 2021, and 2024 evidence recorded in the docs. The `0.1.0a5` release adds the
optional web portal, approved-artifact checksum verification, authenticated deployment staging, and
the GitHub issue-filing assistant workflow on top of the existing notebook and FreshForge surfaces.

The public API is intentionally small while the FABLE-specific conventions are being discovered from
real country calculators.

## Build Docs

```bash
.venv/bin/python -m pip install -e '.[dev,notebook,docs]'
.venv/bin/sphinx-build -b html docs _build/html -W
```

The `Docs Pages` GitHub Actions workflow builds Sphinx docs on pull requests to `main` and deploys
the built HTML to GitHub Pages after merges to `main`.

GitHub Pages URL:

```text
https://ubc-fresh.github.io/fable-pyculator/
```

See `docs/guides/release-deployment.rst` for the release and deployment runbook.

## Release Checks

Build and inspect release artifacts locally:

```bash
scripts/check_release_artifacts.sh
```

Release checks write build outputs under ignored `tmp/release-checks/`.

## Default Checks

```bash
.venv/bin/python -m ruff check .
.venv/bin/python -m pytest
.venv/bin/sphinx-build -b html docs _build/html -W
.venv/bin/python scripts/verify_docs_theme.py _build/html
scripts/check_release_artifacts.sh
sha256sum -c benchmarks/fable-calculator/checksums.sha256
```
