Metadata-Version: 2.4
Name: libby
Version: 0.3.0
Summary: Radiocarbon calibration tool with LLM-powered narrative explanations, Bayesian modelling, and GIS curve selection
Author: Mark Bouck
License: MIT
Project-URL: Homepage, https://github.com/mabo-du/libby
Project-URL: Repository, https://github.com/mabo-du/libby
Project-URL: Documentation, https://github.com/mabo-du/libby#readme
Keywords: radiocarbon,calibration,archaeology,bayesian,llm
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: fastapi>=0.115.0
Requires-Dist: uvicorn[standard]>=0.32.0
Requires-Dist: iosacal>=0.6.0
Requires-Dist: pydantic>=2.0
Requires-Dist: pydantic-settings>=2.0
Requires-Dist: aiosqlite>=0.20
Requires-Dist: numpy>=2.0
Requires-Dist: httpx>=0.28
Requires-Dist: openai>=1.0.0
Requires-Dist: reportlab>=4.5.1
Requires-Dist: matplotlib>=3.5.0
Requires-Dist: scipy>=1.10.0
Requires-Dist: requests>=2.34.2
Requires-Dist: beautifulsoup4>=4.12.0
Requires-Dist: lxml>=5.0.0
Requires-Dist: slowapi>=0.1.9
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.24; extra == "dev"
Requires-Dist: ruff>=0.7; extra == "dev"
Requires-Dist: httpx>=0.28; extra == "dev"
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.40.0; extra == "anthropic"
Provides-Extra: gemini
Requires-Dist: google-genai>=0.3.0; extra == "gemini"
Provides-Extra: mistral
Requires-Dist: mistralai>=1.0.0; extra == "mistral"
Provides-Extra: cohere
Requires-Dist: cohere>=5.5.0; extra == "cohere"
Provides-Extra: bedrock
Requires-Dist: boto3>=1.34.0; extra == "bedrock"
Provides-Extra: all-providers
Requires-Dist: anthropic>=0.40.0; extra == "all-providers"
Requires-Dist: google-genai>=0.3.0; extra == "all-providers"
Requires-Dist: mistralai>=1.0.0; extra == "all-providers"
Requires-Dist: cohere>=5.5.0; extra == "all-providers"
Requires-Dist: boto3>=1.34.0; extra == "all-providers"
Dynamic: license-file

# Libby

<p align="center">
  <img src="assets/LibbyLogo.png" alt="Libby logo" width="200"/>
</p>

<p align="center">
  <a href="https://pypi.org/project/libby/"><img src="https://img.shields.io/pypi/v/libby?color=2c6e49" alt="PyPI"></a>
  <a href="https://github.com/mabo-du/libby"><img src="https://img.shields.io/github/license/mabo-du/libby?color=2c6e49" alt="License"></a>
  <a href="https://www.python.org/"><img src="https://img.shields.io/pypi/pyversions/libby?color=2c6e49" alt="Python"></a>
</p>

**Modern radiocarbon calibration with LLM-powered narrative explanations, Bayesian modelling, and GIS curve selection.**

Libby is a full-featured web application for calibrating radiocarbon dates with
a modern Svelte 5 frontend, a Python/FastAPI backend, and unique features —
including plain-English narrative explanations of calibration results, summed
probability distributions, and a lat/lng-based curve selector for IntCal vs
SHCal vs mixed curves.

<p align="center">
  <img src="assets/libby-demo.webp" alt="Libby WebUI demo — single date calibration workflow" width="700"/>
  <br/>
  <em>Calibrating the Shroud of Turin (691 ± 31 BP) — from input to narrative result</em>
</p>

---

## Features

- **Single-date calibration** — BP age + σ → calibrated PDF chart with 1σ/2σ HPD ranges
- **Batch calibration** — paste multiple dates at once, view SPD/KDE chart and results table
- **Kernel Density Estimation (KDE)** — alternative to SPD for batch date visualisation; avoids artifactual peaks from calibration curve steepness
- **LLM-powered narratives** — plain-English explanation of every calibration result
- **Multi-language support** — narratives in 12 languages (English, French, German, Spanish, Italian, Portuguese, Japanese, Arabic, Chinese, Dutch, Swedish, Polish)
- **Multi-provider LLM support** — 13 providers: Nvidia NIM, Anthropic Claude, Google Gemini, Mistral, Cohere, DeepSeek, Groq, Together AI, Perplexity, Azure AI Foundry, AWS Bedrock, Ollama (local), Hugging Face
- **Bayesian Phase & Sequence modelling** — Metropolis-Hastings MCMC with optional seed for reproducibility, Agreement Index (A ≥ 60 = acceptable)
- **GIS curve selector** — Leaflet map pin-drop auto-suggests the appropriate calibration curve based on latitude
- **Marine reservoir correction (ΔR)** — automatic ΔR lookup from the 14CHRONO Marine Reservoir Database (calib.org); Bevington error-weighted mean, deposit-feeder filtering, 24h cache
- **Project management** — group dates by site, context, or research project
- **Summed Probability Distribution (SPD)** — combined probability across all dates
- **Taphonomic warnings** — material-aware notes (old-wood effect, collagen preservation, marine reservoir effect, etc.)
- **CRM report automation** — generate Section 106 (US), UK HER, and generic compliance reports (PDF + DOCX)
- **Export:** formatted PDF report, CRM compliance reports (PDF/DOCX), CSV, OxCal CQL2 script, publication-ready text, **publication-quality figures (PNG/SVG/TIFF)**, auto-generated journal captions
- **Multi-chart output** — tab-selectable chart types: interactive PDF curve, OxCal-style calibration plot (curve + measurement overlay), multi-panel publication figure (curve overview + SPD)
- **Dynamic curve registry** — SQLite-backed curve metadata; add new curves (IntCal25, SHCal25) by dropping .14c files into the curves directory — no code changes needed
- **Calibration curves:** IntCal20, SHCal20, Marine20 (CC BY 4.0)
- **GIS map** — OpenStreetMap with zone overlays (journal-safe, no Google Maps)
- **Docker deployment** and **pip install** packaging
- **Optional API-token authentication** — `LIBBY_API_TOKEN` for non-localhost deployments (timing-attack safe, SPA + `/health` stay public)
- **216 tests** — 162 backend pytest (153 unit + 9 live calib.org integration) + 54 frontend Vitest

---

## Tech Stack

| Layer | Technology |
|---|---|
| **Backend** | Python 3.13+ / FastAPI |
| **Frontend** | Svelte 5 / SvelteKit / Chart.js / Leaflet |
| **Calibration engine** | iosacal + direct IntCal curve data |
| **Bayesian MCMC** | Pure numpy/scipy Metropolis-Hastings (seedable, runs in dedicated ProcessPoolExecutor) |
| **KDE** | Pure numpy/scipy Gaussian KDE (Silverman's rule) |
| **Delta-R** | calib.org async proxy query (httpx) + Bevington weighted mean + asyncio.Lock rate limiter |
| **LLM** | 13 providers — Nvidia NIM, Anthropic Claude, Google Gemini, Mistral, Cohere, DeepSeek, Groq, Together AI, Perplexity, Azure AI Foundry, AWS Bedrock, Ollama, Hugging Face |
| **Auth** | Optional API-token middleware (`secrets.compare_digest`, timing-attack safe) |
| **Database** | SQLite (aiosqlite) — curves, projects, calibrations, ΔR cache |
| **PDF** | ReportLab (academic + CRM reports) |
| **DOCX** | python-docx (CRM compliance reports) |
| **Charts** | Chart.js (frontend), matplotlib (figure export + PDF reports) |
| **Curve registry** | DB-backed with auto-ingestion; IntCal25-ready |

---

## Quick Start

### Install via pip

```bash
pip install libby
libby
```

Open **http://localhost:50001** in your browser.

### Run with Docker

```bash
docker compose up -d
# Open http://localhost:50001
```

Or pull from GitHub Container Registry (once published):

```bash
docker pull ghcr.io/mabo-du/libby:latest
docker run -p 50001:50001 ghcr.io/mabo-du/libby:latest
```

To publish the Docker image yourself:

```bash
docker build -t libby .
docker tag libby:latest ghcr.io/mabo-du/libby:latest
docker push ghcr.io/mabo-du/libby:latest
```

### Build and run from source

```bash
git clone https://github.com/mabo-du/libby.git
cd libby

# Backend + frontend (single server)
uv sync
uv run uvicorn libby.main:app --reload --port 50001
# Open http://localhost:50001

# Frontend dev server (for UI development, requires backend on 50001)
cd frontend
npm install
npm run dev -- --port 50002
# Open http://localhost:50002 (points at backend API on :50001)
```

### LLM Narratives (Optional)

Libby works without any LLM — a deterministic fallback generates coherent
narratives from the numbers alone. For AI-powered narratives, pick one of
13 supported providers:

| Provider | `LIBBY_LLM_PROVIDER` | Install | API key env var |
|----------|----------------------|---------|-----------------|
| **Nvidia NIM** (default, free tier) | `nim` | — (core) | `LIBBY_NIM_API_KEY` or `NVIDIA_NIM_API_KEY` |
| **Anthropic Claude** | `anthropic` | `pip install 'libby[anthropic]'` | `LIBBY_ANTHROPIC_API_KEY` |
| **Google Gemini** | `gemini` | `pip install 'libby[gemini]'` | `LIBBY_GEMINI_API_KEY` or `GOOGLE_API_KEY` |
| **Mistral AI** | `mistral` | `pip install 'libby[mistral]'` | `LIBBY_MISTRAL_API_KEY` or `MISTRAL_API_KEY` |
| **Cohere Command-R+** | `cohere` | `pip install 'libby[cohere]'` | `LIBBY_COHERE_API_KEY` or `COHERE_API_KEY` |
| **DeepSeek** | `deepseek` | — (core, OpenAI-compat) | `LIBBY_DEEPSEEK_API_KEY` or `DEEPSEEK_API_KEY` |
| **Groq** | `groq` | — (core, OpenAI-compat) | `LIBBY_GROQ_API_KEY` or `GROQ_API_KEY` |
| **Together AI** | `together` | — (core, OpenAI-compat) | `LIBBY_TOGETHER_API_KEY` or `TOGETHER_API_KEY` |
| **Perplexity Sonar** | `perplexity` | — (core, OpenAI-compat) | `LIBBY_PERPLEXITY_API_KEY` or `PERPLEXITY_API_KEY` |
| **Azure AI Foundry** | `azure_foundry` | — (core, OpenAI-compat) | `LIBBY_AZURE_FOUNDRY_API_KEY` + `LIBBY_AZURE_FOUNDRY_BASE_URL` |
| **AWS Bedrock** | `bedrock` | `pip install 'libby[bedrock]'` | `LIBBY_BEDROCK_ACCESS_KEY` + `LIBBY_BEDROCK_SECRET_KEY` (or standard AWS chain) |
| **Ollama** (local) | `ollama` | — (core, httpx) | none — runs at `LIBBY_OLLAMA_BASE_URL` (default `http://localhost:11434`) |
| **Hugging Face** | `huggingface` | — (core, OpenAI-compat) | `LIBBY_HUGGINGFACE_API_KEY` or `HF_TOKEN` |

**Quick start with NIM (free):**
```bash
echo 'LIBBY_NIM_API_KEY=nvapi-...' > .env
```

**Quick start with Anthropic Claude:**
```bash
pip install 'libby[anthropic]'
echo 'LIBBY_LLM_PROVIDER=anthropic' >> .env
echo 'LIBBY_ANTHROPIC_API_KEY=sk-ant-...' >> .env
```

**Install all native-SDK providers at once:**
```bash
pip install 'libby[all-providers]'
```

**OpenAI-compatible providers (DeepSeek, Groq, Together, Perplexity, Azure AI Foundry, Hugging Face)** need no extra install — they use the `openai` SDK (already a core dependency) as an HTTP client. The OpenAI SDK is used purely as an HTTP client; OpenAI-the-provider is intentionally not supported per project policy.

**Excluded providers** (per project policy): OpenAI, x.ai / Grok.

Each provider also accepts a `LIBBY_<PROVIDER>_MODEL` env var to override the default model — see `config.py` for the full list.

### Authentication (Recommended for deployment)

By default Libby runs without authentication (back-compat with single-user
local dev). For any non-localhost deployment, set an API token:

```bash
# Generate a strong token
echo "LIBBY_API_TOKEN=$(openssl rand -hex 32)" >> .env

# Clients must now send it as either:
#   Authorization: Bearer <token>
#   X-API-Token: <token>
```

The SPA frontend (served from the same origin) and the `/health` endpoint
remain public so load-balancers can probe without a token.

### Database Location

The SQLite database defaults to `./libby.db` in the current working
directory. For Docker deployments it's pinned to `/data/libby.db` (in the
mounted volume). Override with:

```bash
export LIBBY_DATABASE_PATH=/var/lib/libby/data.db
```

### CORS

Allowed origins default to `localhost:5173,localhost:50001,localhost:50002`
(the Svelte dev server + production ports). Override for production:

```bash
export LIBBY_CORS_ORIGINS="https://libby.example.com,https://staging.libby.example.com"
```

### Environment Variables Reference

All env vars use the `LIBBY_` prefix. API keys fall back to the provider's
standard env var (e.g. `LIBBY_GEMINI_API_KEY` → `GOOGLE_API_KEY`).

| Variable | Default | Purpose |
|----------|---------|---------|
| `LIBBY_LLM_PROVIDER` | `nim` | One of: `nim`, `anthropic`, `gemini`, `mistral`, `cohere`, `deepseek`, `groq`, `together`, `perplexity`, `azure_foundry`, `bedrock`, `ollama`, `huggingface`, `none` |
| `LIBBY_LLM_MAX_TOKENS` | `512` | Max tokens for narrative generation |
| `LIBBY_LLM_TEMPERATURE` | `0.2` | Sampling temperature |
| `LIBBY_LLM_TIMEOUT_SECONDS` | `30` | LLM API call timeout (1–300) |
| `LIBBY_API_TOKEN` | (empty) | When set, requires `Authorization: Bearer <token>` or `X-API-Token: <token>` on every API request |
| `LIBBY_DATABASE_PATH` | `./libby.db` | SQLite database location (Docker pins to `/data/libby.db`) |
| `LIBBY_CORS_ORIGINS` | `localhost:5173,50001,50002` | Comma-separated allowed CORS origins |
| `LIBBY_TRUSTED_PROXY` | (empty) | IP of trusted reverse proxy for `X-Forwarded-For` (rate limiter) |
| `LIBBY_ENV` | `development` | Set to `production` for production behaviour |
| `LIBBY_NIM_API_KEY` | (empty) | Nvidia NIM API key (falls back to `NVIDIA_NIM_API_KEY`) |
| `LIBBY_ANTHROPIC_API_KEY` | (empty) | Anthropic Claude API key |
| `LIBBY_GEMINI_API_KEY` | (empty) | Google Gemini API key (falls back to `GOOGLE_API_KEY` / `GEMINI_API_KEY`) |
| `LIBBY_MISTRAL_API_KEY` | (empty) | Mistral AI API key (falls back to `MISTRAL_API_KEY`) |
| `LIBBY_COHERE_API_KEY` | (empty) | Cohere API key (falls back to `COHERE_API_KEY` / `CO_API_KEY`) |
| `LIBBY_DEEPSEEK_API_KEY` | (empty) | DeepSeek API key (falls back to `DEEPSEEK_API_KEY`) |
| `LIBBY_GROQ_API_KEY` | (empty) | Groq API key (falls back to `GROQ_API_KEY`) |
| `LIBBY_TOGETHER_API_KEY` | (empty) | Together AI API key (falls back to `TOGETHER_API_KEY`) |
| `LIBBY_PERPLEXITY_API_KEY` | (empty) | Perplexity API key (falls back to `PERPLEXITY_API_KEY` / `PPLX_API_KEY`) |
| `LIBBY_AZURE_FOUNDRY_API_KEY` | (empty) | Azure AI Foundry API key |
| `LIBBY_AZURE_FOUNDRY_BASE_URL` | (empty) | Azure AI Foundry endpoint URL (required when provider is `azure_foundry`) |
| `LIBBY_BEDROCK_ACCESS_KEY` | (empty) | AWS Bedrock access key (falls back to `AWS_ACCESS_KEY_ID`, or standard boto3 chain) |
| `LIBBY_BEDROCK_SECRET_KEY` | (empty) | AWS Bedrock secret key (falls back to `AWS_SECRET_ACCESS_KEY`) |
| `LIBBY_BEDROCK_REGION` | `us-east-1` | AWS region for Bedrock |
| `LIBBY_OLLAMA_BASE_URL` | `http://localhost:11434` | Ollama server URL |
| `LIBBY_HUGGINGFACE_API_KEY` | (empty) | HF token (falls back to `HF_TOKEN`) |

Each provider also accepts a `LIBBY_<PROVIDER>_MODEL` env var to override
the default model — see `src/libby/config.py` for the full list.

---

## Usage

### Single Calibration
Enter a BP age and standard deviation, select a curve, click **Calibrate**.
View the probability distribution chart, 1σ/2σ ranges, and plain-English narrative.

**Chart types** — use the tabs above the chart to switch between:
- **PDF Curve** — interactive probability density with hover crosshair and HPD fills
- **OxCal Plot** — publication-style calibration plot showing the IntCal curve with
  graduated uncertainty bands, your radiocarbon measurement overlaid, and the
  calibrated probability distribution (matching the format archaeologists expect
  in journal publications)
- **Multi-Panel** — double-column publication figure with calibration curve
  overview and summed probability distribution

**Export options** — once calibrated, download your results as:
- PNG / SVG / TIFF figures (300–600 DPI) in your selected chart type
- Journal-compliant figure caption with CC BY 4.0 curve attribution
- Publication-ready text block
- PDF report, CSV data, OxCal CQL2 script

### Batch Calibration
Click **Batch** in the nav, paste dates as CSV-like text, calibrate all at once
with Summed Probability Distribution.

### GIS Curve Selector
On the calibration page, expand **Curve selector by site location**.
Click anywhere on the map to place a pin — Libby suggests the correct curve
based on latitude and explains the reasoning.

### Marine Reservoir Correction (ΔR)
When calibrating marine samples against Marine20, Libby automatically looks up
the local ΔR (Delta-R) correction from the 14CHRONO Marine Reservoir Database
at calib.org. Enter your site coordinates and Libby:
- Queries nearby ΔR measurements within 500 km
- Filters out deposit feeders (organisms that ingest old carbonates)
- Computes the Bevington error-weighted mean and conservative uncertainty
- Displays source citations per the database's attribution requirements
- Caches results for 24 hours for speed and resilience
- Falls back to manual ΔR entry if no data is available

### CRM Reports
Generate compliance-ready radiocarbon dating appendices for:
- **US Section 106 (NHPA)** — letter-size, regulatory language
- **UK HER** — A4, NPPF-compliant formatting  
- **Generic CRM** — grey literature appendix

Output as PDF or editable DOCX. Each report includes project metadata,
a date table with calibrated 2σ ranges, contextual notes, and methodology
boilerplate with CC BY 4.0 curve citations.

### Projects
Create projects to group dates by site. Each project shows:
- All dates with full calibration data
- Summed Probability Distribution chart
- Project summary narrative
- Bayesian modelling (Phase or Sequence)
- PDF report, publication-quality figures, OxCal export, CSV export

### Bayesian Modelling
With 2+ dates in a project:
- **Phase model** — estimates start/end boundaries for an activity period
- **Sequence model** — applies stratigraphic ordering constraints
- Agreement Index (A) per date, following OxCal's convention (A ≥ 60 = acceptable)
- **Optional seed** — pass `seed` in the `/model` request body for reproducible posterior samples (essential for publication verification)

### Lab Report Import
Navigate to **Import**, paste CSV/TSV from your radiocarbon lab.
Auto-detects columns, handles "3000±30" embedded sigma format,
previews all rows, then batch-calibrates with one click.

---

## Project Structure

```
libby/
├── src/libby/              # Python backend
│   ├── main.py             # FastAPI app factory and routes
│   ├── calibration.py      # iosacal integration (async, curve-resolver)
│   ├── bayesian.py         # MCMC Phase/Sequence models (validated)
│   ├── kde.py              # Kernel Density Estimation (Silverman's rule)
│   ├── reservoir.py        # ΔR marine correction (calib.org proxy)
│   ├── crm.py              # CRM report generation (PDF + DOCX)
│   ├── narrative.py        # LLM + fallback narrative generation
│   ├── models.py           # Pydantic schemas (curve field is free-form str)
│   ├── database.py         # SQLite (projects, calibrations, curves, ΔR cache) — path via LIBBY_DATABASE_PATH
│   ├── curve_registry.py   # DB-backed dynamic curve discovery
│   ├── gis.py              # Curve suggestion by latitude
│   ├── spd.py              # Summed Probability Distribution
│   ├── export.py           # CSV/text formatters
│   ├── auth.py             # Optional API-token middleware
│   ├── rate_limiter.py     # Per-IP sliding-window rate limiter
│   ├── figures/            # Multi-chart figure generation (PDF, OxCal-style, multi-panel, KDE)
│   ├── figure.py           # Backward-compat shim → figures/
│   ├── oxcal.py            # OxCal CQL2 script export (forward-compatible)
│   ├── report.py           # Academic PDF report generation
│   ├── importer.py         # Lab report CSV/TSV parser
│   └── providers/          # LLM providers (13 supported — see providers/__init__.py)
├── scripts/
│   └── ingest_curve.py     # CLI tool for registering .14c curve files
├── frontend/src/           # Svelte 5 frontend
│   ├── routes/             # Pages (/, /batch, /import, /projects)
│   └── lib/                # Components (form with ΔR panel, chart, map, etc.)
├── tests/                  # pytest (216 tests: 207 unit + 9 live calib.org)
├── docs/research/          # Deep research reports + risk assessments
├── Dockerfile              # Multi-stage Docker build
├── docker-compose.yml      # Docker deployment
└── MANIFEST.in             # PyPI package manifest
```

---

## Calibration Curves

This tool uses the internationally ratified calibration curves:
- **IntCal20** — Northern Hemisphere terrestrial (Reimer et al. 2020)
- **SHCal20** — Southern Hemisphere terrestrial (Hogg et al. 2020)
- **Marine20** — Global marine (Heaton et al. 2020)

All curves are CC BY 4.0 licensed. Calibration engine: iosacal (GPLv3).

---

## Licence

MIT License. See [LICENSE](LICENSE) for details.

## Documentation

- **[User Guide](USER_GUIDE.md)** — step-by-step calibration workflow, curve selection, ΔR, Bayesian modelling, exports, troubleshooting
- **[Changelog](CHANGELOG.md)** — versioned release notes (latest: 0.3.0)
- **[Research notes](docs/research/)** — deep dives on calibration curve rendering, ΔR risk assessment, multi-chart architecture, Bayesian maths

## Citation

If you use Libby in published research, please cite:
- IntCal20: Reimer et al. 2020, *Radiocarbon* 62(4)
- SHCal20: Hogg et al. 2020, *Radiocarbon* 62(4)
- Marine20: Heaton et al. 2020, *Radiocarbon* 62(4)
