Metadata-Version: 2.4
Name: cjeu-py
Version: 0.6.0
Summary: Python toolkit for empirical research on the Court of Justice of the European Union
Author: Niccolò Ridi
License-Expression: MIT
Project-URL: Repository, https://github.com/niccoloridi/cjeu-py
Project-URL: Issues, https://github.com/niccoloridi/cjeu-py/issues
Project-URL: Changelog, https://github.com/niccoloridi/cjeu-py/blob/main/CHANGELOG.md
Keywords: CJEU,Court of Justice,European Union,case law,citation analysis,CELLAR,empirical legal studies
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Text Processing
Classifier: Operating System :: OS Independent
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas>=2.0
Requires-Dist: numpy>=1.24
Requires-Dist: pyarrow>=14.0
Requires-Dist: requests>=2.31
Requires-Dist: beautifulsoup4>=4.12
Requires-Dist: lxml>=4.9
Requires-Dist: aiohttp>=3.9
Requires-Dist: SPARQLWrapper>=2.0
Requires-Dist: rdflib>=7.0
Requires-Dist: regex>=2023.0
Requires-Dist: tqdm>=4.65
Requires-Dist: jsonlines>=4.0
Requires-Dist: networkx>=3.1
Provides-Extra: llm
Requires-Dist: google-genai>=1.0; extra == "llm"
Provides-Extra: openai-llm
Requires-Dist: openai>=1.0; extra == "openai-llm"
Provides-Extra: analysis
Requires-Dist: scipy>=1.11; extra == "analysis"
Requires-Dist: scikit-learn>=1.3; extra == "analysis"
Requires-Dist: statsmodels>=0.14; extra == "analysis"
Provides-Extra: viz
Requires-Dist: matplotlib>=3.7; extra == "viz"
Requires-Dist: seaborn>=0.12; extra == "viz"
Provides-Extra: gui
Requires-Dist: streamlit>=1.30; extra == "gui"
Provides-Extra: mcp
Requires-Dist: mcp>=1.0; extra == "mcp"
Provides-Extra: all
Requires-Dist: cjeu-py[analysis,gui,llm,openai-llm,viz]; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=7.4; extra == "dev"
Requires-Dist: ruff>=0.4; extra == "dev"
Requires-Dist: mypy>=1.8; extra == "dev"
Requires-Dist: types-requests; extra == "dev"
Requires-Dist: build>=1.0; extra == "dev"
Dynamic: license-file

<p align="center">
  <img src="https://raw.githubusercontent.com/niccoloridi/cjeu-py/main/docs/logo.svg" alt="cjeu-py" width="400">
</p>

<p align="center">
  A Python toolkit for empirical research on the Court of Justice of the European Union (CJEU).<br>
  Collects structured data from the EU's CELLAR endpoint, parses judgment texts, extracts case-law citations, and builds research-ready datasets.
</p>

<p align="center">
  <a href="https://pypi.org/project/cjeu-py/"><img src="https://img.shields.io/pypi/v/cjeu-py" alt="PyPI"></a>
  <a href="https://pypi.org/project/cjeu-py/"><img src="https://img.shields.io/pypi/pyversions/cjeu-py" alt="Python"></a>
  <a href="https://github.com/niccoloridi/cjeu-py/blob/main/LICENSE"><img src="https://img.shields.io/github/license/niccoloridi/cjeu-py" alt="License"></a>
  <a href="https://github.com/niccoloridi/cjeu-py/actions/workflows/test.yml"><img src="https://img.shields.io/github/actions/workflow/status/niccoloridi/cjeu-py/test.yml?label=tests" alt="Tests"></a>
</p>

## Why Python

cjeu-py is designed to be idiomatically Pythonic: `pip install cjeu-py`, a CLI entry point, pandas DataFrames in and out, standard logging, and a flat module structure you can import piecemeal. Data flows through the pipeline as Parquet files and JSONL logs, so each stage is independently inspectable and resumable. The optional LLM layer returns structured JSON output rather than hand-coded labels, so its results merge straight back into the pandas pipeline.

The goal is a toolkit that fits naturally into the workflows that computational social scientists and NLP researchers already use – Jupyter notebooks, pandas, scikit-learn, HuggingFace – without requiring a separate ecosystem.

## What it does

| Stage | What | How |
|-------|------|-----|
| **Collect** | Case metadata, citation networks, subject matter (4 taxonomies), procedural links, legislation links, AG opinion links, academic citations, referring court details. Configurable document types: judgments, orders, AG opinions, and more via `--doc-types` | CELLAR SPARQL endpoint (CDM ontology) |
| **Download** | Full judgment and AG opinion texts | CELLAR REST API (content negotiation, XHTML) |
| **Parse** | Court composition, parties, representatives, procedural dates | XHTML header parser (structural, not heuristic) |
| **Derive** | Judge assignments, case names, operative parts | Flattened from parsed headers |
| **Extract** | Case-law citations resolved to CELEX/ECLI with a confidence tier (A/B/C/U); footnote-aware, since AG opinions cite mostly in footnotes | CourtLink hyperlinks → CELLAR-catalogue-guided search → 13-pattern regex safety net → optional party-name enrichment (`--enrich`) |
| **Scrape** *(optional)* | Judge biographical data (all current and former members) | Curia.europa.eu + LLM structured extraction (requires API key) |
| **Curia** | Live case-law search, pending cases, procedural records, hearing calendar, doctrine notes | InfoCuria JSON API (`cjeu-py curia search\|procedures\|enrich`) |
| **Search** | Full-text, party, citation graph, topic, legislation, and live headnote queries (CELLAR or Curia) | `cjeu-py search text "proportionality"` |
| **Network** | Interactive citation network with centrality metrics, community detection, subject/procedure/year filters | Self-contained HTML (D3.js), GEXF (Gephi / Gephi Lite), D3 JSON |
| **Browse** | List tables, preview rows, column info, statistics, judgment texts | `cjeu-py browse decisions --stats` |
| **Export** | All tables as CSV or Excel | `cjeu-py export --format csv` |
| **GUI** | Browser-based interface for download, browse, search, ontology/headnotes, and network export | Streamlit (`cjeu-py gui`, `pip install 'cjeu-py[gui]'`) |

## Interactive citation network

`export-network` builds a directed citation graph from cached pipeline data (nodes = cases, edges = citing → cited) and exports it in three formats:

- **HTML** – a self-contained interactive visualisation (D3.js force-directed graph) that opens directly in a browser, with no server or dependencies required
- **GEXF** – for Gephi desktop and [Gephi Lite](https://gephi.org/gephi-lite/), with all node attributes (centrality, subjects, formation, procedure) preserved as typed attributes
- **D3 JSON** – for custom web visualisations or programmatic analysis

Each node carries PageRank, betweenness centrality, in-degree, out-degree, and a Louvain community assignment. The graph includes **external cited cases** (authorities cited by your downloaded decisions but not part of the downloaded set) to preserve full citation neighbourhoods. External nodes have year and court derived from the CELEX number; run `enrich-network` (below) to add full metadata.

The HTML export provides:

- **Interactive controls** – node sizing (by any centrality metric, with user-definable min/max radius), node colouring (community, procedure type, year, court, or formation), edge thickness scaling, toggleable community hull shading
- **Filters** – year range slider, court checkboxes (CJ/GC/CST), subject matter checkboxes (36 case-law subject codes with human-readable labels, sorted by frequency), procedure type checkboxes, with All/None toggles
- **Detail sidebar** – click any node to see collapsible sections: case metadata (CELEX, ECLI, date, court, formation, procedure, judge-rapporteur, AG, subjects), centrality metrics, procedural links (joined cases, appeals, interveners, annulled acts), legislation links, academic citations, and citing/cited-by lists. Available sections depend on which metadata tiers have been downloaded

<p align="center">
  <img src="https://raw.githubusercontent.com/niccoloridi/cjeu-py/main/docs/network_screenshot.png" alt="Interactive citation network" width="800">
</p>

A pre-built example is available at [`examples/grand_chamber_network.html`](examples/grand_chamber_network.html) - download and open in a browser. It contains the 500 most central Grand Chamber cases (by PageRank) plus cases they cite.

**Handling external nodes:** By default, nodes outside your downloaded set appear with limited metadata. Two options:

```bash
# Option 1: Enrich external nodes by fetching their metadata from CELLAR
# (one-time, cached - fetches ECLI, date, court, formation)
cjeu-py enrich-network

# Then export as usual - external nodes now have full metadata
cjeu-py export-network --format html --max-nodes 500

# Option 2: Restrict to downloaded decisions only (no external nodes)
cjeu-py export-network --format html --internal-only
```

```bash
# Full network as interactive HTML
cjeu-py export-network --format html

# Cap at 500 most central cases (PageRank) for fast rendering
cjeu-py export-network --format html --max-nodes 500

# Filter by subject and date
cjeu-py export-network --format html --topic "competition" --date-from 2010-01-01

# GEXF for Gephi
cjeu-py export-network --format gexf --formation GRAND_CH
```

Networks above 5,000 nodes trigger a performance warning; above 10,000 a stronger warning suggests using `--max-nodes` or filters. Gephi desktop handles large networks without issue.

## Streamlit GUI

A browser-based GUI covers the core data-collection and exploration workflow without a terminal. It ships in the package - install the `gui` extra and launch it as a CLI command:

```bash
pip install 'cjeu-py[gui]'
cjeu-py gui
```

<p align="center">
  <img src="https://raw.githubusercontent.com/niccoloridi/cjeu-py/main/docs/gui_screenshot.png" alt="cjeu-py Streamlit GUI" width="800">
</p>

The GUI provides five tabs:

- **Download** - base and extended metadata from CELLAR, with filters for court, date range, formation, judge-rapporteur, AG, and document types (judgments, orders, AG opinions, and more). Also: network enrichment and text download.
- **Browse Data** - select any Parquet table, preview it as a dataframe, view descriptive statistics, search judgment texts by CELEX, and download individual tables or all tables as a zip archive (CSV or Excel).
- **Search** - all 8 search modes (text, headnote, party, citing, cited-by, topic, legislation, list) with result tables and CSV export.
- **Ontology & Headnotes** - download the CELLAR subject-matter taxonomy, browse/filter it, and search case headnotes live via SPARQL.
- **Network** - filter and export citation networks as GEXF or interactive HTML, with an embedded preview.

Citation extraction, the InfoCuria (`curia`) commands, and the MCP server are CLI-only for now.

## See it in action

A real session, from an empty data directory to an Excel-ready dataset with
live procedural records from the new CURIA portal:

<p align="center">
  <img src="https://raw.githubusercontent.com/niccoloridi/cjeu-py/main/docs/cli_demo.gif" alt="cjeu-py CLI session (animated)" width="800">
</p>

## Quick start

```bash
# Install from PyPI
pip install cjeu-py

# Or install with all optional dependencies (LLM-assisted judge-bio extraction, statistical analysis, visualisation)
pip install cjeu-py[all]

# Set your Gemini API key (only needed for LLM-assisted judge-bio extraction)
export GEMINI_API_KEY="your-key-here"

# Download metadata from CELLAR (cached to disk – runs once, then instant)
cjeu-py download-cellar --max-items 100

# Filter by court, formation, judge, date range
cjeu-py download-cellar --court CJ --formation GRAND_CH --date-from 2020-01-01
cjeu-py download-cellar --judge Lenaerts --date-from 2015-01-01 --date-to 2020-12-31

# Download judgments + orders + AG opinions (default: judgments only)
cjeu-py download-cellar --doc-types CJ,TJ,FJ,CO,TO,FO,CC,TC

# Re-download even if local data exists
cjeu-py download-cellar --force

# Download extended metadata (joins, appeals, legislation links, AG opinions, case names, etc.)
cjeu-py download-cellar-meta

# Download only high-value metadata (skip academic citations and referring judgments)
cjeu-py download-cellar-meta --detail high

# Include rare legislation link types
cjeu-py download-cellar-meta --detail all

# Supplementary research data (dossiers, summaries, successor chains, etc.)
cjeu-py download-cellar-meta --detail exhaustive

# Every remaining CDM property (administrative metadata bulk dump)
cjeu-py download-cellar-meta --detail kitchen_sink

# Download judgment texts (20 concurrent connections, resumable)
cjeu-py fetch-texts --max-items 50

# Download with language fallback (try English, then French)
cjeu-py fetch-texts --lang eng,fra

# Parse judgment headers (composition, parties, representatives)
cjeu-py parse-headers data/texts/xhtml/

# Extract citations (CourtLink hyperlinks + catalogue search + regex safety net)
cjeu-py extract-citations

# ...with a recall report against the CELLAR citation graph
cjeu-py extract-citations --report

# Merge data sources
cjeu-py merge

# Scrape judge biographical data from Curia
cjeu-py scrape-judges --cache-dir data/raw/judges/cache/

# Extract structured bios via LLM
cjeu-py extract-judge-bios --max-items 10

# Export citation network as interactive HTML (see section above)
cjeu-py export-network --format html --max-nodes 500

# Export all pipeline data as CSV
cjeu-py export --format csv

# Generate variable codebook
cjeu-py codebook

# Download CELLAR subject-matter taxonomy
cjeu-py download-taxonomy

# Query the InfoCuria portal directly (live search, no download needed)
cjeu-py curia search "data protection" --date-from 2024-01-01
cjeu-py curia procedures C-311/18
cjeu-py curia enrich --max-items 100    # attach procedural records to downloaded decisions

# Launch the browser GUI
cjeu-py gui

# Run the read-only MCP server (Claude Desktop, Claude Code, other MCP clients)
cjeu-py mcp

# Browse downloaded data in the terminal
cjeu-py browse                          # list all tables with row counts
cjeu-py browse decisions                # preview first 20 rows
cjeu-py browse decisions --stats        # descriptive statistics
cjeu-py browse decisions --columns      # column names, types, nulls
cjeu-py browse text 62019CJ0311        # read a judgment text
```

All variable definitions are documented in [`CODEBOOK.md`](CODEBOOK.md). For full argument details on every command, see the [`CLI reference`](docs/cli-reference.md).

## Data directory

By default, all downloaded data is stored in `~/.cjeu-py/data/`. You can override this with the `CJEU_DATA_DIR` environment variable or the `--data-dir` flag on any command:

```bash
# Use a custom directory
export CJEU_DATA_DIR="/path/to/my/data"

# Or per-command
cjeu-py download-cellar --data-dir ./my-project/data
```

## Optional dependencies

The base install (`pip install cjeu-py`) includes everything needed for data collection, text extraction, citation parsing, network export, and search. Optional extras:

| Extra | What it adds | Install |
|-------|-------------|---------|
| `llm` | LLM-assisted judge-bio structuring - Gemini provider | `pip install 'cjeu-py[llm]'` |
| `openai-llm` | LLM-assisted judge-bio structuring - OpenAI-compatible / OpenRouter providers | `pip install 'cjeu-py[openai-llm]'` |
| `analysis` | scipy, scikit-learn, statsmodels | `pip install 'cjeu-py[analysis]'` |
| `viz` | matplotlib, seaborn | `pip install 'cjeu-py[viz]'` |
| `gui` | Streamlit browser GUI (`cjeu-py gui`) | `pip install 'cjeu-py[gui]'` |
| `mcp` | Read-only MCP server (`cjeu-py mcp`) | `pip install 'cjeu-py[mcp]'` |
| `all` | `llm` + `openai-llm` + `analysis` + `viz` + `gui` (not `mcp` - install that separately) | `pip install 'cjeu-py[all]'` |

## Search

Query collected data, or a live backend, directly from the command line.

**Local searches** (`text`, `party`, `citing`, `cited-by`, `topic`, `legislation`, `list`) query data you have already downloaded. **Remote searches** (`headnote`) query a live backend and require no local data - the CELLAR SPARQL endpoint by default, or the InfoCuria API via `--source curia` (see [Curia](#curia) below).

```bash
# Full-text search across downloaded judgment paragraphs
cjeu-py search text "common market"
cjeu-py search text "proportionality" --limit 50

# Search by party name
cjeu-py search party "Google"
cjeu-py search party "Commission v Germany" --date-from 2015-01-01

# Citation graph queries
cjeu-py search citing 62014CJ0362      # cases citing Schrems I
cjeu-py search cited-by 62014CJ0362    # cases cited by Schrems I

# Search by subject matter (code or label)
cjeu-py search topic "State aid"
cjeu-py search topic PDON

# Cases linked to a piece of legislation
cjeu-py search legislation 32016R0679

# Live headnote/title search (no local data needed) -- CELLAR by default
cjeu-py search headnote "data protection"
cjeu-py search headnote "state aid" --source curia

# List available values
cjeu-py search list topics
cjeu-py search list judges
cjeu-py search list formations
```

All search modes support `--format csv` and `--format json` for piping into other tools. Local searches also accept `--date-from`, `--date-to`, and `--court` filters.

## Data sources

### CELLAR SPARQL

Metadata is collected via the CELLAR SPARQL endpoint at `https://publications.europa.eu/webapi/rdf/sparql` using CDM ontology properties:

- **Core metadata** – CELEX, ECLI, date, court formation, judge-rapporteur, advocate general, procedure type, procedural classification, published-in-eReports flag, authentic language, EEA relevance
- **Extended metadata** – defendant/applicant agents, referring court, treaty basis, date lodged
- **Relational data** – citation network, joined cases, appeals, interveners, annulled acts
- **Legislation links** – which legislation a case interprets, confirms, amends, annuls, etc. (17 link types)
- **AG opinion links** – direct judgment-to-AG-opinion pairing via CELEX
- **Subject matter** – four taxonomies: EuroVoc (260 broad categories), case-law subject matter, hierarchical case-law directory (fd_578, ~3,800 codes), and old directory (fd_577)
- **Academic citations** – journal articles discussing each case (bibliographic references from CELLAR)
- **Referring national judgments** – court name, decision type, date, and reference number for preliminary rulings
- **Case names** – short popular names (`expression_title_alternative`, older cases) and full party names (`expression_case-law_parties`, newer cases), queried at the expression level
- **Supplementary data** (`--detail exhaustive`) – dossier groupings, case summaries, miscellaneous information, successor chains, legislative incorporation links
- **Administrative metadata** (`--detail kitchen_sink`) – all remaining CDM properties (authoring institution, creation dates, transmission timestamps, collection memberships, obsolete identifiers, etc.) as a long-format table

Each query is paginated and saves results as Parquet. Queries target specific CDM properties rather than fetching all triples. Five detail levels control how much metadata to collect: `high`, `medium` (default), `all`, `exhaustive`, `kitchen_sink`.

### Caching

All CELLAR downloads are **cache-first**: if a Parquet file already exists on disk, it is reused without hitting the network. This makes re-running the pipeline instant after the first download and protects against network interruptions.

```bash
# First run: downloads from CELLAR (~2 min for Grand Chamber)
cjeu-py download-cellar --court CJ --formation GRAND_CH

# Second run: loads from disk in <1s
cjeu-py download-cellar --court CJ --formation GRAND_CH

# Force re-download (overwrites cached files)
cjeu-py download-cellar --force
```

The same applies to `download-cellar-meta` – each table (joined cases, appeals, legislation links, AG opinions, case names, etc.) is cached independently. Text downloads via `fetch-texts` are resumable via a checkpoint file, so interrupted downloads pick up where they left off.

**Strict by default.** A SPARQL query that keeps failing after retries used to come back as an empty result, which the pagination loop read as end-of-results - a network blip could silently truncate a dataset with no error. As of 0.6, `download-cellar`, `download-cellar-meta`, `download-taxonomy`, and `enrich-network` instead raise `CellarQueryError` and abort before writing the failing table (already-cached tables are unaffected). Pass `--allow-partial` to restore the pre-0.6 behaviour (a failed query yields `[]` instead of aborting).

### CELLAR REST API

Full judgment and opinion texts are downloaded via content negotiation against the canonical CELLAR resource URI:

```
GET http://publications.europa.eu/resource/celex/{CELEX}
Accept: application/xhtml+xml, text/html
Accept-Language: eng
```

This returns the official XHTML representation – well-structured, with semantic CSS classes (`coj-normal`, `coj-bold`, `coj-italic`, `coj-count`) that preserve document structure. No website scraping involved. With 20 concurrent connections (default), the full Grand Chamber corpus (976 documents) downloads in under 2 minutes. Downloads are resumable – interrupted fetches pick up where they left off via a checkpoint file.

`fetch-texts` also saves the raw XHTML to `raw/texts/xhtml/{CELEX}.xhtml` by default - citation extraction needs it for CourtLink hyperlinks and footnote-aware paragraph parsing (see [Citation extraction](#citation-extraction) below). Pass `--no-save-html` to skip this and keep only the parsed paragraph JSONL. Documents downloaded before 0.6 have no saved XHTML on disk; `extract-citations` detects this per document, falls back to regex-only extraction over the cached paragraph text, and logs how many documents were affected.

### Judgment header parser

The XHTML header (everything before paragraph 1) contains structured metadata that CELLAR SPARQL does not fully expose. The parser extracts:

- **Court composition** – full panel with roles (President, Vice-President, Presidents of Chambers, Rapporteur, Judges)
- **Parties** – applicants, defendants, and interveners
- **Representatives** – lawyers and agents for each party, with professional titles
- **Procedural dates** – hearing date, AG opinion delivery date
- **Operative part** – the Court's ruling, extracted via the "On those grounds" delimiter

The parser handles both pre-2016 (`class="normal"`) and post-2016 (`class="coj-normal"`) XHTML formats. From parsed data, `parse-headers` also produces:

- **Assignments table** – one row per judge per decision (Parquet)
- **Case names** – "applicant v defendant" format (Parquet)
- **Operative parts** – full dispositif text (JSONL)

### Curia biographical data

Judge biographical data is scraped from the Court's official member pages at curia.europa.eu. Raw bios are then structured via LLM into: birth year, nationality, gender, education, prior careers, CJEU roles with dates, and death year. Covers all current and former members of the Court of Justice, General Court, and Civil Service Tribunal (~260 individuals).

## Curia

`cjeu-py curia` talks to a different Curia surface than the biographical scraper above: the redesigned InfoCuria portal (infocuria.curia.europa.eu), relaunched in January 2026 with a JSON API behind its search UI. It complements CELLAR rather than replacing it - CELLAR remains the authoritative bulk source for closed-case metadata, texts, and the citation network; InfoCuria adds what CELLAR does not carry: a live, next-day-fresh full-text index, **pending** cases (no CELEX yet), procedural timelines, hearing calendar events, judge-rapporteur/AG/formation per case, and academic "doctrine notes".

```bash
# Live full-text search (covers pending cases too)
cjeu-py curia search "state aid" --date-from 2025-01-01

# Full procedural record for one case: state, dates, rapporteur, AG,
# formation, referring court, hearing calendar, doctrine note count
cjeu-py curia procedures C-311/18

# Attach procedural records to every decision already downloaded from
# CELLAR (cache-first, resumable)
cjeu-py curia enrich

# Route a live headnote search through Curia instead of CELLAR
cjeu-py search headnote "data protection" --source curia
```

**What to use when:**

| Need | Source |
|------|--------|
| Bulk metadata, judgment/AG opinion texts, the citation network | CELLAR (`download-cellar`, `download-cellar-meta`, `fetch-texts`) |
| Pending cases, procedural timelines, hearing calendar, doctrine notes, live/next-day-fresh search | Curia (`curia search`, `curia procedures`, `curia enrich`) |

`cjeu-py search headnote --source {auto,cellar,curia}` picks the backend: `auto` (default) tries Curia first and falls back to CELLAR on failure, `cellar`/`curia` pin one source. `curia enrich` writes three tables under `data/raw/curia/` - `curia_procedures`, `curia_calendar`, `curia_doctrine` - which `browse`, `export`, and `codebook` pick up alongside the CELLAR-derived tables.

**Honest caveats:** the InfoCuria JSON API is undocumented and unofficial - this client pins its backend host from the portal's own runtime config at start-up (falling back to a frozen snapshot) and keeps a polite default rate limit of 1 request/second. InfoCuria data is © European Union (Court of Justice); keep the rate limit and attribution if you redistribute it. The API refuses result windows beyond 10,000 rows; searches that would exceed it are partitioned automatically by bisecting the date range, but very broad, high-volume queries can still be slow.

## MCP

`pip install 'cjeu-py[mcp]'` then `cjeu-py mcp` runs a read-only [MCP](https://modelcontextprotocol.io) server on stdio, so Claude Desktop, Claude Code, and other MCP clients can query your local dataset (and live Curia lookups) directly. Seven tools: `search_cases`, `get_decision`, `get_text`, `get_citations`, and `list_tables` read local pipeline data; `curia_search` and `curia_procedures` query InfoCuria live, with the same polite rate limit as the CLI. Nothing writes, and no LLM calls are made.

Add it to an MCP client config as:

```json
{"command": "cjeu-py", "args": ["mcp"]}
```

What a client sees - real calls against the sample corpus, with real results:

<p align="center">
  <img src="https://raw.githubusercontent.com/niccoloridi/cjeu-py/main/docs/mcp_demo.gif" alt="cjeu-py MCP tools exercised over stdio (animated)" width="800">
</p>

Every call above was produced by [`examples/mcp_demo.py`](examples/mcp_demo.py),
which exercises all seven tools end to end; the full outputs (CSV/JSON per
call, plus a `calls.json` transcript) are downloadable from
[`examples/sample_data/mcp/`](examples/sample_data/mcp/). Run it against your
own data directory to smoke-test the server:

```bash
python examples/mcp_demo.py --out mcp_results/          # all tools
python examples/mcp_demo.py --out mcp_results/ --offline  # skip live Curia calls
```

## Citation extraction

Four extraction phases run in sequence, from highest to lowest confidence:

1. **CourtLink hyperlinks** – CJEU judgment XHTML links case references to their target via `<a class="coj-CourtLink">` tags, which carry an exact ECLI. Highest-confidence layer: no string matching involved.
2. **Catalogue-guided text search** – every case CourtLinks found, plus every case the CELLAR citation graph (`cdm:work_cites_work`) says the document should cite, is searched for directly across all paragraphs - including cases CourtLinks missed.
3. **Regex safety net** (13 patterns) – formal case references not already resolved by the two layers above: ECLI identifiers, Case C-xxx/xx, joined cases, ECR references, bare case numbers, paragraph pinpoints.
4. **Party-name enrichment** *(optional, `--enrich`)* – for cases the regex layer newly discovered, fetches party names on demand from CELLAR and searches for informal mentions (e.g. "the *Bosman* principle"). Network access during extraction.

Extraction is **footnote-aware** end to end - citations are captured with `location` (`main_text`/`footnote`) and `footnote_num`, and context windows follow the parent paragraph into its footnotes. This matters most for Advocate General opinions, which place the overwhelming majority of their citations in footnotes rather than the main text; earlier versions parsed main-text paragraphs only and silently dropped them.

Every citation resolves to a `cited_celex` (and `cited_ecli` where known) with an explicit `resolution_tier` and `resolution_confidence`: **A** (exact identifier - CourtLink/ECLI, 1.0), **B** (case number derived and confirmed against locally-known CELEX, 0.9), **C** (derived but unconfirmed, or a party-name match, 0.5), **U** (unresolved, 0.0). This is what lets text-derived citations join the authoritative CELLAR citation network instead of remaining bare strings. Each citation also gets a deterministic `citation_id` (`{citing_celex}:{anchor}:{seq}`, e.g. `62018CJ0078:p47:1`), stable across re-runs, which is what checkpointing keys on.

`cjeu-py extract-citations --report` writes `extraction_report.json`, comparing what was extracted against the CELLAR citation graph per document and giving a mean-recall summary - a sanity check on coverage, not just a count. Citations are anchored to their source paragraph (or footnote) and enriched with configurable context windows. `cjeu-py merge` then folds extracted citations together with decision metadata into `citations_for_classification.parquet` - a classification-ready dataset for downstream labelling and analysis.

## LLM features

cjeu-py includes an optional LLM layer that currently powers judge-bio structuring (`extract-judge-bios`), using Gemini by default. The underlying client (`cjeu_py.llm`) supports three provider options: **Gemini** (`GEMINI_API_KEY`), any **OpenAI-compatible endpoint** (`OPENAI_API_BASE` / `OPENAI_API_KEY` / `OPENAI_MODEL` - hosted OpenAI, Ollama, vLLM, LM Studio), and **OpenRouter** (`OPENROUTER_API_KEY` / `OPENROUTER_MODEL`). All keys are read from environment variables only.

An LLM citation-classification module implementing a published precedent taxonomy is in development as part of a companion research project and will return in a future release. Until then, `extract-citations` + `merge` already produce `citations_for_classification.parquet` (above), a classification-ready dataset you can label and analyse downstream with tools of your choice.

## Project structure

```
cjeu-py/
├── cjeu_py/                           # Core library (pip-installable)
│   ├── main.py                        # CLI entry point (19 commands)
│   ├── config.py                      # Central configuration
│   ├── mcp_server.py                  # Read-only MCP server (7 tools)
│   ├── data_collection/               # CELLAR SPARQL + REST clients, InfoCuria client, header parser, Curia judge-bio scraper
│   ├── citation_extraction/           # CourtLink/catalogue/regex extraction, resolution tiers, context windows, party name matching
│   ├── search.py                      # CLI search (8 modes: text, headnote, party, citing, etc.)
│   ├── browse.py                      # CLI data browser (tables, stats, text viewer)
│   ├── llm/                           # Gemini + OpenAI-compatible + OpenRouter client wrapper (used by extract-judge-bios)
│   ├── gui/                           # Streamlit browser GUI, shipped in the wheel (`cjeu-py gui`)
│   └── utils/                         # XHTML parsing, logging utilities
│
├── gui/                               # Compatibility shim → cjeu_py/gui/app.py (`streamlit run gui/app.py` still works)
├── examples/                          # Pre-built example outputs
├── docs/                              # Logo, screenshots, CLI reference
├── data/                              # Pipeline output (Parquet, JSONL, cached XHTML)
├── tests/                             # 278 tests
├── CHANGELOG.md                       # Release notes
├── CODEBOOK.md                        # Variable definitions for all tables
├── CITATION.cff                       # Academic citation metadata
├── LICENSE                            # MIT
├── pyproject.toml
└── requirements.txt
```

## Current limitations

- **Header parser coverage** – tested on Grand Chamber cases (2013–2025). Earlier cases and smaller formations may have formatting variations.
- **Citation extraction tuned for English** – texts can be downloaded in all 24 EU official languages via `--lang`, but the regex safety net and party-name matching are tuned for English-language judgments; `--lang fra` (and other non-English) texts yield near-zero citations from the regex layer. Unchanged in 0.6 - CourtLink extraction and catalogue-guided search are language-agnostic, but the regex patterns are not.
- **No built-in citation classification** – the LLM classification module is being developed against a companion research paper and ships separately.

## Testing

```bash
python -m pytest tests/ -v
```

## If you use R

If R is your preferred language, see Michal Ovádek's [eurlex](https://michalovadek.github.io/eurlex/) package, which provides access to EUR-Lex data including CJEU case law via the CELLAR SPARQL endpoint.

## Acknowledgements

- **Marc Jacob** – *Precedents and Case-Based Reasoning in the European Court of Justice* (Cambridge, 2014). Taxonomy of citation use and treatment informing the project's research programme.
- **EU Publications Office** – CELLAR SPARQL endpoint, CDM ontology, and REST API.
- **Court of Justice of the European Union** – biographical data from official member pages at curia.europa.eu.
