Metadata-Version: 2.5
Name: traceintel
Version: 0.4.0
Summary: Citation-first OSINT research engine.
Project-URL: Homepage, https://github.com/masi-navi/traceintel-
Project-URL: Repository, https://github.com/masi-navi/traceintel-
Author: TraceIntel contributors
License: MIT
License-File: LICENSE
Keywords: citation,evidence,intelligence,osint,research
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Requires-Python: >=3.11
Requires-Dist: beautifulsoup4<5,>=4.12
Requires-Dist: httpx<1,>=0.27
Requires-Dist: pydantic>=2.6
Requires-Dist: pyyaml>=6.0
Requires-Dist: typer>=0.12
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == 'dev'
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Requires-Dist: twine>=7.0; extra == 'dev'
Requires-Dist: types-pyyaml>=6.0; extra == 'dev'
Description-Content-Type: text/markdown

# TraceIntel

[![TraceIntel CI](https://github.com/Masi-NAVI/TraceIntel-/actions/workflows/ci.yml/badge.svg)](https://github.com/Masi-NAVI/TraceIntel-/actions/workflows/ci.yml)

> **Citation-first OSINT research engine.**

An open-source research engine that analyzes **explicitly supplied** public
material, extracts structured **evidence**, **claims**, **entities** and
**events**, and generates traceable analysis backed by verifiable citations.
It does not crawl or search autonomously — every Source, Document, and
Evidence item is provided by the caller.

---

## Status: v0.3.1

**TraceIntel v0.3.1 is a publication-record patch on top of v0.3.0.** It
changes documentation and packaging metadata only — no module under `src/`, no
schema, no transport format, no analytical behaviour. See
`docs/release-notes-v0.3.1.md` for what it corrects and why the correction had
to be a new release rather than an edit.

**v0.3.0 is the third stable release of an early-stage project, and it is
published.** The v0.3 roadmap (*Research Workspace & Longitudinal Evidence
Management*, Issues #56–#64) is implemented: the `traceintel.projects` workspace domain
contract, its additive SQLite persistence boundary, the `traceintel project`
command group, deterministic longitudinal comparison, local-only full-text
search, the read-only Evidence Graph projection, and recorded external
validation of all of it.

Every v0.3 feature reads material that is **already stored**. None of them
calls a provider, touches the network, or produces a new analytical inference —
so the 17-stage pipeline, every transport format, and all provider behaviour
are exactly what v0.2.0 shipped. Early-stage caveats remain: analytical output
depends on provider responses, remote providers are nondeterministic even at
temperature 0, and no result is ever a claim of truth or authenticity — see
`docs/reproducibility.md`.

- **Version:** `0.3.1` (codename *Research Workspace*) — a record-only patch
  over `0.3.0`, whose feature set it shares exactly
- **Primary language:** Python 3.11+
- Release notes: `docs/release-notes-v0.3.1.md` and
  `docs/release-notes-v0.3.0.md`; readiness audit:
  `docs/release-readiness.md`; roadmap: `docs/roadmap-v0.3.md`
- **v0.3.0 is published.** `v0.3.0` Git tag →
  `a3c9c2ab9a823b9aa28d1bea24d371eccad83a3e`, a GitHub Release
  (https://github.com/Masi-NAVI/TraceIntel-/releases/tag/v0.3.0), and
  `traceintel 0.3.0` on PyPI (https://pypi.org/project/traceintel/0.3.0/,
  unyanked, `requires-python >=3.11`), uploaded via **Trusted Publishing**
  (GitHub OIDC) with no API token created, stored, or used.
  - `traceintel-0.3.0-py3-none-any.whl` —
    `85f2f822032ecffed3a69d4bd8f76076a89cdf476bdfe25b2eb4495c35a65989`
  - `traceintel-0.3.0.tar.gz` —
    `b021236efe83e90f3a5226af7ec25cccac93d517b29a317c469dd163a6a62b1a`
- **Install:** `pip install traceintel==0.3.1` for the corrected record, or
  `pip install traceintel` for the latest published version. The published
  0.3.0 page on PyPI still carries preparation-time wording that says the
  release does not exist; a PyPI description is immutable, so v0.3.1 corrects
  it forward instead. No 0.3.0 file was deleted, replaced, or yanked.
- **Upgrade path.** SQLite `SCHEMA_VERSION` moves **3 → 5** in two additive
  steps for databases written by v0.1.0 or v0.2.0 — new tables only, nothing
  dropped, renamed, rewritten, repaired, or reordered, with existing checkpoint
  rows byte-unchanged (`docs/storage-schema.md`). Transport formats stay at
  **v1**; v0.3 adds no field to any of them. A database at a future schema
  version is refused, never opened optimistically.
- Previous stable release: `docs/release-notes-v0.2.0.md` — published as the
  `v0.2.0` Git tag → `f1c1e250d055551374a8147ae03ac504f36bfef5`, a GitHub
  Release (https://github.com/Masi-NAVI/TraceIntel-/releases/tag/v0.2.0), and
  `traceintel 0.2.0` on PyPI (https://pypi.org/project/traceintel/0.2.0/,
  unyanked, `requires-python >=3.11`).
  - `traceintel-0.2.0-py3-none-any.whl` —
    `9052fc4713d361b9393aa011d6da41a16f18646ab9cf388a2df4fad9bf8d6bcf`
  - `traceintel-0.2.0.tar.gz` —
    `7eb83900e3b6c7fa247d39f2f44c5675672c9d5b2436c3091ae5833f79cb2ceb`
  - That upload used **Trusted Publishing** (GitHub OIDC): no API token was
    created, stored, or used, and no credential appears in either artifact.
- Before that: `docs/release-notes-v0.1.0.md`. Every v0.1.0 and v0.2.0 request,
  snapshot, checkpoint, and export still loads unchanged and re-serializes
  byte-identically.
- **v0.4.0 is prepared — `docs/release-notes-v0.4.0.md`.** The theme is
  *Cross-Run Evidence Graph & Multi-Run Longitudinal Analysis*: Issues #66–#73
  in `docs/roadmap-v0.4.md`. `SCHEMA_VERSION` moves **5 → 6**, additively —
  every prior version upgrades in place with checkpoint rows byte-unchanged,
  and a v0.3 build refuses a v6 database fail-closed rather than reading it
  optimistically. Existing transport formats stay at **v1**. Whether v0.4.0 has
  been published is recorded in `docs/release-readiness.md`, which is updated
  after the fact.

Publication is a distribution fact, not an analytical one: it says these
exact bytes are what the project shipped. It is not a claim about the truth
or authenticity of any analyzed content.

### Architecture at a glance

```
Source / Document / Evidence   (explicitly supplied input)
              ↓
          ResearchRun
        ┌─────┼─────┐
      Entity Claim Event       (Evidence-grounded analytical pipelines)
        └─────┼─────┘
          Provenance
              ↓
        ResearchReport
              ↓
        ExportEnvelope
              ↓
      materialize / verify     (explicit CLI steps)
```

Runs are executed by a deterministic orchestrator over an explicitly injected
LLM provider, checkpointed append-only into SQLite, and resumable only through
explicit, auditable resume requests.

### Capability matrix

**Implemented:**

- explicit HTTP ingestion API (fetch + parse of caller-named public URLs)
- Evidence-grounded Entity / Claim / Event analytical pipelines
  (extraction → resolution → canonicalization, subject/support/contradiction,
  temporal normalization and participant roles)
- unified citation & provenance service with deterministic rendering
- deterministic report assembly, Markdown rendering, and canonical export
  (SHA-256 manifest, envelope, four-file materialization, offline verification)
- canonical ResearchRun orchestration, deterministic executor, append-only
  SQLite checkpoints, explicit resume transitions, deterministic resume
  executor
- operational CLI: `traceintel run execute / inspect / checkpoints / resumes /
  resume` and `traceintel report verify / materialize`
- research workspace: `ResearchProject` grouping runs by ID under a stated
  objective, additive SQLite persistence with append-only membership, and the
  `traceintel project create / list / inspect / runs / attach` command group
- deterministic longitudinal comparison between two runs in a project
  (`traceintel project compare`) — no provider, no network, no fresh inference
- local-only full-text search over a derived, rebuildable index
  (`traceintel project index / search`)
- read-only per-run Evidence Graph projection (`traceintel project graph`) and
  a cross-run, project-scoped projection over the runs a Project attaches
- a derived, deletable cache for the cross-run graph, authoritative for nothing
  and rebuildable with no network access (Issue #67)
- a deterministic multi-run timeline over three or more runs, composing the
  cross-run graph and the existing pairwise comparison (Issue #68)
- bounded, fail-closed graph queries and a canonical, verifiable graph export
  (`traceintel project graph-query / graph-export / graph-verify`)

**Not implemented (deliberate scope limits, not defects):**

- autonomous crawling, search, or topic-driven collection
- scheduled monitoring or background execution
- browser automation
- vector databases, embeddings, or relevance-ranking models
- persisted graph stores or graph databases (the Evidence Graph is a
  read-only projection, never written back)
- project deletion, run detachment, or membership reordering
- cross-run contradiction verdicts (comparison reports differences; it never
  adjudicates them)
- Web UI
- automatic publication, signing, or encryption
- autonomous use of cloud LLMs (remote OpenAI/Anthropic execution exists but
  only via explicit canonical requests with runtime-only credentials — see
  `docs/remote-provider-operation.md`; research content leaves the machine
  only when a request explicitly names a remote provider)

### Quickstart

```bash
# 1. Install (Python 3.11+)
pip install traceintel==0.3.1

# 2. Confirm the CLI
traceintel --version
traceintel --help

# 3. Explore the operational commands
traceintel run --help
traceintel report --help

# 4. Prepare a canonical execution request (see examples/golden_demo/):
#    explicit Sources/Documents/Evidence + report spec + local Ollama provider.

# 5. Execute with an explicit checkpoint database
traceintel run execute examples/golden_demo/execution-request.json \
    --database ./aurora-demo.db

# 6. Inspect the persisted run
traceintel run inspect golden-run-aurora --database ./aurora-demo.db
traceintel run checkpoints golden-run-aurora --database ./aurora-demo.db

# 7. If a stage failed, resume explicitly (see examples/golden_demo/README.md)
# 8/9. Materialize + verify the report export as separate explicit steps
traceintel report materialize --help
traceintel report verify --help
```

TraceIntel analyzes **explicitly supplied** material; step 5 performs no
collection beyond what the request file contains. Production execution
requires a compatible local Ollama model (the offline test fixtures use a
scripted provider that is not reachable through the CLI).

Prefer pinning the exact version as above. Source contributors can instead
use `pip install -e ".[dev]"` from a checkout.

## Design principles

TraceIntel is built around a small set of principles that do not change:

1. **Citation first** — evidence is preferred over model output; every claim is
   traceable back through Document → Source → original URL.
2. **Evidence and inference separation** — `FACT`, `CLAIM`, `INFERENCE`,
   `ESTIMATE`, `OPINION`, and `UNKNOWN` are kept distinct. Model inferences are
   never stored as primary-source facts.
3. **Contradictions are preserved** — conflicting figures/dates/names from
   different sources are kept as separate claims, not silently merged.
4. **Source independence** — syndicated copies of the same article are not
   counted as independent corroboration.
5. **Provider independence** — LLM features go through a provider abstraction;
   no hard dependency on a single vendor.

## Core domain models

The canonical internal representation lives in `src/traceintel/models/` as
Pydantic v2 models. They are **pure domain objects** — no database, network, or
LLM dependencies — and preserve the citation-first provenance chain:

```
Source → Document → Evidence → Claim → Entity / Event
```

- **`Source`** — where information came from (URL, publisher, dates, type, coarse
  reliability tier).
- **`Document`** — a parsed artifact from a source. `raw_text` is stored
  **verbatim** and is the anchor that evidence offsets point into.
- **`Evidence`** — a **verbatim excerpt** of a document. `exact_text` is never
  trimmed, normalized, translated, or summarized; character offsets locate it in
  the document. Evidence is the provenance-bearing material a reader inspects to
  verify a claim.
- **`Claim`** — a tracked assertion, optionally backed by `evidence_ids`, with a
  verification `status` and an epistemic `information_type`.
- **`Entity`** / **`Event`** — real-world subjects and dated occurrences that
  claims and evidence attach to.

### Information type & inference separation

Every claim carries an `information_type` (`FACT`, `CLAIM`, `INFERENCE`,
`ESTIMATE`, `OPINION`, `UNKNOWN`) so that AI-generated reasoning is never stored
as directly observed fact. A claim whose type is `INFERENCE` **must** carry an
`InferenceMetadata` record (`inference_type`, `supporting_claim_ids`,
`confidence`, `generated_by`); the model layer rejects an `INFERENCE` claim that
lacks it. This keeps source-supported claims and generated inference structurally
distinct.

## Web ingestion (pre-alpha)

TraceIntel can fetch and parse **public, unauthenticated** HTML pages via
`src/traceintel/collectors/` and `src/traceintel/parsers/`.

```python
from traceintel.collectors import HTTPCollector
from traceintel.parsers import parse_html

with HTTPCollector() as collector:      # SSRF-checked, robots-respecting, bounded
    page = collector.fetch("https://example.test/article")   # -> FetchedPage
parsed = parse_html(page)               # -> ParsedHTML (text + metadata)
```

The collector enforces HTTP/HTTPS-only URLs, rejects embedded credentials and
internal/private targets, revalidates every redirect hop, applies finite
timeouts and a streamed body-size limit, requires an HTML content type, sends a
`TraceIntel/<version>` User-Agent, and honors `robots.txt` (failing closed on
server/network errors). The parser is pure (no network) and extracts readable
text plus conservative metadata (title, canonical URL, language, publisher,
publication time). See [`SECURITY.md`](SECURITY.md) for the ingestion threat
model and its limitations.

## Processing (normalization, hashing & domain mapping)

`src/traceintel/processing/` turns ingestion output into canonical domain
objects:

```
FetchedPage + ParsedHTML → normalized text → SHA-256 → Source + Document
```

```python
from traceintel.processing import map_document

mapped = map_document(page, parsed)     # -> MappedDocument(source=..., document=...)
```

**Two text representations, different purposes.** `Document.raw_text` is copied
**verbatim** from `ParsedHTML.raw_text` (the provenance-bearing text — never
trimmed, re-cased, or whitespace-/Unicode-normalized). `Document.normalized_text`
is a **separate** representation used only for fingerprinting/comparison.

**Normalization algorithm** (deterministic and idempotent — documented so the
same hash is reproducible by another implementation), applied in order:

1. Unicode **NFC** (not NFKC — compatibility distinctions are preserved).
2. Line endings `\r\n` and `\r` → `\n`.
3. Per line, collapse runs of ASCII horizontal whitespace (space, `\t`, `\f`,
   `\v`) to a single space (non-ASCII spaces such as NBSP are left intact).
4. Strip leading/trailing spaces from each line.
5. Collapse runs of more than two consecutive blank lines to at most two.
6. Strip leading/trailing blank lines from the document.

Nothing else is changed: **case, punctuation, numbers, emoji, and non-ASCII
text are preserved**. An empty/whitespace-only result is rejected.

**Content hash.** `Document.content_hash` is
`sha256(normalized_text.encode("utf-8")).hexdigest()` — a 64-character lowercase
hex string over the normalized text **only** (no URL, title, IDs, timestamps, or
metadata). Hash equality is only a **duplicate-content signal**: TraceIntel does
**not** auto-delete, reject, or merge anything, and there is no
`UNIQUE(content_hash)` constraint. Two documents with different IDs and the same
hash (e.g. syndicated copies) are valid.

Mapping is a pure transformation (no HTTP, no database, no LLM): `Source.url` is
the final fetched URL, `source_type` defaults to `OTHER` (never inferred),
`reliability_tier` stays `None`, only safe ingestion metadata is retained (never
response headers/cookies), and IDs are opaque (injectable for tests). Persisting
mapped objects remains the caller's choice — the processing layer never writes to
the database.

## Evidence extraction (provenance)

`src/traceintel/evidence/` turns exact ranges of source text into `Evidence`,
completing the provenance chain:

```
Source → Document.raw_text → Evidence [start, end) → exact_text
```

```python
from traceintel.evidence import TextSpan, extract_evidence_span, extract_evidence_by_text

ev = extract_evidence_span(document, TextSpan(0, 14))     # by explicit range
ev = extract_evidence_by_text(document, "Project Aurora") # by exact literal quote
```

**Offset contract.** Offsets are **0-based** and the end offset is **exclusive**
— the span `[start, end)` selects `raw_text[start:end]`. Offsets are **Python
`str` indices**, not UTF-8 byte offsets, grapheme indices, or visual columns
(this matters for Japanese, accented text, emoji, and combining characters).

**Evidence is extracted from `raw_text`, never `normalized_text`.** The core
invariant, which a provenance helper verifies, is:

```
document.raw_text[evidence.start_offset:evidence.end_offset] == evidence.exact_text
```

`exact_text` is always derived from the raw-text slice (the caller cannot supply
a conflicting quote) and is never trimmed, normalized, or altered. Literal
lookup (`find_literal_spans` / `extract_evidence_by_text`) is **exact and
case-sensitive** with no fuzzy or LLM matching: repeated quotes preserve
ambiguity (an occurrence index must be given to disambiguate), and a missing
quote fails visibly. Overlapping and duplicate `Evidence` records are allowed;
nothing is auto-merged or persisted.

## LLM provider abstraction

Everything above — `Source → Document → Evidence` — is deterministic and exists
**independently of any AI**. That is the provenance boundary:

```
Source → Document → Evidence
────────────────────────────── deterministic provenance boundary
LLMProvider → validated structured output   (untrusted analytical layer)
```

`src/traceintel/providers/` defines a **provider-neutral** interface for
**validated structured generation**. A caller supplies a Pydantic output model
and receives a validated instance of it — there is deliberately no free-form
`chat`/`complete` method, no streaming, no tools, and no automatic retries.

```python
from traceintel.config import load_config, create_llm_provider
from traceintel.providers import StructuredGenerationRequest
from pydantic import BaseModel

class ProjectStatus(BaseModel):
    project: str
    status: str

provider = create_llm_provider(load_config("config.yaml"))   # config.llm.provider/model
result = provider.generate_structured(
    StructuredGenerationRequest(prompt="…"), ProjectStatus,
)  # -> a validated ProjectStatus
```

- **Ollama is the first (and only) provider**, and **only local Ollama** is
  supported (`http://127.0.0.1:11434`, non-streaming `POST /api/chat`). The
  JSON Schema sent to Ollama derives from `output_type.model_json_schema()`.
- **Output is validated with Pydantic.** Malformed output — invalid JSON, a
  Markdown code fence, a missing/extra/wrong-typed field — **fails visibly**
  (`StructuredOutputValidationError`); it is never silently repaired.
- **Model output is untrusted analytical material, not Evidence.** It never
  modifies `Source`/`Document`/`Evidence`, their offsets, `exact_text`, or
  `content_hash`. See [`SECURITY.md`](SECURITY.md).
- **No cloud providers exist yet**; nothing is added to
  `config.yaml`/`.env`. Prompts and responses are not logged by default.
- **Runtime secret contract (Issue #47):**
  `traceintel.providers.secrets` defines how credentials are handled —
  resolved at runtime only, from the process environment
  (`EnvironmentSecretSource`, exact-name lookup, fail-closed on
  missing/empty) or from an explicitly injected `RuntimeSecretSource`. The
  `RuntimeSecret` wrapper never prints, serializes, pickles, or rides on a
  Pydantic model; provider specs reject credential-shaped fields
  fail-closed; and a `SecretRedactor` scrubs registered secrets from CLI
  output as a second defense. Credentials are never stored in requests,
  snapshots, checkpoints, exports, error records, or logs. There is no
  automatic `.env` loading. The contract is consumed by the remote
  OpenAI/Anthropic providers, which resolve their own fixed environment
  variable per call; local Ollama remains loopback-only and
  authentication-free, and an Ollama request never creates a secret
  source. Redaction never licenses storing or printing research content,
  prompts, or provider responses.
- **Provider conformance suite (Issue #48):** every production provider
  must pass the shared, parameterized contract tests in
  `tests/conformance/providers/` (structured output, bounded responses,
  finite timeouts, credential-leakage prevention, environment trust, no
  retry, endpoint discipline, invalid-configuration rejection) — fully
  offline, over recordable mock transports. Registration steps:
  `docs/provider-conformance.md`.
- **OpenAI provider (Issue #49):** `OpenAIProvider` speaks the official Chat Completions API
  (`https://api.openai.com/v1/chat/completions`, strict Structured
  Outputs, `store: false`, finite `max_completion_tokens`) with the
  Issue #47 runtime-only credential lifecycle (`OPENAI_API_KEY` resolved
  per call, revealed only into the `Authorization` header). Since
  Issue #51 canonical execution/resume requests may name
  `kind: "openai"` explicitly (`docs/remote-provider-operation.md`).
  Configuring a remote provider is an explicit egress decision: prompts
  containing research content leave the local environment. See
  `docs/provider-openai.md`.
- **Anthropic provider (Issue #50):** `AnthropicProvider` speaks the
  official Messages API
  (`https://api.anthropic.com/v1/messages`, `x-api-key` +
  `anthropic-version` headers, structured outputs via
  `output_config.format` with a fail-closed documented-subset schema
  gate, top-level `system`, finite `max_tokens`) with the same
  runtime-only credential lifecycle (`ANTHROPIC_API_KEY` resolved per
  call) and Issue #49-R error-isolation contracts. Since Issue #51
  canonical requests may name `kind: "anthropic"` explicitly. Same egress
  caveat as OpenAI. See `docs/provider-anthropic.md`.
- **Cross-provider validation suite (Issue #52):** `tests/validation/`
  validates OpenAI and Anthropic over the *real* canonical path — real
  loaders, the production factory, the real provider classes, and the real
  `run execute` / `run resume` / `report materialize` / `report verify`
  commands — with only the socket layer replaced by a recording mock
  transport. It records a content-free `CrossProviderValidationRecord`
  (identifiers, statuses, counts, durations; never prompts, provider
  responses, Document/Evidence bodies, Source URLs, or credentials) and
  distinguishes `mock` from `live` mode explicitly. Offline validation is
  complete.
- **Live validation (Issue #52-LIVE): PASSED on both remote providers.**
  `gpt-4.1-mini` (run `31853638249`, 20 s) and `claude-haiku-4-5` (run
  `31865900844`, 34 s) each executed a full run through the canonical CLI
  against a synthetic single-document request: exit code 0, 17 checkpoints
  in canonical order, final status COMPLETED, report materialized, export
  verified. The two runs sit on different commits whose production trees
  are byte-identical, so they describe one tree. Request counts and
  provider charges were **not directly measured**; the recorded bounds are
  at most 4 requests per provider, at most 8 cumulative, under a 500 JPY
  authorized ceiling. Credentials were runtime-only and were never
  displayed, stored, or committed; the validation environment and its
  secrets were deleted afterwards. **This is orchestration evidence, not a
  truth claim** — execution success and export verification describe
  reproducible mechanical processing and say nothing about the truth or
  authenticity of analyzed content, and `mock` results are never a
  substitute for `live` ones. Procedure:
  `docs/cross-provider-validation.md`; recorded results:
  `docs/cross-provider-validation-results.md`.

## Entity extraction (Evidence-grounded)

`src/traceintel/entities/` uses the LLM boundary to propose entity mentions, then
**deterministically verifies** each one against the supplied Evidence:

```
Evidence → LLM entity extraction → deterministic grounding check → EntityCandidate
```

```python
from traceintel.entities import extract_entity_candidates

candidates = extract_entity_candidates(document, evidence, provider)  # -> list[EntityCandidate]
```

- **`EntityCandidate` is not a canonical `Entity`.** It carries a raw
  `surface_form` (never a canonical name), never has aliases, and records
  analytical provenance (`provider_name`, `model_name`, `prompt_version`)
  alongside the `evidence_ids` that ground it. Turning candidates into `Entity`
  records — **Entity Resolution** (alias merging, canonical naming) — is a
  separate, not-yet-implemented issue.
- Every supplied Evidence is provenance-checked **before** the LLM is called
  (`raw_text[start:end] == exact_text`); bad input aborts without contacting the
  model. Only `evidence_id` + `exact_text` are sent (not the whole document),
  as **untrusted quoted data**.
- After generation, a candidate is accepted only if it cites supplied Evidence
  IDs **and** its exact `surface_form` occurs literally (exact, case-sensitive,
  no normalization) in at least one cited `Evidence.exact_text`. Grounding is
  all-or-nothing; hallucinated or wrong-case surface forms are rejected.
- The model's `EntityType` is an **analytical classification** (it may be wrong)
  — only the surface-form grounding is deterministic. Duplicate candidates are
  preserved (no auto-merge), nothing is persisted, and Evidence is never
  modified.

### Entity resolution (deterministic, first-stage)

`resolve_entity_candidates` groups `EntityCandidate`s that share a strict
deterministic surface key **and** a compatible `EntityType`:

```
EntityCandidate[] → deterministic surface key → type-compatible grouping → EntityResolutionGroup[]
```

- The **resolution key** is: Unicode **NFC** (not NFKC) → collapse ASCII
  horizontal whitespace → trim → `casefold()`. That's all — **no** fuzzy
  matching, edit distance, acronym expansion, accent stripping,
  suffix/word removal, tokenization, embeddings, or LLM. So
  `Example Robotics Corp.` and `Example Robotics Corporation` stay **separate**
  (punctuation/words are never normalized away). The key is internal comparison
  metadata; it never replaces the observed `surface_form`.
- **Type compatibility:** same type groups; `OTHER` joins a key that has exactly
  one specific type; all-`OTHER` forms one `OTHER` group; but conflicting
  specific types (e.g. `COMPANY` vs `GOVERNMENT`) stay in **separate** groups,
  and an `OTHER` candidate is never attached arbitrarily under such a conflict.
- Grouping is **additive and non-destructive**: candidates are never modified,
  every candidate ID appears in **exactly one** group, and a group stores
  candidate IDs + the union of their Evidence IDs (not copies of candidates).
- `EntityResolutionGroup` is **not** a canonical `Entity` — no canonical name,
  aliases, description, or attributes are chosen here (that is a later issue). A
  shared resolution key does **not** prove real-world identity; grouping is
  conservative and may later be split or merged by contextual/human review.

### Canonical entity mapping

`map_resolution_group` turns one `EntityResolutionGroup` (plus a candidate
lookup) into a canonical `Entity` and an `EntityResolutionRecord`:

```
EntityResolutionGroup → canonical-name selection → Entity + EntityResolutionRecord
```

- **Canonical names are observed surface forms only.** `Entity.canonical_name`
  is chosen from an exact `EntityCandidate.surface_form` already in the group —
  **never** synthesized, cleaned, title-cased, de-punctuated, expanded, or
  translated (so `Example Robotics Corp.` never becomes
  `Example Robotics Corporation`). Selection is **deterministic, not aesthetic**:
  highest exact-surface frequency, ties broken by the smallest supporting
  candidate ID, then by the surface string.
- **Aliases are observed surface forms only** — the group's distinct surface
  forms excluding the canonical name, in first-seen order (casing/punctuation
  preserved). `Entity.entity_type` is taken from the group's `resolved_type`;
  `description` is `None` and `attributes` is `{}` (no enrichment).
- **`EntityResolutionRecord` preserves provenance** the `Entity` deliberately
  doesn't carry: `resolution_group_id`, all `candidate_ids`, the Evidence-ID
  union, `canonical_source_candidate_ids`, `resolution_method`, and
  `mapping_method` (`canonical-entity-mapping-v1`) — so
  `Entity → record → group → candidates → Evidence` is traceable in memory.
- Exactly **one group maps to one Entity** (groups are never merged); candidates
  and groups are never mutated; **nothing is persisted** and **no Claim linking**
  happens yet.

## Claim extraction (Evidence-grounded)

`src/traceintel/claims/` extracts grounded propositions. Unlike entities, a
claim's *statement* may **paraphrase** — so grounding is anchored on verbatim
**supporting quotes** instead:

```
Evidence → LLM claim extraction → literal support-quote validation → ClaimCandidate
```

```python
from traceintel.claims import extract_claim_candidates

candidates = extract_claim_candidates(document, evidence, provider)  # -> list[ClaimCandidate]
```

- **The claim `statement` may be a paraphrase** (it need not appear literally in
  Evidence and is never treated as Evidence). Every claim must carry at least one
  **supporting quote** that is an **exact, case-sensitive, non-normalized**
  substring of a cited `Evidence.exact_text`. Hallucinated, wrong-case, or
  whitespace-mismatched quotes are rejected; grounding is all-or-nothing.
- For a uniquely-occurring quote, TraceIntel derives Document-level offsets
  (`document.raw_text[start:end] == quote`); when a quote repeats within one
  Evidence the location is ambiguous and offsets are left `None` (no arbitrary
  pick). Offsets are computed by TraceIntel, never supplied by the model.
- **`ClaimCandidate` is not a canonical `Claim`.** It has no `status`,
  `subject_entity_id`, `predicate`, `object_value`, or inference metadata — those
  depend on Entity Resolution and support/contradiction assessment, which are
  not implemented. `InformationType` (FACT/CLAIM/INFERENCE/…) is an **analytical
  classification** and may be wrong; a literal quote does **not** prove a claim
  is objectively true.
- Duplicate and contradictory candidates are both preserved (no dedup, no
  `CONFLICTING` detection); nothing is persisted; Evidence is never modified.

### Claim normalization & entity linking

`normalize_and_link_claim` connects the Claim and Entity branches
deterministically (no LLM):

```
ClaimCandidate + canonical Entities → Evidence-local literal matching → NormalizedClaimCandidate
                                                                          ├── ClaimEntityLink
                                                                          └── UnresolvedEntityMention
```

- The claim's `original_statement` is preserved verbatim; a `normalized_statement`
  is produced with the same Issue #7 `normalize_text` (versioned
  `claim-normalization-v1`). `information_type` and provider/model/prompt
  provenance are carried through unchanged.
- **Entity links are mention-level and come from supporting quotes**, never the
  paraphrased statement. A candidate's exact `surface_form` links in a quote only
  when that `EntityCandidate` is grounded to the **same `evidence_id`** as the
  quote, and matching is **exact, case-sensitive, non-normalized** (the
  resolution key is *not* used here — `Project   Aurora` won't match
  `Project Aurora`). All literal occurrences are found, with quote-local (and,
  when the quote has document offsets, document-level) offsets.
- A `ClaimEntityLink` means "this Entity's observed surface form appears in this
  supporting quote" — **not** that the Entity is the subject/object. No
  `subject_entity_id`, predicate/object, or `ClaimStatus` is assigned. When one
  surface maps to **multiple** canonical Entities, no link is made and an
  `UnresolvedEntityMention` preserves the ambiguity instead of guessing.
- Fully deterministic and offline: inputs are never mutated, output is
  order-independent, nothing is persisted, and no final `Claim` is created.

### Claim subject resolution & canonical drafting

The first claims stage to consult the LLM — through the `LLMProvider`
**abstraction** only (never a concrete provider):

```
NormalizedClaimCandidate → resolve_claim_subject → ClaimSubjectResolutionRecord → draft_canonical_claim → CanonicalClaimDraft
```

- **The subject is chosen only from the claim's existing entity links.** A
  resolved `subject_entity_id` is always one of the `ClaimEntityLink` entities —
  never an `UnresolvedEntityMention`, never an invented ID. There is **no**
  "one link means one subject" shortcut: whenever the claim has at least one
  link, the provider is asked to make the semantic judgement (versioned
  `claim-subject-resolution-v1`). The only provider-free path is *no links at
  all*, which is deterministically `UNRESOLVED`.
- **Every provider decision is re-grounded deterministically.** `RESOLVED`
  requires a linked subject plus a supporting link referencing it; `AMBIGUOUS`
  requires the subject to be absent and ≥2 distinct linked candidates;
  `UNRESOLVED` names no subject. Unknown entity/link IDs are rejected. Provider
  failures propagate unchanged with no automatic retry.
- **No domain `Claim` is fabricated.** `draft_canonical_claim` assembles a
  `CanonicalClaimDraft` (normalized statement, first-seen `evidence_ids` union,
  supporting quotes, link/mention IDs, resolved subject when any). It does **not**
  invent a `ClaimStatus` or `InferenceMetadata`, so `claim` stays `None` and
  machine-readable `claim_blockers` (`CLAIM_STATUS_NOT_ASSESSED`, and
  `INFERENCE_METADATA_REQUIRED` for `INFERENCE`) record exactly what is missing.
- Subject resolution and drafting are deterministic given fixed inputs; inputs
  are never mutated and nothing is persisted.

### Evidence-support assessment & comparison prep

Assesses how a draft's own cited quotations relate to its statement, then prepares
an indexable record for future contradiction analysis:

```
CanonicalClaimDraft
      ↓ (per-quote, model-assisted)
Evidence/Claim semantic assessment  → QuoteClaimRelation (SUPPORTS/CONTRADICTS/NEUTRAL/UNCLEAR)
      ↓ (deterministic aggregate)
ClaimEvidenceSupportState
      ↓
ClaimComparisonRecord
```

- **"Quote exists" is not "claim supported".** Issue #11 proved only that a quote
  literally occurs; a verbatim quote can still *contradict* the claim
  (`Project Aurora was not approved.` grounds a claim that it approved — and
  contradicts it). This stage detects that distinction via the `LLMProvider`
  abstraction only (versioned `claim-support-assessment-v1`).
- **Support state is about cited Evidence, not truth.**
  `SUPPORTED_BY_CITED_EVIDENCE` means only that the cited quotes semantically
  support the claim *as formulated* — never that it is objectively verified,
  corroborated, or true. It is **not** a domain `ClaimStatus`; Issue #15's
  `CLAIM_STATUS_NOT_ASSESSED` blocker stays valid.
- **The model classifies each quote; TraceIntel computes the aggregate.** Any
  mixture of support and contradiction is `MIXED_CITED_EVIDENCE` — no majority
  voting, no weighting, no confidence, no source-reliability weighting. Only
  `NEUTRAL`/`UNCLEAR` yields `INSUFFICIENT_CITED_EVIDENCE`. The response must
  assess every prompt-local `quote-NNNN` ref exactly once (missing/duplicate/
  unknown refs are rejected); Evidence IDs are attached by TraceIntel, not the
  model. Provider failures propagate unchanged with no retry.
- **`ClaimComparisonRecord` is comparison *input*, not contradiction analysis.**
  It carries the original + normalized statement, a `statement_hash`
  (SHA-256 of the normalized statement — an exact normalized-text identity
  signal only, never semantic equivalence, never auto-merged), the copied
  `subject_entity_id`, the deterministic `evidence_ids`, and the support state.
  It holds **no** Claim-vs-Claim verdict, similarity, polarity, or entailment —
  no pairwise comparison exists yet.

### Contradiction detection

Detects semantic contradiction relationships **between** prepared comparison
records, within a conservative scope:

```
ClaimComparisonRecord[]
      ↓ (deterministic, same-subject scoping)
ClaimPairCandidate[]  (canonical order: left_comparison_id < right_comparison_id)
      ↓ (identical statement_hash → deterministic NOT_CONTRADICTORY)
      ↓ (remaining pairs, one bounded batch)
semantic contradiction assessment  → ClaimContradictionRelation (CONTRADICTS/NOT_CONTRADICTORY/UNCLEAR)
      ↓
ClaimContradictionAssessmentRecord[]
```

- **Only same resolved-subject Claims are compared.** Both records must share the
  same non-null `subject_entity_id`. Unresolved-subject Claims are excluded
  conservatively (and surfaced in `unresolved_subject_comparison_ids`) — the
  contradiction subsystem never re-runs subject/Entity resolution. Different
  subjects are never compared, even if statements look similar.
- **Contradiction is not truth adjudication.** `CONTRADICTS` means only that two
  propositions cannot both hold in the same relevant context *as formulated* — it
  never picks a winner, marks one Claim false, or ranks source reliability. The
  model is told to be conservative about time, scope, modality, quantity, actor,
  and conditionals; a later state change (approved in May, withdrawn in June) is
  *not* automatically a contradiction, and missing context favors `UNCLEAR`.
- **Evidence support does not determine contradiction.** Issue #16 support states
  are copied onto the output record as context only — never sent to the model and
  never used to classify. Two Claims both `SUPPORTED_BY_CITED_EVIDENCE` can still
  contradict each other (an important OSINT case).
- **Identical normalized statements are deterministically `NOT_CONTRADICTORY`.**
  Equal `statement_hash` pairs skip the LLM entirely — and the two Claims are
  *not* merged (both comparison IDs, draft IDs, Evidence chains, and support
  states are preserved separately). Only same-subject pairs with *different*
  hashes reach the model, in a single bounded call (finite pair-count and
  character limits; over-limit fails before any provider call). Every prompt-local
  `pair-NNNN` ref must be assessed exactly once; provider errors propagate with no
  retry. No contradiction graph or clustering exists yet.

## Event extraction (Evidence-grounded)

A separate analytical branch (distinct from Claims) that pulls grounded event
candidates directly from Evidence:

```
Evidence
   ↓ LLM event extraction (paraphrase allowed)
   ↓ literal supporting-quote validation
   ↓ literal temporal-expression validation
EventCandidate
```

- **`EventCandidate` is not a canonical `Event`.** It has no canonical event ID,
  no normalized `event_date_start`/`event_date_end`, and no Entity/participant
  links. The existing domain `Event` model is left untouched; turning candidates
  into canonical Events (and Entity linking) is later work.
- **Description is analytical, supporting quotes are verbatim.** The model's
  `description` may paraphrase and is never treated as Evidence; every event must
  carry at least one supporting quote that is an **exact** (case-sensitive,
  non-normalized) substring of a cited Evidence item. Document offsets are derived
  deterministically by TraceIntel (and left `None` when a quote occurs more than
  once in its Evidence). Grounding is all-or-nothing; the existing `EventType`
  enum is reused (analytically — its correctness is not deterministically proven).
- **Temporal expressions are preserved at source precision.** Dates/times are
  captured as literal `EventTemporalMention`s with a `TemporalExpressionKind`
  (`DATE`/`MONTH`/`YEAR`/`TIME`/`RANGE`/`RELATIVE`/…) — never converted to a
  `datetime`. `May 2026` stays `May 2026`, `2026` stays `2026`, `later that year`
  stays relative; no day/month/time/timezone is invented and no cross-Evidence
  date synthesis occurs. A temporal mention must be literal text occurring inside
  a **same-Evidence supporting quote** of that event (intentional locality), and
  an event with no temporal information (`temporal_mentions == []`) is valid.
- Uses the `LLMProvider` abstraction only; Evidence is validated (provenance,
  bounds, duplicate IDs) before the provider is called; provider errors propagate
  with no retry; inputs are never mutated and nothing is persisted. **No Event
  entity linking and no Event persistence exist yet.**

### Event normalization (temporal + Entity linking)

Two independent, fully **deterministic** transforms over an `EventCandidate` — no
LLM, no network — combined into a `NormalizedEventCandidate`:

```
EventCandidate
   ├─ temporal_mentions
   │       ↓ conservative temporal normalization
   │  EventTemporalNormalization[]  (NormalizedTemporalPoint | UNRESOLVED)
   │
   └─ supporting_quotes  +  EntityCandidate provenance  +  CanonicalEntityMapping
           ↓ exact Evidence-local surface matching
      EventEntityLink[] / UnresolvedEventEntityMention[]
           ↓
NormalizedEventCandidate
```

- **Temporal normalization is conservative and precision-preserving.** Only a
  narrow set of unambiguous formats is normalized, per the mention's
  `TemporalExpressionKind`; everything else stays `UNRESOLVED` with a machine
  reason (never guessed). No false precision is introduced — `2026` → `YEAR`,
  `May 2026` → `MONTH` (no day), `14:30` → `MINUTE` (no date) — and **no timezone
  is inferred** (absent offset stays `None`; an explicit `Z`/`+09:00` is recorded
  numerically and the time is **not** converted to UTC). Invalid calendar
  dates/times (`2026-02-30`, `25:00`), `RELATIVE`, `RANGE`, `OTHER`, and
  kind/format mismatches all remain `UNRESOLVED`. The original source text and
  kind are always preserved; each mention gets exactly one normalization record,
  in source order (never combined across mentions or Evidence).
- **Entity linking is Evidence-local and exact.** Links come only from the event's
  **supporting quotes** (never the analytical `description`), matching only
  previously observed `EntityCandidate` surface forms that are grounded to the
  **same Evidence** as the quote — exact, case-sensitive, non-normalized (the
  resolution key is not used). Repeated occurrences and overlapping forms are kept
  distinct; duplicate candidates aggregate into one link; a surface that resolves
  to multiple Entities becomes an `UnresolvedEventEntityMention` rather than a
  guess. An `EventEntityLink` records **mention provenance only — not a participant
  role** (no actor/target/location semantics). Inputs are never mutated, output is
  order-independent, and **no domain `Event` is created**.

### Event participant roles & canonical drafting

Classifies a semantic role for each Entity already linked to the event, then
assembles a provenance-preserving draft:

```
NormalizedEventCandidate
        ↓ closed-set participant-role resolution (LLM)
EventParticipantRoleResolutionRecord
        ↓ deterministic drafting
CanonicalEventDraft
```

- **Roles are assigned only to existing `EventEntityLink` entities.** The LLM
  operates over a **closed set** of already-linked canonical Entity IDs
  (versioned `event-participant-role-resolution-v1`): it cannot introduce a new
  Entity, promote an `UnresolvedEventEntityMention` (those are marked
  non-selectable), or invent an `EventEntityLink`. Every returned role must cite a
  supporting link ID that belongs to that same Entity; unknown/duplicate/missing/
  unresolved-only IDs are rejected. Provider errors propagate with no retry.
- **One linked Entity is not automatically ACTOR.** There is no single-entity
  shortcut — whenever ≥1 definitive link exists, the provider is consulted; the
  only provider-free path is *no links at all* (empty roles, deterministic
  method, no provider metadata). Each Entity gets exactly one
  `EventParticipantRole` (`ACTOR`/`TARGET`/`PARTICIPANT`/`LOCATION`/`OTHER`) or
  `UNRESOLVED` — no multi-role output, no confidence score, and no
  `EntityType`→role heuristic (`LOCATION` is a semantic judgement, not derived
  from the entity's type).
- **`CanonicalEventDraft` is not a domain `Event` and not resolved Event
  identity.** It is a canonicalized analytical draft for one `EventCandidate`
  lineage (candidates are never merged — Event Resolution is later work). It
  preserves the description, `EventType`, supporting quotes, **all** temporal
  mentions + normalizations unchanged (no new temporal reasoning — precision and
  unresolved RELATIVE/RANGE state are untouched), **every** `EventEntityLink` and
  `UnresolvedEventEntityMention` ID, and the Issue #18 extraction provenance kept
  **separate** from the role-resolution provider provenance. No domain `Event` is
  created (a canonical title/identity and resolved datetimes cannot be populated
  without fabrication).

### Event resolution grouping (deterministic, conservative)

A **precision-over-recall** first-stage that groups only Event drafts that are
*identical* under strict deterministic identity signals — no LLM, no fuzzy or
semantic similarity:

```
CanonicalEventDraft[]
        ↓ strict deterministic identity signals
EventResolutionSignature (EventType + normalized description + hash + temporal points + participant roles)
        ↓ exact signature equality (no transitivity)
EventResolutionGroup[]
```

- **Exact description alone is insufficient.** Grouping requires an exactly-equal
  `EventType`, an exactly-equal normalized description (via Issue #7 `normalize_text`
  — case/punctuation preserved, so `announced.` ≠ `announced!`; the normalized
  string *and* its SHA-256 are both part of the signature), **and** at least one
  high-precision discriminator: a non-empty normalized temporal signature or a
  non-empty assigned-participant signature. Two recurring events with the same
  description/type but no temporal/participant discriminator stay in separate
  singleton groups.
- **Uncertainty blocks auto-merge.** Any `UNRESOLVED` temporal normalization
  (`RELATIVE`/`RANGE`/invalid/unsupported), any `UNRESOLVED` participant role, or
  any unresolved Entity mention makes a draft ineligible → it becomes a singleton
  (signature `None`). Temporal points come only from Issue #19 (deduplicated,
  deterministically ordered, timezone-preserving — `Z` ≠ `+09:00`, no UTC
  conversion); participants come only from Issue #20 assigned `(entity_id, role)`
  pairs. When both signals are present, **both** must match exactly.
- **Provenance is preserved, not proof of identity.** Every input draft survives
  in exactly one group; ineligible/unmatched drafts remain singletons. Groups keep
  all member draft IDs, source `EventCandidate`/`NormalizedEvent` IDs, and a
  deterministic first-seen Evidence union — so drafts from **different** Evidence
  lineages may still group, and sharing Evidence never by itself forces a group. An
  identical signature is a conservative signal, **not** proof of the same
  real-world Event; an `EventResolutionGroup` is not yet a domain `Event`.

### Canonical Event mapping

Maps exactly **one** `EventResolutionGroup` (plus the original `CanonicalEventDraft`
lookup) into one provenance-preserving resolved-Event identity — deterministic, no
LLM:

```
EventResolutionGroup + CanonicalEventDraft lookup
        ↓ strict group/draft consistency validation
        ↓ observed canonical-description selection
ResolvedEventRecord + EventResolutionMappingRecord   (domain Event still deferred)
```

- **One group → one resolved Event.** No merging, splitting, re-resolution, or
  fuzzy similarity. The group is first re-validated against its member drafts
  (source `EventCandidate`/`NormalizedEvent` lineage, `EventType`, the Issue #21
  Evidence union, and — for eligible groups — the recomputed Issue #21 signature);
  a stale/tampered or invalid null-signature multi-member group is **rejected**.
- **Canonical description is selected, never synthesized.** It is always an *exact*
  observed member `description`, chosen deterministically (highest exact frequency,
  then smallest `draft_id`, then string) with the contributing draft IDs recorded —
  no longest/prettiest/title-case/LLM rewriting.
- **No title, no false precision.** No Event `title` is invented (canonical
  description is *not* a title), common temporal points are copied from the Issue
  #21 signature exactly (partial precision and explicit offsets preserved — `2026`
  stays `YEAR`, `+09:00` is not converted to UTC), and unresolved `RELATIVE`/`RANGE`
  states stay unresolved. All draft/candidate/Evidence/Entity-link/unresolved-mention/
  role-resolution provenance is preserved.
- **The domain `Event` remains deferred.** Because no defensible independent title
  exists, the default result carries `domain_event = None` with an explicit
  `EVENT_TITLE_NOT_AVAILABLE` blocker — a `ResolvedEventRecord` +
  `EventResolutionMappingRecord` is the deliverable. Nothing is persisted.

## Citation & provenance service

A **pure, read-only view layer** (`src/traceintel/provenance/`) projects the
analytical records produced above into deterministic *citation bundles*. It
performs **no LLM calls, no network, no persistence, and no configuration** —
it mints no IDs and stamps no timestamps, so a bundle is a pure function of its
inputs.

```
analytical root (Entity mapping / claim draft / support / contradiction / resolved Event)
        ↓ resolve each analytical reference to its Evidence
        ↓ Evidence → Document → Source  (+ raw-text provenance revalidation)
ProvenanceBundle { citations[], paths[] }
```

- **Evidence is the atomic citation anchor.** `resolve_evidence_citation` walks
  Evidence → Document (`evidence.document_id`) → Source (`document.source_id`),
  and for offset-bearing Evidence revalidates the raw-text slice through the
  shared `validate_evidence_provenance` before emitting a `ProvenanceCitation`.
  Analytical artifacts (drafts, assessments, mappings) are never treated as
  citations — only Evidence is.
- **Five roots, one bundle shape.** `build_entity_provenance`,
  `build_claim_draft_provenance`, `build_claim_support_provenance`,
  `build_claim_contradiction_provenance`, and `build_resolved_event_provenance`
  each return a `ProvenanceBundle`: `citations` deduplicated by Evidence, plus
  `ProvenancePath`s (typed node chains) that keep every distinct analytical
  lineage **separate** — a contradiction's left and right sides are never merged.
- **Fail-closed and all-or-nothing.** A missing referenced record raises
  `ProvenanceConsistencyError`; a lookup key that disagrees with its stored
  object raises `ProvenanceInputError`; a stale/tampered record (e.g. a draft's
  `evidence_ids` that no longer match its quotes, or Document text that drifted)
  aborts the **whole** bundle. Extra lookup entries are ignored; partial bundles
  are never emitted.
- **No ranking, reliability, style, or truth.** The service is a typed adapter,
  not a graph engine. `CitationSourceMetadata` deliberately omits
  `reliability_tier`; `CitationDocumentMetadata` carries identity + `content_hash`
  but never `raw_text`; nothing renders a citation string or orders sources by
  trust. It exposes *where* evidence came from, never a judgement about it.

## Citation rendering & provenance export

A deterministic presentation/export layer (`src/traceintel/provenance/rendering.py`,
`export.py`, `render_models.py`) turns a `ProvenanceBundle` into machine- and
human-readable forms **without changing provenance semantics**. It consumes only
the bundle — it never traverses analytical artifacts, resolves Evidence, opens
Documents, loads Sources, calls an LLM, touches the network, or persists
anything.

```
ProvenanceBundle
      ↓
Citation Rendering
      ├─ canonical JSON        (export_provenance_json)
      ├─ plain citation list   (render_plain_citations)
      └─ Markdown footnotes    (render_markdown_footnotes)
```

- **Citation numbering derives from `bundle.citations` order** — never from
  alphabetizing Evidence IDs. If the bundle lists `ev-002, ev-001, ev-003`, they
  become citations `1, 2, 3` in that order. A `CitationIndex` gives O(1)
  `evidence_id → number/reference` lookups for downstream callers.
- **Shared Evidence gets one citation number.** When several provenance paths
  reach the same Evidence (e.g. a contradiction's left and right sides, or two
  Entity candidates), it appears once in the citation list; every path resolves
  to that single number while the paths themselves stay distinct.
- **No bibliographic metadata is invented.** Labels use a TraceIntel-native
  labeled-field format built only from fields actually present
  (`Source: <locator>; Document: <document-id>; Evidence: <evidence-id>`, with a
  Source title qualifier when one exists). No author, date, publisher, access
  date, or DOI is fabricated; minimal metadata falls back to stable TraceIntel
  IDs, which are real information.
- **Canonical JSON is deterministic and lossless.** Object keys are sorted;
  `citations`/`paths`/`steps` order is preserved; Unicode stays human-readable
  (`ensure_ascii=False`); there is no trailing newline and no generated
  timestamp/ID, so the same bundle always produces byte-identical output. It
  round-trips back to an equal `ProvenanceBundle`.
- **Render-safe by construction.** Source/Document metadata is untrusted display
  text: presentation newlines are collapsed to spaces and Markdown-special
  characters are escaped so metadata can never inject an extra citation line or
  footnote definition. The *stored* provenance values and Evidence `exact_text`
  are never mutated — sanitization affects the presentation label only.
- **No report generation yet.** This layer prepares citation material and
  indexes; it does not write prose, place citation markers inside sentences, or
  emit PDF/DOCX/HTML.

## Deterministic report assembly

A cross-domain layer (`src/traceintel/reports/`) assembles already-produced
analytical artifacts and their `ProvenanceBundle`s into a structured,
machine-readable `ResearchReport`. It performs **no new analysis** and makes
**no LLM/network/persistence** calls — it is a projection, not a writer.

```
Analytical artifacts + ProvenanceBundles
        ↓
Deterministic Report Assembly
        ↓
ResearchReport
├─ Entity findings
├─ Claim findings
├─ Claim contradiction findings
├─ Event findings
└─ Report-wide citation registry + provenance paths
```

- **The report is structure, not narrative.** It holds ordered sections of typed
  findings (`EntityReportFinding`, `ClaimReportFinding`,
  `ClaimContradictionReportFinding`, `EventReportFinding`) — no summary,
  conclusion, recommendation, or aggregate confidence, and no generated prose.
- **Caller-defined order.** Section titles, section order, and finding order are
  all supplied by the caller and preserved exactly — findings are never
  regrouped by kind, importance, or chronology (there is no importance ranking).
- **Report-wide, first-seen citations.** One `ReportCitationRegistry` numbers
  Evidence once for the whole report, in first-seen order across
  section → finding → each finding's `ProvenanceBundle.citations`. Evidence
  shared by several findings gets a single report-wide number; each finding
  carries the resolved global numbers parallel to its Evidence IDs.
- **Provenance is preserved and validated.** Every finding must be paired with
  the `ProvenanceBundle` whose root matches its artifact (kind and ID); the same
  root artifact may not appear twice. All provenance paths are preserved
  (annotated with their finding and global citation number). Cross-bundle
  conflicts fail closed: the same Evidence ID with different citation content
  across bundles, a mismatched support assessment, or a dangling path aborts the
  whole report.
- **Analytical state is preserved verbatim.** `InformationType`
  (FACT/CLAIM/INFERENCE/ESTIMATE/OPINION/UNKNOWN), claim support state,
  contradiction relation, Event temporal precision, and domain blockers are
  copied unchanged. FACT is never promoted to verified truth, a contradiction
  never selects a winner, and no Event title is invented.
- **No document rendering yet.** Assembly produces structured Pydantic models
  only — no Markdown/HTML/PDF/DOCX report, and no citation placement in prose.

## Markdown report rendering

`render_research_report_markdown` (`src/traceintel/reports/rendering.py`) turns a
`ResearchReport` into one deterministic, human-readable Markdown document. It
consumes only the report (plus Issue #24's citation-label primitive) and makes
**no LLM/network/persistence** calls.

```
ResearchReport
      ↓
Deterministic Markdown Renderer
      ↓
Human-readable Markdown
```

- **No narrative synthesis.** The renderer emits a titled document with a
  metadata block, caller-ordered sections (`##`) and findings (`### Entity`,
  `### Claim`, `### Claim Contradiction`, `### Event`) as neutral labeled-field
  bullet lists, and a final `## Citations` block — no summary, conclusion,
  recommendation, or generated prose.
- **Caller order and analytical wording preserved.** Section titles, section
  order, and finding order are rendered exactly as stored; Claim statements and
  Event descriptions appear verbatim (presentation-escaped only). `InformationType`,
  claim support state (`Not assessed` shown when none was supplied), contradiction
  relation, and domain blockers render as their exact enum values — FACT is never
  labelled "verified", and a contradiction never marks a winner.
- **Temporal precision preserved.** `NormalizedTemporalPoint`s render only the
  components present, tagged with their precision (e.g. `2026-05 (MONTH)`,
  `2026-05-12T14:30+09:00 (MINUTE)`) — a YEAR never becomes a full date and no
  timezone is inferred.
- **Report-wide footnote numbering.** Findings render `[^n]` markers using the
  report's stored citation numbers (never recomputed); Evidence shared across
  findings reuses a single number and is defined once. Citation definitions reuse
  Issue #24's `markdown_citation_label`, so the citation style cannot drift.
- **Markdown-safe.** All report text is untrusted display data: newlines collapse
  to spaces and Markdown-active characters are escaped, so content cannot inject
  headings, list items, code fences, links, or footnote definitions. The
  underlying `ResearchReport` is never mutated. Output is deterministic (no clock,
  no IDs) and ends with exactly one newline.
- **No PDF/DOCX/HTML yet.** The renderer returns a Markdown string; file writing
  and other formats are out of scope.

## Report export & reproducibility

`build_research_report_export` (`src/traceintel/reports/export.py`) packages a
`ResearchReport` into an in-memory `ResearchReportExportEnvelope`. It is pure and
deterministic — no LLM, network, persistence, filesystem writes, archives,
signing, clock, or random source — and consumes only the report.

```
ResearchReport
      ↓
Deterministic Export
      ├─ canonical report JSON      (export_research_report_json)
      ├─ deterministic Markdown     (Issue #26 renderer, reused)
      ├─ canonical provenance JSON  (export_report_provenance_json)
      └─ reproducibility manifest   (SHA-256 hashes + method/version metadata)
```

- **Canonical, lossless report JSON.** The structured report serializes with
  sorted object keys, preserved list order, human-readable Unicode, and no
  trailing newline; it round-trips back to an equal `ResearchReport`. It is the
  machine-readable source of truth; the Markdown is its deterministic human
  presentation.
- **Compact provenance snapshot.** `ReportProvenanceSnapshot` copies the report's
  citation registry and provenance paths verbatim (no re-traversal, no Evidence/
  Source/DB lookup) and canonicalizes to its own audit JSON.
- **Exact-byte SHA-256.** The manifest records `report_json_sha256`,
  `markdown_sha256` (including Issue #26's single final newline), and
  `provenance_json_sha256` as lowercase 64-char hex over the exact UTF-8 bytes,
  plus the producing TraceIntel version, the assembly and Markdown-renderer method
  identities, deterministic source-artifact refs (in section→finding order),
  Evidence IDs (in report citation order), and counts. It carries **no export
  timestamp, ID, host, path, or environment data**.
- **Hashes prove identity, not truth.** A SHA-256 match means *these exact bytes
  came from this exact report under these methods* — never that Evidence is true,
  a Source is reliable, a Claim is verified, or the report is authoritative. There
  is no signature, certificate, or trust field.
- **Reproducibility verification.** `verify_research_report_export` re-checks the
  hashes *and* regenerates the Markdown and provenance JSON from the parsed report
  JSON, so an internally hash-consistent but non-reproducing envelope (e.g. edited
  Markdown with a recomputed hash) is still rejected. This is internal
  reproducibility integrity, not third-party authenticity — a party who can alter
  both the content and the unsigned manifest can produce a different consistent
  envelope.
- **No archive/file output yet.** The envelope is in-memory Pydantic models and
  strings; ZIP packaging is out of scope. Writing the envelope to disk is the
  separate, opt-in file writer below.

## Export file writer

`write_research_report_export` (`src/traceintel/reports/writer.py`) is a narrow,
opt-in filesystem boundary that materializes one already-built
`ResearchReportExportEnvelope` into a deterministic directory. It performs no
analysis and makes no network/upload/signing/archive calls.

```
ResearchReportExportEnvelope
        ↓
Explicit filesystem writer
        ↓
<caller destination>/
├── report.json
├── report.md
├── provenance.json
└── manifest.json
```

- **Opt-in, caller-chosen destination.** The writer never picks a location
  (home, Downloads, cwd, `/tmp`); the caller passes the export directory
  explicitly. The destination's parent must already exist; a missing destination
  directory is created.
- **Verified before writing.** The envelope is reproducibility-verified
  (Issue #27) before any filesystem mutation — an invalid envelope writes
  nothing.
- **Fixed filenames.** Output is always exactly `report.json`, `report.md`,
  `provenance.json`, `manifest.json` — never derived from the report title, ID,
  or any content, so there is no title/ID path-traversal surface.
- **Exact bytes.** Files hold the envelope's exact UTF-8 bytes (canonical JSON
  with no trailing newline; `report.md` keeps Issue #26's single final newline);
  `manifest.json` uses the Issue #27 serializer. Written files are read back and
  checked against the envelope bytes, the manifest SHA-256s, and the reparsed
  manifest.
- **Overwrite protection & symlink rejection.** `overwrite` defaults to `False`;
  if any owned target already exists the whole operation aborts before writing.
  A destination that is a symlink or non-directory, and any target that is a
  symlink, directory, or special file, is rejected (even with `overwrite=True`).
  Unrelated files in the destination are always preserved; nothing is recursively
  deleted.
- **Staged materialization.** Files are written to a private staging directory
  first; a missing destination is published by atomically renaming the whole
  staged directory into place, and an existing directory is updated by per-file
  `os.replace`. Staging is cleaned up on failure. **Replacing four files in an
  existing directory is best-effort per file, not a single atomic multi-file
  filesystem transaction**, and there is **no cross-process locking** — concurrent
  writes to the same destination are unsupported.
- **Operational vs reproducibility metadata.** The write returns a
  `ResearchReportExportWriteResult` (destination and file paths, byte count,
  hashes) — the destination path lives only here, **never** inside `manifest.json`
  (which stays environment-independent). No ZIP, upload, signing, or CLI command
  is included.

## Export directory verifier

`verify_research_report_export_directory` (`src/traceintel/reports/directory_verifier.py`)
is a fully offline, **read-only** auditor that takes only an existing export
directory and independently decides whether it is a valid, canonical, reproducible
TraceIntel export — no original in-memory `ResearchReport` or envelope is required.

```
export directory
├── report.json
├── report.md
├── provenance.json
└── manifest.json
        ↓
offline directory verifier
        ↓
verification result
```

- **Directory-only input.** The directory is the complete verification input; the
  verifier reconstructs the export envelope from the four files and reuses the
  Issue #27 reproducibility verifier for the export-semantics layer.
- **Two layers.** Layer A checks the materialized files (present, regular,
  non-symlink; exact bytes read once via `read_bytes`; strict UTF-8; models parse;
  each JSON file equals its **canonical** serialization byte-for-byte). Layer B
  delegates to `verify_research_report_export`, which re-checks the SHA-256 hashes
  and **regenerates the Markdown and provenance JSON from `report.json`** — so a
  pretty-printed JSON, an added trailing newline, a UTF-8 BOM, a CRLF-converted
  Markdown, or an edited payload with a recomputed hash all fail.
- **Read-only, fail-closed.** It never writes, repairs, recanonicalizes, or
  creates temp/sidecar files; the directory is byte-for-byte untouched on success
  and on failure. Invalid or non-canonical content raises a typed error.
- **Safe by construction.** A destination that is itself a symlink, and any
  required child that is a symlink or non-regular file, are rejected; filenames
  are the fixed constants, so the report title/ID never affect paths. Unrelated
  files in the directory (e.g. `notes.txt`) are allowed and ignored.
- **Consistency, not authenticity.** A successful result means the directory is
  internally consistent and reproducible under the *current* renderer/export
  implementation — not that its content is true or cryptographically authentic.
  The manifest is unsigned, so a party who replaces **all four** files together
  can produce a different internally consistent export. An export from an
  incompatible past/future renderer may fail even if valid under that version.
  Verification assumes the four files remain stable during one run (no
  cross-process locking).

### CLI verification

The verifier is exposed as a read-only CLI command:

```
traceintel report verify ./traceintel-export
traceintel report verify ./traceintel-export --json
```

- The CLI is an interface adapter only: it delegates every check to the
  directory verifier (validating the canonical materialized files and their
  reproducibility) and touches no database or network. It never repairs or
  modifies the directory, and it does not prove truth or authenticity — only
  export integrity/reproducibility.
- **Exit codes:** `0` for a valid export; `2` (stable) for an invalid or
  inconsistent export. Unexpected internal errors are not disguised as
  verification failures.
- Human output is concise (report ID, producer TraceIntel version, counts) and
  prints no Evidence text, Source URLs, report title, hashes, or local paths.
  Expected failure messages redact the supplied export-directory path to a
  `<export-directory>` placeholder while keeping owned child filenames (e.g.
  `manifest.json`) for diagnosis.
  `--json` writes exactly one deterministic JSON object to stdout — on success
  (`"status": "ok"` with identity/counts) *and* on expected failure
  (`"status": "failed"` with `error_type`/`message`, still exiting non-zero) —
  so scripts can always parse stdout.
- `traceintel --version` continues to report the runtime package version; the
  `Producer TraceIntel version` in verification output is the version recorded
  by whoever produced the export.

## Standalone envelope transport

A canonical, versioned single-file JSON representation for a
`ResearchReportExportEnvelope` (`src/traceintel/reports/envelope_transport.py`),
intended as the clean input boundary for later materialization workflows (e.g. a
future `traceintel report materialize envelope.json <destination>`) without any
DB or analytical reconstruction.

```
ResearchReportExportEnvelope
        ↓
export_research_report_envelope_json   (verifies first)
        ↓
canonical transport JSON  (format_version + envelope)

envelope.json
        ↓
load_research_report_envelope_file / load_research_report_envelope_json
        ↓
verified ResearchReportExportEnvelope
```

- **Versioned transport wrapper.** The payload is
  `{"envelope": …, "format_version": "traceintel-research-report-envelope-v1"}` —
  the version identifies the transport format (distinct from the manifest
  version); unknown versions are rejected with no automatic migration. The
  existing envelope model is unchanged, and the transport carries no ID,
  timestamp, self-hash, or file path.
- **Canonical, deterministic JSON.** The shared canonical contract applies:
  sorted object keys, preserved list order, human-readable Unicode (no
  base64/compression), compact separators, **no trailing newline** — byte-identical
  across calls. The serializer re-verifies the envelope (Issue #27) first, so an
  invalid envelope never receives a canonical transport form.
- **Strict, read-only loading.** The loader is not a permissive parser: the
  payload must byte-for-byte equal its own canonical reserialization
  (pretty-printing, key reordering, leading/trailing whitespace, and a BOM all
  fail; nothing is silently recanonicalized), must be strict UTF-8, and the
  embedded envelope must pass Issue #27 reproducibility verification — a
  tampered payload with recomputed hashes is still rejected because the Markdown/
  provenance no longer reproduce from the embedded report JSON. The file loader
  only accepts an existing regular non-symlink file, reads exact bytes, performs
  no repair or writes, and never embeds the file's location in the result.
- **No signing.** The transport is unsigned; canonicality and reproducibility
  establish internal consistency, never authenticity.

### CLI materialization

The transport is materializable via a narrow CLI command:

```
traceintel report materialize ./envelope.json ./traceintel-export
traceintel report materialize ./envelope.json ./traceintel-export --json
```

```
canonical envelope file
→ verified Issue #31 loader
→ Issue #28 writer
→ standard four-file directory
```

- The CLI is a pure adapter — it performs no DB access, analysis, network, or
  repair, parses no envelope JSON itself, and touches no files directly: the
  loader verifies the transport and its embedded reproducibility, and the writer
  verifies the envelope again before staging/publishing (intentional defense in
  depth).
- **Overwrite is off by default**; `--overwrite` explicitly enables replacement
  of TraceIntel-owned export files under the writer's existing symlink/conflict
  policies (unrelated files preserved, nothing recursively deleted).
- **Exit codes:** `0` success, `2` invalid/unloadable envelope transport, `3`
  destination/write failure. Unexpected internal errors remain distinguishable.
- Output is concise (report ID, files/bytes written, layout version) with no
  report content; expected failures redact the caller's local paths to
  `<envelope-file>` / `<destination>` placeholders (child filenames such as
  `report.md` stay visible for diagnosis). Materializing does not prove
  authenticity or truth.

## Canonical research runs

`traceintel.runs` defines the canonical record binding one complete research
execution together. The model layer (Issue #33) is a structural orchestration
contract only; the deterministic executor (Issue #34, below) fills it in.

```
Explicit Sources/Documents/Evidence
          ↓
Canonical ResearchRun
          ↓
typed stage lineage
          ↓
analytical artifacts
          ↓
Provenance / Report / Export
```

- **Explicit inputs only.** A `ResearchRunInput` embeds the caller-supplied
  Sources, Documents, and Evidence in caller order (order is part of run
  reproducibility). Nothing is loaded from a database, configuration, previous
  runs, the working directory, or the web — no hidden fallback, no autonomous
  crawling. Input references are validated (Document→Source,
  Evidence→Document) and offset-bearing Evidence is revalidated with the shared
  exact raw-text provenance check.
- **Deterministic typed stages.** `ResearchRunStage` maps one-to-one onto
  existing components (entity/claim/event extraction through canonicalization,
  provenance build, report assembly, export build), with a fixed canonical
  order (`RESEARCH_RUN_STAGE_ORDER`) and static dependencies
  (`RESEARCH_RUN_STAGE_DEPENDENCIES`) — no scheduler, no workflow engine.
  Stage records are lineage/index entries (status, artifact IDs, typed concise
  errors), never payload containers.
- **Typed artifacts, preserved identities.** `ResearchRunArtifacts` holds the
  existing analytical model types (candidates, resolution groups, canonical
  mappings, drafts, assessments, `ProvenanceBundle`s, `ResearchReport`,
  `ResearchReportExportEnvelope`) by reference — never clones with new IDs.
  Assembly cross-validates: completed stages require completed dependencies,
  stage output IDs must resolve to actual artifacts, provenance roots must
  match artifacts, and report/export identities must agree with their stages.
- **Derived status.** `READY`/`PARTIAL`/`FAILED`/`COMPLETED` is computed from
  the stage records, never independently set. Zero findings do not imply
  failure — and **`COMPLETED` means the pipeline stages completed, not that any
  finding is objectively true**.
- Runs are immutable snapshots assembled by `assemble_research_run` (injectable
  ID factory and clock, one clock read); no DB persistence, filesystem I/O, or
  CLI command exists for runs yet.

## Deterministic run executor

`execute_research_run` (in `traceintel.runs`) executes one research run end to
end **in memory**, orchestrating only the existing analytical functions —
entity/claim/event extraction through canonicalization, provenance build,
report assembly, and export build — in canonical `RESEARCH_RUN_STAGE_ORDER`.
It introduces no new analytical logic, prompts, or scoring.

- **Everything is explicit.** The caller supplies the `ResearchRunInput`
  snapshot, an `LLMProvider` (the protocol only — the executor never imports a
  concrete provider), a `ResearchRunReportSpec`, and the injectable ID
  factory/clock. Nothing is read from configuration, storage, the filesystem,
  or the network by the executor itself; the run's report and export envelope
  are returned in memory and materialized (if at all) by the existing
  writer/CLI layers.
- **Caller-defined report structure.** `ResearchRunReportSpec` names the
  report title and its sections, each selecting *finding kinds* (`ENTITY`,
  `CLAIM`, `CLAIM_CONTRADICTION`, `EVENT`). A kind may appear in at most one
  section; omitted kinds stay out of the report (their artifacts remain in the
  run); an empty section is allowed and stays empty. Findings appear in
  deterministic artifact order — the executor never ranks, summarizes, or
  invents sections.
- **Sequential and stage-atomic.** Stages run one at a time with no
  parallelism, retries, or timeouts added. A stage's outputs are committed
  all-or-nothing: if it fails partway (e.g. on the second document), nothing
  from that stage enters the run.
- **Fail-closed failure semantics.** The first *expected* failure (the typed
  analytical error families plus `LLMProviderError`) marks that stage `FAILED`
  and every later stage `SKIPPED` (`upstream-stage-failed:<STAGE>`), makes no
  further provider calls, and returns a `FAILED` run that preserves all
  completed-stage artifacts. Stage error records carry only the exception
  class, stage, and (where relevant) a document ID — never raw error text,
  Evidence bodies, prompts, or provider responses. Unexpected errors
  (`RuntimeError`, etc.) propagate — they are never converted into a "clean"
  failed run.
- **Deterministic.** With the same input snapshot, the same provider responses,
  and the same injected ID factory/clock, executing twice yields identical
  serialized runs. Stage records carry stable orchestration method identities
  (`research-run-<stage>-v1`); analytical provenance stays on the artifacts.
- The final run is assembled through `assemble_research_run`, so every Issue
  #33 cross-stage validation applies to executor output. **A `COMPLETED` run
  means the stages completed — it never means the findings are true.**

## Run checkpoint persistence & resume preparation

ResearchRun snapshots can be durably checkpointed — explicitly, and only when
the caller asks for it:

```
execute_research_run
       ↓
immutable ResearchRun snapshots
       ↓
optional explicit checkpoint sink
       ↓
SQLite append-only checkpoint store
```

- **Checkpointing is optional and explicit.** `execute_research_run` accepts a
  `checkpoint_sink` (the `ResearchRunCheckpointSink` protocol — one `record(run)`
  method) defaulting to `None`. Without a sink, nothing is persisted and
  behavior is exactly as before; there is no automatic persistence, no hidden
  database, and no configuration flag that silently enables checkpointing. With
  a sink, the executor emits only canonical committed snapshots: the initial
  READY snapshot, one snapshot after each successfully committed stage, and the
  final FAILED snapshot on an expected failure — 17 snapshots for a fully
  successful run, never a pre-commit partial state.
- **Versioned canonical snapshot format.** `export_research_run_snapshot_json`
  first runs the full Issue #33 cross-stage consistency validation
  (`validate_research_run_consistency` — a hand-built inconsistent run never
  gets a snapshot), wraps the run in the versioned
  `ResearchRunSnapshotTransport` (`traceintel-research-run-snapshot-v1`), and
  serializes with the shared canonical JSON contract.
  `load_research_run_snapshot_json` rejects anything non-canonical (pretty
  printing, reordered keys, whitespace, trailing newline, BOM), unknown format
  versions, extra fields, and any embedded run that fails consistency
  validation. The `ResearchRun` model itself carries no persistence metadata.
- **Append-only checkpoints.** `ResearchRunCheckpointStore` (in
  `traceintel.storage`) persists `(run_id, checkpoint_index, snapshot_json,
  snapshot_sha256)` rows starting at index 0, allocating the next index and
  inserting in one transaction. Checkpoints are never updated or deleted (no
  deletion/pruning API exists); appending a byte-identical snapshot is
  idempotent; and each run ID is one linear lineage — its input, `created_at`,
  title/objective, and every already-COMPLETED stage record and committed
  artifact are frozen across later checkpoints. Terminal snapshots (FAILED or
  COMPLETED) accept no materially different continuation in v1.
- **SHA-256 is integrity, not authentication.** The stored digest (over the
  exact UTF-8 canonical bytes) detects accidental or content-level
  modification; it proves nothing about producer identity, authenticity,
  trust, or research truth, and everything remains unsigned.
- **Reads fail closed.** `load(run_id, index)` / `load_latest(run_id)` verify
  the digest, canonicality, full run consistency, and that the payload's run
  ID matches the row key; `list_checkpoints` returns ascending verified
  metadata only. Any mismatch is a typed error, never a silently repaired run.
- **Resume preparation is read-only.** `prepare_research_run_resume` describes
  where a loaded run could continue (READY/PARTIAL → first NOT_STARTED stage;
  FAILED → only through an explicit `RETRY_FAILED_STAGE` request, never plain
  continuation; COMPLETED → nothing to resume). The persistence layer never
  invokes an analytical function.

## Explicit resume contract & resume executor

A persisted FAILED or PARTIAL run is continued only through an explicit,
auditable transition — there is no implicit or automatic retry:

```
FAILED/PARTIAL checkpoint
        ↓
explicit ResumeRequest
        ↓
resume transition
        ↓
prepared ResearchRun
        ↓
resume_research_run
        ↓
new immutable checkpoints
```

- **Resume is always explicit.** `ResearchRunResumeRequest` names the run, the
  exact checkpoint the caller inspected, one of exactly two modes
  (`CONTINUE_PARTIAL` for READY/PARTIAL runs, `RETRY_FAILED_STAGE` for FAILED
  runs), and the stage the caller expects to execute next (stale UI/automation
  state fails fast). COMPLETED runs are never reopened. No automatic retry, no
  backoff, no max-retry policy: one explicit request is one execution attempt.
- **History is never rewritten.** The FAILED checkpoint stays stored verbatim.
  `prepare_research_run_for_resume` (pure — no provider, SQLite, IDs, or
  clock) produces a *new* snapshot in which only the failed stage and its
  downstream SKIPPED stages become canonical NOT_STARTED records; completed
  stage records and all committed artifacts stay exactly equal, run identity
  (`run_id`, `created_at`, title/objective, input) is frozen, and the status
  is re-derived (PARTIAL, or READY when the first analytical stage failed).
  PARTIAL/READY continuation changes no state at all — the prepared snapshot
  equals the source. Lineage stays linear: no branches, no alternate heads.
- **Atomic persisted transitions.** `ResearchRunCheckpointStore.
  append_resume_transition(request)` verifies the exact requested checkpoint,
  requires it to still be the latest (stale requests are rejected) and not
  already consumed by an earlier resume record, then — in one transaction —
  appends the prepared checkpoint (retry only; continuation references the
  existing checkpoint instead of duplicating bytes) plus an append-only
  `ResearchRunResumeRecord` audit row (`run_id` + `resume_index`, linking
  source → prepared checkpoint; no UUID, no timestamp). Ordinary `append`
  stays fully strict — only this explicit API can persist a FAILED → prepared
  transition. `list_resume_records(run_id)` returns the ascending audit trail.
- **The resume executor reuses the engine.** `resume_research_run(run=…,
  resume_stage=…, provider=…, report_spec=…, id_factory=…, clock=…,
  checkpoint_sink=…)` runs the *same* stage runners as `execute_research_run`
  from the run's first NOT_STARTED stage. Completed stages are never
  re-executed (an export-only retry makes zero provider calls); existing
  artifacts are inputs, not something to regenerate; failure semantics,
  atomicity, and determinism are identical to initial execution. Newly minted
  IDs are checked against every existing run ID — collisions fail closed.
  With a checkpoint sink, only newly committed stages emit snapshots (the
  prepared snapshot is presumed already persisted). Works fully in memory —
  SQLite is optional.
- **Report spec remains an explicit input.** ResearchRun snapshots do not
  persist the `ResearchRunReportSpec`, so resume takes it explicitly: it
  shapes report assembly only when REPORT_ASSEMBLY has not yet completed, and
  an existing `ResearchReport` is preserved untouched. For pre-report
  resumptions TraceIntel cannot prove the supplied spec equals the original
  intent — a documented v1 limitation.

## Operational run CLI

The mature execution/recovery system is exposed through a narrow CLI that is
purely an adapter over the existing APIs:

```
canonical run request
        ↓
traceintel run execute <request-file> --database <sqlite-path>
        ↓
SQLite checkpoints
        ↓
traceintel run inspect <run-id> --database <sqlite-path>
traceintel run checkpoints <run-id> --database <sqlite-path>
traceintel run resumes <run-id> --database <sqlite-path>
        ↓
explicit resume request
        ↓
traceintel run resume <request-file> --database <sqlite-path>
```

- **Canonical request files, not flags.** Initial execution takes a versioned
  `traceintel-research-run-execution-request-v1` file (explicit
  `ResearchRunInput`, `ResearchRunReportSpec`, and provider specification;
  optional `run_id`/title/objective); resume takes a separate
  `traceintel-research-run-resume-request-v1` file (the Issue #36 resume
  request naming the exact source checkpoint, plus report spec and provider).
  Both loaders enforce the exact canonical JSON form — pretty printing,
  reordered keys, BOM, trailing newline, unknown fields, and unknown versions
  all fail — and read files strictly (no symlinks, strict UTF-8, no writes).
  Request files carry **no credentials**; the v1 format has no credential
  fields at all.
- **Explicit provider, explicit database.** The request names the provider
  through an explicit `kind`-discriminated union (Issue #51): `"ollama"`
  with `model` and a loopback-only `base_url` (all existing Ollama safety
  constraints unchanged), or the remote `"openai"`/`"anthropic"` kinds with
  `kind` and `model` only — never credentials or endpoints. No environment
  overrides, no fallback, no automatic selection; unknown kinds fail
  closed. Remote execution is an explicit research-content egress decision
  (`docs/remote-provider-operation.md`), and every persistence
  command requires an explicit `--database` path — there is no default, cwd,
  home-directory, or config database. **No autonomous collection occurs**:
  the CLI executes exactly the supplied input snapshot.
- **Stable exit codes.** `0` completed; `2` request/provider/input failure;
  `3` the analytical run returned FAILED (a valid persisted outcome, never
  conflated with a malformed request); `4` storage/checkpoint failure.
  `--json` emits exactly one machine-readable object on stdout (expected
  failures included); human expected failures go to stderr. Expected errors
  redact the caller's paths as `<request-file>` and `<database>`; outputs
  never contain Evidence text, Source URLs, report bodies, or report titles.
- **Inspect and listings are read-only.** `inspect` summarizes the verified
  latest checkpoint (status, checkpoint index, `created_at`, last completed
  stage, next stage, failed stage, report ID); `checkpoints`/`resumes` list
  verified metadata in ascending order. No analytical payloads are printed
  and nothing is mutated.
- **No automatic anything.** One `run resume` invocation is one explicit
  attempt (no `--retry`, no retry-latest); a completed `run execute` does not
  materialize report files — `traceintel report materialize` remains the
  separate explicit workflow.

## Operational project CLI

The workspace layer is exposed through a second narrow adapter over the
Issue #56 domain contract and the Issue #57 store:

```
traceintel project create  --database <sqlite-path> --title <text>
                           [--objective <text>] [--project-id <id>]
traceintel project list    --database <sqlite-path>
traceintel project inspect <project-id> --database <sqlite-path>
traceintel project runs    <project-id> --database <sqlite-path>
traceintel project attach  <project-id> <run-id> --database <sqlite-path>
```

- **A project is a container, not an executor.** No command in this group
  runs, resumes, schedules, monitors, or selects a provider for anything, and
  there is no path from here to one — the module imports no provider, no
  executor, and no network client. `ResearchRun` remains the authoritative
  execution unit.
- **Only existing, verified runs can be attached.** `attach` names a run that
  is already stored in that same database, and the store requires its latest
  checkpoint to load and verify before recording it. Only the run ID is kept;
  the run, its checkpoints, snapshots, exports, and request are not modified.
- **Membership is append-only.** Each attachment takes the next sequence,
  earlier membership is never rewritten, and re-attaching a member run fails
  closed rather than silently succeeding. There is **no delete, detach,
  reorder, or update-in-place command** — that is a deliberate absence, not a
  gap to be filled later.
- **Everything is explicit.** Every command requires `--database`; the project
  ID and run ID are always arguments. Nothing is discovered from the working
  directory, read from an environment variable, or defaulted to the most
  recent anything.
- **Read-only commands are read-only.** `list`, `inspect`, and `runs` change
  no project, no membership, and no run data; they will not even bring a
  database file into existence. They do apply the store's ordinary additive
  schema initialization, which is the same path every other entry point uses.
- **Safe output only.** Commands print project ID, title, objective,
  `created_at`, assembly method, attached run IDs, and their sequences. No
  Evidence text, Document text, Source URL, report body, report title,
  prompt, provider response, or credential is printed — none of it is
  reachable from a project, because the domain model has nowhere to hold it.
  Public/private data separation is unchanged.
- **Stable exit codes, matching the run taxonomy.** `0` success; `2`
  request failure (invalid input, unknown project, unknown run, duplicate
  project, duplicate attachment); `4` storage failure (unknown schema
  version, corrupt project row or digest, an unverifiable run checkpoint, a
  contended write lock). Code `3` means "analytical failure" in the run group
  and is deliberately unused here — a project never analyzes anything.
  `--json` emits exactly one machine-readable object on stdout, structured
  rather than stringified human output; human expected failures go to stderr.
  Expected errors redact the caller's database path as `<database>`, report
  validation failures by count rather than echoing the rejected value, and
  never surface a traceback or a raw `sqlite3` exception.

### `project compare` — deterministic longitudinal comparison (Issue #59)

```
traceintel project compare <project-id> <left-run-id> <right-run-id> --database <sqlite-path> [--json]
```

Answers "what changed from this run to that run?" using **only material the
two runs already contain**. Pure: no provider, no network, no clock, no minted
ID, no fresh inference, and nothing written. Both run IDs are explicit;
neither is defaulted to the earliest or most recent attachment, and the
project's membership order supplies each run's reported sequence rather than
any re-sorting of run IDs.

Cross-run identity reuses the **existing** normalization — the entity
resolution key, the normalized claim statement, the canonical event
description — never run-local opaque IDs, which differ between runs by
construction. Those IDs are still carried on every change record so a reader
can traverse back to that run's provenance.

Three categories: `ADDED`, `REMOVED`, `CHANGED`. Artifacts present on both
sides and identical are reported as an `unchanged_count`, not as records. A
cross-run key occurring twice inside one run makes the pairing ambiguous and
fails closed rather than being resolved arbitrarily; only COMPLETED runs are
comparable, because an incomplete run's missing artifacts reflect where it
stopped rather than what changed.

Repeated invocation is byte-identical, including across processes with
different hash seeds. Output carries keys, identifiers, and changed-field
names — never Evidence text, Document text, or Source URLs.

**A comparison asserts nothing about truth or authenticity.** A difference
between two runs means the two runs differ — not that reality changed, not
that either run is correct. It creates no Evidence and no provenance bundle:
it is an analytical relationship, not a source.

### `project index` / `project search` — local full-text search (Issue #60)

```
traceintel project index  <project-id> --database <sqlite-path> [--json]
traceintel project search <project-id> <query> --database <sqlite-path>
                          [--limit N] [--offset N] [--json]
```

**Local-only.** Exact token matching over an index built on this machine from
material already stored on it. There is no embedding, no vector store, no
relevance model, no remote service, no provider, and no network client in this
path — neither the query nor what it finds leaves the machine.

**The index is derived state, authoritative for nothing.** It is recomputed
from stored material, no analytical path reads it, and deleting it loses no
information: search returns nothing until it is rebuilt, and every other
command is unaffected. Rebuilding is idempotent, and building writes only to
the index tables — checkpoints, snapshots, exports, project rows, and
membership rows are read and left byte-identical.

Tokenization is Unicode NFKC, case folding, and a split on non-word
characters, keeping tokens of 2–64 characters. Matching is **conjunctive** —
every term must occur. Results are ordered by matched-term count, then
occurrences, then stable identifiers: a **stated ordering rule, not a
relevance score**. An empty query matches nothing and is not an error.

**Hits identify, they do not quote.** A result names the run, artifact kind,
artifact ID, and field, plus how many terms matched and how often — no
snippet, no window, no field content. Evidence text, Document text, and Source
URLs therefore stay out of the output by construction. Read the material
itself through the existing export and verification commands, where the
content rules already live.

A **stale** index — one built before the stored material changed — fails a
query closed and is never rebuilt implicitly, because answering from an index
that silently disagrees with the material is worse than answering not at all.
Rebuild it explicitly with `project index`.

### `project graph` — read-only Evidence Graph (Issue #61)

```
traceintel project graph <project-id> <run-id> --database <sqlite-path>
                         [--node <node-id>] [--json]
```

Exposes the Evidence → Claim → Event relationships that stored provenance
already records, as a traversable projection.

**Derived, never persisted.** There is no graph table, no graph database, and
no persisted edge anywhere in TraceIntel. The graph is computed from the run's
verified latest checkpoint, printed, and discarded; recomputing it from the
same material yields the same graph, including across processes with different
hash seeds. It is authoritative for nothing and can be discarded freely.

**It invents nothing.** Seven edge kinds, each mapping to one concrete stored
field. No inference, no similarity matching, no automatic entity merging, no
edge weighting, and no relationship type beyond what the material records.
Node IDs are `KIND:artifact_id` and edge IDs are `KIND:source->target`, so
duplicate elimination is exact and ordering is deterministic. Direction is
meaningful and never symmetrized. Isolated nodes are **kept** — an Evidence
supporting nothing is a real fact about the run. A stored reference to an
artifact the run does not contain **fails closed** rather than being silently
dropped.

**It proves nothing about truth or authenticity.** An edge records that the
stored material links two artifacts — not that the Evidence is accurate, the
Claim true, or the Source genuine. Output identifies rather than quotes: node
IDs, kinds, and artifact IDs, never Evidence text, Document text, or Source
URLs.

#### Cross-run projection (Issue #66)

`traceintel.graph.project_evidence_graph(project, runs)` projects the runs of
one Project into a single graph, merging nodes by the **same cross-run identity
longitudinal comparison pairs on** — one shared definition in
`traceintel.projects.identity`, so a merge here and an "unchanged" verdict
there are one judgement rather than two that agree by coincidence.

Merging never hides a run: every node records each run that produced it,
together with that run's own artifact ID, so two runs disagreeing can never
look like agreement. Node IDs are a **digest** of the identity key rather than
the key itself, because claim and event keys embed normalized statements and
canonical descriptions — a node ID built from them would quote content into a
structure whose whole discipline is to identify rather than quote. An ambiguous
join — one run holding two artifacts of the same identity — **fails closed**
rather than picking one.

Same asserts-nothing-about-truth contract as the per-run graph. The projection
itself is still computed on demand and stored nowhere; Issue #67 added an
optional derived *cache* for it, described below.

#### `project graph-query` — bounded queries (Issue #69)

```
traceintel project graph-query <project-id> --database <sqlite-path>
                               [--node <node-id>] [--node-kind <kind>]
                               [--edge-kind <kind>] [--run <run-id>]
                               [--artifact <artifact-id>]
                               [--neighbourhood <node-id>] [--depth <1-3>]
                               [--limit <1-1000>] [--json]
```

**Every query is finite.** There is no "return everything" flag. The default
limit is 50 and the hard maximum 1000; a larger request is **refused rather
than clamped**, because a caller who asked for 10,000 and silently received
1,000 would read a truncated answer as a complete one. Results always state
whether they were truncated. The limit must be a real positive `int` — `True`,
`5.0`, `"5"`, NaN and infinity are each refused rather than coerced.
Neighbourhood walks are bounded to three hops, breadth-first, visiting each
node at most once.

**No query language exists.** Filters name structural facts the graph already
carries and are matched by equality. No SQL is issued, no expression parsed, no
caller text evaluated, no URL fetched, and a query cannot cross into another
project. Reading writes nothing — including no cache.

#### `project graph-export` / `graph-verify` — canonical export (Issue #69)

```
traceintel project graph-export <project-id> --database <sqlite-path>
                                --output <file> [--json]
traceintel project graph-verify <export-file> [--json]
```

Writes the project-scoped graph as one canonical
`traceintel-project-graph-export` v1 file: project ID, graph method, covered run
IDs, attached run count, producer and producer version, deterministically
ordered nodes and edges with their per-run occurrences and provenance
references, and a content digest over everything else. The same material always
produces byte-identical bytes.

**The export carries no research content.** No Evidence text, Document text,
Source body or URL, prompt, provider response, credential, or filesystem path —
not by filtering on the way out, but because a cross-run graph node never held
any of those. This is asserted against the export bytes themselves.

**Verification checks the bytes, not just the object.** It re-serializes the
parsed payload canonically and compares it to the input, which catches
reordered keys, reordered arrays, injected whitespace, and truncation together —
including a payload that *claims* canonical ordering while carrying another.
Malformed JSON, a missing or unknown field, a wrong type, a duplicate ID, a
dangling reference, a digest mismatch, a coverage mismatch, an unsupported
future format version, and an unsupported producer are each refused.

**Verification proves nothing about truth.** Success means these bytes are
internally consistent and unmodified — not that any claim is true, any source
authentic, any analysis correct, or any provider good. The export says so in
its own payload.

**File safety.** An existing file is never overwritten, a symlink is never
written through, a missing parent directory is never created, and a failed
write leaves no partial file behind. Failure messages redact both the database
and the output path, and exit codes follow the existing taxonomy: 2 request, 3
analytical, 4 storage.

The workspace features have been validated end to end from a **fresh package
install, outside the source checkout, against real SQLite, through this CLI**,
including upgrade from databases written by the actual released v0.1.0 and
v0.2.0 builds — 81/81 checks, with zero provider calls and zero network
requests (measured under a socket guard, not assumed). Procedure, results, and
stated limits: `docs/external-validation-v0.3.md`. That record is about
execution, storage integrity, and fail-closed behaviour — **not** about whether
any analyzed content is true or any Source authentic.

RC readiness (#63, remediated by #63-R) and release preparation (#64) are
complete: the readiness verdict is `RC_READY`, packaging metadata is validated
by a mandatory CI gate on every push, and the version is `0.3.0`. Publication
itself has not happened and requires explicit authorization.

## Configuration

Runtime settings are loaded from a YAML file into a **typed, validated**
`TraceIntelConfig` (Pydantic v2) via `src/traceintel/config/`. Configuration is
loaded explicitly by the caller — there is no global config object and importing
the package performs no filesystem I/O.

```python
from traceintel.config import load_config, create_database

config = load_config("config.yaml")   # typed & validated; unknown keys rejected
db = create_database(config)           # config -> SQLiteDatabase (side-effect free)
db.initialize()
```

- YAML is parsed only with `yaml.safe_load` (arbitrary Python-object tags are
  rejected); files are read as UTF-8. An empty file yields the documented
  defaults, and the document root must be a mapping.
- **Unknown keys are rejected** (`extra="forbid"`), so a typo like
  `max_documnts` fails loudly instead of being ignored.
- The database `path` is obtained from configuration and passed to the storage
  layer **verbatim** (relative paths are not rewritten; `:memory:` is supported).
  The dependency direction is one-way: `config → storage`, never the reverse.
- **Secrets are not stored in YAML.** No API keys/tokens are modeled here;
  provider credentials come only from the Issue #47 runtime secret contract
  (process environment or an injected `RuntimeSecretSource` — see the LLM
  provider section). There is **no `.env` loading and no
  environment-variable override system.**

See `config.example.yaml` for the full schema. There is still no OSINT
collection pipeline — configuration only validates and stores settings.

## Persistence

Domain models can be persisted to a local **SQLite** database via
`src/traceintel/storage/`. SQLite is the initial local-first backend (stdlib
`sqlite3`, no ORM). Persistence is kept **separate from the domain models** —
the models contain no storage code — behind a small `Repository` abstraction so
the backend can be replaced later (e.g. PostgreSQL).

```python
from traceintel.storage import SQLiteDatabase, SourceRepository

db = SQLiteDatabase(":memory:")   # or a filesystem path
db.initialize()
sources = SourceRepository(db)
sources.add(source)               # persists the model as-is
loaded = sources.get(source.source_id)  # semantically equal to the original
```

Each model is stored as a JSON payload plus extracted primary/foreign-key
columns; reconstruction roundtrips through Pydantic, preserving enums,
timezone-aware datetimes, nested JSON, and verbatim text. The direct provenance
foreign keys — `Document.source_id → Source` and `Evidence.document_id →
Document` — are enforced by SQLite; secondary references (e.g. `evidence_ids`)
are stored but not yet FK-validated.

> **Note:** There is **no runtime configuration loader yet** (the database path
> is supplied explicitly in code, not read from `config.yaml`/`.env`), and no
> collectors/OSINT pipeline. `content_hash` is stored but is **not** treated as
> unique — content-level duplicate detection is a later phase.

### Schema versioning and upgrades

The schema only ever grows: every historical version is a strict prefix of the
current one, upgrading means "create the tables that are missing", and no step
drops a table, renames a column, reinterprets a value, or rewrites a stored
row — checkpoints in particular stay append-only. That is what makes a code
rollback safe: reverting an older build leaves unused tables behind, never a
database the older build cannot read.

`initialize()` is the upgrade path. It detects the database's version first —
refusing a version newer than the build supports, or a half-applied one,
*before any statement executes* — then creates what is missing and verifies the
result. Databases written before v4 carry no recorded version, so theirs is
inferred from the tables they hold; absence is the legacy signal, not
corruption.

Full table definitions, constraints, the upgrade matrix, fail-closed
behaviour, and the migration/rollback policy: `docs/storage-schema.md`.

### Research project persistence

`ResearchProjectStore` persists the `traceintel.projects` workspace contract —
projects and their run membership — over the same database.

```python
from traceintel.projects import create_research_project
from traceintel.storage import ResearchProjectStore, SQLiteDatabase

db = SQLiteDatabase(":memory:")
db.initialize()
store = ResearchProjectStore(db)
store.create_project(create_research_project(title="Aurora workspace"))
```

Runs are attached **by ID** through `attach_run(project_id, run_id)`, and only
after that run's latest checkpoint has loaded and verified. Attachment is one
write-locked transaction covering verification, sequence allocation, the
membership insert, and the canonical update, so it commits whole or not at all.
Every domain invariant is also a database constraint, and stored state is
re-validated on every read rather than trusted. There is no delete, detach,
reorder, or update-in-place API, and nothing here can execute, resume,
schedule, or monitor a run.

The stored SHA-256 establishes **integrity, not authenticity**: it detects
accidental or content-level modification, and proves nothing about who wrote
the row or whether any analyzed content is true.

## Planned architecture (target)

```
src/traceintel/
  collectors/  parsers/  models/  evidence/  claims/  entities/  events/
  provenance/  scoring/  providers/  storage/  exporters/  pipelines/  cli/  utils/
```

Directories are created as each phase is implemented — the tree above is the
**target**, not the current state.

## Data & privacy policy

This repository must contain **only** source code, tests, documentation,
configuration templates, and **synthetic / fictional** example data. Real
research data, collected datasets, credentials, cookies, tokens, and private
notes must never be committed. See [`SECURITY.md`](SECURITY.md).

## Development

Requires Python 3.11+.

```bash
# Install in editable mode with dev tooling
python -m pip install -e ".[dev]"

# Verify the CLI entry point
traceintel --version

# Run the checks
pytest
ruff check .
mypy
```

## Out of scope for v0.3

Web/mobile UI, HTTP servers or any listening socket, graph/vector databases,
embeddings, autonomous browsers, social-media scraping, login-required
collection, scheduling, background workers, PDF OCR, image recognition, and
full RAG chatbots are explicitly **out of scope** for the `v0.3` line, as are
provider auto-selection, fallback chains, cost optimization, and streaming
APIs.

Within the workspace theme specifically: project deletion, run detachment, and
membership reordering are out of scope for this line, and a project never
executes, resumes, schedules, or monitors a run.

## License

[MIT](LICENSE)
