Metadata-Version: 2.4
Name: klode
Version: 0.5.1
Summary: A grounded knowledge system: an engine (klode.lib) and a supervising agent (klode.gate) over a corpus of knowledge bases.
Author: Xiaolai Li
License-Expression: MIT
Project-URL: Homepage, https://github.com/xiaolai/klode
Project-URL: Repository, https://github.com/xiaolai/klode
Project-URL: Issues, https://github.com/xiaolai/klode/issues
Keywords: knowledge-base,citations,grep,provenance,supervising-agent,rag
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Text Processing :: Markup :: Markdown
Classifier: Topic :: Software Development :: Documentation
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: entail
Requires-Dist: minicheck>=1.1; extra == "entail"
Dynamic: license-file

# klode

A **klode** — a rich vein — of grounded, verifiable knowledge, with the machinery to encode it and to
*supervise work* against it. Every claim is anchored to a verbatim source and checked by a fail-closed
linter, so a citation can't quietly rot: **cite, don't recall.**

That guarantee is **referential, not semantic** — it proves the quoted text is still there, in a
current source, in exactly one place. It does *not* prove the quote supports the claim built on it.
Green `klode check` means "no citation rot," never "this is true."

A second, narrower limit sits beside it. `klode ingest` verifies an extraction against a control and
against the rendered page, which catches dropped pages, duplicated blocks, and scrambled reading
order — but **no signal detects a value transposition** that preserves the words, the length, and
the order. A table extractor that swaps two cells produces this, and every check passes:

```
control:   Alice status approved   Bob status rejected
candidate: Alice status rejected   Bob status approved
```

Both claims are inverted; containment, inflation, order, and coverage are all perfect. **A
table-derived claim needs human verification.** Recorded rather than hoped away.

Zero runtime dependencies — Python 3.11+, standard library only.

## The two loops

- **Loop A — encode expertise** (`klode.lib`): turn sources into cited, retrievable knowledge. Every
  card claim carries a verbatim `(grep: …)` anchor; `klode check` fails if any citation stops resolving.
- **Loop B — supervise work** (`klode.gate`): submit a draft, score it against a **CriterionSpec** —
  an authored, corpus-pinned, human-approved rubric with stable ids and behaviorally anchored levels
  — and return **Go / Recycle**. Each cited defect is grounded through `klode.lib.verify`, so the
  judge's citations are un-fakeable, and every authored field declares whether it is the source's
  words, a paraphrase, an inference, or *not stated at all*. The boundary is enforced: `klode.gate`
  consumes only the `klode.lib` public API, never its internals.
  See [`dev-docs/SPEC-criterion.md`](dev-docs/SPEC-criterion.md).

## Install

```bash
pip install klode
# or from a source checkout:
pipx install -e .              # provides the `klode` and `klode-mcp` commands
```

## Three surfaces over one engine

A knowledge base is a `library.toml` plus a corpus of cited cards. Point klode at one with `-c`, or
register several in a manifest and address them by id (`--kb <id>` / `--registry`).

**CLI** — `klode`:

```bash
klode -c path/to/library.toml check                 # citation-rot linter (exit 1 on any broken citation)
klode -c path/to/library.toml search pacing         # retrieval over the cards
klode -c path/to/library.toml consult brevity       # read a craft lens
klode -c path/to/library.toml verify brevity "the exact quote"   # prove a quote against its source
klode ingest paper.pdf --shelf papers               # ingest a source -> clean, grep-ready text
klode kbs                                            # list the KBs in a registry
```

Add `--json` to any read verb for machine-readable output (the same structured result the MCP renders).

**MCP server** — `klode-mcp` (stdio): exposes the read/verify surface to an agent — `list_kbs`,
`search_sources`, `consult_dimension`, `consult_framework`, `zoom_card`, `verify_quote`, `diagnose`,
`list_lenses`. Serves one KB (`-c`) or many (`--registry`), and every grounded result names its KB.

**Library** — `import klode.lib`: the stable public-API facade — `verify`, `search`, `consult`,
`resolve`, `diagnose`, `Config`. Cheap to import; pulls in no frontends or optional backends.

## Ingestion — any format to grep-ready text

`klode ingest` detects the format by content signature (not the extension) and converges every source
on one clean-text pipeline: **PDF, EPUB, DOCX, HTML/XHTML, Markdown, TXT**. EPUB/DOCX/HTML/Markdown/TXT
are pure stdlib; only PDF's OCR tiers are optional.

Markdown handling is syntax-aware, not a text passthrough: YAML front matter and HTML comments
(including DocC's JSON metadata block) are dropped, `[text](dest)` becomes `text`, images become
their alt text, and code fences and inline code are preserved untouched. That matters for corpora
such as an offline Apple documentation mirror, where 281k Markdown pages would otherwise carry
their metadata and link destinations into every grep.

Point `ingest` at a **directory** to import a corpus instead of one file:

```bash
klode -c library.toml ingest ~/docs/archive --shelf docs --pattern '**/*.md'
klode -c library.toml ingest ~/docs/archive --shelf docs --pattern '**/*.md' --limit 1000
```

Each file gets a stable id derived from its path relative to the directory; existing sources are
skipped, slug collisions are disambiguated with a digest rather than overwritten, and the run
reports `ingested / skipped / failed` and exits nonzero if any file failed.

For table-heavy or multi-column PDFs, point klode at a
[docling-serve](https://github.com/docling-project/docling-serve) endpoint — the GPU runs
server-side, so klode stays dependency-free — and pass `--tier docling`:

```toml
# ~/.klode/settings.toml
[ingest]
docling_url = "http://<host>:15001"   # docling-serve
marker_url  = "http://<host>:15002"   # marker_server, for --tier marker
tier = "auto"                          # escalate only when the measured text layer is bad
```

**Prefer a remote endpoint over a local install.** A remote conversion has a deadline that scales
with the document; the in-process backends have no wall-clock bound, so a pathological PDF can
wedge a local ingest indefinitely. It also keeps this client dependency-free, which is the point of
the project.

`KLODE_DOCLING_URL` overrides the file, and `klode settings` prints every value with the source
that won. The URL is topology, not a credential: **bind docling-serve to a private interface**, and
do not rely on the URL being unguessable.

[marker](https://github.com/datalab-to/marker) is supported the same way (`marker_url`, `--tier
marker`) and is **remote-only** — it pulls torch and layout models, which klode does not depend on.
It is deliberately *not* in the `auto` escalation ladder: a backend earns a ladder slot by
measuring better than the one it would displace, not by being installed.

Backends are chosen by measurement, never by intuition — `eval/extract_bakeoff.py` ranks them
against the *rendered page* (`pdftoppm` + `tesseract`), the one signal not downstream of another
extractor. The committed result over 20 two-column academic papers
(`eval/results/extract-bakeoff-2026-08-11.json`): docling and `pdftotext` tie on recall (0.931 vs
0.945), and docling wins decisively on **reading order** — median 1.000 against 0.697, better on 10
documents of 17 and worse on 1. That is what earns it the tier-3 slot. marker failed 16 of the same
20 documents on the deployment tested, so it has no paired basis to be ranked at all.

## Architecture

One operations registry projects a single core to both surfaces (CLI + MCP), so they can differ in
formatting but never in behaviour. Every grounded result carries structured provenance (which KB,
which source, which policy). See `dev-docs/` for the design record and `SPEC-operations.md` for the
machine-readable operation table.

## Status

`0.5.1` — beta. The engine (`klode.lib`) is solid: **1,008 tests**, a stable public-API facade, an AST
layering guard, a content-sniffing multi-format ingester, and an MCP server, all with zero runtime
dependencies. `klode.gate` (Loop B) is now a fail-closed supervising gate — freshness/review-aware
grounding (`verify_evidence`), a bounded evidence-context op (`verify_context`), a fail-closed
verified-context bundle (`build_context_bundle`), a shared structured anchor contract
(`parse_markers`/`Marker`, regex/context/`#n`), and **CriterionSpec v1** as its sole rubric input
(field-level epistemics, behaviorally anchored levels, a computed corpus fingerprint, and a
human-approval admission gate) are in place, and the rubric judge is **real** (`LLMJudge`:
G-Eval two-step form-filling, position-bias-debiased by balanced permutation, transport-injectable,
stdlib-only). What is **not** done is *calibration*: no rubric has yet been measured against human
scores, so `Verdict.calibrated` is False and every verdict is marked non-production — by mechanism,
not by convention. The optional entailment check
(`klode check --entail`) pulls a small NLI model behind `klode[entail]` and is advisory, warn-only.

## License

MIT — see [LICENSE](LICENSE).
