Metadata-Version: 2.4
Name: agent-pdf-workspace
Version: 0.3.0
Summary: Offline, agent-oriented PDF exploration workspaces
Author-email: Dark Light <darklight@noreply.com>
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: agents,document-processing,ocr,offline,okf,pdf
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Python: <3.15,>=3.11
Requires-Dist: liteparse<2.7,>=2.6
Requires-Dist: numpy<2.5,>=1.26
Requires-Dist: onnxruntime<2,>=1.24
Requires-Dist: pdfplumber<0.12,>=0.11
Requires-Dist: pydantic<3,>=2.11
Requires-Dist: pypdf<7,>=5.7
Requires-Dist: pyyaml<7,>=6
Requires-Dist: rapidocr<3.10,>=3.9
Requires-Dist: regex<2027,>=2024.11
Requires-Dist: typer<1,>=0.16
Description-Content-Type: text/markdown

# agent-pdf-workspace

`agent-pdf-workspace` converts local PDFs into persistent, agent-oriented workspaces. One PDF
becomes one workspace; several PDFs become one **collection** that is searched as a unit. Text,
layout, tables, OCR, metadata, exact/regex/full-text search, rendering, and integrity checks run
locally. The package contains no LLM, vision model, or embedding dependency and
performs no network requests while processing documents.

An external agent can inspect queued image crops with its own vision capability and return short,
schema-validated descriptions. PDF content is always treated as untrusted data.

## Install

Python 3.11–3.14 is supported.

```bash
uv tool install .
# or, for development from this checkout
uv sync
uv run pdfws --version
```

LiteParse supplies native PDF parsing and page rendering with its OCR disabled.
RapidOCR and ONNX Runtime perform OCR through Python packages installed from
PyPI. RapidOCR's default PP-OCRv6 models are included in its wheel, cover
English, German, and other Latin-script languages, and require no runtime model
download. No Tesseract executable, Tesseract language pack, GitHub access, or
separate system OCR installation is used.

In `auto` mode the package first inspects native extraction and sends only image-dominated or
text-empty complex pages through local OCR; both layers retain their provenance when merged.

## Quick start

```bash
pdfws ingest report.pdf --target ./report-workspace --ocr auto --language deu+eng --json
pdfws inspect ./report-workspace --json
pdfws search ./report-workspace "operating income" --json
pdfws read ./report-workspace --page 12 --json
pdfws visuals next ./report-workspace --json
pdfws verify ./report-workspace --json
pdfws export okf ./report-workspace --target ./report-okf --json
```

For several PDFs, pass more than one source or add them to a collection over time:

```bash
pdfws ingest q1.pdf q2.pdf q3.pdf --target ./library --ocr auto --json
pdfws collection add ./library q4.pdf --json
pdfws collection list ./library --json
pdfws search ./library "operating income" --json
```

Temporary parser sessions now default to
`./report-workspace/cache/tmp/pdfws-worker-*/`, so parser workers do not use
`%TEMP%`, `%TMP%`, or `/tmp`. Select another owned cache root with an
absolute path when ingesting, rendering, or submitting a visual audit:

```bash
pdfws ingest report.pdf --target ./report-workspace --cache-dir /absolute/cache --json
pdfws render ./report-workspace --page 12 --bbox 72,90,520,420 --cache-dir /absolute/cache --json
```

The Python API accepts the same override as
`Workspace.ingest(..., cache_directory=...)` and
`Workspace.open(..., cache_directory=...)`. An already-open instance can switch
subsequent worker sessions with `workspace.set_cache_directory(...)`. A new
external directory receives the `.agent-pdf-workspace-cache-v1` ownership
marker. Reusing an existing directory without that marker, or passing a
relative path, is refused.

The bundled OpenCode integration additionally exports
`set_cache_directory(workspace, cache_directory)`. It validates the path and
remembers it for later render and audit calls in the current OpenCode session.

For encrypted input, provide the password through standard input so it is not exposed in process
arguments or persisted:

```bash
printf '%s\n' "$PDF_PASSWORD" | pdfws ingest protected.pdf --target ./workspace --password-stdin
```

### Visual descriptions

`pdfws visuals next` returns either a `describe_region` task or an annotated `audit_page` task.
For a region, have the calling agent inspect `asset_path` and submit JSON like:

```json
{
  "visual_id": "visual-p0001-…",
  "crop_sha256": "…",
  "kind": "chart",
  "decorative": false,
  "short_description": "Revenue rises through Q3 and dips slightly in Q4.",
  "text_in_visual": "| Quarter | Revenue (EUR m) |\n|---|---:|\n| Q1 | 12 |\n| Q2 | 15 |",
  "agent_id": "document-agent",
  "model_id": "vision-model"
}
```

```bash
pdfws visuals submit ./workspace --task-id TASK --input description.json --json
```

For charts and plots, `text_in_visual` should be a Markdown table with one row per visible data
point and explicit series/category, value, and unit columns. Preserve signs, decimals, years, and
label/value associations. Mark estimates as estimates instead of inventing precision. To correct an
already submitted description, recheck the crop and existing JSON and pass `--replace`; a
formatting-only revision must not recompute or change verified values.

Page audits can add bboxes missed by deterministic raster/vector detection; every added region then
becomes its own description task. Accepted descriptions are persisted as both readable Markdown
and a versioned JSON sidecar under `visuals/descriptions/`.

## Python API

```python
from agent_pdf_workspace import Workspace

workspace = Workspace.ingest(
    "report.pdf",
    "report-workspace",
    ocr="auto",
    ocr_languages=("deu", "eng"),
)
hits = workspace.search("cash flow", mode="fts")
page = workspace.read_page(hits[0].page)
crop_path = workspace.render_region(hits[0].page, (72, 90, 520, 420))
okf_bundle = workspace.export_okf("report-okf")
```

Search hits include a stable hit ID, page, page-coordinate bounding box, kind, and snippet. Custom
ingestion limits supplied with `IngestConfig` are persisted without secrets and remain effective
when the workspace is reopened.

## Several PDFs in one workspace

A collection holds any number of PDFs under one root. Every member stays a complete, separately
verifiable schema-1 workspace below `members/<document_id>/`; the collection adds a shared
cross-document search index, one manifest, and one agent entry point.

```python
from agent_pdf_workspace import Collection

collection = Collection.ingest(["q1.pdf", "q2.pdf"], "./library", ocr="auto")
collection.add_pdf("q3.pdf", "q4.pdf")
collection.add("q5.pdf", "./old-report-workspace")   # ingests PDFs, adopts workspaces
document_id = collection.members()[0].document_id

hits = collection.search("cash flow")                  # every PDF; hits carry document_id
page = collection.read_page(12, document_id=document_id)
collection.export_okf("./library-okf")
collection.verify()
```

`document_id` is `<filename-slug>-<source_sha256[:12]>`, so adding the same PDF file twice is
idempotent and the member directory name always equals the document ID. The same bytes under a
different file name form a separate member sharing the hash suffix, and adding the same file with
different extraction settings is refused.

An existing single-PDF workspace joins a collection without re-ingestion. `collection add` detects
a workspace directory and adopts it; `collection attach` is the explicit form. Both copy by default
and only relocate with `--move`:

```bash
pdfws collection add ./library new.pdf ./report-workspace --json
pdfws collection attach ./library ./report-workspace --json
pdfws collection attach ./library ./report-workspace --move --json
```

The workspace is verified before and after adoption; a workspace that fails integrity verification
is refused and nothing is written.

`inspect`, `search`, `read`, `render`, `verify`, `visuals *`, and `export okf` detect a collection
path automatically and accept `--document ID`. `read` and `render` require it once a collection
holds more than one PDF; `search` and `visuals` span every document without it.

```bash
pdfws read ./library --document q1-1a2b3c4d5e6f --page 12 --json
pdfws visuals next ./library --json     # returns a task tagged with its document_id
```

Visual task assets stay member-relative: resolve a task crop as
`COLLECTION/members/<document_id>/<asset_path>`.

`pdfws collection remove ./library --document ID` detaches a member into `removed/<ID>`, where it
remains a valid standalone workspace. Data is deleted only with the explicit `--delete` flag.

The main public contracts are Pydantic models exported from `agent_pdf_workspace`. Versioned JSON
Schemas and the bundled agent skill are included in the wheel; export the latter with
`pdfws skill export TARGET`.

### OpenCode installation

OpenCode calls its configurable home `OPENCODE_CONFIG_DIR`. The installer reads that variable
first and otherwise asks the installed CLI via `opencode debug paths`; it does not assume that
`~/.config/opencode` is the active directory.

```bash
pdfws opencode path --json
pdfws opencode install --json
```

The install command writes the skill to `<config>/skills/agent-pdf-workspace` and native tools to
`<config>/tools/agent_pdf_workspace.ts`. Use `--config-dir DIR` for an explicit directory or
`--force` to replace only those two package-owned artifacts. The OpenCode tools invoke `pdfws`
without a shell and expose bounded inspect, search, page-read, region-render, and visual-task
operations. Each of those accepts an optional `document` argument so it can address one PDF inside
a collection, and `collection_add(collection, sources)` plus `collection_list(collection)` manage
membership. It also exposes `export_okf(workspace, target)` for portable OKF
0.2 output.

## OKF 0.2 export

Version `0.2.0` exports any integrity-valid schema-1 workspace to the official
[Open Knowledge Format 0.2](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md):

```bash
pdfws export okf ./report-workspace --target ./report-okf --json
```

Version `0.3.0` exports a whole collection the same way. The bundle root links to
`documents/index.md` and holds one `documents/<document-id>/` subtree — `document.md`, `pages/`,
`tables/`, `visuals/`, and `references/` — per PDF:

```bash
pdfws export okf ./library --target ./library-okf --json
```

The target must be a new directory outside the workspace. Existing paths and
symbolic links are refused. The exporter verifies `manifest-sha256.txt` first,
copies only the known original PDF and referenced visual crops, and never
modifies the source workspace.

```text
OKF_BUNDLE/
├── index.md                         # declares okf_version: "0.2"
├── document.md                      # type: PDF Document
├── pages/
│   ├── index.md                     # reserved OKF index; no concept frontmatter
│   └── page-0001.md                 # type: PDF Page
├── tables/
│   ├── index.md
│   └── <table-id>.md                # type: PDF Table
├── visuals/
│   ├── index.md
│   └── <visual-id>.md               # type: PDF Visual
└── references/
    ├── original.pdf                 # byte-identical source
    └── visuals/<sha256>.png          # referenced crops
```

Every non-reserved Markdown file contains parseable YAML frontmatter with a
non-empty `type`. Page, table, and visual concepts retain PDF-specific IDs,
page numbers, bounding boxes, extraction provenance, hashes, and the existing
untrusted-content warnings as producer extensions. Submitted visual
descriptions are included as stable concepts; visuals still awaiting an agent
description remain explicit `draft` concepts. The OKF bundle is a separate
export and does not change persistent workspace schema `1.0`.

## Workspace format

The source PDF is copied unchanged to `source/original.pdf`. Human-readable Markdown provides
navigation while JSON sidecars preserve coordinates and provenance. Tables are also emitted as CSV.
SQLite FTS and rendered query crops live under `cache/` and are regenerable.

### Processing flow

`agent-pdf-workspace` creates a persistent, page-oriented workspace rather than
separate per-page PDF files:

```mermaid
flowchart LR
    A["Source PDF"] --> B["Copy unchanged<br/>source/original.pdf"]
    B --> C["pypdf inventory<br/>pages, metadata, attachments, active content"]
    B --> D["LiteParse<br/>native extraction and page rendering<br/>OCR disabled"]
    D --> K["RapidOCR plus ONNX Runtime<br/>selected local OCR<br/>models bundled in PyPI wheel"]
    C --> E["pdfplumber geometry<br/>tables, regions, page previews"]
    D --> E
    K --> E
    E --> F["Per-page Markdown and JSON"]
    E --> G["Table MD, JSON, and CSV"]
    E --> H["Visual PNG assets and tasks"]
    F --> I["SQLite full-text index"]
    G --> I
    H --> I
    I --> J["Integrity manifest and reusable workspace"]
```

### Workspace and cache versions

The persistent workspace schema is `1.0` in every package version to date.
Package releases keep the same persistent workspace schema while extending
runtime and export behavior:

| Package versions | Persistent workspace | Temporary parser-worker location |
| --- | --- | --- |
| `0.1.0`–`0.1.1` | Schema `1.0`; regenerable search and render data below `WORKSPACE_ROOT/cache/` | OS temporary directory selected by Python (`%TEMP%`, `%TMP%`, or `/tmp`) |
| `0.1.2` | Schema `1.0`; existing workspaces open without conversion; OCR moves from LiteParse/Tesseract to Python-packaged RapidOCR/ONNX Runtime | `WORKSPACE_ROOT/cache/tmp/` by default, or an absolute owned cache selected through Python, CLI, or OpenCode |
| `0.2.0` | Schema `1.0`; adds a separate, conformant OKF 0.2 export through Python, CLI, and OpenCode | Same workspace-local or explicitly selected owned cache behavior as `0.1.2` |
| `0.2.1` | Schema `1.0`; closes SQLite search handles before atomic index replacement for Windows compatibility | Same workspace-local or explicitly selected owned cache behavior as `0.1.2` |
| `0.3.0` | Schema `1.0` unchanged per PDF; adds an additive collection layer (`collection.json` schema `1.0`) that holds many such workspaces below `members/` | Same workspace-local or explicitly selected owned cache behavior as `0.1.2` |

Old `pdfws-worker-*` directories contain stage-specific `result.json` scratch
without a reliable source-PDF identity. They are not reusable document caches
and are never imported as authoritative workspace data. A complete workspace
from `0.1.0` or `0.1.1`, however, remains directly usable because its schema did
not change.

### Persistent directory structure

One target directory represents one source PDF. Files below `cache/` are
regenerable; the other listed records are canonical unless noted otherwise:

```text
WORKSPACE_ROOT/
├── index.md
├── document.md
├── pages/
│   ├── index.md
│   └── page-0001.md
├── layout/pages/
│   └── page-0001.json
├── tables/
│   └── <table-id>.{md,json,csv}
├── visuals/
│   ├── index.md
│   ├── assets/<sha256>.png
│   └── descriptions/<visual-id>.{md,json}
├── tasks/
│   ├── visuals.jsonl
│   └── audits/<task-id>.json
├── source/original.pdf
├── metadata/
│   ├── workspace.json
│   └── diagnostics.jsonl
├── cache/                                      # regenerable and integrity-excluded
│   ├── .agent-pdf-workspace-cache-v1
│   ├── search.sqlite
│   ├── renders/page-<n>-<key>.png
│   ├── audit-staging/<sha256>.png
│   ├── rejected-agent-input/
│   └── tmp/
│       └── pdfws-worker-<random>/result.json   # temporary; removed after each worker
├── manifest-sha256.txt
├── log.md                                      # runtime log; integrity-excluded
└── .pdfws.lock                                 # temporary writer lock
```

Page splitting produces `pages/page-0001.md` and
`layout/pages/page-0001.json`, not `page-0001.pdf`. The original PDF remains
byte-for-byte available at `source/original.pdf`.

### Collection directory structure

A collection wraps unmodified workspaces; it never rewrites a member tree except through the
normal workspace API:

```text
COLLECTION_ROOT/
├── index.md                                # overview of every document
├── collection.json                         # collection manifest, schema 1.0
├── members/
│   └── <document-id>/                      # a complete PDF workspace as listed above
├── removed/<document-id>/                  # detached members, no longer part of the collection
├── cache/                                  # regenerable and integrity-excluded
│   ├── .agent-pdf-workspace-cache-v1
│   ├── search.sqlite                       # cross-document index
│   ├── search-state.json                   # per-member index fingerprints
│   └── tmp/
├── manifest-sha256.txt                     # covers index.md and collection.json only
└── .pdfws-collection.lock                  # temporary collection writer lock
```

Integrity is split: the collection manifest covers only the collection's own files, and every
member proves itself through its own `manifest-sha256.txt`. `pdfws verify COLLECTION` reports both
and is valid only when all parts are valid.

When `--cache-dir` or `cache_directory=` selects an external cache, only the
temporary worker branch moves:

```text
<ABSOLUTE_CACHE_ROOT>/
├── .agent-pdf-workspace-cache-v1
└── tmp/
    └── pdfws-worker-<random>/result.json       # temporary
```

Canonical output, `search.sqlite`, and rendered query crops remain below the
selected PDF workspace. Worker sessions contain bounded intermediate JSON and
are deleted on normal completion or timeout; they are not importable document
workspaces.

OpenCode integration creates files only when explicitly installed:

```text
<OpenCode config directory>/
├── skills/agent-pdf-workspace/
│   ├── SKILL.md
│   └── agents/openai.yaml
└── tools/agent_pdf_workspace.ts
```

The persistent workspace layout is optimized for lossless PDF exploration and
is not itself an OKF bundle. Use `pdfws export okf` to produce the separate,
conformant OKF 0.2 tree described above. ODT/ODF export remains outside the
workspace v1 contract.

The reproducible local comparison and its limitations are documented in
[benchmarks/parsebench/results/2026-07-15-test-suite.md](benchmarks/parsebench/results/2026-07-15-test-suite.md).

See [docs/format-v1.md](docs/format-v1.md) for the format contract and
[SECURITY.md](SECURITY.md) for the trust model.

## Development

```bash
uv sync --python 3.12
uv run pytest --cov=agent_pdf_workspace --cov-fail-under=80
uv run ruff check .
uv run ruff format --check .
uv run mypy src/agent_pdf_workspace
uv run python -m build
```

No public PyPI release or remote repository is created automatically.
