Metadata-Version: 2.5
Name: docgate
Version: 0.3.0
Summary: Bounded, receipt-first document extraction.
Project-URL: Homepage, https://github.com/BEASTSHRIRAM/docgate
Project-URL: Documentation, https://beastshriram.github.io/docgate/
Project-URL: Source, https://github.com/BEASTSHRIRAM/docgate
Project-URL: Issues, https://github.com/BEASTSHRIRAM/docgate/issues
Project-URL: Changelog, https://github.com/BEASTSHRIRAM/docgate/blob/main/CHANGELOG.md
Author: docgate contributors
License: Apache-2.0
License-File: LICENSE
Keywords: cost-control,document-ai,ocr,pdf,vision
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.10
Requires-Dist: pymupdf<1.27,>=1.24
Requires-Dist: pypdf>=4.0
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff>=0.8; extra == 'dev'
Provides-Extra: docs
Requires-Dist: mkdocs-material<10,>=9.5; extra == 'docs'
Provides-Extra: office
Requires-Dist: openpyxl>=3.1; extra == 'office'
Requires-Dist: python-docx>=1.1; extra == 'office'
Requires-Dist: python-pptx>=1.0; extra == 'office'
Description-Content-Type: text/markdown

# docgate

<p align="center">
  <img src="docs/images/logoo.png" alt="docgate logo" width="360">
</p>

**Spend the minimum on document AI, and know exactly what you didn't check.**

[Documentation](https://beastshriram.github.io/docgate/) ·
[PyPI](https://pypi.org/project/docgate/) ·
[GitHub](https://github.com/BEASTSHRIRAM/docgate)

Most document pipelines extract every page and return an empty string when a
page could not be read. docgate is deliberately different: it extracts the
free text layer first, bounds raster output before allocating memory, and always
returns a receipt that distinguishes blank, skipped, and unknown pages.

## Formats

| Format | Local extraction | Model call needed |
| --- | --- | --- |
| PDF | Embedded text per page | Only for pages without usable text |
| DOCX | Paragraphs and tables | No |
| XLSX | One logical page per worksheet | No |
| PPTX | One logical page per slide | No |
| TXT, Markdown, HTML | Native text | No |

Install Office readers when needed:

```bash
pip install "docgate[office]"
```

For a non-PDF byte stream, pass a file path today; stream adapters are an
intentional extension point for future formats.

```python
import docgate

result = docgate.read("bundle.pdf")
print(result.text)
print(result.receipt.complete)
print(result.receipt.unknown)
```

## Install

```bash
pip install docgate
```

## What it can do

- Audit a document locally without an API key, render, or provider call.
- Extract usable embedded PDF text for free, before calling any model.
- Output-bounded page rasterisation. A pathological declared page size cannot
  allocate an unbounded image.
- A `Clamp` is reported whenever requested output is reduced.
- A `Receipt` accompanies every result. Pages that were not read are explicit
  `Unknown` records, never silently omitted.
- Send only pages without a usable text layer to your OCR/vision provider.
- Fall back across providers and record every failed attempt as a `Degradation`.

## Audit before processing

Run a local preflight before selecting or paying a provider:

```bash
docgate audit bundle.pdf
docgate audit bundle.pdf --json
docgate audit bundle.pdf --report audit.html
```

The audit reports text-layer coverage, pages requiring OCR, embedded images,
predicted output clamps, and bounded pixel-buffer estimates. Its JSON excludes
the filename, source path, document text, images, and credentials. An incomplete
inspection exits with code 2.

The HTML report is a static, offline page with a color-coded coverage map. It
contains no document content or external resources and can be shared with a
team as an operational preflight artifact.

## Process many documents safely

`read_many()` provides bounded concurrency, per-document isolation, stable
input ordering, and aggregate receipts. One corrupt document cannot discard the
rest of the batch.

```python
batch = docgate.read_many(
    documents,
    providers=[ocr, vision],
    max_concurrency=8,
    per_document_budget=Decimal("0.02"),
    total_budget=Decimal("10.00"),
)

print(batch.receipt)
```

Sources are consumed lazily with bounded in-flight work. Concurrent total-budget
reservations are atomic, and batch progress events contain only a zero-based
input index and operational status.

## Bring any OCR or vision model

docgate is provider-neutral. It is not limited to OpenAI, Anthropic, Mistral,
or GLM. The ecosystem includes AWS Textract, Azure Document Intelligence,
Google Document AI and Gemini, specialist parsers such as LlamaParse and
Reducto, hosted vision services such as Together and Groq, and local engines
such as Tesseract, PaddleOCR, EasyOCR, Surya, and docTR.

Today Docgate ships lightweight native Mistral and Anthropic adapters, an
OpenAI-compatible vision route, and `CallableProvider` for any existing SDK or
local/private engine. Native capability-aware adapters are being added without
turning their SDKs into core dependencies. See the researched
[provider landscape](docs/provider-landscape.md) for current support versus the
adapter roadmap.

```python
from decimal import Decimal
import docgate

vision = docgate.OpenAICompatibleProvider(
    model="your-vision-model",
    api_key="...",
    base_url="https://your-openai-compatible-host/v1",
    cost_per_page=Decimal("0.004"),
)

result = docgate.read("bundle.pdf", providers=[vision])
```

Claude uses its native Messages API:

```python
claude = docgate.AnthropicProvider(model="your-claude-vision-model", api_key="...")
result = docgate.read("bundle.pdf", providers=[claude])
```

For a local or custom engine:

```python
local_ocr = docgate.CallableProvider("my-ocr", lambda png_pages: [run_ocr(p) for p in png_pages])
result = docgate.read("bundle.pdf", providers=[local_ocr])
```

## Control spend and retain private cache hits

Pass a hard budget to stop before an extra paid page. The partial result stays
useful and every page not attempted is listed as `BUDGET_EXHAUSTED`.

```python
result = docgate.read("bundle.pdf", providers=[vision], budget=Decimal("0.10"))
assert result.receipt.spent <= Decimal("0.10")
```

Use `SQLiteCache` for durable local cache hits across runs. Cache entries use a
SHA-256 digest of the rendered PNG bytes plus provider identity — never a path
or filename. Failed and empty responses are never stored.

```python
cache = docgate.SQLiteCache(".docgate-cache.db")
result = docgate.read("bundle.pdf", providers=[vision], cache=cache)
```

## Stay available when providers fail

Provider fallbacks are tried in order. Repeated failures open a circuit breaker,
so a dead key does not consume a full timeout for every page. After its cooldown,
docgate automatically allows one recovery probe; a successful call closes the
breaker. Every skipped or failed provider is recorded in `receipt.degraded`.

```python
docgate.configure(
    providers=[primary_vision, backup_vision],
    breaker_failures=3,
    breaker_cooldown_s=120,
)
```

Use `ExecutionPolicy` when provider calls need bounded waiting and retries:

```python
policy = docgate.ExecutionPolicy(
    timeout_s=30,
    max_attempts=3,
    initial_backoff_s=0.5,
    max_backoff_s=4,
)
result = docgate.read("bundle.pdf", providers=[primary, fallback], execution_policy=policy)
```

Timeouts and failed attempts remain visible in `receipt.degraded`. For progress
reporting, pass `on_progress`; events include page/provider/attempt metadata but
never document text, images, filenames, or credentials. See the
[reliability guide](docs/reliability.md) for the thread-timeout boundary and
fallback behavior.

## Escalate OCR to vision only when needed

A provider returning weak text is not the same as a provider outage. Use a
`TextQualityGate` to reject visibly insufficient OCR output and try the next
provider, usually a vision model. The receipt records this as
`quality_gate_rejected`, while preserving the OCR provider's health.

```python
ocr = docgate.MistralOcrProvider(api_key="...", cost_per_page=Decimal("0.004"))
vision = docgate.OpenAICompatibleProvider(model="your-vision-model", api_key="...")

result = docgate.read(
    "employment-bundle.pdf",
    need="CV and experience letter",
    selection="top_k",
    max_paid_pages=3,
    providers=[ocr, vision],
    quality_gate=docgate.TextQualityGate(min_characters=40, min_alphanumeric_characters=25),
)
```

See [OCR-to-vision routing](docs/ROUTING.md) for the complete mixed-document
workflow and its limits.

## Enforce where documents may go

Companies can prevent sensitive pages from crossing a trust boundary while
still using the same provider ladder. Policy runs before cache lookup or model
invocation; a denial is visible on the receipt and never silently bypassed.

```python
local = docgate.CallableProvider("local-ocr", run_ocr, trust_zone="local")
remote = docgate.OpenAICompatibleProvider(model="your-vision-model", api_key="...")

result = docgate.read(
    "confidential.pdf",
    providers=[local, remote],
    provider_policy=docgate.TrustZonePolicy(("local",)),
)
```

If local OCR cannot read a page, it becomes `Unknown`; docgate will not send it
to the external fallback. Custom policies receive cost and provider metadata,
never document text, images, filenames, paths, or credentials. See
[provider governance](docs/governance.md).

## Keep estimates auditable

Pricing is supplied by your application because provider pricing changes. Attach
the date and source you verified; every provider call records its estimate in
`receipt.quotes`. Cache entries include the exact image bytes plus a non-secret
provider/model/prompt identity, so a model or prompt change cannot reuse old
output.

```python
from datetime import date
from decimal import Decimal

pricing = docgate.PricingProfile(
    per_page=Decimal("0.004"),
    verified_on=date(2026, 8, 30),
    source="https://provider.example/pricing",
)
ocr = docgate.MistralOcrProvider(api_key="...", pricing=pricing)
```

`read()` is a convenience wrapper around `plan()` then `execute()`. Planning is
local-only and does not invoke any provider.

```python
plan = docgate.plan("bundle.pdf", need="degree certificate")
print(plan.estimated_cost)       # Decimal('0') in v0.1
print(plan.candidate_pages)      # ranked from local text-layer signals
result = docgate.execute(plan)
```

Plans and receipts also have versioned JSON forms for CI, job metadata, and
audit logs. They exclude document text, source paths, images, and secrets by
default.

```bash
docgate plan bundle.pdf --need "degree certificate" --selection top_k \
  --max-paid-pages 1 --json

docgate read notes.docx --json
```

An incomplete `docgate read` exits with code 2. Use `--include-text` only when
the JSON consumer intentionally needs extracted document content.

### Select before you spend

Pass `selection="candidates"` to read only pages with local keyword evidence,
or `selection="top_k"` with `max_paid_pages` to cap work at the highest-ranked
pages. Every excluded page is present in `receipt.unknown` as
`SKIPPED_BY_PLAN`; partial output is always visible.

```python
result = docgate.read(
    "bundle.pdf",
    need="degree certificate",
    selection="top_k",
    max_paid_pages=1,
    providers=[vision],
)
```

## What docgate does not do

- It is not an OCR engine or hosted model service.
- It is not RAG, a vector store, or an agent framework.
- It does not interpret, score, verify, or make decisions about documents.
- It has no telemetry or phone-home behaviour.

## Status

v0.3 is in development. The current main branch adds local document audits,
offline HTML coverage reports, and bounded multi-document execution on top of
selective paid-page routing, hard budgets, stable receipts, and the CLI. A local
cost/savings quote and policy-as-code are the next production controls.

## License

Apache-2.0.

## Releases

Releases are published from GitHub Actions using PyPI Trusted Publishing—no
long-lived PyPI token is stored in this project. See [the release guide](docs/RELEASING.md).
