Metadata-Version: 2.4
Name: fpf-chunking
Version: 0.2.0
Summary: FPF-guided (holon / bounded-context) semantic document chunking
Author: Valentin Alekseev
Author-email: Valentin Alekseev <valentin.alekseev@gmail.com>
License-Expression: MIT
Requires-Dist: anthropic>=0.120.2
Requires-Dist: openai>=2.50.0
Requires-Dist: google-genai>=1.0.0 ; extra == 'gemini'
Requires-Python: >=3.11
Provides-Extra: gemini
Description-Content-Type: text/markdown

# fpf-chunking

FPF-guided (holon / bounded-context) semantic document chunking: parses a
document's structure, uses an LLM to validate which structural boundaries
make good chunk boundaries, and assembles the result into token-bounded
chunks — without ever exceeding the LLM's context window on a single
oversized structural unit.

## Install

    pip install fpf-chunking

## Usage

From a file:

```python
from anthropic import Anthropic
from fpf_chunking import MarkdownLoader, chunk_document

doc = MarkdownLoader().load("path/to/doc.md")
chunks = chunk_document(
    doc,
    anthropic_client=Anthropic(api_key="..."),
    anthropic_model="claude-sonnet-4-5-20250929",
    max_chunk_tokens=400,
)
```

From content already in memory (e.g. fetched over the network by your own
wrapper — no filesystem path required):

```python
from fpf_chunking import Document, chunk_document

doc = Document(doc_id="my-doc", text=buffer_from_network)
chunks = chunk_document(doc, anthropic_client=..., anthropic_model=..., max_chunk_tokens=400)
```

Then embed however you like:

```python
from openai import OpenAI
from fpf_chunking import embed

embedded = embed(chunks, client=OpenAI(api_key="..."), model="text-embedding-3-small")
```

See the module docstrings for the full API: `parse_holons`,
`extract_candidate_boundaries`, `split_oversized_holons`, `assemble_chunks`,
`FPFBoundaryValidator`, `fixed_size_chunks` (naive baseline), and
`semantic_chunks` (embedding-similarity baseline) are all exported for
callers who want the individual pipeline stages instead of the composed
`chunk_document`.
