Metadata-Version: 2.5
Name: bigcontext-schema
Version: 0.9.0
Summary: Tools for BigContext Schema: a checker with plain-English reasons, a coverage check, an extraction prompt for any LLM, a knowledge-graph export, and exports to OKF, OSI, dbt, OWL and ODRL.
Project-URL: Specification, https://github.com/BigContext-OpenSource/bigcontext-schema
Author: Big Context & Company
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Requires-Python: >=3.11
Requires-Dist: jsonschema<5,>=4.20
Requires-Dist: pyyaml<7,>=6
Requires-Dist: sqlglot<31,>=25
Provides-Extra: generate
Requires-Dist: anthropic<3,>=2.3; extra == 'generate'
Requires-Dist: deepagents<0.8,>=0.7.32; extra == 'generate'
Requires-Dist: google-genai<3,>=2.34; extra == 'generate'
Requires-Dist: langchain-anthropic<1.8,>=1.7.7; extra == 'generate'
Requires-Dist: langchain-core<2,>=1.7; extra == 'generate'
Requires-Dist: langchain-google-genai<4.5,>=4.4.2; extra == 'generate'
Requires-Dist: langchain-openai<1.8,>=1.7.3; extra == 'generate'
Requires-Dist: langchain<1.5,>=1.4.7; extra == 'generate'
Requires-Dist: langgraph<1.3,>=1.2.18; extra == 'generate'
Requires-Dist: openai<4,>=3.41; extra == 'generate'
Requires-Dist: pydantic<3,>=2.9; extra == 'generate'
Provides-Extra: graph
Requires-Dist: pyshacl<0.41,>=0.40.2; extra == 'graph'
Requires-Dist: rdflib<8,>=7.1; extra == 'graph'
Requires-Dist: reasonable==0.5.0a1; extra == 'graph'
Description-Content-Type: text/markdown

# BigContext Schema tools

Free tools for [BigContext Schema](https://github.com/BigContext-OpenSource/bigcontext-schema), a specification for recording business rules, procedures,
decisions and responsibilities as versioned records with their source text:

- **Generate:** turns a Markdown or plain-text file into a checked document with the AI model you choose (Anthropic,
  OpenAI or Google) and your own key, and shows every line of your text beside the statements that quote it.
- **Checker:** validates a document and explains every problem in plain English, including references that point
  at nothing and quotes that are not in the source; its notes point out where a document departs from a convention
  the guide states, and where a quote names a table, channel, party or state that its statement does not link.
- **Coverage check:** lists the lines of a source text that a document leaves unaccounted for.
- **Extraction prompt:** one brief that any LLM can follow to turn a text into a valid document.
- **Exports:** knowledge pages in OKF; metric and join definitions to and from dbt's semantic layer and OSI (Apache
  Ossie); the stated facts as OWL; rules to the W3C policy language ODRL; whatever a format cannot hold is reported.
- **Graph export:** documents as an RDF knowledge graph (record, stated facts, worked-out facts and their
  provenance), checked with SHACL, reasoned over with an OWL 2 RL reasoner, and asked questions that follow
  chains, each answer traced back to its quotes. See [graph.md](https://github.com/BigContext-OpenSource/bigcontext-schema/blob/main/graph.md).
- **Cypher export:** the same facts and their provenance for Neo4j, Memgraph, PuppyGraph and other Cypher engines:
  CSV tables, a portable Cypher script, a DuckDB loader, a PuppyGraph schema and the questions in openCypher.

## Install

```bash
pip install bigcontext-schema                     # check, coverage, prompt, and the okf, osi, dbt and odrl exports
pip install "bigcontext-schema[graph]"            # adds graph, cypher and owl
pip install "bigcontext-schema[generate]"         # adds generate
pip install "bigcontext-schema[generate,graph]"   # all of them; uv add takes the same names
uvx bigcontext-schema check document.yaml         # runs once without installing
```

A command whose libraries are not installed says which install adds them. Python 3.11 or later.

## Try it on your own text

```bash
pip install "bigcontext-schema[generate]"
export ANTHROPIC_API_KEY=...            # or OPENAI_API_KEY, or GOOGLE_API_KEY
bigcontext-schema generate notes.md     # writes notes.yaml, notes.html and, if anything did not fit, notes.unmapped.md
```

It prints each statement read back in plain English, what the writer was unsure of, and whether the document passes
the checker with every line accounted for. `notes.html` shows each line of your text beside the statements that quote
it. Anything the standard could not express (or fit only roughly) is kept in the document's `unmapped` list and
also written to `notes.unmapped.md`, each item with its line, quote, the statement type it would need and why,
ready to paste into the specification's enhancement form, which the closing message links to with the kind,
statement types and version filled in, so the open standard can grow to cover it (remove anything confidential
first). Exit code 0 when the document passes and every line is accounted for, 1 when something is left (the report
says what), 2 for a setting, key or file problem.

Each section of the text (about 80 lines; longer texts are cut at headings and written in parallel) is written by an
AI agent that checks its own work with the checker, the coverage check and a plain-English read-back. A second pass
grades the document before it finishes: it first answers, from the text alone, who each statement names (who
escalates, who must comply), then reads every statement and its links back against its quote, with the guide in
hand, for up to two rounds.

| Setting | Values |
|---|---|
| `BIGCONTEXT_PROVIDER` | `anthropic`, `openai` or `google`. Unset: the provider whose key is set. |
| `BIGCONTEXT_MODEL` | Any model id of that provider. Unset: `claude-opus-5-5` for Anthropic, `gpt-6.1-sol` for OpenAI; Google has no tested default yet. |
| `BIGCONTEXT_MAX_CALLS` | Model calls per section before it stops and reports what is left (default 40). |
| key | `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, or `GOOGLE_API_KEY` / `GEMINI_API_KEY`. Never written to any output. |

Tested on four data-team documents (about 45 lines each) against a 147-item answer key; every run passed the checker
with every line accounted for:

| Model | Matches the key | Time per document | About the cost for all four |
|---|---|---|---|
| `claude-opus-5-5` | 98% | 3 to 4 min | $3.60 |
| `gpt-6.1-sol` | 93% | 1.5 to 3 min | |
| `claude-sonnet-5-5` | 93% | 2.5 to 4 min | $2.40 |
| `claude-haiku-5-5` | 90% | 3 to 4 min | $0.25 |
| `gpt-6-luna` | 86% | 4 to 5 min | |

Other current model ids: Anthropic `claude-fable-5-1`; OpenAI `gpt-6.1-terra`, `gpt-6-astra`, `gpt-6-sol`, `gpt-5.5`,
`o3`, `o4-mini`; Google `gemini-4-argon`, `gemini-3.8-flash`, `gemini-3.6-flash`, `gemini-3.5-flash-lite`,
`gemini-3.1-pro-preview`. Any id the provider serves works.

Before it starts, generate asks the provider (free) whether the key works and the model exists. A wrong setting is
reported with what is wrong and how to fix it, such as "OpenAI has no model 'gpt-6.1-soll' that this key can use.
Did you mean gpt-6.1-sol?" or "ANTHROPIC_API_KEY holds what looks like an OpenAI key".

Options `--provider`, `--model`, `--max-calls` and `-o` override the settings.

## Use

```bash
bigcontext-schema generate notes.md [-o notes.yaml] [--provider anthropic|openai|google] [--model ID]
bigcontext-schema check document.yaml [--source source.txt] [--require-quotes]   # --require-quotes: written from a text
bigcontext-schema coverage document.yaml --source source.txt [--lines 127-300]
bigcontext-schema prompt --source source.txt [--lines 127-300] [--tag C2] > prompt.md
bigcontext-schema graph document.yaml [--as-of 2026-10-01] [--validate] -o graph.trig    # address: the document's own
bigcontext-schema cypher document.yaml [--as-of 2026-10-01] -o graph/
bigcontext-schema odrl document.yaml [--as-of 2026-10-01] -o policy.jsonld
bigcontext-schema okf document.yaml [...] [--as-of 2026-10-01] -o pages/      # a new or empty folder
bigcontext-schema osi document.yaml [...] [--name NAME] -o model.yaml
bigcontext-schema dbt document.yaml [...] -o semantic_models.yaml
bigcontext-schema owl document.yaml [...] [--as-of 2026-10-01] -o facts.ttl   # needs [graph]
```

graph, cypher and odrl join element ids to the source's `base_iri`, or to `urn:bigcontext-schema:<source id>/`
when it has none, so the same document always gives the same graph (`--base` sets another address). They need
`--as-of` when the document gives no `as_of`: the date is never taken to be today.

### Export

| Format | Command | From Python (`from bigcontext_schema import export`) | What it holds |
|---|---|---|---|
| OKF 0.2 knowledge pages | `okf` | `export.to_okf(documents, as_of=, generated_at=)` | One page per element and procedure, a table's fields as rows of its `# Schema`, each statement in force read back with its quote and source, and `index.md` |
| OSI semantic model | `osi` | `export.to_osi(documents, name=)` | Datasets, fields, relationships and metrics, as YAML |
| dbt semantic models | `dbt` | `export.to_dbt(documents)` | `semantic_models` and `metrics`, as YAML |
| OWL | `owl` | `export.to_owl(documents, base=, as_of=)` | The stated facts in force, each individual declared with its label |
| ODRL policy | `odrl` | `export.to_odrl(documents, base=, as_of=)` | The rules in force, as JSON-LD |

`documents` is `{name: document}`. Each function returns `(output, report)`; the report names, by id, what the format
could not hold (a statement not in force on the date, a rule part ODRL has no place for), and each command prints it.
OKF, OSI and dbt take the documents of one source (its sections, merged first) and stop on documents from several
sources or on a document that fails the checker. OKF and OWL hold what is in force on the date; OSI and dbt take, for
each metric, the one definition the converters export, with no date. Exit code 0 when written, 1 when the format got
nothing (OSI needs a table; dbt a table or a metric), 2 for a setting or input to fix.

The OWL file imports the fact ontology by its address, `https://w3id.org/bigcontext-schema/facts`; to work offline,
load the packaged `bigcontext_schema/spec/bigcontext_facts.owl.ttl` in its place. An OWL reasoner over the two works
out the same facts the `graph` command's reasoner does. To trace a fact to its statement and quote, use `graph`, whose
record and provenance graphs the OWL file leaves out.

From Python:

```python
from bigcontext_schema.check import Checker
from bigcontext_schema.coverage import quotes, uncovered
from bigcontext_schema.prompt import extraction_prompt
from bigcontext_schema.generate import generate, write
from bigcontext_schema.interop.dbt import core_to_dbt
from bigcontext_schema.interop.osi import core_to_osi
from bigcontext_schema.rdf import dataset, facts_graph, record_graph, validate
from bigcontext_schema.facts import ask, path, trace
from bigcontext_schema.links import of, facts_of
```

`rdf`, `facts`, `cypher` and `export.to_owl` need `[graph]`; `generate` needs `[generate]`.

A document from a text in memory, with the key your code reads:

```python
import pathlib
from dotenv import dotenv_values          # python-dotenv; the key is not put in os.environ
from bigcontext_schema.generate import write

key = dotenv_values(".env")["ANTHROPIC_API_KEY"]
result = write(pathlib.Path("notes.md").read_text(), provider="anthropic", model="claude-opus-5-5", key=key)
```

`result` holds `document`, `yaml`, `passes`, `problems`, `missing_lines`, `sections`, `used`, `report_html` and
`unmapped_markdown`; no file is written. Without `key`, the key is read from the process environment, as the command
reads it; without `provider`, it follows from the key. Keep `.env` out of git. A wrong setting, key or model raises
`SettingsError` before any paid call, with what is wrong and how to fix it. The first run for a provider registers
deepagents' harness profile for that provider in the whole process: every deep agent of that provider then runs
without the general-purpose subagent and without the `ls`, `glob`, `grep`, `delete`, `task` and `execute` tools.

The graph's questions, from Python: `ask(facts_graph(data, base), "who_answers_for", "t.customers.email", base)`.
Every link of the statements in force (1.2): `ask(record_graph(data, base), "record/connections", "t.orders", base)`,
or `of(statement, kind_of)` for one statement without a graph. `dataset(..., in_force=[ids])` gives the statements a
caller holds in force, in place of the specification's rule.
For Cypher engines, `bigcontext-schema cypher` writes `graph.cypher` (run it in Neo4j, Memgraph, FalkorDB or
Neptune), CSV tables for bulk loaders, and for PuppyGraph `duckdb.sql` plus `puppygraph_schema.json`: load the tables
into DuckDB, mount the file, upload the schema. The questions are in `graph/questions/`.

The output loads into any RDF store (Jena, GraphDB, RDFox, Stardog, Neptune) or into Neo4j with neosemantics; with
`--reasoner none` your store's own reasoner works out the facts from `bigcontext_facts.owl.ttl`.

## Development

```bash
uv sync --all-extras                         # the tests need both extras
uv run pytest
uv run python scripts/build_spec.py          # after changing ../spec/bigcontext_schema.yaml, guide.md or bigcontext_link_roles.yaml
uv run python scripts/build_spec.py --check  # fails if a generated file or the packaged specification is out of date
uv run python scripts/build_conformance.py   # after a change the specification's conformance cases (../spec/tests/) must follow
```

Build the Docker image with `docker build -t bigcontext-schema .` from this folder; the Dockerfile's first lines show
how to run it. Check the fact ontology with HermiT (needs Java): `JAVA_HOME=<a JDK> uv run python
scripts/check_ontology.py`.

The package carries a copy of the specification in `src/bigcontext_schema/spec/`, written by `scripts/build_spec.py`;
edit the specification in `../spec/` only. LinkML is needed only to generate.

## Licence

Apache License 2.0. Copyright 2026 Big Context & Company. The specification files the package carries are under
the same licence. See `LICENSE` and `NOTICE`.
