Metadata-Version: 2.4
Name: contextgc
Version: 0.4.6
Summary: Deterministic context compiler for AI agent transcripts. Retires superseded facts, compacts tool output, pins declared invariants. No dependencies, no model calls.
Author: Jay Gopal Tripathy
License: MIT
Project-URL: Homepage, https://context-hackdevengers.vercel.app
Project-URL: Repository, https://github.com/j4yop/context-hackdevengers
Project-URL: Issues, https://github.com/j4yop/context-hackdevengers/issues
Keywords: llm,context-engineering,context-management,agents,token-reduction,prompt-compaction,kv-cache,deterministic
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: server
Requires-Dist: fastapi>=0.110; extra == "server"
Requires-Dist: uvicorn>=0.27; extra == "server"
Requires-Dist: pydantic>=2.0; extra == "server"
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: fastapi>=0.110; extra == "dev"
Requires-Dist: httpx>=0.27; extra == "dev"
Requires-Dist: uvicorn>=0.27; extra == "dev"
Provides-Extra: bench
Requires-Dist: pandas>=2.0; extra == "bench"
Requires-Dist: pyarrow>=15.0; extra == "bench"
Dynamic: license-file

# contextgc

**A deterministic context compiler for AI agent transcripts.**

Long agent sessions accumulate two kinds of dead weight: facts that were replaced
but never removed, and tool payloads far larger than the signal in them. The
model then has to reconcile two different values for the same key — and you pay
for every token of it.

`contextgc` compiles a transcript down to what is currently true. No model call,
no network, no API key, no dependencies. Same input, same output, microseconds.

```bash
pip install contextgc
```

> **On PyPI** as `contextgc`, published via Trusted Publishing with no stored
> credentials. `pip install contextgc` gets you the library and nothing else —
> it has **zero runtime dependencies** by design, and it is a library, not a
> CLI: there is no `contextgc` command. The entry points are
> `compile_messages`, `load_schema`, `patch_openai` and `StateDAG`.
>
> If you found an earlier version of this README saying "Not on PyPI yet", it
> was true when written and then stopped being true, which is how a README
> starts lying in the most load-bearing spot it has. Every number in this file
> is reproducible from the repository; see below for how to get it.

```python
from contextgc import compile_messages, load_schema

messages = [
    {"role": "user", "content": "Deliver order ORD-1 to 402 Oak Street, Apt 5, Springfield, IL 62704. Pay with my credit card."},
    {"role": "assistant", "content": "Confirmed: 402 Oak Street, Apt 5, on the credit card."},
    {"role": "user", "content": "Actually reroute to 900 Pine Avenue, Suite 12, Chicago, IL 60601."},
    {"role": "assistant", "content": "Rerouted to 900 Pine Avenue, Suite 12."},
    {"role": "user", "content": "Thanks."},
]

compiled, telemetry = compile_messages(messages, schema=load_schema("logistics"))

telemetry["active_state_slots"]
# {'delivery_address': '900 Pine Avenue, Suite 12', 'payment_method': 'credit card'}

telemetry["retired_turn_indices"]
# [1, 2]  -- the two turns that asserted a now-stale address
```

```
  system: [ACTIVE_AGENT_STATE]
  - delivery_address = "900 Pine Avenue, Suite 12" (turn 3 [inferred])
  - payment_method = "credit card" (turn 0 [inferred])
     user: Deliver order ORD-1 to 402 Oak Street, Apt 5, Springfield, IL 62704. Pay with my credit card.
  assistant: Rerouted to 900 Pine Avenue, Suite 12.
     user: Thanks.
```

Five turns in, three out. The address that was current for one turn is gone, and
the one that superseded it is in a header the model can read without re-reading
the transcript.

> **Two things this example has to get right, and the version that was here
> first got both wrong.** It did not pass a `schema`, and it cannot: with no
> schema the read path has no vocabulary and extracts **nothing** — that is
> deliberate, not a bug, since a built-in default domain would match prose in
> every other domain. And the content has to be something the shipped schema
> actually matches. The earlier version of this example used *"Tower B, Flat
> 402"*, *"Gate 2"* and *"the Clubhouse desk"*, and claimed an output of
> `{'destination_address': 'Gate 2', 'dietary_allergy': 'peanut'}`. Those two
> slot names exist nowhere in this project, and none of that prose matches the
> `logistics` patterns, which want a US street address (`Oak Street`, `Pine
> Avenue`) and a phrase like *"my credit card"*. Run it and you got `{}` and the
> transcript straight back. **The first example in a README is the whole
> product for someone evaluating it in thirty seconds, and it was returning
> nothing while looking like it worked.**

Try it on your own transcript at **[context-hackdevengers.vercel.app](https://context-hackdevengers.vercel.app)** —
paste a conversation on the left, get the compiled context back on the right.

---

## Two ways to get state out of a transcript

### 1. Read path — infer it (default, works with any model)

Patterns scan the text for assertions. Deterministic, zero cost, works on
transcripts you did not produce. Its ceiling is hard: it cannot resolve *"send it
to the new place instead"*, because "it" and "the new place" are not patterns.
Anything it misses leaves the state silently incomplete.

### 2. Write path — the agent declares it

With `teach_protocol=True` the agent is asked to report what it concluded, as a
structured side-effect of the turn it was already making:

```
<contextgc-state>
{"assert": {"destination_address": "Gate 2"},
 "pin":    {"dietary_allergy": "peanut"},
 "revoke": ["order_id"],
 "unsure": {"rider_location": "maybe the west gate"}}
</contextgc-state>
```

**No extra model call.** The block rides along in a completion the agent was
going to make anyway. It is not free, though: the instruction is ~200 tokens
added to the system prompt on every request, and in `compact` mode that
invalidates the prefix cache for the first message. Use `cache_friendly` if you
want both.

The block is stripped before the transcript is sent onward, including from inside
`tool_calls` arguments, so a model never sees protocol markup.

#### What the write path actually buys

| | Read path | Write path |
|---|---|---|
| Provenance | guessed | **declared**, and inference cannot overwrite it |
| Void facts | inexpressible | `revoke` — void, not merely stale |
| Unsettled facts | indistinguishable | `unsure`, surfaced and flagged |
| Pinned rules | hardcoded list | `pin`, lifted only by an explicit re-pin |
| Pronoun references | **not resolved** | **not resolved either** |

That last row is the honest one. The write path was motivated by coreference,
and it does **not** reliably fix it: the agent has to notice the reference and
report it. When it does, the result is authoritative; when it does not, the
state is quietly incomplete. There is currently no measurement of how often that
happens, which is the main open question about this feature.

#### Who may declare

**Only the assistant.** Tool output is a fetched page or an MCP result, user
text may be a pasted injection or a quote of this README, and a system message
is a summarised transcript that inherits whatever those contained. If any of
them could assert state, `declared` would be a mark of forgery rather than of
authority. Markup in an untrusted role is still stripped from the prompt, but
never applied, and is counted in `blocks_in_untrusted_roles`.

#### Read `conflicts`, not `declared_share`

```python
compiled, telemetry = compile_messages(messages, teach_protocol=True)
telemetry["declarations"]["declared_share"]   # provenance mix, NOT a score
telemetry["conflicts"]                        # declared vs. what the text says
```

`declared_share` is the fraction of tracked facts that came from the agent. It is
**not** a confidence measure, and it is specifically not a safety one: a declared
fact always outranks an inferred one, so the score goes *up* when the model's
opinion wins a disagreement. A transcript where the agent declared a stale value
and was never contradicted reads `1.0`.

`conflicts` is the signal that matters. It fires when the agent asserted a key
and the transcript's own text still says something different:

```
conflicts: [{'entity': 'gate_code', 'declared': '1111', 'declared_turn': 1,
             'inferred': '9999', 'inferred_turn': 2,
             'note': 'the declaration won; verify it is still correct'}]
```

Read that as: *the agent may be quoting an older turn; the human has since said
otherwise in plain text.* A model that keeps getting this wrong is not
declaring better, it is declaring more confidently than the evidence supports.

`rejected_writes` is the other one: writes the compiler refused — pinned keys,
lifted guardrails, markup from an untrusted role.

#### Cost and bounds

An agent that re-declares its entire state every turn is the most likely real
failure: every re-assert supersedes the last, so every declaring turn becomes
prunable and the compiler deletes the conversation. The register is therefore
capped at 64 facts and 512 characters per value; `state_keys_dropped_over_limit`
reports what was cut. When the register costs more than it saves,
`context_grew` and `token_growth` say so — `compression_ratio_pct` is clamped at
0 and would hide it.

---

## Measured against real agent transcripts

Everything above is a design claim. This is what happens when the compiler is run
over 40 real SWE-agent trajectories from
[`nebius/SWE-agent-trajectories`](https://huggingface.co/datasets/nebius/SWE-agent-trajectories)
— 1,936 turns, 2.26M characters, 20 repositories, real model output and real
tool output. The shard is repository-ordered, so the loader caps trajectories per
repository (`--per-repo`); without that cap the first 40 transcripts come from three
repositories and every number below inherits that narrowness.

**Two corpora, because one is not evidence of generalisation.**

| | coding | airline support | retail support |
|---|---|---|---|
| corpus | `nebius/SWE-agent-trajectories` | `APIGen-MT-5k` (airline) | `APIGen-MT-5k` (retail) |
| what it is | SWE-agent bug-fixing | reservation tool use | order / exchange tool use |
| sample | 40 transcripts, 20 repos | 60 conversations | 80 conversations |
| facts tracked | 39 | 72 | 47 |
| key re-assertions | 77 | 10 | 2 |
| token reduction | **66.5%** | **59.0%** | **62.5%** |
| turns retired | 161 | 9 | 2 |
| tool payloads compacted | 668 | 138 | 275 |
| retirement violations | 0 | 0 | 0 |
| contexts that grew | 0 | 0 | 0 |
| precision (independent units) | **90% (n=130, CI 84–94%)** | **99% (n=149, CI 95–100%)** | 100% (n=47, CI 92–100%) |

Run with `--corpus swe-agent`, or `--corpus apigen --domain airline|retail`.

**Reduction transfers. Supersession does not.** Coding re-asserts a key 77 times
across 40 transcripts; retail does it twice across 80. That is the honest limit of
this library: compaction of bulky tool output helps everywhere, but the
last-write-wins state register only earns its keep where a conversation keeps
changing its mind about one thing.

```bash
git clone https://github.com/j4yop/context-hackdevengers
cd context-hackdevengers
pip install -e '.[bench]'
python -m benchmarks fetch          # 85 MB parquet shard
python -m benchmarks run --limit 40 --schema contextgc/schemas/coding.json
```

> **A clone is required, and `pip install 'contextgc[bench]'` is not enough.**
> The `bench` extra installs `pandas` and `pyarrow` and nothing else. The
> harness lives in `benchmarks/`, which is not in the wheel — the wheel ships
> `contextgc` only, deliberately, so the library stays dependency-free. So the
> command above reads
> `ModuleNotFoundError: No module named 'benchmarks'` from a pip install, and
> that is the correct behaviour, not a bug to report. Every figure in this
> README is reproducible from a clone; none of them is reproducible from the
> wheel, because the wheel does not contain the thing that measured them.

```
CORPUS  swe-agent-trajectories
  transcripts      40
  turns            total 1390, median 32, range 12-92
  characters       1,958,156
  models           swe-agent-llama-70b x38, swe-agent-llama-8b x2

  facts_extracted            40   n=40
  keys_reasserted            77   n=40
  token_reduction          66.5%  n=40
  tool_payloads_compacted   668   n=40
  turns_retired              161   n=40
  retirement_violations       0   n=40
  compile_ms_p50            2.96  n=40
  compile_ms_p95           11.97  n=40

PRECISION (hand-labelled sample)
  labels supplied        39
  matched an extraction  39
  JUDGED                 39   <- the denominator
    correct              39
    incorrect             0
  PRECISION (rows)       100%   (n=39)
  n is inflated          39 rows collapse to 36 independent units
  PRECISION (clusters)   100%   (n=36, 95% CI 90-100%)
  repositories covered  20
```

Every count above is deterministic and reproduces exactly. The two `compile_ms`
figures are wall-clock on one machine and move run to run — treat them as "single
-digit milliseconds", not as a benchmark.

**The interval is the finding, not the point estimate.** 36 independent judgements
cannot distinguish 95% from 100%, and one repository contributes a handful of them.
This still catches gross regression; it is not a claim about unseen transcripts. The
labels and the loader settings that produced them are committed
(`benchmarks/labels/`), so the number can be regenerated and argued with.

### Using it in a loop

The library is a compiler plus a wrapper. The wrapper is the part you wire in,
and it is worth knowing that the state register is re-derived per call rather
than carried in memory -- so a multi-turn agent does not need a session object
to keep track of anything:

```python
from openai import OpenAI
from contextgc import load_schema, patch_openai

client = patch_openai(OpenAI(), schema=load_schema("coding"))
# every client.chat.completions.create(...) call now compiles first,
# and the response carries what it did as `.context_gc`
```

`tests/test_agent_loop.py` drives exactly this shape -- a growing history
through the real wrapper -- and asserts the property that matters: across 40
turns in which the agent moves between ten files and contradicts itself, the
register always names the file the previous turn announced, never two at once,
and no retirement ever strands a value. The same file also checks that a
declaration arriving in a user or tool turn is stripped rather than believed.

### When it does *not* help

Two behaviours worth knowing before you point this at a live agent. Both are
deliberate; neither was documented until they were measured.

**The last two turns are never retired.** A correction that arrives immediately
after the wrong claim leaves both turns in place, because the trailing turns are
the model's most recent exchange and cutting them would strip the reply it is
about to continue. So a short session can hold a visible contradiction and still
report `retired_turn_count: 0`; there, the state register is what resolves it.

```python
# adjacent: nothing retired, but the register states the current value
H + [asst("editing `wrong.py`."), asst("actually editing `right.py`.")]
#   -> retired 0, register says current_file = "right.py"

# one turn between them: the wrong claim is gone
H + [asst("editing `wrong.py`."), user("ok"), asst("actually editing `right.py`.")]
#   -> retired 1, "editing `wrong.py`" no longer reaches the model
```

**A short session can cost more than it saves.** The state register is a fixed
header plus one line per slot. On a 7-turn transcript the compiled output is
*larger* than the input, and the telemetry says so rather than rounding it away:
`compression_ratio_pct: 0.0`, `context_grew: True`. The 70% figure comes from
40 real transcripts averaging 48 turns. If your sessions are shorter than that,
measure before you adopt this.

### What this measurement changed

Running it was not a formality. It overturned five things.

**1. The default entity schema was producing confident nonsense.** The shipped
default was a logistics schema — `destination_address`, `refund_claim`,
`gate_code` — applied to everything. On 60 real coding transcripts it produced
75 facts, and every sampled one was prose matched by accident:

```
destination_address = "of parentheses"
destination_address = "it, otherwise do a lookup using type"
destination_address = "a placeholder dictionary"
```

The patterns were written to match a fixture, and a coding transcript is full of
the words they look for. **`ENTITY_PATTERNS` is now empty**, and a schema is
opt-in per domain. The old patterns are kept in
`contextgc/schemas/logistics.json` with the measurement that condemns them.

**2. The sanitizer never fired on real data.** It keyed on a `TOOL_OUTPUT` marker
and on `role in (tool, function)`. In the corpus, **0% of messages carry that
marker** and tool output is filed under `user`. So 668 real tool outputs were
being passed through untouched. Detection is now content-based — a stack trace
is a stack trace whatever role it is filed under — and picks up 19.9% of
messages. Token reduction on the same corpus went from **17% to 58%**.

**3. Reading state out of tool output was a precision failure.** A file path in a
grep listing (`Found 14 matches for X in /path/to/dispatcher.py:`) is not a
statement about which file the agent is editing, but the pattern promoted it
anyway. State is no longer inferred from machine-generated output.

**4. Two more precision defects, found by the second labelling pass.**
A weak `in <path>` trigger matched prose — *"Upon reviewing `main.py` again …
`dispatcher.py` uses a helper"* became `current_file = dispatcher.py` — and a
single-letter `c` extension parsed `example.com` as `example.c`. Patterns now
require an explicit verb acting on the path, and a real path prefix. Both
defects were found by hand-labelling, not by the aggregate numbers.

**6. A repeat was being treated as a contradiction.** There was no value
comparison at all: any new extraction for a live entity superseded the previous
one. An agent that said `**Cabin Class:** Business` and then `business class` had
not changed its mind, yet the earlier turn was retired as superseded — taking
whatever else it carried with it. A repeat is now a reaffirmation, and a value
that differs only in case is recorded as one rather than acted on. Genuinely
different values supersede as before.

This one is worth stating plainly because it changed the headline. Fixing it
removed phantom supersession that had been inflating every figure: coding
re-assertions fell from 187 to 77 and retirements from 271 to 161, and travel
retirements from 58 to 9. The numbers above are the corrected ones. The earlier
figures were not wrong arithmetic — they were measuring repeats as changes.

**7. A schema-free extractor survived the rewrite.** Behind the registered
patterns sat a generic `set <key> to <value>` scraper and a harvester that turned
any `{"k": "v"}` in any turn into a `slot_k` fact. Both contradicted claims this
README makes: that the default schema is empty *because* a default matching
everything produces confident nonsense, and that state is never inferred from
machine output. The JSON harvester's only guard was `role not in ("tool",
"system")`, and every corpus measured here files tool results under `user`. It
was found by extracting `slot_symbol = "€"` out of `currency = {"symbol": "€"}`.
One extraction in 40 transcripts, so removing it barely moved any aggregate —
which is exactly why it survived: the numbers never showed it.

**5. The precision sample was too small, too narrow, and partly fake.** The
first label set drew all 24 rows from **two** repositories, and several rows were
the *same turn* of the same issue read twice from two trajectories of that issue.
Counting those as independent evidence inflated the denominator. The corpus loader
made this worse: the shard is repository-ordered, so "the first 40 transcripts"
were 40 trajectories from **3** repositories, and every aggregate number
inherited that.

Both are fixed rather than caveated. The loader takes `--per-repo` (default 2), so
a run spans as many repositories as the shard allows — **20** for the same 40
transcripts. The label set was re-read from scratch, one extraction per
`(repository, turn)` — 39 rows, which collapse to 36 independent units, across 20
repositories. The
report now prints the row count *and* the clustered count, so a reader can see
when `n` is inflated, plus a 95% Wilson interval so a point estimate is not
mistaken for a measurement.

This changed the headline: the previous **88% (n=16)** was, on independent units,
**100% (n=36, 95% CI 90–100%)** from a far wider sample — and a naive re-run of
the old labels against the widened corpus collapsed to **n=2**, which is what
exposed the problem in the first place. It then had to be re-read a second time,
because fixing the repeat bug in defect 6 changed what the compiler extracts.

#### What the labels said, which is not what the 100% said

The labels arrived — 104 new rows, every `failing_test` one a person had never
looked at — and the headline moved. It is no longer 100%, and the number that
was flattering is the number that was narrow:

| | before | after |
|---|---|---|
| coding precision | 100% (n=36) | **90% (n=130, CI 84–94%)** |
| `current_file` | 100% (n=36) | **98% (n=98)** |
| `failing_test` | *unmeasured* | **75% (n=44)** |

**`failing_test` is four times worse than the slot beside it**, and it is worse
for one reason, visible in the notes: nine of its eleven errors are a test file
the agent *mentioned* rather than *acted on*. "Found 47 matches… in
`tests/unittests/test_dispatcher.py`" is a grep result, not a failing test. The
read path cannot tell the difference, because telling the difference is the
whole content of the slot.

That is the direct cost of the reachability fix below, which replaced a verb
requirement with a noun phrase. A noun phrase matches mention; a verb matches
action. The old pattern was accurate and unreachable. The new one is reachable
and 25% wrong, and **the reachable one is worth more**, because you cannot
improve what you cannot see. It is also why the reported figure is the combined
90% and not `current_file`'s 98%: the console shows one number, and it has to be
the one covering the whole label set.

**And the tenth note found a bug no aggregate ever would have.** One label read
*"extracted value is corrupted with leading dot stripped (`/tests/...` instead of
`./tests/...`)"*. The cause was `val.strip().strip(".,;")` in `state_dag.py`:
`strip` removes characters from **both** ends, so a legal relative path lost its
leading dot and stopped naming a file that exists. It occurred **once in 159,124**
pattern matches, because agents mostly write absolute or bare paths — so no
precision figure, at any sample size, could ever have surfaced it. Fixed to
`rstrip`, with a test.

#### The second coding slot, unreachable and then not

`coding.failing_test` matched **740 times** in the coding corpus and **not once in
speech**. Every hit sat inside a pytest result — `FAILED tests/test_x.py::test_y` —
which is machine output the read path is *forbidden* to infer from. So the slot
could never fire, could never be labelled, and had no evidence behind its
contract. It was carried in a shipping schema by a number that was large and
true and meant nothing.

The number hid this because a corpus-wide count cannot tell you where the
matches were. `test_the_one_slot_that_never_fires_from_speech_says_so` existed
precisely to catch it, and the fix was to the **pattern**, not to the
measurement. Requiring the noun phrase that actually introduces a file — *the
test file `test_dispatcher.py`*, *the tests in `test_run.py`*, *the failing test
`test_x.py`*:

| `coding.failing_test` | before | after |
|---|---|---|
| speech observations | **0** | **78** |
| distinct values | 0 | **49** |
| observations in tool output | 740 | 14 |

Across 174,815 agent turns there are 81 noun-phrase hits, 71 of them not beside a
shell command. Those are assertions, not commands — which is the distinction the
old pattern had collapsed. Nothing the read path actually used is lost: it was
already discarding the other 740.

The read path now extracts `test_dispatcher.py`, `test_cognitive_complexity.py`
and `tests/test_cognitive_complexity.py` from the first 40 transcripts, which
before produced **zero**.

Two honest costs. The derived contract is now `[...]test[...].py`, which accepts
any path containing `test` — so it also accepts `latest_file.py` and
`my_test_helper.py`, and those are recorded as residual false accepts rather than
fitted away with a lookup table. And the slot is named `failing_test` while the
prose it now matches says only *a test file*; that was true of the old pattern
too, and renaming a shipped key is a breaking change not made to improve a
sentence. **49 distinct values is thin** for precision, and the worksheet below
exists to thicken it. It is measurable now, which it was not.

#### A cheaper check that turns out not to be good enough

With the slot reachable, the obvious next move is a check that needs no human.
Every `failing_test` value the read path produces from speech can be tested for
existence: does that path appear anywhere in what the agent's own tools listed?

| slot | read from speech | names a file the tools listed | rate |
|---|---|---|---|
| `failing_test` | 78 | 78 | **100.0%** |
| `current_file` | 38,375 | 38,253 | 99.7% |

`failing_test` scores 100%. **So does `current_file`, and that is the whole
finding: the check does not discriminate.** Both slots are near-perfect at it
because both name files in a repository the agent is actively working in, so
the file existing is nearly guaranteed and near-worthless as evidence that the
read path chose *correctly*.

It does rule out one real failure — the read path is not inventing paths that
were never there — and it rules out nothing else. A `failing_test` naming the
wrong-but-real test file passes it. So it is recorded as a floor, not a
precision figure, and the human labels remain the only route to the number.

#### The worksheet was a trap, and it is disarmed

`failing_test` is now reachable, so the next step is a worksheet. The default
window returns **2 rows**:

```
--limit  60 ->  2 rows       --limit 2000 -> 21 rows
--limit 500 ->  7 rows       --limit 6000 -> 44 rows
```

Two rows read as *this slot cannot be labelled* when the shard holds 49 distinct
values. A person who believed that would drop the slot and report coding
precision as single-slot permanently — which is how it was single-slot in the
first place, and the reason the 100% said so little. So the worksheet now names
any slot short of its budget, and **refuses to guess the cause**: the window may
be too small, or the corpus may have no more, and only raising `--limit` tells
them apart. At `--limit 6000` the shortfall is real exhaustion — 44 of the 49
available values, the other 5 already labelled — which is the answer the tool
would not assert on its own.

`ws.json` is built at `--per-entity 60 --limit 6000`: 44 `failing_test` rows and
60 `current_file`, all `unlabelled`, no suggestions attached. It is waiting on
a person. **Coding precision stays at 100% (n=36, all `current_file`) until
someone fills it in**, and that sentence is the honest state of the number, not
a rounding of it.

#### The write path, checked at last

Everything above scores what the **read path** inferred. The **write path** is the
model declaring facts, and until now nothing had ever checked whether a declared
fact was *true* — which is the gap that mattered, because a declared value wins
over the read path by design. An unchecked declaration silently overrides
everything the read path got right.

The same records settle it: a capture is keyed by transcript and turn, the corpus
has the booking record for that transcript, and the turn indices line up.

```
python -m benchmarks.ground_truth travel --declarations captures/airline-14b.json
```

| 14B airline capture | count |
|---|---|
| declared values | 109 |
| corroborated by the record | **70** |
| **contradicted by the record** | **12** |
| unverifiable (no record saw the outcome) | 2 |
| unchecked (slot has no record field) | 25 |

**So the write path is wrong 12 times in 82 decidable declarations — 85.4%
corroborated.** And the value gate rejects **11 of those 12**. Those it catches
are the ones the earlier capture already showed: `passenger_id = "None"`,
`flight_number = "H0MVIE"` (a reservation id declared as a flight),
`flight_date = "2024-05-07T13:04:42"` (a timestamp, not a date), `passenger_id =
"Yusuf Jackson"` (a name where an id belongs).

**The twelfth is the one worth reading twice.** `cabin_class = "basic economy"`
where the record says `economy`. It satisfies its contract, passes the gate, and
is still wrong — the exact failure this project has said all along a contract
cannot catch, now with a row in a table rather than an assurance. A contract
checks shape, never truth.

#### And the twelfth is not fixable, which was worth measuring

The obvious repair is a **vocabulary gate**: collect the values the outcomes
actually take, reject anything outside them. So the booking records were
counted, and they admit three cabins:

```
basic_economy    business    economy
```

`basic economy` is **not** outside that vocabulary. It is `basic_economy`
written the way a person writes it. A vocabulary gate would accept it, catch
nothing, and look like an improvement in a diff.

And it *should* accept it. The model did not invent a cabin. It attributed a
**real** cabin to the **wrong booking** — `basic economy` is admissible, and
attributing it to a reservation that is `economy` is the error. Catching that
requires a check on meaning, and this project has no oracle for meaning in this
domain.

So the write path is 12 wrong in 82 decidable declarations, and the twelfth is
unfixable by construction. That is a better thing to know than a gate that
looks better on a table, and it is asserted in
`test_a_value_gate_cannot_catch_the_one_declaration_it_missed` so nobody
re-attempts it.

The read path's 99.6% is not resting on the same gap: `_normalise_cabin`
already collapses the two spellings, so the enum-versus-prose comparison is
sound. **This was checked rather than assumed**, because a headline figure
sitting next to a known normalisation gap is exactly the number that should not
be trusted until someone looks.

Two of the numbers above are only trustworthy because of bugs found while
producing them:

- The record was looked up by **corpus** name (`airline`) while the marker is
  keyed by **schema** (`travel`), so every airline record resolved to nothing and
  109 declarations scored 0 corroborated. That reads like a finding about the
  write path and was a lookup that found nothing.
- `2024-05-20` and `May 20, 2024` are the same flight date. Scoring them
  differently called a *true* declaration a contradiction three times in one
  conversation.

**What this does not establish.** 25 declarations sit in slots with no record
field at all — `total_baggages`, `cancellation_possible` — and nothing here can
settle them either way. And n=82 across 6 transcripts on one model is a small
base. The write path is *measured* now, which it was not, and it is still not
*proven*: one well-formed false value in twelve is the number to remember.

#### The same check on the third shipped schema, and what it cannot settle

`python -m benchmarks.ground_truth logistics --corpus apigen-retail --limit 1000`
runs the same scoring over the retail half of APIGen-MT. The records are shaped
differently enough that reusing the airline mapping would have been a lie about
them, so both sides are reduced instead: `payment_method_id` is
`credit_card_5843230` / `gift_card_8633125` / `paypal_5334408` while the read path
says "Credit Card" / "Gift Card" / "Paypal" / "paypal account" / "gift card
balance", and an address is tested on the house number and street rather than the
whole line.

| | count | share |
|---|---|---|
| corroborated by the record | 560 | **97.4%** |
| **contradicted by the record** | **15** | **2.6%** |
| unverifiable | 17 | 3.0% |

Three things that number needs beside it.

**A record has to be scored at the right point in the conversation.** This is
where most of the error was, and it was in the *measurement*, not the read path.
`payment_history` says what was already charged and `address` says where the order
is now, so a `delivery_address` slot — whose whole purpose is a *requested*
change — looked like it was being judged against a record that was behind by
definition. But the record is not behind. The agent applies the change, and the
**next** tool result shows the new state:

    apigen-1751 turn 13  "760 Elm Avenue"   record before: 592 Elm     after: 760 Elm
    apigen-1817 turn 11  "123 Oak Street"   record before: 463 Main     after: 123 Oak
    apigen-1865 turn 12  "828 River Road"  record before: 388 Spruce   after: 828 River

So a value is checked against the state as of **just before** the turn, and against
every state recorded **at or after** it. That took retail contradictions from **70
to 19** — and all three cases above were in the 70, and all three had been read by
hand before the data was blamed on the read path.

**And a record that never saw the outcome is silence, not disagreement.** If nothing
is recorded at or after the turn, the order had no chance to reflect the change, so
a value matching nothing is *unverifiable*. A customer who says "charge the
difference to my credit card" and then ends the conversation leaves no trace, and
calling the read path wrong for agreeing with them measures nothing. That split
took the 19 down to 15 — and it applies to the airline corpus too, where it moved
contradictions from 5 to **2**.

**A partial address is not a contradiction.** "464 Oak Street, Suite 664" is
correct, and the record also holds "San Diego, CA 92135". 25 of the 51
corroborated addresses stopped at the street line; they count as corroborated and
the incompleteness is reported beside the rate.

The two travel contradictions that remain are the irreducible ones: a cabin class
taken off a baggage-allowance table, and one turn where the agent says Economy
while the record says `basic_economy`. The second is not a pattern bug at all — the
read path faithfully recorded what the agent said.

#### A ground truth that is not a person: the booking database

Every airline tool result in the corpus is the environment's own record of a
reservation — `{"reservation_id": "0U4NPP", "origin": "PHL", "cabin": "economy",
"flights": [{"flight_number": "HAT076", "date": "2024-05-09", ...}]}`. That is
ground truth for exactly the slots the travel schema tracks, and it is already in
the conversation.

It is worth being precise about why using it is legitimate when the read path
refuses to. The library must not *infer* state from a machine payload — a booking
record is a database dump, not a statement of what the customer wants. Scoring
against it is the opposite operation: the payload is the answer key, held aside
from the system under test. Refusing to read the key and refusing to score
against the key would be one and the same mistake.

`python -m benchmarks.ground_truth travel --limit 200` puts an extraction in one
of three buckets, and reports them apart:

| | count | share |
|---|---|---|
| corroborated by the record | 557 | **99.6%** |
| **contradicted by the record** | **2** | **0.4%** |
| unverifiable (no record for that slot) | 56 | 9.1% |

The rate is over corroborated + contradicted. The unverifiable column is **not** a
pass — it is extractions this check says nothing about, and folding it in would
flatter the number.

**This is a different measurement from the hand-labelled one, and it is not
interchangeable with it.** It asks "is this value consistent with what the
environment recorded", not "did a person read the turn and judge it true". It is
stricter in one way, because the record is authoritative rather than a reader's
impression, and looser in another, because it cannot see whether the customer had
*moved on* by that turn. It scores a real but out-of-date value as correct — the
same limitation that sank the staleness attempt.

It also found two defects that 39 single-slot hand labels had not:

- **`abella_anderson_9682`.** The `passenger_id` pattern's optional verb
  `(?:is|:)?` matched the leading `is` of the *name*, so the read path produced a
  corrupted user id that passed every key-level check. 15 occurrences in the
  corpus, and the mechanical worksheet suggestion read `correct` on all of them,
  because the string is in the turn. It now needs a word boundary after the verb.
- **`cabin_class = "first"`.** "rescheduling my return flight to **the first**
  one-stop option" matched `to\s+(?:the\s+)?(first)` and recorded a First Class
  cabin. The database holds only `economy`, `business` and `basic_economy` across
  3,030 records and never `first`. This is row 051 of the worksheet, where the
  suggestion again said `correct`.

**What is left, honestly.** 4 of the 5 remaining contradictions are
`cabin_class`, and they are the residue of three fixes that were **measured
rather than reasoned about**. Two were tried and reverted because they were worse:

| attempt | effect |
|---|---|
| require the mention near "your" / "you are flying" | lost 50 of 122 corroborated to fix 4 of 6 |
| take the **last** mention in a turn as the conclusion | contradictions 6 → **10** |
| treat 2+ distinct cabins in a turn as an enumeration | lost 16 of 122 corroborated to fix 3 of 6 |
| **negative lookahead for policy framing** | contradictions 7 → **5**, coverage unchanged |

That last one is the keeper: "basic economy **flights cannot be modified**" is a
rule about the category, not the passenger's cabin. It removes 69 of 1,361
`basic economy` matches, and reading all 69 found every one is a policy statement
with no genuine mention among them — which is why it cost nothing in coverage
while the other three cost a lot.

The last-wins result is the instructive one. Taking the final mention as the
turn's conclusion is intuitively right — an agent lists categories, then states
the answer — and it made things worse, because agents state the current cabin and
*then* offer an upgrade: "you are in economy, would you like business?" First-wins
happens to suit this corpus, and the reason is a property of airline
conversations rather than a principle.

The 4 that remain are two turns where the correct value is also present and the
wrong one was picked, one hypothetical ("would two separate bookings work, one in
economy and one in business?"), and **one genuine agent-versus-record
disagreement** where the agent says Economy, the record says `basic_economy`, and
the read path faithfully recorded what the agent said. That last one is not a
pattern bug at all, and it is the most interesting row in the table.

#### The travel sample, labelled across all seven slots

The worksheet was filled in — 80 rows, 77 `correct`, 3 `incorrect`, every note
written — and merged into `benchmarks/labels/travel.json`. Precision is now
**99% (n=149, 95% CI 95–100%)** and, for the first time, it is a statement about
all seven travel slots rather than one. The three `incorrect` are informative:

| row | extraction | why it was marked incorrect |
|---|---|---|
| 002 | `cabin_class = 'economy'` | a hypothetical — "if I book a flight in economy class…" |
| 051 | `cabin_class = 'first'` | the ordinal in "the first one-stop option" |
| 068 | `passenger_id = 'abella_lopez_2185'` | the `is`-prefix bug, a truncated id |

Rows 051 and 068 are the two bugs the booking-database check found, judged
independently by a person reading the turns. Row 002 is the more interesting one:
**the booking record agreed with it.** `cabin_class` was extracted as `economy`
and the record held `economy`, so the record-based check scored it corroborated
and never flagged it. A human reading the turn saw a conditional that asserts
nothing. That is the clearest available statement of what the record cannot do —
it confirms a value was real, and has no opinion on whether the sentence meant
it.

The three `incorrect` verdicts are recorded on extractions that **no longer
reproduce**, because both were bugs that have since been fixed. They are kept as
regression records: if either reappears, the label says it was wrong before.

#### A memory cap was quietly moving the precision number

The 99% above was initially reported as **98% (n=131)**. The cause was not the
library. `harness.run` capped retained extractions at 400 as a memory guard for a
200 MB corpus; the airline corpus yields 615 active facts, so **21 of 152 labels
had no extraction to match** and the denominator fell from 149 to 131. Because
transcripts are compiled in order, the labels lost were not a random sample —
every transcript past the cap contributed nothing scoreable at all.

The cap is now counted (`extractions_dropped`), exposed as `--max-extractions`,
and when it bites the report says the figure is over a truncated sample and is
an artefact of the harness. The default was raised so it does not bite at this
corpus's size. It is still a cap: pass `--max-extractions` deliberately and the
notice appears.

#### Making labelling cheap enough to stop being the bottleneck

`python -m benchmarks.label_worksheet travel --limit 200 --per-entity 12` writes
a worksheet a human can fill in, and
`python -m benchmarks.label_worksheet travel --merge <file>` folds the judged
rows into the committed label file. Three things it does that hand-picking does
not:

- **It spreads the budget across slots.** Sampling "the extractions" reproduces
  the existing shape, because `current_file` is the easiest slot to sample and
  the others are not. The worksheet samples round-robin, so 12 per slot is 12
  per slot — 80 rows across all seven travel slots, of which 60 are for the five
  slots that have **no labels at all**.
- **It quotes the registering turn and its neighbours.** Both "confirmed errors"
  this project once carried were mislabelled by someone reading a value without
  the sentence that said the agent had moved on. The quote is the fix, and the
  merge refuses a row that does not carry one.
- **It will not invent a verdict.** Every row ships `unlabelled`, and the merge
  exits while any row is unlabelled, so a precision number can never acquire a
  denominator nobody judged. Rows are de-duplicated by value and by transcript
  first, because inflating `n` with repeats was this project's first precision
  bug.

`--suggest` additionally attaches a **mechanical suggestion** to each row — a
text check, with its evidence recorded (is the value in the registering turn, what
role is that turn, did the sanitizer call it machine output). It is not a
judgement and it cannot become one: `verdict` stays `unlabelled`, the merge exits
while anything is unlabelled and says so explicitly, and a suggestion that was on
offer is recorded *as offered* on the committed label so a later reader can see the
reviewer was not working blind. Its own limitation is stated on every row: the
"confirmed errors" this project once carried were all cases where the value *was*
in the turn and the judgement was still wrong, because the agent had already moved
on. A string check cannot see that.

The committed label file is the only thing that produces a number. The worksheet
carries none until a person fills it in.

#### One precision question the labels structurally cannot answer

All 39 labels are `current_file`, and every one judges a value at the turn it was
*registered*. That design cannot see the failure a fact tracker is actually
judged on: whether it followed the agent afterwards. If the tracker says
`current_file = memset.py` on turn 51 and the agent is editing `cli.py` on turn
52, the value was true when written and wrong by the next turn — and every hand
label on turn 51 still reads "correct".

So there is a second, mechanical precision question, and it was attempted and
**does not survive contact with the corpus**. `python -m benchmarks.staleness`
keeps the attempt and the reason:

- The action signal has to be an **edit**, because reading a file is not editing
  it. Taking any file named against a command gives 28.6% support, and reports a
  44.9% disagreement rate. Reading the turns shows most of it is the agent
  opening a file to diagnose an import error, or running a script to verify a
  fix, while the tracker correctly holds the file it is working on. `open
  lexicon/config.py` is a read; `python reproduce.py` is a run.
- An **edit-only** signal is correct and rare: explicit edit markers appear in
  1.7% of agent turns and in-place `sed -i` writes in 0.0%. On 60 transcripts
  that leaves 69 comparable transitions — and **25 of the 31 disagreements come
  from a single transcript**, an agent ping-ponging between `api.py` and
  `common_types.py`. Drop it and the rate falls from 44.9% to 14.3%.

**So no staleness figure is quoted anywhere.** A rate that one transcript moves
by 30 points is that transcript's behaviour, not the tracker's, and the tool now
reports the concentration instead of the average rather than letting a
convenient number through. The honest position is that this dimension is
currently unmeasured: it needs either a corpus where agents announce their edits
in a parseable way, or a human reading pairs of turns. The read path's refusal
to infer from tool output is a large part of why — it holds no value on the
`user` turn preceding an agent action, so the comparison has to be between
consecutive agent actions, which is a definition chosen here and not given by
the data.

#### The one staleness test that needs no edit signal

The reason above is that a rate needs a ground truth for *when the file changed*.
But there is a weaker question with a free answer, and it is the one that
actually matters: **does the read path's value match the agent's most recent file
statement?** If it does, the tracker is not stale, whatever the agent was doing
in between. The vendored coding transcript answers it outright — 18 distinct
file statements, in order:

```
memset.py -> reproduce.py -> memset.py -> lexicon/lexicon/cli.py
```

The read path reproduces that sequence exactly, including the round trip: the
agent creates `reproduce.py`, works in it, then goes back to `cli.py`, and the
read path follows it out of `reproduce.py` and back. Its final value equals the
agent's final statement, so **the read path is not stale, and it is a faithful
last-statement tracker.** 81% of its matches come from the agent's speech and
19% from tool output; it takes the final one either way.

Two things this does *not* license. It does not produce a staleness **rate**,
because a rate needs the edit signal the corpus lacks. And it does not validate
the confounded 44.9%: one trajectory agreeing with itself is not a measurement.

The turn-by-turn version **is** available in the 14B capture, which records
`declared` on every turn — and it is still not measurable, for a reason worth
writing down. The capture has 6 transcripts; the vendored agent transcript is
**one** file. Aligning each transcript's model declarations to a matching agent
prefix requires slicing that one file six ways, which invents six trajectories
out of one. I did it, got 8 apparent disagreements, and discarded all 8: they
were artifacts of the slicing, and reporting them would have been the exact
failure this section is about. **A measurement that needs data the corpus does
not have is not a weak measurement, it is a fabricated one.**

One oddity to leave alone on purpose. Half the values are doubled
(`lexicon/lexicon/cli.py`, where the first `lexicon` is the conda prefix and the
second is the repo). That is the **agent's own text**, faithfully captured, and
normalising it would be fitting the tool to a transcript's confusion. Recorded
here so it is not later "fixed".

**Where this leaves the dimension.** Not unmeasured any more — measured, once,
and negative: the tracker does not go stale on the trajectory available. Still
no rate, and still n=1 trajectory. That is a better answer than the one it
replaces, and it is still not a headline.

#### The rate that can be quoted, and why the earlier ones were not

The reason there was no rate is that the read path's value could not be lined up
with the model's declaration **at the same turn**. It can now, and the reason is
not a cleverer scorer — it is that the read path's value is recorded *per turn
during the capture*, from that trajectory's own prefix:

```python
"read_path": _read_path_state(schema, history)
```

Previously it had to be reconstructed afterwards, which means slicing whatever
text is nearby. The old 14B capture has **6 transcripts and one vendored agent
file**, so any re-derivation has to invent six trajectories out of one. Slicing
it produced 8 apparent disagreements, all artifacts — which is why they were
discarded rather than reported.

`benchmarks/capture.py` now records the read path alongside every declaration,
over three **genuinely distinct** real trajectories:

```
python -m benchmarks.staleness --capture captures/coding-14b-distinct.json
```

| | turns |
|---|---|
| model declared `current_file` | 33 |
| exact agreement | 10 |
| same file, model less precise (`memset.py` vs `lexicon/providers/memset.py`) | 20 |
| **genuinely different file** | **3** |

**9.1% disagreement over 3 trajectories.** It qualifies where the 44.9% did not:
no action signal, so not confounded by opening a file being a read; no rebuilt
alignment; and the contributing trajectories are counted, so a rate one agent
moves is visible as one. **The worst trajectory still holds 67% of them** — 2 of
3, down from 25 of 31 — so n=3 is three trajectories' worth of evidence, not
thirty turns'.

**Agreement is not correctness.** Both sides are inference, so this counts
*disagreements*, which is the staleness signal. It does not adjudicate them:
coding has no external record saying which file the agent was in, so a
disagreement is counted, not resolved. This is a **disagreement rate**, not a
staleness rate — staleness still needs to know when the file changed.

**The two "confirmed errors" this project carried were mislabelled.** Both were
recorded as `current_file` extractions judged incorrect because the turn's earlier
prose discussed a different file. Re-reading the turns against the doctrine the
labels themselves state — *"agent opens/edits/creates or restates this exact
value"* — shows the opposite. At each turn the agent says it is navigating to
`dispatcher.py`, emits

```
open azure_functions_worker/dispatcher.py
```

and **the very next turn is the tool result confirming the open happened**:
`[File: .../dispatcher.py (717 lines total)]`. By the doctrine, both extractions
are correct. The label judged the turn by the file its prose discussed; the
doctrine asks what the agent *did*, and the next turn answers that.

This mattered beyond tidying, because it is also the answer to why a proposed fix
had to be abandoned. Stripping fenced blocks before pattern matching — treating a
shown command as a quotation rather than an action — cost **48% of the measured
supersession signal** (77 → 40 re-assertions). In this corpus a fenced command
in an assistant turn *is* the action, not a hypothetical, and the measurement was
right to stop that change.

`benchmarks/labels/known_failures.json` is now empty, with the correction
recorded beside it. Neither file is the right home for the two cases: they are
not errors, and they sit in rows `--per-repo 2` skips, so adding them to
`precision.json` would trip the label-drift check — the same check that caught 20
of 24 labels matching nothing when it was written. The file and its tests stay,
so the first real error is checkable from the moment it is recorded.

### What the corpus says the problem actually is

The measurable, high-frequency mutable entity in real coding transcripts is
**which file the agent is working on** — across 77 re-assertions,
with the agent moving between them and correcting itself:

> *"It seems that I attempted to edit the wrong file again. I need to edit the
> `memset.py` file instead of the `reproduce.py` file."*

That is precisely the last-write-wins case the library exists for, and 77 key
re-assertions fired across the 40 transcripts. `contextgc/schemas/coding.json`
is derived from that observation, not from what would have been convenient.

The logistics scenario in the demo is not representative of coding work. This is
a domain-specific mechanism, and the corpus says which domain it actually
applies to.

### Retail: the original sin, measured at scale

`contextgc/schemas/logistics.json` used to be the library's *default*. On 80 real
retail conversations it extracted 46 facts, and **every one read in context was
wrong**:

```
destination_address = "perfectly suit my needs"
destination_address = "for any price difference"
destination_address = "my existing PayPal account"
destination_address = "the Visa card ending in 2364"
```

The `use` trigger is the cause: in a retail conversation "use" is nearly always
about money. The original evidence for emptying the default schema was three
examples on 60 *coding* transcripts; this is the same defect at scale, on data
that is actually the domain the schema was written for. The patterns are kept at
`benchmarks/schemas/condemned/logistics-original.json`, and a nightly job asserts
they still fail, so a future reader can reproduce the claim rather than take it
on trust.

The replacement was derived by measuring 400 retail conversations, and **the
result is mostly negative**:

| entity | spoken | machine output | machine share | supersedes |
|---|---|---|---|---|
| `delivery_address` | 51 | 1,048 | 95% | 15% (26 convs) |
| `payment_method` | 141 | 3,701 | 86% | 1% |
| `order_id` | 625 | 2,340 | 79% | 5% |
| `refund_amount` | 2,280 | — | — | 93% |

`refund_amount` is deliberately **absent** despite being the most-mentioned
quantity in the data. It "changes" in 93% of conversations because a retail
conversation contains many different dollar amounts — item prices, totals, price
differences — not because one is being corrected. Putting them in a single
last-write-wins slot would be the same error as modelling origin and destination
as one airport code, and the same error as the schema it replaces. A slot that
changes constantly because it means several things at once tracks nothing.

So retail ships two slots, and the supersession rate stays in the schema file
where it cannot be quietly forgotten.

### What the compactor actually covers

"It compacted 668 payloads" says nothing about *which* payloads. Measuring every
shape in the three corpora:

| payload shape | n | share of JSON bytes | saved | compacted |
|---|---|---|---|---|
| dict wrapping records under any key | 370 | **84%** | 56% | yes |
| a tool **call** | 507 | 14% | 0% | no, by design |
| a lookup map of strings | 8 | **2.5%** | 0% | no — see below |
| text listings, test output, stack traces | — | — | 36–93% | yes |

Counted over 60 airline + 80 retail conversations. The coding corpus contains
**no JSON payloads at all** (2,976 candidates, none parse), so its compaction
figure comes entirely from the non-JSON shapes above.

**The uncovered shapes are 2.5% of the bytes, and that is deliberate.** A
lookup map of strings — a product catalogue, a currency table — has no repeated
fields, so there is genuinely nothing to summarise; the only way to shrink one is
to drop entries, and a model that needed the catalogue would then be reasoning
from a truncated one. Trading 2.5% of JSON payload bytes, about 0.4% of tokens,
for silently discarding lookup data is the wrong side of this project's own
rule. Earlier revisions of this table listed "dict wrapping a map of strings" as
an uncovered *shape*; measuring it showed the 507 instances of that shape in the
corpora are all tool calls, which are deliberately untouched, and the genuine
case is 8 payloads.

Two things that table changed.

**The dict branch guessed key names.** It looked for a list under one of eight
hardcoded keys — `items`, `products`, `records`, `data` and so on — and fell
through to a flat distillation for anything else. The airline payloads wrap their
flights under `flights` and their search results under `results`; neither is on
the list. So **18% of airline and 27% of retail payloads went uncompressed for no
reason other than the spelling of a key.** It now finds the largest collection
under any key. Guessing key names is a list of tomorrow's bugs.

**A tool call is not tool output.** A `function_call` turn is the agent stating
what it did, and compressing its `arguments` would obscure the action rather than
the output it describes. That is exempt however large it is.

Corrected on the corpora: airline +10.3% and retail +18.9% of payload characters
saved, coding unchanged because its payloads are text and the JSON branch
correctly never fires.

### What the second domain settled about the write path

A capture now exists (see below), so the write path is no longer unmeasured. But
the second domain answers the question that motivated it, and the answer is not
subtle. In customer-service transcripts, **82–99% of every mutable entity's
mentions live in tool output**, not in speech:

| entity | in speech | in machine output | machine share |
|---|---|---|---|
| `reservation_id` | 610 | 2,922 | **83%** |
| `airport` | 1,060 | 4,768 | **82%** |
| `payment_method` | 18 | 1,395 | **99%** |
| `cabin_class` | 271 | 96 | 26% |

The read path is forbidden to infer state from machine output — a grep listing is
not a statement about intent, and that rule was measured on the coding corpus. So
a pattern-based tracker structurally *cannot* see reservation ids or payment
methods here. That is a property of tool-heavy domains, not a defect in the
tracker, and it remains the strongest argument for the write path that this project
has produced: the agent knows those values, and only the agent can say them.

The caveat is now a measurement rather than a hope. A model that *can* state those
values still has to state them correctly: the coding capture below shows a model
filling 19 slots with invented keys, none of them schema fields. An agent that
declares a `payment_method` wrongly is worse than an agent that declares nothing,
because the compiler will treat it as authoritative. Making declarations earn their
place in the schema is a prerequisite for the travel case, not a detail.

What is left reachable is cabin class, which is spoken often and changes in 41% of
conversations.

#### What the corpus could actually reach, and what it cost

The first capture on this domain was refused by `benchmarks capture --verify`: only
**7 of 19** declared keys (37%) named a slot the two-slot travel schema defined.
Not because the model was wrong — an airline conversation is about
`flight_number`, `origin`, `destination`, `flight_date` and `passenger_id`, and
the schema had no pattern for any of them, so the read path could not corroborate
what the model said. The fix is to derive those slots from the corpus, **not**
from the model's output, which would be fitting the schema to the metric.

Five slots were added, each fitted to the APIGen airline corpus and each carrying
its own observation count. Two precision defects surfaced in the process, both
found by measuring rather than by reading:

- An airport pattern written as `to ([A-Z]{3})` is compiled case-insensitively, so
  **"to the" was read as the airport `the`**, and the slot produced 52 distinct
  values where the corpus has 20 IATA codes. This is the same defect the schema
  already documents for reservation ids, so the fix is the same: scope the class
  case-sensitively with `(?-i:...)`.
- `passenger_id` written as a bare `name_name_1234` token fired **543 times
  inside tool output**, contradicting this schema's own rule that a pattern must
  require a phrasing and never a bare token. It survived only because the
  sanitizer happened to catch those messages.

A third defect was pre-existing and is described next, because it is the more
interesting one: an agent's *options menu* was becoming state.

#### An options menu is a listing, and a listing is not a booking

The shipped `cabin_class` pattern has `(?<!Seats: )` and `(?<!Prices: )`
lookbehinds, added when a turn listed every cabin with its price and the pattern
took `Basic Economy` off the price list as the passenger's cabin. Those guards
cover one layout. The airline corpus also writes `Price: $142` and
`**Flight 1:**`, and across 134 menu-shaped agent turns the shipped schema leaked
**905 extractions**; the new slots would have added about 1,470 more.

The obvious fix — a content-based "this is a listing" guard — was measured and
rejected. A content-only guard **deletes the single most important turn in the
corpus**: 2 customer turns contain option-menu markers and both are a booking,
including "Let's go with Option 1: Flight HAT148 from Miami to Denver". The same
strings in an agent turn mean it is listing what it could offer, and nothing in
the content tells the two apart.

So the guard is conditioned on the role, in `ToolSanitizer.looks_like_option_menu`:
only an *assistant* turn can be an agent listing its own options, so a customer
turn is never claimed. Measured on the airline corpus that suppresses 440 of 6,538
agent turns (6.7%) and **0** customer turns. An options menu is a listing in
exactly the sense a search listing is — "flights from DFW to SEA are available"
does not mean the passenger is flying DFW to SEA — and what the menu *does* support,
that these routes exist, is not what this library tracks.

#### A birthdate read as a flight date, found by a disagreement

The shadow replay's single disagreement on the airline capture was the read path
producing `August 10, 1995` against a declared `2024-05-07T13:04:42`. The read
path was right about the disagreement existing and wrong about the value: it had
matched a passenger's *date of birth*. No aggregate had shown this. A bare
month-name date pattern matched 1,494 dates of which 93 (6.2%) were birthdates.

Requiring flight context, plus a lookbehind for "born", took that to 11 hits whose
matched date was a genuine flight date with a birthdate nearby — and dropped the
slot from 82 distinct values to 32, while keeping 1,636 observations. Filtering
on the year would have been easier and wrong: it fits the corpus's fixed "current
time" rather than the slot's meaning. After the fix the replay reports 0 changed
keys and 16 agreements.

The travel schema is now 7 slots, 18,800 read-path observations.

#### The second capture, on the airline corpus

With the corpus-derived slots in place, the same model on the same domain
(`qwen2.5:14b`, 6 transcripts, 36 turns) goes from **37%** of declared keys
naming a schema slot to **76%** — 80 of 105. Every turn declared, and the six
off-schema keys it invents are genuine airline entities the schema still does not
cover (`total_baggages`, `nonfree_baggages`, `passenger_check_in_status`).

Two numbers here are worse than on the coding corpus, and both are findings rather
than noise:

- **6 of 36 blocks do not parse as JSON**, so `capture --verify` still refuses
  this capture. On the coding corpus it was 0 of 42. Malformed output is worse in
  a domain with longer, more structured turns.
- **16 of 37 shadow additions are off-schema**, so nothing can say whether any of
  them is true. That fraction is the same problem as before, smaller.

### Measuring the write path, when there is a model

`benchmarks shadow` replays captured declarations. Producing a capture needs a real
model, and one will not be invented here — a synthetic declaration set would
measure the harness rather than the write path. There is one committed capture at
`captures/run1.json`, and these are the commands that produced the numbers below:

```bash
# any OpenAI-compatible endpoint; a local ollama works and needs no key
export CONTEXTGC_CAPTURE_BASE_URL=http://localhost:11434/v1
export CONTEXTGC_CAPTURE_MODEL=qwen2.5:7b
python -m benchmarks capture --transcript benchmarks/corpus/sample.txt \
    --out captures/run1.json --limit 6 --schema coding
python -m benchmarks capture --verify --out captures/run1.json
python -m benchmarks shadow --corpus synthetic \
    --corpus-path benchmarks/corpus/sample.txt \
    --captures captures/run1.json --schema contextgc/schemas/coding.json
```

Note `--corpus-path` on the shadow run. Capture and replay must come from the same
corpus under the same id scheme; pointing shadow at the downloaded `--corpus
swe-agent` shard instead would replay nothing while printing a clean comparison,
so it refuses instead.

The endpoint is read from the environment rather than an argument, so a key cannot
land in a shell history or a CI log. `capture --verify` refuses a capture that
recorded no declarations, one that does not say which endpoint produced it — a
capture that cannot be attributed is an anecdote — and one whose declarations name
a schema slot less than half the time.

Capture is resumable and checkpoints after every turn, because a local server that
drops on the fortieth of forty-two calls should cost one call. A capture records a
fingerprint of everything that shapes its prompt, so a re-run after a prompt change
starts fresh rather than quietly mixing two experiments into one file.

### Is the write path worth it? Measured, and the answer is no

There is now a capture, made by a real model against a real trajectory file, and it
is committed at `captures/run1.json` so the comparison replays without a GPU. Two
more sit beside it: the 7B run before the instruction fix, and the same run after
it, because that pair is the evidence for the bug below rather than for the write
path.

```bash
python -m benchmarks capture --verify --out captures/run1.json
python -m benchmarks shadow --corpus synthetic \
    --corpus-path benchmarks/corpus/sample.txt \
    --captures captures/run1.json --schema contextgc/schemas/coding.json
```

| | |
|---|---|
| model | `qwen2.5:14b` (Q4_K_M), local, via ollama |
| transcripts | 6 real SWE-agent trajectories, 42 assistant turns |
| turns where the model emitted a state block | 42 / 42 (**100%**) |
| blocks that were not valid JSON | **0** |
| declared keys naming a schema slot | 53 / 55 (**96%**) |
| **keys the declaration added** | **9** |
| — of those, in the schema | **7** |
| — of those, outside it | 2 |
| **keys the declaration changed** | **4** |
| keys the declaration agreed with | 1 |

The first run of this used `qwen2.5:7b` and produced 34% in-schema, 19 off-schema
additions and no verified information at all. That number was a bug in the
instruction, not a property of the model — see below. Both captures are kept.

**Every one of the 19 added keys was outside the schema.** Not one was a
`current_file` or a `failing_test`. The 4 agreements were all `current_file`, and
the declaration never once corrected the read path.

That `19` used to be the headline, and it was the wrong number to lead with. It
counted `key`, `value`, `auth_token`, `line_144` and `headers_set` as recovered
facts. Shadow mode now splits it, and the split is the finding:

```
  keys added      19
      in schema    0   (the read path had patterns for these and still missed them)
      off schema   19   (nothing could corroborate these)
```

The write path's real contribution, at this model size, was confirming four facts
the read path had already found, and inventing nineteen keys nobody asked for —
including `key` and `value`, which are the literal placeholders from the format
example in the instruction, copied through as if they were entity names.

The read path found `current_file` in 5 of the 6 transcripts by pattern. The model
declared it 12 times, and the two agreed 4 times. In no transcript did the
declaration supply a `current_file` the read path had missed. **Net verified
information from the write path: zero.**

#### What this does and does not say

This is a measurement of *a 7B model's ability to use the protocol*, and it is
separable from the protocol's design in one direction only. Two things bound it:

- **It is a lower bound.** In the capture the model is a bystander inferring what
  an *observed* assistant turn changed. In real use the agent emits the block
  inline, with direct knowledge of its own actions. That is an easier task, and
  this number does not predict it.
- **It is one model.** A frontier model would very likely behave differently. The
  claim is not "declarations do not work"; it is "declarations from a 7B model do
  not survive contact with a real trajectory."

What the result established, without any model-size caveat, is that the ingestion
path accepted out-of-schema keys without complaint. The 19 junk keys were not
filtered, not flagged and not counted as a problem — they were `added`, which is
the metric that flatters the write path.

#### A second model, and a bug the first one was blaming

`qwen2.5:14b` was captured on the same six trajectories, and the first
comparison said the 34% in-schema rate was not a model-size artifact — 14b
scored 40%. That reading was wrong, and finding out why mattered more than the
number.

`teach_protocol` built the protocol instruction from `sorted(self.dag.active_state)`
— the keys **already in state**. On turn 1 nothing is in state, so a model
running the `coding` schema was told that schema had **no keys at all**, and fell
back to the format example's placeholders. Both models had been copying
`{"key":"value"}` out of the instruction as if `key` and `value` were entity
names. The schema is the definition of which keys exist; telling the model about
it is not hinting it toward an answer, it is the one thing it cannot infer.

Fixed, and re-measured on the same trajectories:

| | 7b before | 7b after | 14b after |
|---|---|---|---|
| compliance | 90% | 93% | **100%** |
| malformed blocks | 7 | 4 | **0** |
| declared keys | 29 | 39 | 55 |
| keys naming a schema slot | 10 | 34 | **53** |
| **in-schema share** | 34% | **87%** | **96%** |

So the vocabulary bug was the dominant factor and the model size a real but
smaller increment on top of it.

#### Why the ratio is not the answer

96% in-schema looks like a working write path. Adjudicating what it actually
produced says otherwise.

| 14b, shadow replay | count |
|---|---|
| keys added, in schema | 7 |
| keys changed (disagreed with the read path) | 4 |
| keys agreed | 1 |
| keys added, off schema | 2 |

**Every one of the 7 in-schema additions is a value of the wrong shape.** Three
are the literal string `None`. The rest are error messages and a method name
placed in `failing_test`, whose patterns are supposed to yield a test
identifier: `HTTPError: 403 Forbidden`, `SyntaxError: '(' was never closed`,
`mocking_memset_api_response`. The model stopped inventing *keys* and started
inventing *values shaped to look like keys*.

And one of the 4 changes is a genuine regression. In transcript 5 the agent
worked through `memset.py` and had moved on to `cli.py`; the read path's final
`current_file` is `lexicon/cli.py`, and the declaration moved it **back** to
`lexicon/lexicon/providers/memset.py`. Sending the model a file the agent has
already left is precisely the failure this project exists to prevent. The other
three are benign — the read path says `memset.py`, the declaration says the full
path, and both name the same file.

**Net verified information from the write path: still zero.** For a different,
and more insidious, reason than before. The first failure mode was caught by
asking whether the key was in the schema. The second passes that check, which is
why an in-schema ratio is necessary and not sufficient.

#### The gate that follows from it: value shape

A key check is necessary and not sufficient, and the second capture proved it by
sitting exactly on the boundary: 96% of declared keys named a real slot, and the
values were still wrong. So a schema may now state what a *declared* value for
each slot may look like.

The first contracts were written by hand from the values one capture happened to
produce, and that was a guess wearing a measurement's clothes. They are now
**derived from the schema's own entity patterns**, by
`contextgc/value_shapes.py`. The patterns were fitted to a corpus, so their
alternations *are* that corpus's vocabulary; reusing them substitutes a
measurement for a guess:

```json
"values": {
  "current_file": "(?:[\\w][\\w./-]*\\.(?:py|js|ts|tsx|jsx|go|rs|java|rb|cpp|hpp))",
  "failing_test": "(?:[\\w./-]*test[\\w./-]*\\.py)"
}
```

The second line is worth pausing on, because it is a contract that accepts
`latest_file.py` — `la`+`test` — and `my_test_helper.py`. The `test_` anchoring
this slot used to have is gone, because the pattern that reaches it from speech
no longer requires a `test_` prefix. Both junk cases are recorded as residual
false accepts. The trade is deliberate: a contract that rejects 2 of 49 real
values and is unreachable buys less than one that accepts everything the read
path can reach and admits its 2 false accepts in the open.

Deriving them mattered more than the hand-written versions looked. The old
`logistics.delivery_address` was `[\w\s.,'/-]+` and `payment_method` was
`[\w ]+`, so on a 48-value set of wrong-shaped declarations — `None`, error
messages, `memset.py`, a bare `1234567` — the hand-written contracts accepted
**34 of 48** and the derived ones accept **0 of 48**. Every one of the **7,929**
distinct values the read path has actually produced across the three shipped
schemas is still accepted, so the tightening cost nothing in coverage.

Two bugs in the derivation were found only by running it, not by reading it, and
both are now pinned by tests in `tests/test_value_shapes.py`:

- The first version read `basic economy` — 14 of 42 observations, the most
  common value in `cabin_class` — as a structural pattern and dropped it. A
  contract that rejects the commonest value in its own slot is worse than none.
- The first version anchored `test_` to a separator and rejected 2 of 126 real
  values, because `base_test_test.py` and `tests.test_flask_pyoidc` put an
  underscore or a dot in front of `test_`. The boundary is "not preceded by an
  alphanumeric".

One false accept is left and recorded rather than fitted away: `failing_test`
still accepts `my_test_helper.py`, since a helper module's name contains `test_`
and telling it from a test would need more than the pattern carries. It is
written into the schema's `_measurement` so the limit is visible.

`value_policy` then behaves like `declaration_policy`: `flag` keeps the value and
reports it, `reject` drops it, `off` disables the check. A slot with no contract
is not gated — absence is not permission to guess.

Contracts are stored in the schema file, where a reviewer can read them, and a
test fails if an entity pattern is edited without re-deriving, so the two cannot
drift apart.

Two further properties were forced by measurement rather than chosen:

- **Contracts are never applied to the read path's own values**, which come out
  of the slot's pattern by construction, so gating them would be circular. A test
  runs every contract over the corpora's extractions and fails the build on any
  wrong rejection. It earned its place immediately: a case-sensitive contract
  rejected **3 of 111** travel extractions the tracker produced itself, including
  `cabin_class = "Business"`.
- **The match is a search, not a full match, and is case-insensitive.** A strict
  match rejects a leading `/` on an absolute path and pytest's `::` separator.

Measured on the canonical 14B capture: **47** declared string values, **45** of
them in slots that have a contract. **16** are of the wrong shape and are
reported; 29 pass. All 16 are `failing_test` values — the 14B model never once
produced a plausible test identifier in this capture, only error text and shell
commands. The remaining 2 declarations are in `auth_token` and `credentials`,
which are not schema slots and so are not gated.

**This does not make the write path proven.** It removes a failure mode that was
previously invisible, and the same standard applies to it as to the key gate: a
value that satisfies its contract can still be wrong. A future model could
satisfy the contract with a *plausible but wrong* test identifier, and nothing
here would catch that. Treat the write path as unproven rather than as working
until truth is checked against labels.

#### The prompt was wrong before it was measured

The first capture of this run scored 55% compliance. That was not the model. The
harness was handing the model the protocol twice — once as a hand-built system
message and once via `teach_protocol=True` — 801 bytes of duplication per turn.
Four of the nineteen non-compliant replies were the model faithfully echoing the
duplicated `[STATE_PROTOCOL]` wrapper back.

`--legacy-prompt` reproduces that defect on demand, so the comparison below is
measured rather than remembered. Both columns are the same model, the same six
trajectories, the same counting method:

| | `--legacy-prompt` | current |
|---|---|---|
| turns emitting a block | 23 / 42 | **38 / 42** |
| compliance | 55% | **90%** |
| blocks that were not valid JSON | 4 / 23 (17%) | 7 / 38 (18%) |
| declared keys | 22 | 29 |
| keys naming a schema slot | 8 | 10 |
| **in-schema share** | **36%** | **34%** |

Compliance went up by 35 points. The share of declarations naming something the
tracker asked for did not move.

That is the finding. A harness reporting the compliance figure would have shown a
clean improvement; the in-schema share is flat, so the extra blocks are noise. And
the malformed rate is flat too, which corrects an earlier reading of this data: a
first pass compared one run at 36% malformed against one at 18% and called it an
improvement. Re-running the legacy prompt put it at 17%. The model is sampled, so
single-run rates carry real noise and the 36%→18% comparison was two draws, not a
trend.

Compliance is not correctness, and neither is a low malformed rate. The number
that predicts whether the write path is worth enabling is the in-schema share, and
`capture --verify` now refuses a capture that scores below 50% on it.

A capture made with `--legacy-prompt` is stamped `legacy_prompt: true` and `verify`
refuses it. It is a valid record of what the broken harness did and an invalid
measurement of the protocol, and it should not sit in `captures/` looking like one.

The committed capture's `schema` field was backfilled to `coding` after the fact,
and the backfill is verifiable rather than asserted: the capture's
`prompt_fingerprint` covers the schema, and only `coding` reproduces the recorded
digest `5c5215b167ec10d4`. A capture with no recorded schema reports its in-schema
ratio as unknown instead of guessing one.

### What this harness refuses to do

- Print a percentage without the `N` it came from.
- Print a precision figure unless one was measured against hand labels.
- Call a declaration "correct" — it cannot know truth, only disagreement.
- Score itself. A compiler grading its own compression is the mistake this
  project was rewritten to remove.
- Report a synthetic corpus as evidence; a vendored slice of real trajectories is
  labelled with its real source.

---

## What it actually does

Four things, in order:

**1. Tracks asserted entity state, last-write-wins.**
When a key is re-asserted, the earlier assertion is marked superseded. When a
turn has been superseded for *every* fact it touched, it is retired from the
context.

**2. Never retires the evidence.**
If a turn is superseded for one key but is still the only support for another,
it **stays in the context**. Retiring it would hand the model a value with
nothing behind it. `get_retirement_violations(proposed)` checks a *proposed*
retirement set and reports any live fact that would be left unsupported.

An earlier version of this check compared the prunable set against the live set
and could therefore only ever return `[]` — it asserted an identity, not a
property, and the test suite "proving" it could not fail. The current test
asserts that the check *can* return a violation, which is the only way to know it
means anything.

**3. Compacts tool output without losing the safety bits.**
Bulk rows are sampled down. Any row carrying an allergen, severity, PII, recall,
or expiry signal is always retained, and the number of dropped rows is reported
explicitly (`truncated=true`).

**4. Pins caller-declared invariants.**
Rules you declare are inlined into the compiled context. There are no built-in
domain rules — shipping a hardcoded "refunds over ₹150" default once meant the
tool silently imported one demo scenario's business rules into every unrelated
session.

### Two modes

| `mode` | Behaviour | Use when |
|---|---|---|
| `compact` *(default)* | Retires superseded turns, inlines state at the head. | Minimising tokens |
| `cache_friendly` | Emits the input prefix **byte-identical**, appends the state register at the tail. Tool payloads are passed through verbatim. | A prefix/KV cache is keyed on the conversation prefix |

`cache_friendly` does not compact tool payloads, because doing so would mutate
the very prefix the cache is keyed on. The reported
`kv_cache_prefix_messages_preserved` is measured by diffing the emitted output
against the input — not inferred from the mode.

---

## Honest scope

This is the part that matters more than the feature list.

**It does not make models reliable.** It removes contradictions from the input.
It cannot add judgement, and it cannot stop a model ignoring a rule you pinned.
Nothing at the prompt level can guarantee compliance.

**The problem it solves is one of three.** "Context rot" is really three separate
failures, and only one is deterministic:

| Failure | Addressable here? |
|---|---|
| **Contradiction** — a stale assertion and its replacement both in context | **Yes.** One key, two bindings, a total order picks the winner. No semantics required. |
| **Irrelevance** — bulk tokens competing for probability mass | No. Needs a relevance judgement, i.e. a model. |
| **Position** — lost-in-the-middle attention decay | No. It is a property of softmax attention over sequence length. Preprocessing cannot fix it. |

**State tracking is regular expressions on the read path.** There is no
coreference resolution, so a fact expressed only through pronouns will not be
tracked. Missed extraction is the main failure mode: the state is *silently
incomplete*. The write path can supply those facts when the agent cooperates —
`conflicts` and `rejected_writes` tell you when it did not. Extend the schema for
your domain:

```python
from contextgc import StateDAG

dag = StateDAG()
dag.register_entity_schema("order_id", [r"order (?:id |number )?([A-Z]{3}-\d+)"])
```

**A declaration the schema cannot account for is not verifiable, whichever policy
you pick.** `declaration_policy="reject"` drops it, which stops junk entering
state — and also means a correct fact under an unexpected name is dropped
silently from your point of view, visible only in `rejected_writes`. The default
`"flag"` keeps it and tells you. Neither makes it true.

And the stronger version of the same warning: even a declaration that *is* in the
schema may be unverifiable. On the 14B capture 96% of declared keys named a real
slot and every one of the resulting additions was a value of the wrong shape —
three literal `None`s, and error messages where a test identifier belongs. A key
check catches an invented key; nothing here yet catches an invented value. Treat
the write path as unproven rather than as working until value shape is checked
too.

**Token counts are `chars/4`, not a BPE tokenizer.** Read them as a ratio, not a
bill. Exact numbers need `tiktoken` against your real model.

**Retired-turn recall is lexical** — token, subword, and phrase overlap. Not
semantic. "What did we decide about billing?" will not find a turn that only
contains "invoice".

**The write-path numbers come from one 7B model on six trajectories.** They are
the project's only real measurement of the write path, and they are a lower bound
rather than a verdict on the design — see the section above for what bounds them.

---

## What was removed, and why

This project previously published numbers that its own code did not support. The
audit that prompted the rewrite is summarised here so the change is legible:

| Was | Reality |
|---|---|
| "40–47% faster inference" | A formula: `500 + raw*0.25` vs `400 + clean*0.15`. The coefficient ratio *was* the result. Nothing measured time-to-first-token. Removed. |
| "100% policy invariant compliance" | A regex auditor run against a sentence the server had just written itself, in a code path never used in production. Removed. |
| A benchmark "vanilla agent" | The baseline state was derived from this tool's own state tracker — the system computing its own counterfactual. Removed. |
| "768-dim vector recall, IVFFlat index" | SHA-256 hashes of words, plus a ClickHouse `DDL_SCHEMA` constant referenced nowhere in the codebase, behind a UI badge. The search was ~80% lexical. Renamed to `RetiredTurnArchive` and described accurately. |
| "sub-millisecond recall" | True only because the archive held six rows in a Python `for` loop. |
| Fabricated completions from `/v1/chat/completions` | With no API key the endpoint returned a valid-looking `chat.completion` with invented `usage` and no disclosure field, silently poisoning real integrations. It now returns **503**. |
| A public endpoint that forwarded caller `Authorization` headers and fell back to the operator's key | Anonymous visitors could spend the operator's money. Removed; the proxy now uses a server-side key only. |
| `allow_origins=["*"]` with `allow_credentials=True` | Replaced with an explicit `CONTEXTGC_ALLOWED_ORIGINS` allowlist. |
| Metrics disagreeing across six files (latency as 39.1/39.8/39.9/46.3/46.8%; tests as 16/27/28/31) | There is now no hardcoded metric anywhere. Every number in the UI is computed per request. |

### Bugs fixed

- **Retiring a turn did not retract the facts it uniquely asserted.** A turn
  superseded for one key could be removed from the prompt while a fact only it
  had asserted stayed in the state summary — the model received an authoritative
  value with no visible support. `get_orphaned_facts()` is the regression test.
- **The sanitizer could drop the safety row.** A 25-item catalogue was truncated
  to the first 3 items, discarding a `PEANUT_ALLERGEN` warning on item 15 — the
  exact signal the invariant guardrail depends on. Safety-relevant rows are now
  always retained and omissions are reported.
- **`cache_friendly` mode mutated the prefix it claimed to preserve.** Tool
  payloads were distilled before the mode was read. The flag was literally
  `mode == "cache_friendly"`.
- **Retired-turn recall was unreachable in the SDK path.** It read the archive
  before the loop that populated it, and the convenience function built a fresh
  engine on every call.
- **Unbounded memory growth.** Module-global engines accumulated archive rows
  across requests (6 → 12 → 18 → 24 → 30). State is now per-invocation.
- **Injection via recalled content.** Raw archived text was interpolated into a
  live user turn without escaping, so it could forge our own bracket delimiters.
- **Three entity patterns missed natural phrasings** — "a severe peanut allergy"
  (adjective-first), "set the gate code **to** 1111", and "change **the** address
  to". The peanut-allergy pattern only matched the inverted "allergy: peanuts"
  form the fixtures happened to use.

---

## API

Two surfaces, deliberately unequal in importance.

### `POST /api/compile` — the product

Stateless. No key, no model call, nothing persisted. 256 KiB body cap,
rate-limited, `Cache-Control: no-store`.

```bash
curl -X POST localhost:8000/api/compile \
  -H 'Content-Type: application/json' \
  -d '{"transcript":"user: deliver to Tower B\nuser: actually use Gate 2, code 4921",
       "mode":"compact",
       "invariants":["refunds over 500 need supervisor approval"]}'
```

Set `"teach_protocol": true` to inject the state-protocol instruction, and
`/api/example` returns a `write_path_example` that demonstrates it.

Set `"entity_schema"` to a shipped schema name to turn state tracking on, and
`"declaration_policy"` to decide what happens when the agent declares a key that
schema does not define:

| `declaration_policy` | effect |
|---|---|
| `"flag"` *(default)* | keep it, report it in `telemetry.declarations.off_schema_keys` and `rejected_writes` |
| `"reject"` | drop it, report it as dropped |
| `"off"` | accept silently |

It has no effect without `entity_schema`: an unconfigured engine has no vocabulary
to judge a key against. Note that `"reject"` also drops a *correct* fact declared
under an unexpected name, so read `rejected_writes` rather than assuming the drop
was right.

Accepts a plain-text transcript or a JSON message array:

```
user: deliver order ORD-1 to Tower B, Flat 402
tool [inventory]: {"items": [{"name": "Milk 1L", "price": 34}]}
assistant: confirmed, routing to Tower B
user: actually reroute to Gate 2. Code 4921.
```

Unparseable input returns **422** with per-line parser warnings rather than
silently dropping turns.

### `POST /v1/chat/completions` — optional proxy

An OpenAI-shaped pass-through that compiles the history, then forwards to a real
upstream. Requires a server-side `CONTEXTGC_UPSTREAM_KEY`.

**It refuses rather than fabricating.** A `503` means the operator has not
configured a key — it does not mean you received a model response. It never
accepts a caller's key.

| Variable | Default | Purpose |
|---|---|---|
| `CONTEXTGC_UPSTREAM_KEY` | *(unset)* | Enables the proxy. Absent ⇒ `/v1` returns 503. |
| `CONTEXTGC_UPSTREAM_BASE` | `https://api.openai.com/v1` | Upstream base URL |
| `CONTEXTGC_UPSTREAM_MODEL` | `gpt-4o-mini` | Default model |
| `CONTEXTGC_ALLOWED_ORIGINS` | *(empty)* | Comma-separated CORS allowlist. Empty ⇒ no CORS headers. |
| `CONTEXTGC_RATE_LIMIT` | `60` | Requests per minute per IP |

Responses carry `X-ContextGC-Telemetry: raw=…; compiled=…; saved=…%; compile_ms=…`.

---

## Development

```bash
git clone https://github.com/j4yop/context-hackdevengers
cd context-hackdevengers
pip install -e ".[dev]"
pytest -q                      # 289 tests
uvicorn server.main:app --reload
```

Zero runtime dependencies. `server/` and the test tooling are optional extras.

Two pages, two jobs: `/` is the overview and `/console` is the tool. They share
one stylesheet (`web/style.css`) deliberately — duplicating the tokens is how two
pages drift into looking like two products. `web/console.html` was extracted from
the old single page rather than rewritten, so no control could go missing in the
split, and `tests/browser/page.mjs` drives both surfaces plus dark mode.

The test suite asserts the claims at tolerances tight enough to fail: the
`< 10ms` compile ceiling is enforced on a 60-turn transcript, growth is asserted
to stay roughly linear from 20 to 200 turns, a monkeypatched `socket` proves
nothing opens a network connection, and one test asserts that no fabricated
telemetry field (`latency`, `hallucination`, `estimated`, `risk_score`,
`embedding`, `ivfflat`) has crept back into the output.

## Publishing

Full procedure, including the PyPI Trusted Publishing setup that has to be done
in a browser: [`docs/RELEASING.md`](docs/RELEASING.md).

`main` is protected: a pull request with `test (3.9)`, `test (3.11)`,
`test (3.13)`, `benchmark`, `build` and `browser` green. Direct pushes and force
pushes are refused, including for admins, because several of the defects in this
project's history were mergeable and invisible until something ran.

Releases are cut by tag and go out through `.github/workflows/release.yml`:

```bash
git tag v0.4.0 && git push --tags
```

One-time setup, which has to be done by the maintainer on pypi.org:
**Publishing → add a trusted publisher** for `j4yop/context-hackdevengers`
with workflow `release.yml`. There is then no API token anywhere.

Before uploading, that workflow refuses unless the tag matches the
`pyproject.toml` version, the version is not already on PyPI, `twine check
--strict` passes, the wheel installs into an empty virtualenv and compiles a
transcript using a schema from the installed package, and the dependency list is
still empty. Run it without publishing via
`workflow_dispatch` with `dry_run=true`.

## License

MIT — see [LICENSE](LICENSE).
