One rule governs every file under paxman/capabilities/*/grammar/ —
learn it once, read them all.
Your mission is reading paxman's recognition layer fluently. Every capability ships several grammar files (paxman/capabilities/SIUnit/grammar/name_recognition.py, …/Country/grammar/alpha2_recognition.py, …), and they all satisfy one interface. Master that interface today and every one of those files becomes 80% familiar.
re.finditerYou already do recognition in Python. When a regex finds "kilogram" inside
"Add 5 kilogram", m.span() hands you back more than the string — it hands
you a location: (6, 14). The start index, an end index, the matched text.
Paxman's entire recognition layer is built on that idea, formalized:
A grammar answers where something appears in raw text — never what it means, never whether it is valid.
ARCHITECTURE.md, “Separation of Recognition and Validation”
In regex terms: think of each shipped grammar as one giant pattern run through
finditer — with the semantics strictly limited so that a second stage can decide
meaning later.
From the source of truth, paxman/core/domain.py:
@dataclass(frozen=True, slots=True)
class RecognitionMatch(Generic[NotationT]):
"""A span-bearing recognition produced by a grammar.
``start`` and ``end`` are half-open character offsets into the input
text passed to ``Grammar.recognize()``; ``raw_text`` is the matched
substring, so ``len(raw_text) == end - start`` always holds.
"""
notation: NotationT
start: int
end: int
raw_text: str(The constructor also validates both facts above — negative spans and mismatched lengths are impossible.) And the abstract method every grammar must implement, paxman/core/domain.py ~line 301:
@abstractmethod
def recognize(self, text: str) -> list[RecognitionMatch[NotationT]]:
"""Extract span-bearing recognition matches from raw text. …"""| Field | Meaning | Regex analogue |
|---|---|---|
| notation | a small structured record of what shape of thing was found (position included implicitly) | group names + their values |
| start | index of first matched character, relative to the original input | m.start() |
| end | one past the last matched character (half-open) | m.end() |
| raw_text | exactly text[start:end], original casing preserved | m.group(0) |
[start, end) means start included, end excluded — the same convention as
Python's range() and slices, which makes text[start:end] and
text[start:end].len() come out right without mental gymnastics. If you ever write an
off-by-one here, the dataclass raises before it leaves the factory.
This is the part that surprises newcomers most. Look at the Country name grammar's own docstring (paxman/capabilities/Country/grammar/name_recognition.py):
Examples: "United States" → value="United States"
"USA" → value="USA"
Non-examples: "840" → [] (no name match)
"XYZ" → [] (unknown name)The grammar emits "USA" unchanged — not "US", not a
canonical code, not a boolean “is valid”. Deciding that USA means ISO alpha-2 US, with
provenance, is the job of the validation rules, a separate layer the engine runs after
recognition. A single input can even be recognized by two grammars at once (think of
01/02/2026 under US and European date readings) — the engine keeps both so ambiguity
stays observable, deduplicating only within one grammar (“longer wins”) and ordering everything into
document order itself (ARCHITECTURE.md §Recognition Pipeline Contract).
So when you read any grammar file, you are asking one question only: “which substrings would this light up?”
Scoring note: fewer clicks = sharper recall. Replay until every round falls on the first try.
Type from memory, then press Enter. If the hint appears, resist scrolling up — struggling to retrieve is what builds durable memory.
span(); notice how little new vocabulary lesson 001 needed beyond it.