store: 313 nodes (136 with aliases) · repo=okl · hook interests=['python', 'method', 'security', 'agent-safety', 'retrieval-design', 'eval-integrity', 'data-quality']
retrieval code: commit 781bf01 · relevance floor on · hybrid linear (313 records with a vector) · every row below, recall and precision alike, is measured with this code against this store

A. RECALL: is the governing lesson in the top-12? (harness config = no interests; hook config = repo interests)  rank = position in the ranked in-scope list
task             variant     harness  hook      rank/live  matched terms
idor_endpoint    authored    yes      yes          1/289   ['add', 'endpoint', 'get', 'id', 'user']
idor_endpoint    ticket      yes      yes          4/289   ['add', 'get', 'id', 'return']
idor_endpoint    paraphrase  MISS     MISS        58/289   ['record']
idor_endpoint    bug-report  yes      yes          3/289   ['other', 'url']
price_tamper     authored    yes      yes          3/289   ['product', 'price', 'catalog', 'service', 'client']
price_tamper     ticket      yes      yes          2/289   ['product', 'price', 'body']
price_tamper     paraphrase  MISS     MISS        22/289   []
price_tamper     bug-report  yes      yes          2/289   ['request', 'devtools']
react_fetch      authored    yes      MISS         1/289   ['react', 'component', 'tanstack', 'query', 'v5', 'api']
react_fetch      ticket      yes      MISS         3/289   ['component', 'load', 'api']
react_fetch      paraphrase  MISS     MISS        39/289   []
react_fetch      bug-report  yes      MISS         3/289   ['re', 'requests', 'render', 'stale', 'data', 'after']
judge_summary    authored    yes      yes          4/289   ['llm', 'eval', 'cases', 'judge', 'prints', 'summary']
judge_summary    ticket      yes      yes          1/289   ['llm', 'eval', 'cases', 'judge', 'score', 'summary']
judge_summary    paraphrase  yes      yes          7/289   ['report']
judge_summary    bug-report  yes      yes          1/289   ['eval', 'printed', 'score', 'cases', 'threw', 'exceptions']
exit_code_trust  authored    yes      yes          2/289   ['data', 'files', 'reports', 'succeeded', 'output']
exit_code_trust  ticket      yes      yes          2/289   ['data', 'success', 'failure']
exit_code_trust  paraphrase  MISS     MISS       105/289   ['data']
exit_code_trust  bug-report  yes      yes          1/289   ['pipeline', 'said', 'succeeded', 'output', 'directory', 'empty']
spa_tokens       authored    yes      yes          2/289   ['react', 'oidc', 'client', 'ts', 'keycloak', 'session']
spa_tokens       ticket      yes      yes          1/289   ['oidc', 'client', 'ts', 'keycloak', 'access', 'token']
spa_tokens       paraphrase  yes      yes          3/289   ['logged', 'refresh', 'browser']
spa_tokens       bug-report  yes      yes         11/289   ['jwt', 'refresh', 'session']
rate_limiter     authored    yes      yes          1/289   ['rate', 'requests', 'per', 'client', 'endpoint', 'replicas']
rate_limiter     ticket      yes      yes          1/289   ['rate', 'limit', 'per', 'replicas', 'redis']
rate_limiter     paraphrase  MISS     MISS        42/289   []
rate_limiter     bug-report  yes      yes          1/289   ['through', 'requests']
ci_linter        authored    yes      yes          9/289   ['step', 'ruff', 'fails', 'build', 'findings', 'output']
ci_linter        ticket      yes      yes         12/289   ['step', 'ruff', 'fail', 'findings']
ci_linter        paraphrase  MISS     MISS        82/289   ['runs']
ci_linter        bug-report  yes      yes          4/289   ['ci', 'linter', 'new']

recall by variant (of 8): {'authored': 'harness 8/8, hook 7/8', 'ticket': 'harness 8/8, hook 7/8', 'paraphrase': 'harness 2/8, hook 2/8', 'bug-report': 'harness 8/8, hook 7/8'}

A2. HELD-OUT RECALL: rephrasings written blind from tasks.jsonl (evals/heldout_variants.json); same columns
idor_endpoint    ticket      yes      yes          2/289   ['add', 'get', 'id', 'endpoint', 'user']
idor_endpoint    paraphrase  MISS     MISS        75/289   ['add']
idor_endpoint    bug-report  MISS     MISS        22/289   ['url']
price_tamper     ticket      MISS     MISS       203/289   []
price_tamper     paraphrase  yes      yes          1/289   ['amount']
price_tamper     bug-report  MISS     MISS         4/289   ['catalog']
react_fetch      ticket      yes      MISS         2/289   ['react', 'component', 'fetch', 'api']
react_fetch      paraphrase  yes      MISS         2/289   ['server', 'library', 'data', 'already']
react_fetch      bug-report  MISS     MISS        43/289   ['request']
judge_summary    ticket      yes      yes          2/289   ['llm', 'eval', 'cases', 'judge', 'score', 'summary']
judge_summary    paraphrase  MISS     MISS        70/289   []
judge_summary    bug-report  yes      yes          1/289   ['summary', 'average', 'score', 'judge', 'threw', 'cases']
exit_code_trust  ticket      yes      yes          2/289   ['data', 'succeeded']
exit_code_trust  paraphrase  MISS     MISS         7/289   ['data', 'job']
exit_code_trust  bug-report  yes      yes          1/289   ['reports', 'success', 'output', 'empty', 'files']
spa_tokens       ticket      yes      yes          1/289   ['oidc', 'client', 'ts', 'access', 'token', 'keycloak']
spa_tokens       paraphrase  yes      yes          3/289   ['library', 'refresh', 'browser']
spa_tokens       bug-report  yes      yes          5/289   ['script', 'page', 'read']
rate_limiter     ticket      yes      yes          1/289   ['rate', 'requests', 'per', 'client', 'replicas']
rate_limiter     paraphrase  MISS     MISS        12/289   ['per', 'shared', 'memory', 'store']
rate_limiter     bug-report  yes      yes          1/289   ['through', 'requests', 'per', 'limit']
ci_linter        ticket      yes      yes          9/289   ['step', 'ruff', 'fail', 'build', 'findings']
ci_linter        paraphrase  MISS     MISS       132/289   ['runs']
ci_linter        bug-report  yes      yes         32/289   ['step', 'new', 'findings', 'changed']

held-out recall by variant (of 8): {'ticket': 'harness 7/8, hook 6/8', 'paraphrase': 'harness 3/8, hook 2/8', 'bug-report': 'harness 5/8, hook 5/8'}

B. PRECISION on six real prompts from this session (hook config). For each of the 12 returned: terms matched, and the rarest matched term's document frequency.

> 'this seems like a major point to scrutinize'
  terms=['major', 'point', 'scrutinize']  returned=5  dropped_by_cutoff=0
   1 term(s) ['point']                                rarest df=19  | An instruction to an agent needs a mechanical counterpart 
   1 term(s) ['point']                                rarest df=19  | Build the adapters (clients, connections, repositories) in
   1 term(s) ['point']                                rarest df=19  | OKL_DATABASE_URL was honoured by the store and ignored by 
   0 term(s) []                                       rarest df=None  | A recorded lesson protects reasoning; only a merged fix pr
   1 term(s) ['major']                                rarest df=4  | A source-compatible build on a major dependency upgrade pr
  would survive: >=2 terms: 0/5 · a term in <=3% of store: 1/5

> 'ok do the updates and update my site and the writing section'
  terms=['updates', 'update', 'site', 'writing', 'section']  returned=9  dropped_by_cutoff=0
   2 term(s) ['updates', 'update']                    rarest df=6  | Docs and diagrams are the review surface, not byproducts —
   1 term(s) ['update']                               rarest df=16  | A state with a verb that records why is refused by every g
   2 term(s) ['updates', 'update']                    rarest df=6  | A hook that calls a CLI must survive an older CLI — plugin
   3 term(s) ['updates', 'update', 'section']         rarest df=6  | Reach for database primitives before distributed locks — t
   2 term(s) ['update', 'site']                       rarest df=8  | Non-sargable predicates defeat indexes — fix at write time
   1 term(s) ['writing']                              rarest df=12  | If the failure mode would surprise the next person who hit
   1 term(s) ['updates']                              rarest df=6  | An identifier sweep is done when grep returns zero — not w
   1 term(s) ['site']                                 rarest df=8  | A 'nobody else does this' claim expires as the field ships
   1 term(s) ['writing']                              rarest df=12  | Writing each function's contract down exposes the bugs tha
  would survive: >=2 terms: 4/9 · a term in <=3% of store: 6/9

> 'what about the org knowwledge ideae is that a nich ?'
  terms=['org', 'knowwledge', 'ideae', 'nich']  returned=0  dropped_by_cutoff=0
  would survive: >=2 terms: 0/0 · a term in <=3% of store: 0/0

> 'give me a doc for that too'
  terms=['give', 'doc']  returned=12  dropped_by_cutoff=0
   1 term(s) ['doc']                                  rarest df=18  | Docs and diagrams are the review surface, not byproducts —
   1 term(s) ['doc']                                  rarest df=18  | Wire a doc claim to a check that governs the doc AND the c
   1 term(s) ['doc']                                  rarest df=18  | The docs' claim that okl record --verified is refused is t
   1 term(s) ['give']                                 rarest df=8  | Type every signature, and tighten mypy one step at a time,
   1 term(s) ['doc']                                  rarest df=18  | Code comments link durable docs, never rolling-state files
   1 term(s) ['doc']                                  rarest df=18  | The README's stated check defaults (limit 12; formats agen
   1 term(s) ['doc']                                  rarest df=18  | The install command in the README and guide names the real
   1 term(s) ['doc']                                  rarest df=18  | A render is a separate artifact from its source and needs 
   1 term(s) ['doc']                                  rarest df=18  | Every enforcement surface needs a mechanical trigger and a
   1 term(s) ['doc']                                  rarest df=18  | A diagram with numbers in it is generated from the system,
   1 term(s) ['give']                                 rarest df=8  | Importing verification records through the normal write pa
   0 term(s) []                                       rarest df=None  | Run the architecture/review agent headless in CI on every 
  would survive: >=2 terms: 0/12 · a term in <=3% of store: 2/12

> 'how do we come up with a good plan for governing github copilot and mcp'
  terms=['come', 'plan', 'governing', 'github', 'copilot', 'mcp']  returned=7  dropped_by_cutoff=0
   2 term(s) ['github', 'copilot']                    rarest df=1  | A 'nobody else does this' claim expires as the field ships
   2 term(s) ['github', 'mcp']                        rarest df=10  | Audits read the repo, not the working tree — a scan of dir
   1 term(s) ['come']                                 rarest df=2  | Verification stamps come from observed checks with stored 
   1 term(s) ['governing']                            rarest df=4  | A lesson's files should include what its behaviour depends
   1 term(s) ['plan']                                 rarest df=4  | Migrations are immutable once applied anywhere; destructiv
   1 term(s) ['mcp']                                  rarest df=11  | A PyPI MCP server listed on the MCP Registry cannot start 
   1 term(s) ['plan']                                 rarest df=4  | Importing verification records through the normal write pa
  would survive: >=2 terms: 2/7 · a term in <=3% of store: 5/7

> 'what about the search ?  and how this thing finds the lessions needed to each prompt?'
  terms=['search', 'finds', 'lessions', 'needed', 'prompt']  returned=12  dropped_by_cutoff=1
   1 term(s) ['search']                               rarest df=19  | A 'nobody measures X' or 'no published system does X' clai
   1 term(s) ['prompt']                               rarest df=15  | Driving claude -p from a script: prompt on stdin, no GNU t
   1 term(s) ['prompt']                               rarest df=15  | An instruction to an agent needs a mechanical counterpart 
   1 term(s) ['search']                               rarest df=19  | Don't infer by resemblance what you can look up — document
   1 term(s) ['search']                               rarest df=19  | A 'nobody else does this' claim expires as the field ships
   1 term(s) ['prompt']                               rarest df=15  | A subprocess that shells out to a coding agent must run in
   1 term(s) ['prompt']                               rarest df=15  | A hook runs in the session's current directory — anchor it
   2 term(s) ['search', 'prompt']                     rarest df=15  | A per-socket timeout is not a deadline, and a failure afte
   1 term(s) ['finds']                                rarest df=4  | A committed drift snapshot with zero rules fails CI, and o
   2 term(s) ['finds', 'prompt']                      rarest df=4  | The encoding loop has no runtime input — every lesson ente
   2 term(s) ['search', 'prompt']                     rarest df=15  | okl stores lessons as typed records found by full-text sea
   1 term(s) ['search']                               rarest df=19  | Research papers are cited from their PDFs on a dated relat
  would survive: >=2 terms: 3/12 · a term in <=3% of store: 2/12

C. The same two policies on the governing lesson of each AUTHORED eval task (would a threshold cost recall?)
  idor_endpoint    matched= 5 rarest df=15  >=2:ok  rare<=3%:LOST
  price_tamper     matched= 5 rarest df=2  >=2:ok  rare<=3%:ok
  react_fetch      matched= 6 rarest df=1  >=2:ok  rare<=3%:ok
  judge_summary    matched= 7 rarest df=3  >=2:ok  rare<=3%:ok
  exit_code_trust  matched= 5 rarest df=1  >=2:ok  rare<=3%:ok
  spa_tokens       matched=11 rarest df=3  >=2:ok  rare<=3%:ok
  rate_limiter     matched= 7 rarest df=3  >=2:ok  rare<=3%:ok
  ci_linter        matched= 6 rarest df=7  >=2:ok  rare<=3%:ok
