store: 313 nodes (136 with aliases) · repo=okl · hook interests=['python', 'method', 'security', 'agent-safety', 'retrieval-design', 'eval-integrity', 'data-quality']
retrieval code: commit 781bf01 · relevance floor on · hybrid off (313 records with a vector) · every row below, recall and precision alike, is measured with this code against this store

A. RECALL: is the governing lesson in the top-12? (harness config = no interests; hook config = repo interests)  rank = position in the ranked in-scope list
task             variant     harness  hook      rank/live  matched terms
idor_endpoint    authored    yes      yes          4/218   ['add', 'endpoint', 'get', 'id', 'user']
idor_endpoint    ticket      yes      yes          1/97    ['add', 'get', 'id', 'return']
idor_endpoint    paraphrase  MISS     MISS        80/103   ['record']
idor_endpoint    bug-report  yes      yes          5/52    ['other', 'url']
price_tamper     authored    yes      yes          1/185   ['product', 'price', 'catalog', 'service', 'client']
price_tamper     ticket      yes      yes          1/59    ['product', 'price', 'body']
price_tamper     paraphrase  MISS     MISS      None/19    []
price_tamper     bug-report  yes      yes          1/63    ['request', 'devtools']
react_fetch      authored    yes      MISS         1/195   ['react', 'component', 'tanstack', 'query', 'v5', 'api']
react_fetch      ticket      yes      MISS         2/91    ['component', 'load', 'api']
react_fetch      paraphrase  MISS     MISS      None/63    []
react_fetch      bug-report  yes      MISS         1/156   ['re', 'requests', 'render', 'stale', 'data', 'after']
judge_summary    authored    yes      yes          1/209   ['llm', 'eval', 'cases', 'judge', 'prints', 'summary']
judge_summary    ticket      yes      yes          1/103   ['llm', 'eval', 'cases', 'judge', 'score', 'summary']
judge_summary    paraphrase  MISS     MISS        61/159   ['report']
judge_summary    bug-report  yes      yes          1/50    ['eval', 'printed', 'score', 'cases', 'threw', 'exceptions']
exit_code_trust  authored    yes      yes          1/172   ['data', 'files', 'reports', 'succeeded', 'output']
exit_code_trust  ticket      yes      yes          1/152   ['data', 'success', 'failure']
exit_code_trust  paraphrase  MISS     MISS        97/108   ['data']
exit_code_trust  bug-report  yes      yes          1/71    ['pipeline', 'said', 'succeeded', 'output', 'directory', 'empty']
spa_tokens       authored    yes      yes          1/153   ['react', 'oidc', 'client', 'ts', 'keycloak', 'session']
spa_tokens       ticket      yes      yes          1/75    ['oidc', 'client', 'ts', 'keycloak', 'access', 'token']
spa_tokens       paraphrase  yes      yes          2/52    ['logged', 'refresh', 'browser']
spa_tokens       bug-report  yes      yes          2/63    ['jwt', 'refresh', 'session']
rate_limiter     authored    yes      yes          1/223   ['rate', 'requests', 'per', 'client', 'endpoint', 'replicas']
rate_limiter     ticket      yes      yes          2/78    ['rate', 'limit', 'per', 'replicas', 'redis']
rate_limiter     paraphrase  MISS     MISS      None/64    []
rate_limiter     bug-report  yes      yes          5/83    ['through', 'requests']
ci_linter        authored    yes      yes          1/157   ['step', 'ruff', 'fails', 'build', 'findings', 'output']
ci_linter        ticket      yes      yes          1/119   ['step', 'ruff', 'fail', 'findings']
ci_linter        paraphrase  MISS     MISS        81/123   ['runs']
ci_linter        bug-report  yes      yes          5/173   ['ci', 'linter', 'new']

recall by variant (of 8): {'authored': 'harness 8/8, hook 7/8', 'ticket': 'harness 8/8, hook 7/8', 'paraphrase': 'harness 1/8, hook 1/8', 'bug-report': 'harness 8/8, hook 7/8'}

A2. HELD-OUT RECALL: rephrasings written blind from tasks.jsonl (evals/heldout_variants.json); same columns
idor_endpoint    ticket      yes      yes          1/109   ['add', 'get', 'id', 'endpoint', 'user']
idor_endpoint    paraphrase  MISS     MISS       118/127   ['add']
idor_endpoint    bug-report  MISS     MISS        53/121   ['url']
price_tamper     ticket      MISS     MISS      None/84    []
price_tamper     paraphrase  MISS     MISS        22/163   ['amount']
price_tamper     bug-report  MISS     MISS         4/19    ['catalog']
react_fetch      ticket      yes      MISS         3/124   ['react', 'component', 'fetch', 'api']
react_fetch      paraphrase  yes      MISS         5/199   ['server', 'library', 'data', 'already']
react_fetch      bug-report  MISS     MISS        71/170   ['request']
judge_summary    ticket      yes      yes          1/148   ['llm', 'eval', 'cases', 'judge', 'score', 'summary']
judge_summary    paraphrase  MISS     MISS      None/127   []
judge_summary    bug-report  yes      yes          1/67    ['summary', 'average', 'score', 'judge', 'threw', 'cases']
exit_code_trust  ticket      yes      yes          1/151   ['data', 'succeeded']
exit_code_trust  paraphrase  MISS     MISS        40/159   ['data', 'job']
exit_code_trust  bug-report  yes      yes          1/111   ['reports', 'success', 'output', 'empty', 'files']
spa_tokens       ticket      yes      yes          1/85    ['oidc', 'client', 'ts', 'access', 'token', 'keycloak']
spa_tokens       paraphrase  yes      yes          6/154   ['library', 'refresh', 'browser']
spa_tokens       bug-report  yes      yes          5/184   ['script', 'page', 'read']
rate_limiter     ticket      yes      yes          1/138   ['rate', 'requests', 'per', 'client', 'replicas']
rate_limiter     paraphrase  MISS     yes         14/202   ['per', 'shared', 'memory', 'store']
rate_limiter     bug-report  yes      yes          2/134   ['through', 'requests', 'per', 'limit']
ci_linter        ticket      yes      yes          2/143   ['step', 'ruff', 'fail', 'build', 'findings']
ci_linter        paraphrase  MISS     MISS       128/177   ['runs']
ci_linter        bug-report  yes      yes          5/187   ['step', 'new', 'findings', 'changed']

held-out recall by variant (of 8): {'ticket': 'harness 7/8, hook 6/8', 'paraphrase': 'harness 2/8, hook 2/8', 'bug-report': 'harness 5/8, hook 5/8'}

B. PRECISION on six real prompts from this session (hook config). For each of the 12 returned: terms matched, and the rarest matched term's document frequency.

> 'this seems like a major point to scrutinize'
  terms=['major', 'point', 'scrutinize']  returned=4  dropped_by_cutoff=0
   1 term(s) ['point']                                rarest df=19  | An instruction to an agent needs a mechanical counterpart 
   1 term(s) ['point']                                rarest df=19  | Build the adapters (clients, connections, repositories) in
   1 term(s) ['point']                                rarest df=19  | OKL_DATABASE_URL was honoured by the store and ignored by 
   1 term(s) ['major']                                rarest df=4  | A source-compatible build on a major dependency upgrade pr
  would survive: >=2 terms: 0/4 · a term in <=3% of store: 1/4

> 'ok do the updates and update my site and the writing section'
  terms=['updates', 'update', 'site', 'writing', 'section']  returned=9  dropped_by_cutoff=0
   3 term(s) ['updates', 'update', 'section']         rarest df=6  | Reach for database primitives before distributed locks — t
   2 term(s) ['updates', 'update']                    rarest df=6  | A hook that calls a CLI must survive an older CLI — plugin
   2 term(s) ['updates', 'update']                    rarest df=6  | Docs and diagrams are the review surface, not byproducts —
   2 term(s) ['update', 'site']                       rarest df=8  | Non-sargable predicates defeat indexes — fix at write time
   1 term(s) ['updates']                              rarest df=6  | An identifier sweep is done when grep returns zero — not w
   1 term(s) ['writing']                              rarest df=12  | If the failure mode would surprise the next person who hit
   1 term(s) ['site']                                 rarest df=8  | A 'nobody else does this' claim expires as the field ships
   1 term(s) ['update']                               rarest df=16  | A state with a verb that records why is refused by every g
   1 term(s) ['writing']                              rarest df=12  | Writing each function's contract down exposes the bugs tha
  would survive: >=2 terms: 4/9 · a term in <=3% of store: 6/9

> 'what about the org knowwledge ideae is that a nich ?'
  terms=['org', 'knowwledge', 'ideae', 'nich']  returned=0  dropped_by_cutoff=0
  would survive: >=2 terms: 0/0 · a term in <=3% of store: 0/0

> 'give me a doc for that too'
  terms=['give', 'doc']  returned=11  dropped_by_cutoff=0
   1 term(s) ['give']                                 rarest df=8  | Type every signature, and tighten mypy one step at a time,
   1 term(s) ['doc']                                  rarest df=18  | Wire a doc claim to a check that governs the doc AND the c
   1 term(s) ['doc']                                  rarest df=18  | Docs and diagrams are the review surface, not byproducts —
   1 term(s) ['doc']                                  rarest df=18  | The docs' claim that okl record --verified is refused is t
   1 term(s) ['doc']                                  rarest df=18  | Code comments link durable docs, never rolling-state files
   1 term(s) ['doc']                                  rarest df=18  | The install command in the README and guide names the real
   1 term(s) ['doc']                                  rarest df=18  | The README's stated check defaults (limit 12; formats agen
   1 term(s) ['doc']                                  rarest df=18  | A render is a separate artifact from its source and needs 
   1 term(s) ['doc']                                  rarest df=18  | Every enforcement surface needs a mechanical trigger and a
   1 term(s) ['doc']                                  rarest df=18  | A diagram with numbers in it is generated from the system,
   1 term(s) ['give']                                 rarest df=8  | Importing verification records through the normal write pa
  would survive: >=2 terms: 0/11 · a term in <=3% of store: 2/11

> 'how do we come up with a good plan for governing github copilot and mcp'
  terms=['come', 'plan', 'governing', 'github', 'copilot', 'mcp']  returned=7  dropped_by_cutoff=0
   2 term(s) ['github', 'copilot']                    rarest df=1  | A 'nobody else does this' claim expires as the field ships
   1 term(s) ['come']                                 rarest df=2  | Verification stamps come from observed checks with stored 
   1 term(s) ['plan']                                 rarest df=4  | Migrations are immutable once applied anywhere; destructiv
   2 term(s) ['github', 'mcp']                        rarest df=10  | Audits read the repo, not the working tree — a scan of dir
   1 term(s) ['governing']                            rarest df=4  | A lesson's files should include what its behaviour depends
   1 term(s) ['plan']                                 rarest df=4  | Importing verification records through the normal write pa
   1 term(s) ['mcp']                                  rarest df=11  | A PyPI MCP server listed on the MCP Registry cannot start 
  would survive: >=2 terms: 2/7 · a term in <=3% of store: 5/7

> 'what about the search ?  and how this thing finds the lessions needed to each prompt?'
  terms=['search', 'finds', 'lessions', 'needed', 'prompt']  returned=12  dropped_by_cutoff=1
   1 term(s) ['prompt']                               rarest df=15  | Driving claude -p from a script: prompt on stdin, no GNU t
   1 term(s) ['search']                               rarest df=19  | A 'nobody measures X' or 'no published system does X' clai
   1 term(s) ['search']                               rarest df=19  | Don't infer by resemblance what you can look up — document
   1 term(s) ['search']                               rarest df=19  | A 'nobody else does this' claim expires as the field ships
   1 term(s) ['prompt']                               rarest df=15  | An instruction to an agent needs a mechanical counterpart 
   2 term(s) ['search', 'prompt']                     rarest df=15  | A per-socket timeout is not a deadline, and a failure afte
   1 term(s) ['finds']                                rarest df=4  | A committed drift snapshot with zero rules fails CI, and o
   1 term(s) ['prompt']                               rarest df=15  | A hook runs in the session's current directory — anchor it
   1 term(s) ['prompt']                               rarest df=15  | A subprocess that shells out to a coding agent must run in
   2 term(s) ['finds', 'prompt']                      rarest df=4  | The encoding loop has no runtime input — every lesson ente
   2 term(s) ['search', 'prompt']                     rarest df=15  | okl stores lessons as typed records found by full-text sea
   1 term(s) ['search']                               rarest df=19  | Research papers are cited from their PDFs on a dated relat
  would survive: >=2 terms: 3/12 · a term in <=3% of store: 2/12

C. The same two policies on the governing lesson of each AUTHORED eval task (would a threshold cost recall?)
  idor_endpoint    matched= 5 rarest df=15  >=2:ok  rare<=3%:LOST
  price_tamper     matched= 5 rarest df=2  >=2:ok  rare<=3%:ok
  react_fetch      matched= 6 rarest df=1  >=2:ok  rare<=3%:ok
  judge_summary    matched= 7 rarest df=3  >=2:ok  rare<=3%:ok
  exit_code_trust  matched= 5 rarest df=1  >=2:ok  rare<=3%:ok
  spa_tokens       matched=11 rarest df=3  >=2:ok  rare<=3%:ok
  rate_limiter     matched= 7 rarest df=3  >=2:ok  rare<=3%:ok
  ci_linter        matched= 6 rarest df=7  >=2:ok  rare<=3%:ok
