living document · updated by the quality loop

The boost roadmap

Every finding from the autonomous quality loop lands here — what shipped, what's mid-flight, and what's queued — scored by complexity, impact, and a little bit of wow.

See what's next → Back to the guide
Shipped merged & released In flight a PR is open Next triaged, starting soon Planned on the list
20Shipped
1Next up
62Planned
478Loop finds

Shipped

// merged to main & published
ShippedLicensing

The copyleft protected nothing and cost the one thing boost needs

Write-up · relicence-apache-2.md

Complexity M Impact High Wow ★★★★ GPL-3.0-only → Apache-2.0, while it was still a one-person decision
ShippedAgents · Parity

bmad on knew about a second host and wrote to one anyway

Write-up · bmad-hooks-all-hosts.md

Complexity S Impact Med Wow ★★★ the autopilot wrote hooks to Claude and nowhere else, long after boost knew a second host
ShippedAgents · Parity

A Gemini user got the heuristic fallback from every AI command

Write-up · ai-backend-fanout.md

Complexity S Impact High Wow ★★★★ the last surface that only spoke Claude — and the one a user can see
ShippedQuality · CLI

All 80 boost commands audited against a disposable HOME — four defects fixed

Write-up · cli-command-audit.md

Complexity M Impact Med Wow ★★★ audited all 80 COMMANDS entries against a disposable HOME; 4 real defects fixed with tests

Next up

// triaged findings, starting soon

Planned

// on the list
DeclinedTesting · Gap

Bring commands/ under mutation testing

The ~8,100-line command layer has zero mutation coverage: mutmut is scoped to core/ only. A blocking 80% floor was attempted and the baseline has now actually been measured — it does not clear the bar, and two of the three constraints recorded here earlier were wrong. What a sample run of commands/taps.py (the best-covered module in the package, 95.8% lines) against tests/unit/ + tests/functional/ reports through this repo's own gate: 570/793 killed — 71.9%, under the 80% floor, with all 223 survivors spread across every one of the module's 7 functions rather than concentrated in one testable gap. Mutant density is 4.2 per statement in taps.py but 2.46 across core/ (9,909 mutants over 4,033 statements), so commands/'s 5,314 statements imply somewhere between 13,000 and 22,300 mutants; at the measured 1.42 mutations/sec — roughly 6.5× slower per mutant than the core/ job, which matches the functional-vs-unit suite cost — that is 2.5 to 4.5 hours on one runner, against an ~18-minute job today. The job sets no timeout-minutes, so it inherits GitHub's 360-minute cap rather than failing fast. Two blockers sit underneath that number. Selecting only tests/unit/, the way core/ does, leaves commands/ at 17.9% line coverage versus 91.5% with functional included — and "no tests" mutants count against the score, so that route floors out near 18%. Selecting tests/functional/ instead crashes the run: mutmut's record_trampoline_hit calls p.resolve(strict=True) on the relative source path, so any test that chdirs into a temp project dies with FileNotFoundError: <tmp>/boost_cli — confirmed on test_verify_sees_a_project_skill. Corrections to the earlier note: pytest_add_cli_args_test_selection is not single-valued, it is a list that configparser splits on newlines (a space-separated line is what errors), and a sibling pytest_add_cli_args takes extra pytest flags. One prerequisite is already fixed: --no-mcp leaked through os.environ and made the suite order-dependent, which mutmut exposes because it runs whichever test subset covers each mutant. Declined 2026-07-29. Not "hard" — blocked, and the block is upstream. mutmut 3.6.0 is still the latest release, and src/mutmut/__main__.py:120 still reads source_paths = [p.resolve(strict=True) for p in Config.get().source_paths] — unconditionally, before the max_stack_depth guard — on paths that configuration.py:102 builds as plain relative Paths. Every chdir-ing functional test therefore kills the run, and the functional suite is precisely what takes commands/ from 17.9% to 91.5% line coverage. So the two candidate configurations are "floors out near 18%" and "crashes"; there is no third. Even granting a fix, the measured numbers already refuse the proposal on their own terms: 71.9% against an 80% blocking floor, with survivors spread evenly rather than pooled in one testable gap, at 2.5–4.5 hours per run against an ~18-minute job. A gate that is 8 points red on the day it lands does not gate anything; it just makes main red. The lead, if anyone reopens this: the crash is a relative-path bug, and source_paths is not required to be relative — an absolute path survives resolve(strict=True) from any working directory. Whether the rest of mutmut's mutants/ copy machinery tolerates one is untested. That, plus a non-blocking scheduled job that rotates one module per week, is the shape worth trying; a blocking 80% floor is not. What is not lost by declining. The architecture already puts behaviour in core/ — which is mutation-gated at 80% — and keeps commands/ as thin CLI glue. The uncovered layer is the one deliberately designed to hold the least logic, and it still carries 91.5% line coverage from the functional suite.

Complexity XL Impact High Wow ★★★ declined 2026-07-29 — 71.9% measured, and the run cannot complete at all
In flightQuality · Eval

a Tier 3 eval for tool-call behaviour, floored in both directions

boost's required gate floors four retrieval metrics — recall@k ≥ 0.78, hit@1 ≥ 0.40, MRR ≥ 0.52, nDCG@k ≥ 0.58 — over a 91-query golden set and a 10,152-entry corpus, every row pinned to a commit SHA. All of it measures what boost returns once it is asked. Nothing measures whether an agent asks. **The call itself is unmeasured**, and it is the step everything downstream depends on. The miss that exposed it. A Gemini CLI session was asked to "create a new, simplified app demonstrating RAG implementation in Python3 using langGraph, langChain, and langSmith" — a new project, an architecture decision and a dependency choice, which is three of the triggers boost_search's description names explicitly. It activated two already-installed skills, built the app, and never called boost. Asked why, it paraphrased boost's own lock-in trigger list back verbatim, so the text was read and was not persuasive. A gate that floors recall@k at 0.78 reported nothing, because retrieval was never invoked. Every claim in the MCP surface is argued, not measured. The triggers, the 10-15s stated cost, the skip list, the three-kind framing, the "already covered is not already checked" defeater — each survived a careful review and none has a number behind it. That is a high-variance lever tuned blind: Tool Preferences in Agentic LLMs are Unreliable (EMNLP 2025, arxiv 2505.18135) measures description-only edits swinging call rate by more than 10×. boost currently ships those edits on reasoning alone. The design constraint that decides whether this is worth building: floor both directions. A tier that measures call rate alone rewards making boost maximally assertive, which is precisely the capture the surface is written to avoid — and boost has already learned this exact lesson one tier down. Flooring recall alone was a hole rather than a simplification: a ranker that finds the right answer every time and never ranks it first scores recall@10 1.000 with hit@1 0.000, and passed. So the prompt set needs two halves — a should-call set (multi-file work, a new subsystem, a config or CI job that outlives the session) and a should-not-call set drawn from the shipped skip list (a question, a one-line edit, a command the user just handed over) — with a false-call ceiling as binding as the call-rate floor. One number without the other is an incentive to ship the thing boost refuses to be. Per host, never averaged. The two registered hosts do not see the same boost text. Claude Code puts server instructions in the system prompt; Gemini CLI never delivers them in interactive mode at all — Config.initialize() does not await mcpInitializationPromise, so getMcpInstructions() returns "", startChat stamps the context entry once with a stable id, and the later refreshMcpContext() re-renders Tier 1 only. A single averaged score would hide a host where 1,786 characters of guidance are simply absent, and would credit or blame wording for a delivery failure. Shape. An opt-in make eval-tools beside eval-ai / eval-rec / eval-explain — real hosts and real LLM calls, so it is non-deterministic and key-gated and must not join the required check gate; same degrade-cleanly contract as the other Tier 2 evals. Because the outcome is stochastic, report N runs per prompt with an interval rather than a single pass/fail, the way golden-set-statistical-power established for retrieval — a one-shot replay cannot tell a wording regression from a sampling wobble. Where this stands (2026-08-31), and a correction. The Claude Code arm shipped in #616: scripts/eval_tools.py, a 16-prompt set halved into should-call and should-NOT-call, Wilson intervals over N runs, and a verdict that floors call rate and ceilings false calls. Its probe was broken, and the finding this card recorded was an artifact of it. An earlier revision of this card reported “3/3 false calls — boost's tools fired on What is the difference between a Python list and a tuple?”. They did not fire. called_boost() substring-scanned the raw event stream, and claude -p --output-format stream-json --verbose opens with a system/init event enumerating every tool available to the session — which on any machine where boost is registered contains mcp__boost__boost_search and the other three CONSULT names. So the check returned true on every run, including runs with no tool call at all. Measured directly: Say OK and nothing else. produced zero tool_use blocks and scored as a boost consult. Two consequences shipped with it. make eval-tools could never pass — eight no-call rows × three runs is 24 forced trues, so the false-call rate's lower bound sat at 1.00 against a 0.20 ceiling, red on every machine forever. And the tier built to retire unfalsifiable claims had produced one. The lesson is the tier's own: the existing tests passed because they fed hand-written one-line fragments with no init event — a fixture the author invented could not catch the author's wrong model of the input. The probe now parses the NDJSON and counts only tool_use blocks inside assistant events, and the regression test drives a captured real stream. The second host arm is still unwritten, and should stay that way until the fixed probe is re-run. Building arm two on a probe that cannot tell an offer from a call would produce two hosts scoring an identical, meaningless 1.00. When it is built, the candidate is Gemini CLI proper, not Antigravity CLI: the delivery claim below is about Gemini's Node bundle, and agy is a third mode again — it receives boost's instructions and writes them to ~/.gemini/antigravity-cli/mcp/boost/instructions.md, pointing the agent at the file rather than inlining it. Substituting it would measure a different mechanism than the one this card argues about. Cost, now measured. Two trivial runs on a real host reported $0.657 and $0.682 of total_cost_usd, so 16 prompts × 3 runs is roughly $30–50 per host per invocation on a machine with a crowded tool surface. --strict-mcp-config with a boost-only config cuts that sharply and controls the surface confound in the same move. 2026-08-31: --strict-mcp-config shipped. eval_tools.py now takes a --strict-mcp-config flag: it writes a boost-only mcpServers config (the same <launcher> mcp --stdio invocation and fork-safety env core.mcphost.register_argv uses for a real registration — confirmed against an actual claude mcp add-json write, not guessed at the schema) to a temp file and passes --strict-mcp-config --mcp-config <path> to every claude -p call, cleaning the file up afterward. This session's sandbox had no network path to PyPI, so the pinned toolchain (pytest, ruff, mypy, …) could not be installed and make check could not be run here; the change was verified by hand instead — direct python3.12 import of the module, the new unit tests executed by eye against the interpreter, py_compile, a manual line-length check against ruff's 88-column default, and an end-to-end dry run with subprocess.run mocked that confirms the flags land on the argv and the temp file is created and removed. CI runs the real gate on the PR. Still unwritten: the second host arm (Gemini CLI). No gemini CLI was reachable in this sandbox to capture a real stream from, and building that arm on an invented model of Gemini's non-interactive output format is the exact mistake this card's own probe fix (2026-08-30) already paid for once — "a fixture the author invented cannot catch the author's wrong model of the input." That arm stays a placeholder until it can be built against a captured real stream, on a machine with the gemini CLI installed. What it unlocks. The first honest answer to "did that description edit help", a baseline the next surface change can regress against, and a way to retire claims that survive only because nobody can check them.

Complexity L Impact High Wow ★★★★ probe fixed; --strict-mcp-config shipped for the surface confound; second host (Gemini CLI) still unwritten
PlannedTech-debt

unpin the [eval] langchain stack when ragas ships its fix

The [eval] extra pins langchain-core<0.4, langchain-community<0.4 and langchain-openai<1 because ragas hard-imports ChatVertexAI from a langchain_community chat-models path that 0.4.x deleted. The LangChain integration card originally made this unpin its phase 0 and was corrected in place: ragas 0.4.3 still carries the import (measured 2026-08-04 — declared bounds are open, but import ragas crashes beside langchain 1.x), while upstream main already has the removal merged. So the unpin is one release of someone else's package away. What to do when it lands. Check pip index versions ragas (or the PyPI JSON) for a release after 0.4.3; verify in a throwaway venv that import ragas succeeds beside langchain>=1; then move [eval] to that floor, delete the three langchain pins, and adapt scripts/eval_explain.py if the 0.4 scoring API moved (its evaluate/to_pandas surface is what test_eval_faithfulness.py stubs in the unit suite). The eval-explain workflow is the live proof — it must stay green with real keys. Re-checked 2026-08-30. pip index versions ragas still reports 0.4.3 as the newest release, so nothing has changed and this card is still not claimable. Recorded here rather than left implicit: a card that says "check before starting" gives a reader no way to tell a check that came back negative from a check nobody ran. Why it stays its own card. The shipped integration card documents the block but will not be re-read; an unpin nobody remembers is how a workaround pin outlives its reason by years. This card is the reminder, and it is deliberately not claimable until the upstream release exists.

Complexity S Impact Low Wow ★ still blocked — re-checked 2026-08-30, PyPI's newest ragas is still 0.4.3
DeclinedRelease

give boost-langchain a release path to PyPI

Declined, deliberately. Both missing pieces below were owner-only or upstream-blocked, and the research they prompted dissolved the premise: a second PyPI project bought a separate release cadence nobody needed (boost releases more often than langchain), while the ecosystem evidence — langchain-community sunset, non-langchain-* names in LangChain's own integrations listing, in-host precedents from ragatouille to mlflow — showed the standalone distribution was never required. The integration now ships inside the boost-skill-cli wheel behind a [langchain] extra instead; see langchain-in-the-wheel. The original card follows for the record. The boost-langchain distribution shipped under integrations/langchain/ with its whole point being a separate release cadence from boost-skill-cli — langchain majors move faster than boost does, and the conformance workflow already builds the sdist/wheel and runs twine check on every touching PR. What does not exist is any way for those artifacts to reach PyPI: the name 404s there, and nothing publishes on any trigger. Two pieces, one of which only the repo owner can do. First, create the PyPI project and configure a Trusted Publisher for it — pending-publisher registration works before the first upload, and the filename-matching rule that pinned boost's own workflow name applies here too. Second, a publish workflow with a deliberate trigger: not boost's every-merge cadence (publish.yml releases boost-skill-cli on every push to main, which is exactly the coupling the separate distribution exists to avoid) — a tag like boost-langchain-v0.1.0 or a manual dispatch that bumps the static version, builds from integrations/langchain/, and publishes with the OIDC token. Remember the repo's own lesson: a release:-triggered workflow can never fire here (GITHUB_TOKEN events do not chain), so trigger on the tag push or dispatch directly. The floor is already honest. The package requires boost-skill-cli>=1.0.320 — measured against the actual API it calls, verified by an adversarial install — so the first published version works against PyPI as it stands today.

Complexity S Impact Med Wow ★★ declined — superseded by shipping the integration inside the boost-skill-cli wheel (see langchain-in-the-wheel)
In flightSearch · Performance

publish the keyword index the way vectors are published

Dense vectors are built once in CI and downloaded. The BM25 index is not: core/rag.py has no export or import function at all, and shards.yml / scripts/publish_shards.py are dense-only end to end. Every install rebuilds the same index from the same registries, at the same pinned commits, to produce the same bytes. Measured, on a real 458-tap machine. The on-disk index is rag_index.json 43.7 MB plus rag_postings.sqlite 653.0 MB — 696.7 MB for 18,619,658 postings. Build cost, timed over a 9,306-entry / 69-tap slice: 4.54 s reading bodies and tokenizing, 3.83 s writing postings, 8.4 s total — about 0.9 ms per entry, so roughly 65 s and ~900 MB extrapolated to the full 71,700-entry catalogue. Which user actually pays it. Not the default one: boost quickstart taps the 7 starter registries and indexes them in about a second. The cost lands on boost quickstart --catalog — 463 registries, 2 min 10 s of parallel cloning and then a minute of indexing on top — and on anyone who taps their way there gradually. The bug that makes this worth doing is not speed. boost catalog --import already exists and already looks like the answer: shareable-catalogue-bundle advertises 10.9 MB replacing a 12 GB clone and "59,972 searchable items in 4 seconds". That 4 seconds is fast for a reason the card does not state. rag.read_body degrades silently to name + description when the item's clone is absent (rag.py: "Missing files degrade to just the catalog metadata"), and a bundle import restores catalogues with zero repositories cloned. So the index it builds is not the full-content index the evals gate floors — it is a frontmatter index wearing the same file name. Measured directly over 3,015 real entries, indexing them with and then without their clones: 3,041,326 tokens versus 182,507. A bundle-only index carries 6.0% of the searchable text, and nothing in the output says so. That is the same failure shape as an unpinned eval corpus — a number that still renders confidently while measuring something else. Why this is easier than the dense shards, not harder. BM25 looks like it needs global statistics, and it does — but none of them are frozen at build time. _bm25 derives n = len(docs) and df = len(plist) on every query, so IDF is computed from whatever corpus is loaded. A per-registry shard therefore merges by offsetting doc_id, unioning the postings, and recomputing avg_len from per-shard totals — arithmetic, not re-derivation. And unlike vectors there is no embedding space to match and no API key to hold, so shards.incompatible() has no analogue here: a published keyword index is importable by everyone, including the keyless user who cannot use vectors at all. Shape. rag.export_shard / rag.import_shard mirroring dense's pair, per-registry assets on the existing shards-latest release, rows carried in the same manifest.json with the same commit pin and sha256 — the carry-forward machinery in publish_shards.py manifest --carry-forward applies unchanged, because a registry whose commit did not move has an index that did not change either. Three invariants transfer verbatim from the dense side and each is load-bearing: verify before replacing, refuse a shard whose commit is not the tap's commit, and never treat a missing digest as a match. The open question is payload size, and it is large enough to be its own decision — see shrink-the-published-index. This card should not ship until that one has an answer, because publishing 697 MB per refresh to save 65 s of CPU is not obviously the right trade, and at the compressed sizes measured there it clearly is. Partly landed, and deliberately still inflight — 2026-09-10. What shipped is the half that needed no size decision: the index now records what it is. read_body_full returns the text and whether it contains the item's body, build() reports metadata_only over every document written (reused ones included, or an incremental build reports zero on the run after a bundle import), index_completeness() reads the share back off disk, and boost reindex says it out loud instead of reporting the same confident count for a 6% index. The share is of tokens, not documents: a bodyless entry still produces a document, so a document share sits at 1.0 until it drops to 0.0. INDEX_VERSION moved to 9, because the flag is written only when a body is missing and absence may only be read as "complete" once no older document can survive. What did NOT land: rag.export_shard / rag.import_shard, the per-registry assets, and the manifest.json rows — the publishing pipeline itself. That half is what the payload-size question governs, and shrink-the-published-index still has no answer: its claim is stale, not active — branch loop/shrink-postings-index was last touched 2026-09-02, carries one commit, has no pull request, and is 394 commits behind main. Someone should un-claim it. One structural finding for whoever takes it: doc ids are positional (_save does enumerate(docs)), so the card's "merge by offsetting doc_id" is sound as written — and the shard format should serialize logical postings (digest → term → tf) rather than the SQLite layout, so the interning that branch was attempting cannot invalidate a published shard.

Complexity L Impact High Wow ★★★★ every machine rebuilds 697 MB of index for a corpus that is byte-identical on all of them
In flightPerformance · Storage

shrink the keyword index before publishing it — structure first, then compression

publish-the-keyword-index is worth doing only if the artifact is small enough to ship weekly. This card is the measurement that decides it, and the first answer is that compression is the second lever, not the first. What the format actually stores. _write_postings creates postings (term TEXT, doc INTEGER, tf INTEGER) and inserts one row per posting, so the term string is repeated in every row. Measured on the real 458-tap store: 18,619,658 rows over 210,422 distinct terms averaging 6.6 characters. That is ~123 MB of term text to carry 1.4 MB of distinct term text — 88× redundancy, before the per-row and B-tree overhead that turns it into a 653 MB file (page_size 4096, 167,174 pages, freelist 0, so it is not slack space). Compression measured on that file, as it stands: rag_index.json 43.7 MB raw · gzip -6 10.6 MB (4.12×).
rag_postings.sqlite 653.0 MB raw · gzip -6 201.7 MB (3.23×) · zstd -3 169.9 MB (3.84×) · zstd -19 106.7 MB (6.11×). So even with no format change, zstd -19 puts the whole index near 117 MB — under half the ~300 MB of dense vectors already published weekly. The trade is already good; the point of this card is that it can be much better, and that the two levers compose. Structure first, and it is the bigger win. Interning terms into terms(id, term) with postings(term_id, doc, tf) removes ~123 MB of duplicated strings and shrinks the postings_term index from a text key to an integer one. Beyond that, the classic inverted-index encodings apply directly because doc ids within a term are ascending: delta-encode them, varint or bitpack the deltas, and store one blob per term rather than one row per posting. Both shrink the file on disk, not just in transit, which is the half a compressed download never gives back — the user still ends up with 653 MB resident after import. What must not regress. read_postings exists precisely so a query touches a handful of terms instead of materialising the whole map — the change that took cold search from 8-13 s and multiple GB resident to 31-70 ms of scoring. A blob-per-term layout keeps that property (one row read per query term, decoded on the spot); a scheme that requires decoding neighbouring terms to find one does not. _bm25 must stay byte-identical, as it did through the SQLite move, and TestBm25Math is what says so. Decompression cost is the thing to measure, not assume. zstd -19 is slow to compress and fast to decompress, which is the right asymmetry for a weekly build feeding many imports — but "fast" needs a number on the import path before it is a claim, next to the 0.12 s that importing dense rows costs today. A zstd dictionary trained across shards is the obvious follow-on for the many-small-registries case, where per-shard compression has little context to work with. Deliverable. A measured comparison — raw, interned, delta+varint, each × none/gzip/zstd — on the real store, with import-side decode time beside each. That table is what tells publish-the-keyword-index what to ship, and it is worth having even if publishing is declined: the on-disk win applies to every install today. Progress — PR 688, merged as f003fa03 in train 691. The structural half shipped: _write_postings now interns terms into their own terms(id, term, df) table, with postings carrying an integer term_id instead of repeating the term string on every row — exactly the "structure first" change this card calls the bigger win, and it bumps INDEX_VERSION so every store picks it up on its next rebuild. stem_expansions now reads the precomputed df column directly instead of a GROUP BY COUNT(*) over postings on every prefix lookup. Not done: the delta/varint doc-id encoding, and the full raw/interned/delta+varint × none/gzip/zstd comparison table with import-side decode times on a real multi-hundred-MB store — this sandbox has no such store to measure against, only a small synthetic one (interning alone cut a 1.8M-posting/20k-term synthetic store from 63.6 MB to 47.9 MB, directionally consistent with the real-store estimate above but not a substitute for it). Left as follow-on work before this card can be called shipped.

Complexity M Impact Med Wow ★★★ 653 MB of postings holds 1.4 MB of distinct terms — the term string is stored 88 times over
ShippedHealth · Robustness

A best-effort log handler prints a traceback over every command's output

Write-up · delay-true-defeats-the-log-suppress.md

Complexity S Impact Medium Wow ★★ handleError overridden on the file handler, scoped to emit (#637); doctor probes open() rather than mode bits (#638)
In flightSearch · Ranking

Near-identical copies survive content-hash dedup and take the whole result page

Content-hash dedup shipped and worked: rag.dedupe_by_content took duplicate result slots from 4.94 to 0.60 per query over a 77-tap corpus. That card closed naming one thing still open — near-identical rather than byte-identical clustering, where core/typosquat.py's confusion machinery would apply — and buried it under a shipped status where nobody would claim it. This card is that remainder, with a measurement that makes it look considerably worse than “refinement”. Observed on a real 466-tap install with hybrid RRF serving (658,131 chunks): for the query exa search, every one of the top ten rows is exa-search, and the descriptions are what give the shape away — one Japanese (Exa MCPによるウェブ、コード、企業調査), two Chinese (通过Exa MCP进行神经搜索), five English variants of Neural search via Exa MCP, plus Use Exa MCP for current web… and AI-powered web search…. All ten are ★ curated. The footer reads 51 matches · ranked by hybrid RRF (BM25 + dense). Every one of those passed dedup correctly. They are not byte-identical: they are the same skill in Japanese, in Chinese, and in five English phrasings across different registries. The body digest differs, so dedupe_by_content keeps them all — which is exactly the behaviour #366 proved must be preserved, since two entries sharing a name can be genuinely different rules. The shipped fix is not misbehaving. It simply does not reach this shape. What the 0.60 residual actually was. The prior card described its leftover as “entries sharing a name whose bodies genuinely differ, which must stay separate” — true as stated, and it reads as a rounding error. At 466 taps the same residual is a full result page. The gap between 0.60 and 10.0 is worth understanding before designing anything: the 77-tap measurement used 50 natural-language queries averaged, and an average hides the shape here. Duplicate pressure was already known to be a step function of which registries are tapped rather than how many; near-identical pressure looks like a step function of which query — harmless across a query set, total on any query that lands on a widely-mirrored skill. Re-measure per-query maxima, not means. The hard part is the safety proof, not the clustering. Content hashing was adoptable because one count settled it: of 14,153 distinct bodies, clusters spanning more than one name numbered zero, so collapsing could not merge two different skills. Near-identical clustering has no such free proof — any similarity threshold loose enough to merge a Japanese translation with its English original is loose enough to merge two genuinely different skills that share boilerplate. Establish the equivalent bound first (over a real corpus, at the chosen threshold, count clusters spanning more than one meaning) or the fix trades a visible problem for a silent one. Three things to get right. Translations are the motivating case and the hardest: they share almost no tokens with the original, so token-overlap similarity will not find them while an embedding will — and the vectors are already on disk, which makes this cheaper here than it would be anywhere else. Collapse before k, and at both the retrieve and retrieve_any seams, for the reason the shipped dedup already documents: fusion reintroduces copies either engine dropped, because the copies are distinct (tap, skill_md) keys and RRF has no reason to treat them as one. The existing quality prior carries over unchanged — rag.source_rank orders on the user's curated flag first and shipped confidence second, and choosing among near-identical copies is the same question as choosing among identical ones: where should the user install from. Not to be confused with #629, which deduplicated vector storage (one row per distinct embedding, 39.7% repeats reclaimed). That is a disk-size fix beneath the index and changes no ranking; this is about which rows reach the user's screen. What shipped, and what did not. rag.collapse_near_duplicate_hits is the same "keep the earliest rank slot, promote a better source" contract as dedupe_by_content, run over cosine similarity of the entries' first-chunk embeddings (dense.entry_vectors, an index probe through chunks_entry on a quantized store) instead of a body hash, at the retrieve_any seam before k is applied. It is covered by unit tests down to the arithmetic (_cosine's dimension- mismatch and zero-vector guards), the clustering contract (rank order, quality-prior promotion, limit-after-collapse), the dense.entry_vectors lookup against a real quantized sqlite-vec store, and the retrieve_any/boost search --collapse-near-duplicates wiring in both directions (on and off). It ships opt-in and off by default — retrieve_any(..., collapse_near_duplicates=True) or boost search --collapse-near-duplicates — rather than replacing dedupe_by_content's output on the default path. Two things this card asks for are still open, and both need a real embedding backend (a built dense index, over a real multi-tap corpus) that the environment this was implemented in cannot reach — no network path to an embeddings provider or to the local ONNX model download, confirmed rather than assumed: huggingface.co and pypi.org both refuse at the network policy layer. First, the safety proof this card itself demands before defaulting the mechanism on — “over a real corpus, at the chosen threshold, count clusters spanning more than one meaning” — has not been run; NEAR_DUPLICATE_THRESHOLD = 0.97 is a starting point, not a validated floor. Second, re-measuring the exa search case (and per-query maxima generally) against the fix needs that same corpus and index. Whoever runs that measurement should flip the CLI flag's default, fold the corpus count into this card's evidence, and only then consider this shipped. The bound has now been measured, and it says the acceptance test in this card is the wrong one. scripts/measure_near_duplicate_bound.py runs the count this card asks for against the pinned 20-repo eval corpus (10,152 entries, 104,271 chunks, BAAI/bge-small-en-v1.5 at 384-d). Those entries reduce to 5,714 distinct chunk-0 vectors — 44% of entries already share a chunk-0 embedding byte for byte — and at NEAR_DUPLICATE_THRESHOLD = 0.97, 162 pairs clear the threshold and 56 clusters span more than one name. Sweeping the threshold moves that number but never to zero: 0.96 → 91, 0.97 → 56, 0.98 → 28, 0.99 → 13, 0.995 → 8, 0.999 → 4. Four of those 56 are not the threshold's doing at all. They are clusters of a single vector shared by several names, so they cluster at any threshold, which is why the sweep bottoms out at 4 rather than 0. The largest is the same at every threshold and is worth naming: 28 differently-named agents from one tap (affaan-m/ECC — architect, code-reviewer, chief-of-staff, database-reviewer, e2e-runner, …) whose chunk 0 is the same Spanish preamble (No cambiar rol, persona ni identidad…) in every file. Chunk 0 is name + description + opening of body, and where a registry opens every file with identical boilerplate, the name does not move the vector enough to separate them. A floor exists that no threshold can reach under, so “count must be zero” was never achievable. Worse for the test: most of the other 52 are the feature working. Hand-classifying all 56 at 0.97, roughly two-thirds are genuinely one skill under two names — twelve are pure hyphen-versus-underscore renderings of one integration (zoho-mail / zoho_mail, google_maps / google-maps, anthropic_administrator / anthropic-administrator), and the rest are suffix variants of one document (tdd / tdd-guide, rust-review / rust-reviewer, testing-patterns / code-showcase-testing-patterns). Collapsing those is precisely what this card exists to do. A metric that counts them as violations would reject every threshold that works. The dangerous merges have a shape, and this card already named it. The ~20 clusters that are real false merges are dominated by near-miss brand names: coinmarketcal with coinmarketcap, bugbug with bugsnag, parsehub with parseur, linkhut with linkup, mx-technologies with mx-toolbox, salesforce-marketing-cloud with salesforce-service-cloud. These are distinct products whose descriptions are boilerplate around a swapped word. That is the core/typosquat.py confusion shape this card's opening paragraph pointed at, arrived at independently from the other end: the guard this needs is not a tighter cosine floor but a name-confusability veto — refuse to collapse two entries whose names are a confusable edit apart, however close their vectors sit. So the default stays off, for a better-supported reason than before. The measurement does not say 0.97 is too loose; it says similarity alone cannot separate tdd/tdd-guide (collapse) from coinmarketcal/coinmarketcap (never collapse), because both pairs sit in the same cosine band. Flipping the default needs the confusability veto first, and a re-count with it applied. And this bound is space-specific: it was measured in bge-small 384-d, while a keyed production install is voyage-4 at 1024-d. Cosine thresholds do not transfer between embedding spaces — rerun the script against each space before trusting a number in it.

Complexity M Impact High Wow ★★★★ shipped opt-in (#639); bound measured (#645) and it refutes the zero-clusters test -- default-on now waits on a name-confusability veto
ShippedQuality · Retrieval eval

The monthly corpus refresh rewrote the pins and the baseline but left every documented number stale — taps.txt now contradicts its own header, and nothing checks it

Write-up · corpus-refresh-updates-data-not-prose.md

Complexity S Impact Low Wow ★★ .github/workflows/eval-corpus-refresh.yml regenerates taps.txt's data rows and baseli…
ShippedOnboarding · Bug

With a corrupt config.json, doctor reports "no registries tapped" and verdicts "● ready to set up" exit 0 while search is dead; heal says "nothing to heal"

Write-up · corrupt-config-reads-as-ready-to-set-up.md

Complexity M Impact Med Wow ★★★ Doctor has no config-integrity check. When ~/.boost/config.json fails to parse, confi…
ShippedQuality · Retrieval eval

BOOST_NO_EMBED has no state in the reason ladder: doctor calls a deliberate kill switch a degraded fault (exit 1) and hands advice that is a measured no-op in both branches

Write-up · dense-kill-switch-has-no-reason-state.md

Complexity M Impact Med Wow ★★★ embed.enabled() short-circuits provider() to None (embed.py:134), so BOOST_NO_EMBED=1…
PlannedQuality · Retrieval eval

fix_hint's no-key guard has been unreachable since the day it was written; a missing API key now prescribes the full re-embed the guard exists to prevent

Measured. On a complete [rag] extra with a voyage-4-built vector store and no API key exported, dense.status() returns reason='provider-changed' (not no-key), so all three status-passing surfaces print rebuild it: \boost reindex --dense --force\` and boost doctor exits 1 — measured verbatim, including doctor's "live key is local; searches are using BM25" line — while the guard written to prevent exactly that (21f28223, #444) fires only under BOOST_NO_EMBED=1` or a partial install, both confirmed by direct probe. Reproduce it. cd <repo>
export HOME=$TMPDIR/verify-dense-f1; export BOOST_HOME=$HOME/.boost; mkdir -p "$HOME"
unset VOYAGE_API_KEY OPENAI_API_KEY BOOST_NO_EMBED
python3 tests/make_fixture.py $TMPDIR/verify-dense-f1-fix >/dev/null
./boost tap $TMPDIR/verify-dense-f1-fix >/dev/null
# NOTE: ./boost execs system python3, which has no sqlite_vec, so it always says
# "no-backend". Every dense command below MUST run under .venv.
.venv/bin/python - <<'PY'
import hashlib, sys; sys.path.insert(0, ".")
from boost_cli.core import catalog, dense, embed
DIM = 1024
def fake(texts, input_type=None, timeout=60): # no network, no API spend
out = []
for t in texts:
h = hashlib.sha256(t.encode()).digest()
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. The defect is real and reproduced verbatim, but five stated details are wrong: 1. SOURCE ATTRIBUTION OF THE COST FIGURE. why_it_matters says "which CLAUDE.md prices at ~1.2 s per chunk on CPU". grep -n 'per chunk\|1\.2 s' CLAUDE.md returns nothing. The figure is real but lives in the roadmap: docs/roadmap/items/keyless-semantic-search-for-everyone.md:298 ("4,431 s — 74 minutes, about 1.2 s per chunk") and :308, plus the-shard-job-that-could-not-finish.md:17 and keyless-dense-tier-local-static-embeddings.md:75. Cite the roadmap, not CLAUDE.md. 2. quality.py:769 is wrong -> the fix_hint call is boost_cli/commands/quality.py:780 (fix = dense.fix_hint(st["reason"], st)). Line 769 sits inside the unrelated search-quantization warning. 3. pyproject.toml:55-57 is wrong -> the three pins are at lines 56, 67 and 68 (sqlite-vec>=0.1.6, onnxruntime>=1.17, tokenizers>=0.15), split by a 10-line comment justifying the fastembed rejection. They are in one extra, which is the load-bearing part, but the range is not 55-57. 4. THE CAUSAL-ORDER COMMIT PAIR NAMES THE WRONG COMMIT FOR THE GUARD. 86163e03 (2026-07-31, #364) created test_dense_fix_hint.py and the _FIX table, but not the guard. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. PRECONDITION — state it on the card so nobody reads it as unconditional. The misroute needs (a) a store built with an API provider, (b) the complete [rag] extra so provider() falls through to local, and (c) the key absent from the *process* environment. #444's own commit message documents that shape as recurring in the wild: "the shell exports one; the spawned server does not inherit it." THE REAL MACHINE IS NOT CURRENTLY IN THE BUG STATE. ~/.boost/cache/rag_vectors.sqlite reads provider="local", model="BAAI/bge-small-en-v1.5", dim=384 — so provider() matches built_provider and its reason is None. The 645,592-chunk / 1.20 GB / 440-tap figure is cost-if-the-hint-is-followed, not present harm. Do not let the card imply the machine is broken today. (The 750,416-chunk voyage-4 store in the #444 message and the test docstring is a past state of the same machine.) THE MISDIAGNOSIS WAS REFUTABLE FROM THE SAME FILE. The _FIX comment — "This reason means 'no key AND no local backend', which in practice is a partial install or BOOST_NO_EMBED" — was already in dense.py from 86163e03 (07-31) when #444 (08-03) added a guard whose docstring asserts the opposite: "an unfinished install with no store and a complete install whose key merely went missing both land here." Two claims about no-key, 30 lines apart, that contradict each other. Why it is worth doing. A user who opens a new shell without exporting their key is told by doctor, search and the MCP server to re-embed their whole store. On the real machine in this repo that is 645,592 chunks / 1.20 GB, which CLAUDE.md prices at ~1.2 s per chunk on CPU — hours of compute, or a real API bill — to fix a problem whose actual remedy is one export. The repo already decided this was unacceptable and wrote six tests plus a paragraph of docstring to prevent it; the guard has simply never been able to fire. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity L Impact High Wow ★★★★ dense.status() checks prov is None before it looks at the store (dense.py:618-621), b…
ShippedQuality · Retrieval eval

status() has no state for "ready but the embedder does not work": doctor green-ticks a tier that never ran, the search hint is suppressed, and every search re-pays the failed model fetch

Write-up · dense-ready-but-embedder-cannot-run.md

Complexity M Impact Med Wow ★★★ localembed.available() (localembed.py:85) tests whether onnxruntime and tokenizers *i…
ShippedUX · Bug

out.err's multi-line hint is coloured as one span, so line 1 ends with no RESET and lines 2+ carry no start code

Write-up · err-multiline-hint-colours-across-the-newline.md

Complexity S Impact Low Wow ★★ err() joins a multi-line hint's lines and then wraps the whole joined string in one c…
PlannedQuality · Retrieval eval

The eval corpus's size and its concentration ceiling are counted with len(scan_dir) — the measure measure_registry.py exists to say is wrong — so 44.7% of the gate's corpus is vendored …

Measured. Two tools in this repo, run on the same clone at the same pinned SHA, give item counts 3.16x apart — scripts/measure_registry.py says est_items=2100 while scripts/eval_corpus.py --ensure records 6634 entries — and the tool that is right is the one the eval gate does not use: measure_registry.py's own docstring (lines 6-11) states that len(catalog.scan_dir(repo)) is not the measurement, "the same rule the eval gate's ranked list uses". eval_corpus.py:299 is still counts[repo] = len(entries), and MAX_SHARE = 0.65 (line 113) ratchets on that number inside CI's required lint job — so 976 extra vendored copies from one stranger's repository, containing zero new content, turn every open pull request red. Reproduce it. cd <repo>
export HOME=$TMPDIR/audit-corpus-verify && mkdir -p "$HOME" && export BOOST_HOME=$HOME/.boost
.venv/bin/python scripts/eval_corpus.py --ensure # -> "corpus: 10731 entries"; "concentration: ... 61.8%"
.venv/bin/python scripts/measure_registry.py "$BOOST_HOME/repos/sickn33__antigravity-awesome-skills" # -> est_items=2100
.venv/bin/python - <<'PY'
import json, pathlib, collections, os
home=pathlib.Path(os.environ["BOOST_HOME"]); ents=[]
for f in sorted((home/"cache").glob("*.json")):
if f.stem.startswith("rag_"): continue
d=json.loads(f.read_text())
for e in d["skills"]: e["_tap"]=d["tap"]; ents.append(e)
alld={e["content"] for e in ents}
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. 1. MATERIALLY WRONG — "The vendoring is 60 plugins/agentic-bundle-*/ directories". There are 58 agentic-bundle-* dirs (60 plugins/* dirs total), and they hold only 449 of the 6,634 entries (6.8%; median 8 entries each). The 3.13x inflation is two FULL CATALOG MIRRORS: plugins/agentic-awesome-skills-claude (2,049 entries) and plugins/agentic-awesome-skills (2,027) = 4,076 entries, digest-identical to skills/ (2,107 entries, all distinct). This reframes the ceiling scenario: "one more bundle render... adds ~2,117 entries" is impossible — a bundle is ~8-10 entries. The correct unit is "one more per-agent mirror", ~2,049 entries -> 67.9%, still over 65%. Sharper still: only 976 extra vendored sickn33 entries are needed to cross the ceiling (< half a mirror). The conclusion survives; the unit does not — and the correct unit is exactly the pattern the est-items card already documents. 2. "Read all 318 titles from grep -h '^title:' docs/roadmap/items/*.md" — there are 432 items / 432 title lines today (verified identical to origin/main except one unrelated file). The not-carded sweep was run over a smaller set than exists. 3. "Every one of them... none measures distinct content" is wrong. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. SCOPE OF MY REPRO — what I actually settled, and what I did not. - I materialised the corpus at the CURRENT taps.txt pins in my own disposable HOME and reproduced every headline number independently. The pre-refresh figures (10,152 / 5,790 / 62.1% / 34.5% / 3.16x) I took from the SHARED read-only eval-home at $TMPDIR/eval-home, which is still at the older pins — a read-only census, no writes. - The repo checkout was on a peer session's branch loop/missing-json, not main. I diffed every file this finding touches (scripts/eval_corpus.py, tests/eval/taps.txt, boost_cli/core/rag.py, .github/workflows/eval-corpus-refresh.yml, all of docs/roadmap/items/) against origin/main: identical except one unrelated roadmap item. The verification holds against main (origin/main af4bdbfd). - I could NOT isolate the causal mechanism of the 22 rank changes. The finding asserts "BM25 statistics (N and document frequency), not slot consumption". My data is CONSISTENT with that — dedupe_by_content does run over the full pool before k (rag.py:1053, confirmed), and recall@k is bit-identical while the rank-sensitive metrics move — but I did not separate idf from avgdl, nor rule out that a different byte-identical copy survives dedupe and carries a different grade_key. Write it as "consistent with", not "confirmed". WHY HIGH RATHER THAN MEDIUM. The metric movement alone cannot fail anything: hit@1 +0.011, MRR +0.010, nDCG +0.008 against 0.06-0.10 of headroom over the floors. The severity is the ratchet's teeth, which the finder never traced. Why it is worth doing. MAX_SHARE is the only shipped guard against the gate's corpus becoming one publisher's house style, and it is measured on a quantity a third party can inflate by ~3x without publishing a single new skill — so it can fire on a repo contributing a third of the content, and cannot fire on a repo that dominates the content without vendoring. The same raw count is the headline everywhere (10,152 entries / 62%) and is what the monthly refresh PR reports as growth, overstating real content growth 4x. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity L Impact High Wow ★★★★ scripts/eval_corpus.py counts a repo's contribution as len(entries) from a raw scan (…
PlannedQuality · Retrieval eval

The corpus's 65.6% duplication is 99.93% inside a single tap, so the required gate never once exercises the cross-tap trust ordering dedup exists for — 0 swaps in 264,735 comparisons

Measured. Over the 91 required golden queries on the current pins, 264,544 of 264,735 collapse comparisons inside rag.dedupe_by_content (99.93%) were between two copies in the SAME tap and the source-preference branch executed 0 times; and forcing it to fire — marking one tap curated produces 191 swaps — leaves recall@k / hit@1 / MRR / nDCG@k at 0.8407 / 0.4835 / 0.6065 / 0.6552 both before and after, with all 91 graded ranked-key lists identical. Reproduce it. cd <repo>
export HOME=$TMPDIR/audit-corpus-verify && export BOOST_HOME=$HOME/.boost # after eval_corpus.py --ensure (see other finding)
.venv/bin/python - <<'PY'
import sys, json
sys.path.insert(0,"."); sys.path.insert(0,"scripts")
from boost_cli.core import rag
from boost_cli.core.rag import source_rank
stats={"collapses":0,"same_tap":0,"diff_tap":0,"swaps":0,"ties":0}
def patched(hits, limit):
best={}; out=[]
for hit in hits:
d=hit.get("content")
if not d: out.append(hit); continue
s=best.get(d)
if s is None: best[d]=len(out); out.append(hit); continue
stats["collapses"]+=1; kept=out[s]
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. 1. THE REMEDY CLAIM IS WRONG, and this is the correction that matters. The finding's why_it_matters says "adding a single mirror registry to taps.txt would be enough to make it visible." It would not. eval_retrieval.grade_key (scripts/eval_retrieval.py:197-209) keys every ranked slot on the content digest (body:<digest>), the exemplar class (cls:...), or the entry name — and a swap only ever replaces hit["entry"] INSIDE a content cluster, where the digest is identical by construction and, because catalog._content_digest hashes name + description + body, so is the name. Every return branch of grade_key is therefore invariant under a swap; the nohash:tap::skill_md branch at :209 is unreachable for a swapped hit because dedupe never collapses a hit with no digest. Proven, not argued: marking composio-community/awesome-codex-skills curated on the same pinned corpus fires 191 swaps (every cross-tap collapse becomes a swap — the composio copy arrives second in all 191) and leaves recall@k / hit@1 / MRR / nDCG@k at 0.840659 / 0.483516 / 0.606517 / 0.655233 before AND after, with all 91 graded ranked-key lists byte-identical. So a trust-ordering regression is invisible to the required gate BY CONSTRUCTION, regardless of corpus shape — the gap is that the harness never grades on source, not that the corpus lacks mirrors. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. SCOPE OF MY REPRO — read this before re-verifying. The finder's numbers reproduce ONLY on the CURRENT pins (10,731 entries). The shared read-only corpus at $TMPDIR/eval-home is PRE-REFRESH: 10 of its 20 clones are not at the SHAs in tests/eval/taps.txt (anthropics/skills, NeoLabHQ, LessUp, affaan-m/ECC, first-fluke, langchain-ai, minio, OneWave-AI, sickn33, anthropics/claude-agent-sdk-python), and it holds 10,152 entries. There the same instrumentation gives 1,986 clusters / 6,348 entries (62.5%) / 240,646 collapses / 240,455 same_tap / 191 diff_tap / 0 swaps. I materialised the current pins into a private copy (eval_corpus.py --ensure, network fetch works despite a harmless failed to store: 100001 commit-graph warning) to get the finder's exact figures. Anyone re-checking against $TMPDIR/eval-home will get the smaller set and should not read that as refuting the finding — the qualitative result (5 cross-tap clusters, all high/high, 0 swaps, ~99.9% same-tap) is identical on both. DOC TRAP, not this finding's fault: tests/eval/taps.txt's own header prose says "10,152 entries" and "sickn33 … is 6,309", and CLAUDE.md repeats 10,152 — but the file's own per-repo rows sum to 10,731 with sickn33 at 6,634. The pins were moved in commit cbc0a58b ("test(eval): refresh the pinned retrieval corpus") and the header prose was not updated. A card author quoting corpus size must take the row sum, not the header. NOT ALREADY CARDED, but read the existing card first. Why it is worth doing. The required corpus contains essentially none of the duplicate shape that dominates a real install. A user's duplicates arrive as mirror registries republishing each other's skills across taps, which is what source_rank decides between and what determines where a user is told to install from; the gate's duplicates are one publisher re-vendoring itself into 60 plugin bundles, where every candidate has the same tap and the tie-break is a no-op. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity S Impact Low Wow ★★ rag.dedupe_by_content is documented against a real install where "registries mirror e…
PlannedQuality · Retrieval eval

The 2026-09-01 pin refresh moved the corpus and re-baselined it, but nothing re-derives the floors — CLAUDE.md's "~10% under measured" is now 7.2%–17.3%

Measured. The required gate's recall floor has 5.52 queries of headroom out of 91 ((0.8407 - 0.78) x 91), against the 6.55 that CLAUDE.md's published 0.852 implies — one golden query of margin the documentation says exists and does not, because commit cbc0a58b re-baselined the corpus to 10,731 entries and left CLAUDE.md, taps.txt and eval_corpus.py all stating 10,152 / 0.852. Reproduce it. cd <repo>
# 1. the refresh commit moved pins + baseline and nothing else
git show cbc0a58b --stat --format='%h %ad %s' --date=short
git show cbc0a58b -- tests/eval/baseline.json | grep -E '^[-+].*(recall|hit@1|MRR|nDCG)'
# 2. what taps.txt records NOW vs what the docs claim
awk '!/^#/ && NF>=3 {s+=$3; n++} END {print "rows:", n, " recorded total entries:", s}' tests/eval/taps.txt
grep -n '10,152\|0\.852' CLAUDE.md tests/eval/taps.txt scripts/eval_corpus.py
# 3. recompute the floor gaps against the committed post-refresh baseline
python3 -c "
import json
b=json.load(open('tests/eval/baseline.json'))['sets']['golden.jsonl@a0617183f8c9']['engines']['BM25 full-content']
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. 1. CAUSAL FRAMING IS WRONG (the headline correction). The title and claim say the 2026-09-01 pin refresh is what made "~10% under" untrue. It was untrue the day it was written. git log -S'~10% under' traces the sentence to 170d52c0 (2026-07-31), whose own commit message says the floors were "re-derived against the 20-tap numbers at the same ~10% relative headroom" against measured 0.863 / 0.473 / 0.607 / 0.662 — at which point the gaps were 9.6% / 15.4% / 14.3% / 12.4%, a 1.60x spread. The refresh WIDENED a spread (1.60x -> 1.82x -> 2.39x); it did not create one. Correct statement: "the floors were never uniformly ~10% under, and the refresh widened the spread from 1.82x to 2.39x." 2. "not the 6.60 the documented number implies" — 6.60 matches nothing. CLAUDE.md's published 0.852 gives (0.852-0.78) x 91 = 6.55 queries; the unrounded pre-refresh baseline 0.8406... gives 6.52. Use 6.55. (The optimism is therefore ~1.03 queries, not 1.1.) 3. "2.4x spread" -> 2.39x (rounding, harmless, but state it as 2.39x on a card). 4. DIRECTION OF DRIFT IS NOT UNIFORM. "every future passing refresh decays the calibration further" is true only of the SPREAD. This refresh moved two floors tighter (recall 8.4%->7.2%, nDCG 11.8%->11.5%) and two LOOSER (hit@1 15.3%->17.3%, MRR 14.0%->14.3%). A card must not say "the floors tightened." 5. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. SCOPE THE REMEDY CAREFULLY — the finding's phrase "nothing re-derives the floors" invites exactly the fix 170d52c0 warns against: auto-moving floors on every refresh ("Lowering a threshold deserves suspicion... a floor calibrated on an unrepresentative corpus measures the corpus"). The repo's design is deliberately fixed absolute floors with a relaxed regression-vs-baseline. The CONFIRMED defect is narrower and should be carded as such: the measured values and the "~10%" characterisation are stated in seven places across four files (CLAUDE.md:67,84; tests/eval/taps.txt:25,43; scripts/eval_corpus.py:30,108 and the runtime-printed operator message at :331-334), nothing re-states or guards them after a PASSING refresh, and no test asserts any of them. SCOPE LIMIT OF MY REPRO #1: I did not re-measure 0.8407 / 0.4835 / 0.6065 / 0.6552. Those are the committed baseline written by cbc0a58b — which is also the finder's only source. Verifying them empirically needs a re-tap at the new pins, which is forbidden here (shared read-only corpus) and needs network. Every derived percentage in this verification inherits that. SCOPE LIMIT #2: the shared $TMPDIR/eval-home is materialised at the PRE-refresh pins — live total 10,152 vs taps.txt's recorded 10,731, with the same 10 taps PIN-DIFFERS. make eval against it would trip eval_corpus.py's CORPUS DRIFT check, not run the gate. Why it is worth doing. The "~10% under" statement is the entire published justification for where the four floors sit — CLAUDE.md calls it "loose enough that upstream drift can't flake the build, tight enough to catch a collapse." That is a claim about a margin, and the margin now differs 2.4x across the four metrics with recall the tightest at 5.52 queries out of 91. Because the monthly refresh re-baselines but never re-derives or re-states the floors, and prompts a human only when the refreshed corpus FAILS, every future passing refresh decays the calibration further with no signal at all. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ CLAUDE.md states the gate's four floors sit "~10% under" measured values of 0.852 / 0…
ShippedQuality · Retrieval eval

make eval scores a corpus with every SKILL.md body missing, reports all four floors PASS, and scores HIGHER than the real corpus

Write-up · eval-gate-scores-a-body-less-corpus.md

Complexity M Impact Med Wow ★★★ The Tier 1 required gate has no check that the corpus it scores actually contains the…
ShippedUX · Bug

boost explain's heuristic fallback prints every heading in the file — 541 lines for one skill — while the sibling list in the same function caps at 12

Write-up · explain-outline-has-no-cap.md

Complexity M Impact Med Wow ★★★ cmd_explain asks the model for "4-6 sentences, no markdown" (info.py:800-803), but th…
ShippedQuality · Retrieval eval

The CI-vs-Makefile floor-parity test compares only the floor VALUES, so changing -k in ci.yml turns a PASS into a FAIL with the test still green

Write-up · gate-parity-test-ignores-k-and-golden.md

Complexity M Impact Med Wow ★★★ the parity test now parses the whole eval_retrieval.py call in the Makefile, ci.yml and the refresh workflow with the script's own parser; a -k 5 in ci.yml alone fails it
PlannedQuality · Retrieval eval

The exemplar mechanism was applied to the ungated set only: golden.jsonl is 0/91 pinned, and 10 of its 43 hit@1 credits are on names the metric cannot adjudicate

Measured. On the 10,731-entry corpus that committed taps.txt materializes today, tests/eval/golden.jsonl — the only set the required eval gate floors — is 0/91 exemplar-pinned while the ungated golden-natural.jsonl is 50/50; 10 of its 44 hit@1 credits (22.7%) are awarded on a name that resolves to more than one distinct body, and the hit@1 floor's entire headroom is 7.60 queries, smaller than the 10 credits the metric cannot adjudicate. Reproduce it. cd <repo>
export BOOST_HOME=$TMPDIR/eval-home
# 0. confirm which corpus you are on (expect 10152)
.venv/bin/python -c "import sys;sys.path.insert(0,'.');from boost_cli.core import catalog;print(len(catalog.all_entries()))"; echo EXIT=$?
# 1. exemplar counts per set
python3 -c "
import json
for f in ['tests/eval/golden.jsonl','tests/eval/golden-natural.jsonl']:
r=[json.loads(l) for l in open(f) if l.strip() and not l.startswith('#')]
print(f,'rows:',len(r),'with exemplar:',sum(1 for x in r if x.get('exemplar')))"; echo EXIT=$?
# 2. the harness's own list of undecided gated rows
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. The defect is real and the spine is exact, but five figures are stale by one corpus refresh. The finding measured on the 10,152-entry corpus CLAUDE.md documents; committed tests/eval/taps.txt moved on 2026-09-01 (cbc0a58b) and now materializes 10,731 entries, which is what make eval/CI builds today. Corrected values, all measured by me on that corpus: - hit@1 credits: 43/91 = 0.473 -> 44/91 = 0.484 - un-adjudicated share of credits: 23.3% -> 22.7% (the count stays 10) - floor headroom: 6.60 queries -> 7.60 queries - worst-case bound stripping all 10: 33/91 = 0.363 -> 34/91 = 0.374 (still below the 0.400 floor) - BM25 four-metric line: 0.852 / 0.473 / 0.605 / 0.657 -> 0.841 / 0.484 / 0.607 / 0.655 The finding's numbers are correct for the corpus it names and it disclosed the provenance honestly; they are nonetheless the wrong numbers to print about "the set the required eval gate floors", because the gate no longer builds that corpus. A card must publish the 10,731 figures and name the corpus. Nothing else in the finding is wrong. 0/91 vs 50/50, the 27 undecided rows / 62 candidate bodies, the same 10 names and 10 queries, the skill-creator worked example, the not-carded check, and the "realistic shift is 1-2 queries, the floor still passes" bound all hold on BOTH corpora. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. 1. WHAT IS CORPUS-INDEPENDENT (identical on 10,152 and 10,731): exemplar counts 0/91 and 50/50; 27 undecided worksheet rows over 62 candidate bodies; the same 10 ambiguous-credit names and the same 10 queries; the skill-creator example. Only the credit count and the two ratios moved. Build the card on the invariant half and quote the 10,731 figures for the rest. 2. TRAP FOR THE NEXT VERIFIER: the shared read-only $TMPDIR/eval-home is a PRE-REFRESH corpus — its .eval-corpus-ready sentinel digest does not match sha256(tests/eval/taps.txt), and 10 of its 20 clones sit at a different commit. Anyone who measures only there reproduces the finding's old numbers verbatim and marks it CONFIRMED without noticing. Rebuilding at the committed pins in a disposable HOME took ~3 minutes and is what produced the correction. 3. DO NOT COPY Makefile:126, which still comments "over twenty it scores 0.863 / 0.473 / 0.607 / 0.662". That matches neither corpus (10,152 gives 0.852/0.473/0.605/0.657; 10,731 gives 0.841/0.484/0.607/0.655). CLAUDE.md:67's figures are right for the old corpus and now also stale. 4. SCOPE OF MY REPRO: I verified the BM25 engine only (--engines bm25), which is what the gate floors; I did not build a dense store (impossible here). I did not run make eval end to end, only its second command against a corpus I materialised with its first. 5. NOT A DEFECT IN THE HARNESS: exemplar_worksheet (scripts/eval_retrieval.py:219) and exemplar grading work correctly and fail loudly on a bad pin. Why it is worth doing. golden.jsonl is the set the required eval gate floors, so it is the only one that can block a merge. 23.3% of its hit@1 credits are awarded on a name that maps to several genuinely different skills, and the floor's whole margin (6.60 queries) is narrower than the un-adjudicated credit count (10). That does not mean retrieval is worse than reported — the realistic shift is 1-2 queries — it means the published margin cannot be read as precision about the intended skill, so anyone tuning blend weights or pool depth against this gate inherits an unquantified slack. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ CLAUDE.md presents exemplar pinning as a live migration — "relevance is still decided…
ShippedUX · Bug

boost info/deps on a not-installed rule or workflow widens the tap's sparse cone for a directory source_dir_for immediately rejects

Write-up · info-deps-materialize-a-dir-they-then-reject.md

Complexity M Impact Med Wow ★★★ cmd_info (info.py:482-486) and _skill_dir_for_deps (info.py:1058-1066) call store.sou…
ShippedInterop

boost_install's description enumerates four of the five enabled agent targets and omits Antigravity CLI, the fourth linking agent

Write-up · mcp-install-description-omits-antigravity.md

Complexity S Impact Low Wow ★★ boost_install's description says installs are "wired into every agent you have enable…
ShippedInterop

boost_search advertises "10-15 seconds" unconditionally; with no AI configured it is 0.013 s median and the rerank never runs

Write-up · mcp-search-cost-overstated-on-keyless-machines.md

Complexity M Impact Med Wow ★★★ boost_search's description and INSTRUCTIONS both state the cost as a flat "10-15 seco…
ShippedInterop

The skip list — the one bound on boost's triggers — ships in INSTRUCTIONS only, in zero of the seven tool descriptions

Write-up · mcp-skip-list-absent-from-tool-descriptions.md

Complexity M Impact Med Wow ★★★ boost's agent-facing surface duplicates six of its seven load-bearing elements from I…
ShippedUX · Bug

out.panel still fits its content to term_width(), so boost count | … clips the line to an assumed 80 columns

Write-up · panel-fits-data-to-an-assumed-80-in-a-pipe.md

Complexity S Impact Low Wow ★★ output.panel computes room = term_width() - 4 (output.py:527) and clips its content t…
ShippedOnboarding · Bug

quickstart silently discards the incompatible shard status, so a user with any API key is never told why zero vectors arrived

Write-up · quickstart-drops-incompatible-shard-status.md

Complexity M Impact Med Wow ★★★ shards.sync() returns status: "incompatible" with a fully-formed reason when the publ…
ShippedOnboarding · Bug

A local manifest read error is reported as "no published shards", and the BoostError's hint — the only actionable line — is discarded

Write-up · quickstart-manifest-error-drops-hint-and-misnames-cause.md

Complexity M Impact Med Wow ★★★ cmd_quickstart catches every BoostError from shards.fetch_manifest() and prints exc.m…
ShippedOnboarding · Bug

With every registry unreachable, quickstart prints "✓ indexed 0 items" and "✓ ready", exits 0 — and the command it recommends exits 1

Write-up · quickstart-says-ready-exit-0-after-every-tap-failed.md

Complexity L Impact High Wow ★★★★ cmd_quickstart warns per failed clone but never tracks failures: it unconditionally r…
PlannedOnboarding · Bug

The shard download is invisible both before and during: --catalog --dry-run never names the 1,604.8 MB, and the live fetch passes no progress callback and has no spinner

Measured. On a virgin HOME, boost quickstart --catalog --dry-run prints exactly two lines — "would tap 464 registries (459 pinned to a published shard's commit)" and "would build the keyword index, then import 459 shard(s)" — and never names the 1,604.8 MB (1,604,753,775 bytes) those same 459 manifest rows sum to, although the dry-run has already read the manifest that carries every row's bytes and shards._size_label() exists to format it. Reproduce it. cd <repo>
grep -n 'on_event' boost_cli/commands/quickstart.py # -> no matches
grep -n 'on_event' boost_cli/commands/discovery.py boost_cli/commands/pkg.py
sed -n '195,215p' boost_cli/commands/quickstart.py # Spinner at 195, bare sync at 209
curl -sSL -o $TMPDIR/mf.json https://github.com/jonnyeclectic/boost/releases/download/shards-latest/manifest.json
python3 -c "import json;d=json.load(open('$TMPDIR/mf.json'));print(len(d['shards']),'rows', '%.1f MB' % (sum(r.get('bytes',0) for r in d['shards'])/1e6))"
export HOME=$TMPDIR/audit-qs-f3 ; export BOOST_HOME=$HOME/.boost ; mkdir -p "$HOME"
env HOME="$HOME" BOOST_HOME="$BOOST_HOME" BOOST_SHARD_MANIFEST="file://$TMPDIR/mf.json" \
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. One stated detail is wrong: "shards.sync() … both other callers pass one." shards.sync has THREE call sites, and only ONE passes on_event: - boost_cli/commands/discovery.py:409 (sync) — passes on_event=None if args.as_json else _shard_event at :414. ✓ - boost_cli/commands/quickstart.py:209 (sync) — passes none, no Spinner. ✗ - boost_cli/commands/pkg.py:941 (_resync_vectors) — shards.sync(list(by_name), by_name, manifest=manifest), passes NO on_event and is inside no Spinner either. ✗ The finding's second citation, pkg.py:997, is the on_event= line of a shards.ingest( call that starts at pkg.py:993 — not a sync caller. Both quoted line numbers (414, 997) are accurate as lines; the framing "both other [sync] callers pass one" is not, and so is the implied "quickstart is the lone outlier". Everything else re-derived and correct to the digit: 459 manifest rows; 1,604,753,775 bytes = 1604.8 MB (1530.4 MiB); the five per-default byte counts and chunk counts verbatim; expo/skills and K-Dense-AI/scientific-agent-skills have NO ROW; 17,088,844 = 17.1 MB for the seven defaults; the two dry-run lines verbatim including "464 registries" / "459 shard(s)"; the --catalog help text verbatim. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. SCOPE LIMITS OF MY REPRO — a card must not overstate these: 1. The live silence was measured on the SEVEN DEFAULTS only, and the sandbox proxy truncates response bodies (every shard ended "failed verification" — bytes were received and hashed, then rejected). My 2.19 s gap ([3.93s] "indexed 962 items" -> [6.12s] first shard line) is a LOWER BOUND; a complete 17.1 MB fetch takes longer. The finder measured 2.53 s and 1.98 s — same shape. 2. --catalog was NEVER exercised live by me (it would pull 1.6 GB, and the proxy truncates). 1,604.8 MB is manifest arithmetic — the sum of bytes over all 459 rows — not a stopwatched transfer. It is the right number for a virgin HOME, where no row is "current" and all 459 download, but write it as a sum, not as a measurement. 3. The dry-run repro requires dense.have_backend() True (the repo .venv has [rag]). Without the extra the manifest is never fetched and the dry-run prints a different "0 because …" line — a different defect, already carded in audit-quickstart-findings.md. 4. The live run needs the REAL manifest URL. With BOOST_SHARD_MANIFEST=file://… the host check refuses every shard ("shard URL … is not on the manifest's own host") and no download is attempted, so that override is fine for the dry-run repro but useless for timing the live path. FIX SCOPE IS WIDER THAN THE FINDING SAYS: because pkg.py:941 (_resync_vectors) is also a silent shards.sync caller, a fix that only touches quickstart leaves two of three sync sites inconsistent. Why it is worth doing. --catalog is the one expensive, mostly-irreversible decision quickstart offers a brand-new user, and both surfaces that exist to describe it — --help and --dry-run — omit its dominant cost. A user on a laptop tether or a metered link is asked to approve "459 shard(s)" with no way to learn that means 1.6 GB, and once it starts there is no spinner, no per-shard line and no byte counter to tell them how far along it is or that anything is happening at all. Every ingredient of the fix is already in the file: pass _shard_event-style on_event, and sum row['bytes'] through the existing _size_label in the dry-run line. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ shards.sync() takes an on_event progress callback and both other callers pass one; qu…
ShippedQuality · Docs

README's "81 commands" table enumerates only 80 — the missing one is quickstart, the README's own first command

Write-up · readme-81-command-table-lists-80-omits-quickstart.md

Complexity S Impact Low Wow ★★ README.md:389 heads a table "## 81 commands, organized into 8 groups", but the eight …
ShippedUX · Bug

boost recommend sizes the description cell per row from that row's because: text, so neither column lines up

Write-up · recommend-recomputes-its-columns-per-row.md

Complexity S Impact Low Wow ★★ cmd_recommend hand-rolls its rows and computes desc_w = max(cols - 2 - width - 2 - (l…
PlannedQuality · Retrieval eval

The corpus refresh re-baselines only golden.jsonl, so golden-natural.jsonl's baseline silently describes a corpus that no longer exists

Measured. On the corpus tests/eval/taps.txt pins today (10,731 entries, 20 taps), eval_retrieval.py --golden tests/eval/golden-natural.jsonl -k 10 exits 1 and prints "REGRESSION vs baseline: catalog.search recall@k: 0.080 -> 0.060 (-0.020)" — while the keyword set on that identical corpus exits 0 and reproduces its baseline to four decimals (0.841/0.484/0.607/0.655), because the September refresh commit cbc0a58b moved golden.jsonl's baseline and left golden-natural.jsonl's describing the 10,152-entry corpus it replaced. Reproduce it. cd <repo>
# 1. the refresh re-baselines with no --golden, and the default is golden.jsonl
sed -n '112,119p' .github/workflows/eval-corpus-refresh.yml
grep -n 'DEFAULT_GOLDEN\|--golden' scripts/eval_retrieval.py
# 2. the refresh commit changed ONLY the golden.jsonl set
git show cbc0a58b -- tests/eval/baseline.json | grep -nE 'golden|^[-+] *"(recall|hit)'
# 3. the natural baseline still holds pre-refresh numbers
python3 -c "
import json;s=json.load(open('tests/eval/baseline.json'))['sets']
for k,v in s.items(): print(k, v['engines']['BM25 full-content'])"
# 4. and they reproduce exactly on the PRE-refresh corpus
export BOOST_HOME=$TMPDIR/eval-home
… What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. The mechanism and every load-bearing measurement are correct. Four stated details are wrong: 1. LINE NUMBER. if drop > eps: is at scripts/eval_retrieval.py:545, not 546. (The other three cited lines are exact: DEFAULT_GOLDEN at 66, --golden default at 656, --regression-eps default 0.02 at 668, and the workflow's bare --save-baseline at eval-corpus-refresh.yml:119.) 2. ROADMAP FILENAME. The finding's not_carded_check names nothing-refreshes-the-eval-corpus-pins; no such file exists. That string is the card's TITLE. The file is docs/roadmap/items/eval-corpus-pins-have-no-refresh-path.md (status: shipped, pr: 431). Its conclusion still stands — I read it, and its only baseline sentence is "regenerate baseline.json", singular, with nothing about which sets a refresh maintains. 3. TITLE COUNT. "Checked all 318 titles" — there are 432 item files (405 board: code, 27 board: design). I re-ran the not-carded check over all 432 and reached the same conclusion. 4. UNDERSTATED, NOT OVERSTATED — the important one. The finding frames the false regression as a future risk ("a 2-query shift (0.040) reports a confident REGRESSION"). It is not future. On the corpus the current taps.txt pins, the natural set ALREADY exits 1 today with REGRESSION vs baseline: catalog.search recall@k: 0.080 -> 0.060 (-0.020) — the drop computes to 0.020000000000000004, which clears eps=0.02. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. SCOPE OF MY REPRO — what it does and does not settle. - It settles the mechanism completely: I read the current workflow and argparse default, walked every commit that ever touched baseline.json, and confirmed cbc0a58b is the first to move one set of two. - It settles the consequence empirically, which the finding did not: I built the post-refresh corpus (network fetch of the 8 moved pins onto copies of the shared clones, in HOME=$TMPDIR/verify-5, since deleted) and got a real exit-1 false regression on the natural set with a clean exit-0 control on the keyword set. That pair is the card's evidence — one command, two sets, same corpus, opposite results. - Measurement noise seen: BM25 MRR on the natural set came out 0.235 on the --build run and 0.237 on the next run against the same index. Do not quote the natural set's post-refresh MRR to three decimals in a card. recall@k (0.360), hit@1 (0.160) and the catalog.search regression line were stable across both runs. BLAST RADIUS — do not overstate it in the card. The natural set is NOT wired into any gate. make eval and CI run golden.jsonl only, and both pass --regression-eps 1, so the required gate is untouched and stays green. grep for golden-natural across *.py/*.yml/Makefile/*.sh finds it only in tests/unit/test_eval_baseline.py and test_eval_grading.py (which test the harness, not the corpus) plus CLAUDE.md and roadmap prose. Why it is worth doing. CLAUDE.md documents baseline keying as the fix for exactly this class of bug: "Before that, running the natural-language set printed eight confident 'REGRESSION vs baseline' lines that were only the gap between two different question sets." The key was made query-set-aware but not corpus-aware, and the monthly refresh only maintains one of the two sets. The natural set is the only instrument the project has for measuring whether a user's plain-English question finds the right skill (it scores BM25 at recall 0.360 / hit@1 0.160, and 0.077 / 0.000 on its 13 workflow rows) — so it is the set that matters most for judging the keyless-dense work, and its reference point is now silently wrong. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ eval-corpus-refresh.yml:119 runs eval_retrieval.py --save-baseline -k 10 with no --go…
ShippedQuality · Docs

The two roadmap boards' install footers still advertise Python 3.9+, four minor versions under the real floor

Write-up · roadmap-boards-advertise-python-39.md

Complexity S Impact Low Wow ★★ docs/roadmap.html:17141 and docs/design-roadmap.html:582 both close with Install with…
ShippedUX · Bug

search caps the name column at 32 and the tap column at 20 at every terminal width, while the description column grows without limit

Write-up · search-caps-the-two-copyable-identifiers.md

Complexity M Impact Med Wow ★★★ output.search_layout (output.py:668-690) hard-caps name_w at 32 and tap_w at 20 and h…
ShippedUX · Bug

The relevance meter and its "one gradient moment" are constant on the default result page: 138/150 rows full bars, 150/150 the same colour

Write-up · search-relevance-meter-is-constant-on-default-page.md

Complexity M Impact Med Wow ★★★ format_search_row renders aurora(meter(frac), meter_hue(frac)) with frac = score / ma…
ShippedQuality · Docs

The contributor-onboarding gate tables state three wrong suite sizes, and README and CONTRIBUTING disagree with each other

Write-up · stale-gate-sizes-in-readme-and-contributing.md

Complexity S Impact Low Wow ★★ README.md:515 says make smoke is "176 checks" and CONTRIBUTING.md:123 says bash tests…
ShippedQuality · Retrieval eval

rag.surface's de-hyphenated name copy is justified by two claims that are both false, and its real effect — an undocumented 3x name / 2x description field weight — is guarded by a test …

Write-up · surface-dehyphenation-rationale-is-false.md

Complexity M Impact Med Wow ★★★ surface() (rag.py:200-208) indexes the name, a de-hyphenated copy of the name, and th…
ShippedQuality · Retrieval eval

Tier 3's false-call ceiling is unreachable at its own default N, and tolerates zero false calls at the N make eval-tools uses

Write-up · tier3-ceiling-unreachable-at-default-n.md

Complexity M Impact Med Wow ★★★ eval_tools.py judges the should-NOT-call ceiling against the Wilson UPPER bound, but …
ShippedOnboarding · Bug

An unreadable tap cache is invisible to doctor ("✓ 1 tap cloned & cached", exit 0) while search, browse and heal all exit 70 with a crash report

Write-up · unreadable-tap-cache-healthy-doctor-crashing-heal.md

Complexity M Impact Med Wow ★★★ fixed by #889 (the cache is replaced, not rewritten in place: search/browse/info/update/heal 70 → 0 for mode 400 and 000) and the #909 train (b1b1bca, eeb2c76: doctor and heal both name an unwritable cache dir; heal no longer claims a rebuild it did not do) — re-measured on a316c6b
ShippedOnboarding · Bug

"! agent dir ~/.cursor/skills is not writable" is the one doctor issue with no next action — heal has no path for it and the next install crashes at exit 70

Write-up · unwritable-agent-dir-has-no-remedy.md

Complexity M Impact Med Wow ★★★ Every other issue doctor raises names a command (boost heal, boost sync, boost update…
ShippedOnboarding · Bug

Nothing boost prints ever names an entry point: bare ./boost is byte-identical on a virgin machine and a working one, and the one command its failure-hints route you to is the only setup …

Write-up · bare-boost-never-names-an-entry-point.md

Complexity M Impact Med Wow ★★★ A newcomer's first screen is 103 lines of 81 commands that is byte-for-byte the same …
PlannedTech-debt

golden-natural.jsonl — the only fully exemplar-graded query set — is invoked by no make target and no workflow, so its numbers can only be produced by a human typing the command

Measured. Running the natural set today against the same 20-tap eval corpus prints "REGRESSION vs baseline: catalog.search recall@k: 0.080 -> 0.060 (-0.020)" (BM25 MRR 0.2447 -> 0.237, nDCG 0.2640 -> 0.259) — the snapshot has already drifted, and no Makefile target or workflow passes --golden, so nothing in the repo can ever emit that line. Reproduce it. cd <repo>
grep -n natural Makefile ; echo "makefile grep exit $?"
grep -rn natural .github/workflows/ ; echo "workflow grep exit $?"
grep -rn 'golden-natural' Makefile .github/workflows/ scripts/ tests/ | sed -n '1,20p'
sed -n '1,23p' tests/unit/test_eval_baseline.py
python3 -c "import json,hashlib,pathlib; b=json.load(open('tests/eval/baseline.json')); print('baseline keys:',list(b['sets'])); print('current digests:',[p+'@'+hashlib.sha256(pathlib.Path('tests/eval/'+p).read_bytes()).hexdigest()[:12] for p in ('golden.jsonl','golden-natural.jsonl')])" What adversarial verification corrected. This card's numbers are the re-measured ones, not the ones first reported. 1. BASELINE NUMBERS ARE WRONG. The claim says the golden-natural row carries "0.350 / 0.160 / 0.245 / 0.259". It does not. tests/eval/baseline.json's golden-natural.jsonl@0d91b0cd8e41 BM25 row is 0.360 / 0.160 / 0.2447 / 0.2640. The quoted 0.350/0.160/0.245/0.259 are the *pre-migration* ("name-graded (before)") figures at docs/roadmap/items/golden-set-grades-by-name-not-by-skill.md:125; the baseline holds the *post*-migration row from line 126-127 (0.360 / 0.160 / 0.245 / 0.264). The claim read the wrong line of the card. 2. "the only tests that touch it are explicitly synthetic" IS WRONG. tests/unit/test_eval_grading.py reads the REAL shipped file in three live assertions: test_the_shipped_set_has_nothing_left_to_decide (line 217-228, asserts len(rows) == 50 and no unpinned row) and class TestTheMigrationIsFinished (line 247+): test_every_row_pins_an_exemplar, test_every_exemplar_is_well_formed, test_no_exemplar_is_a_localised_copy. Only tests/unit/test_eval_baseline.py is synthetic. The defect survives the correction, because those tests grade the file's SHAPE (row count, exemplar presence and syntax) and never run retrieval — so they cannot move or invalidate a single number in the baseline row. The accurate statement is: automation pins the query set's shape, nothing re-measures its scores. 3. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. Scope and limits of my repro: - The structural half (no --golden in any Makefile target or workflow; --golden defaults to golden.jsonl at scripts/eval_retrieval.py:66/:656) is exhaustive over Makefile, .github/workflows/ and scripts/ — read, not inferred. - The drift measurement was run against a PRIVATE COPY of the shared 20-tap eval corpus (I copied cache/ + config.json into $TMPDIR/lens2-exemplar and never wrote to the shared home). I did not verify that this corpus is byte-identical to the one that produced the committed baseline row: Makefile:130-133 says the corpus "tracks upstream HEAD rather than pinned commits", while CLAUDE.md says every taps.txt row pins a SHA — the repo disagrees with itself there and I did not resolve it. So the -0.020 delta could be corpus movement rather than ranker movement. That distinction does not affect the finding: corpus movement is exactly the thing a re-run is supposed to report, and no re-run happens. - I did not run make eval, make check, or any workflow; I read their recipes. - I did not measure how long a make eval-natural would add (the eval corpus was already materialised for me, so I never paid the tap cost). - catalog.search's numbers here are near the floor of resolution (3 of 50 queries vs 4 of 50), so its -0.020 is one query; the BM25 MRR/nDCG drift is below the default 0.02 eps and would not have been flagged even if someone ran it. Why it is worth doing. CLAUDE.md documents the two-set design ("Baselines are keyed by query set (name@content-digest), so one file holds both golden.jsonl and golden-natural.jsonl without either overwriting the other") as if both sets are exercised. Only one is. The set that received 50 hand-made relevance judgments — the project's best available measurement of retrieval quality, and the one whose grading key actually identifies a skill — contributes to no gate and no scheduled monitor, so the investment decays silently. Adding a make eval-natural target (or a non-blocking scheduled run alongside eval-stats.yml) would cost one recipe and turn a frozen snapshot back into a signal. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CORRECTED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ The 50-row natural-language golden set is the one place where the project's content-c…
PlannedTech-debt

The exemplar migration never reached the query set the required gate runs: 0 of 91 rows, and 27 of them have an ambiguous target on the gate's own corpus

Measured. Both invocations of the merge-blocking retrieval gate — the Makefile eval target and ci.yml's "retrieval quality gate" step — run scripts/eval_retrieval.py with no --golden, so both take DEFAULT_GOLDEN = tests/eval/golden.jsonl; 0 of that file's 91 rows carry an exemplar, and on the gate's own 20-tap corpus 27 of the 91 rows have at least one target name that resolves to more than one content digest (21 of 77 distinct target names are ambiguous). Reproduce it. cd <repo>
python3 -c "import json; rows=[json.loads(l) for l in open('tests/eval/golden.jsonl') if l.strip() and not l.startswith('#')]; print(len(rows),'rows;',sum(1 for r in rows if r.get('exemplar')),'with exemplar')"
python3 -c "import json; rows=[json.loads(l) for l in open('tests/eval/golden-natural.jsonl') if l.strip() and not l.startswith('#')]; print(len(rows),'rows;',sum(1 for r in rows if r.get('exemplar')),'with exemplar')"
grep -n 'DEFAULT_GOLDEN' scripts/eval_retrieval.py
sed -n '134,139p' Makefile
sed -n '269,279p' .github/workflows/ci.yml
export BOOST_HOME=$TMPDIR/eval-home
python3 -c "
import json,glob,collections,os
n2c=collections.defaultdict(set)
… Verification found nothing to correct. Every number, file:line and command output above was independently re-derived and matched exactly. What is NOT established. Recorded because a card that overstates its own evidence is worse than no card. Severity lowered from the claim's High to Medium on a measurement the claim did not make. I inspected the bodies behind the ambiguous names and they are two different things. Genuine homonyms exist — brand-guidelines is Anthropic brand colors vs OpenAI brand colors vs Sentry *copy writing* (three different jobs), and pdf is a general PDF skill vs a workflow that converts PDF to Markdown with one specific tool — but they are a minority, and neither appears among the rank-1 hits. The names that actually score hit@1 through an ambiguous target (skill-creator, slack-gif-creator, theme-factory, prompt-engineering, mcp-builder) are re-publications of one upstream skill across anthropics/skills, composio-community/awesome-codex-skills and sickn33/antigravity-awesome-skills, differing by a few bytes. A "wrong body" there is still a correct answer to the user, so the looseness is much smaller in practice than in principle. I did not adopt the shipped card's "the looseness was latent" finding as a reason: that was measured at hit@1 0.160, where there was nothing for ambiguity to inflate, whereas this set measures 0.484 — a different regime, so it does not transfer. Floor-margin exposure, with that qualification attached: hit@1 measures 44/91 = 0.484 against a floor needing 37 rows (margin 7 rows) with 10 rank-1 hits landing on an ambiguous name; recall@10 measures 77/91 = 0.846 against a floor needing 71 rows (margin 6 rows) with 20 rows matched only via ambiguous names. Why it is worth doing. This is the only gate in make check that can fail a merge on retrieval quality, and every retrieval decision validated against it (RRF fusion vs preferring dense, pool depth, whether the LLM rerank earns its keep) inherits the looseness. The project already established the fix and built the harness for it; the required set simply never got migrated, and because the card is closed as shipped nothing tracks the remaining work on the set that matters most. Found by an automated audit of retrieval/eval quality, the search & browse surfaces, and first-run onboarding; every finding was then re-measured from scratch by an independent adversarial verifier whose instruction was to refute it. Verdict: CONFIRMED. No fix is prescribed here — the measurement is the contribution.

Complexity M Impact Med Wow ★★★ CLAUDE.md states "Relevance is still decided by name (or by content class when a gold…
ShippedTech-debt

The "taps last refreshed N days ago" hint can never fire on a machine that tapped and never ran boost update — the only writer of the marker is update itself

Write-up · stale-tap-hint-dead-for-tap-only-installs.md

Complexity M Impact Med Wow ★★★ CLAUDE.md's rule is that "search must never refresh taps in the background: _hint_sta…
ShippedDocs · Bug

CLAUDE.md instructs the wrong licence: every new source file is told to carry GPL-3.0-only in a repo whose 373 headers, LICENSE and pyproject all say Apache-2.0

Write-up · claude-md-instructs-the-wrong-licence-header.md

Complexity S Impact Med Wow ★★★ CLAUDE.md tells every contributor and agent to open a new source file with GPL-3.0-o…
ShippedUX · Bug

At a narrow pane boost hooks list drops name, the argument hooks remove -n takes, while host and scope survive

Write-up · hooks-list-drops-the-name-before-the-host.md

Complexity S Impact Low Wow ★★ fixed — `name` now leads the table, so a narrow pane drops it after `host`, `scope`, `event` and `matcher`, never before
ShippedOnboarding · Bug

An unwritable agent rules/ or commands/ dir still crashes a rule or workflow install at exit 70

Write-up · unwritable-rule-or-workflow-dir-crashes-install.md

Complexity M Impact Med Wow ★★ an unwritable ~/.cursor/rules or ~/.cursor/commands still crashes a rule/workflow install at exit 70, leaving files the lock never records
ShippedRobustness · Bug

Two config.json shapes still slip past the corrupt-config handling. Invalid UTF-8 crashes every command, doctor included. {"taps": "x"} is a fresh install to doctor and a configured machine to --help.

Write-up · config-json-edge-shapes-crash-or-disagree.md

Complexity S Impact Med Wow ★★ A config.json holding invalid UTF-8 crashes every command, doctor included; {"taps": "x"} reads as a fresh install to doctor and as configured to --help…
ShippedRobustness · Bug

Two cache writers still crash on what one sudo boost leaves behind. A read-only _names.txt fails heal, update and untap at exit 70 while doctor says healthy.

Write-up · cache-writers-that-still-crash-on-a-read-only-cache.md

Complexity S Impact Med Wow ★★ fixed: refresh_names replaces _names.txt through atomic_write_text and warns once when it cannot write at all (heal/update/untap/tap 70 → 0 on a 444 file); reindex in a 500 cache dir prints one error naming the dir and `chmod u+w ~/.boost/cache` (70 → 1)
PlannedUX · Bug

Just before out.table drops a column, it shrinks the widest one with an ellipsis. In hooks list that is often name, and bmad-r… is not a name hooks remove -n accepts.

Measured on the branch that moved name to the front of boost hooks list, with the autopilot's hook names (bmad, bmad-route) piped at COLUMNS 40–100. name now shows at 52 widths, where it showed at 27. At 24 of those it is shortened, e.g. bmad-r…, in the widths 49–51, 57–59, 65–67, 74–79 and 83–91, the card's own width of 65 included. Before the reorder the same happened at 13 of 27. The cause is the fitter's shrink-before-drop rule (output._fit_columns). Before it drops a column, it shrinks the widest one that can still shrink, down to _MIN_COL. For prose that is the right trade. For an identifier, a clipped value is worse than an absent one, because it reads like data and is not: boost hooks remove -n bmad-r… removes nothing. keep= cannot express "may drop, never shrink". It means never drop and never shrink, and two such columns together can outgrow the pane, the overflow PR 902 closed. Likely fix: out.table takes a third class of column, whole or absent. The fitter never shrinks it and drops it in right-to-left order like any other column. hooks list would mark name, and taps and list would mark their NAME columns, the other arguments people copy. Measure every caller before and after: wide panes must stay byte-identical.

Complexity M Impact Low Wow ★★ In the few columns before out.table drops a column, the widest one that can shrink is squeezed with an ellipsis, and in hooks list that is the name remove -n takes…
PlannedRobustness · Bug

Under a read-only ~/.boost with no cache dir, update, heal and doctor crash at exit 70, and heal --dry-run says 0 for a run that crashes

Measured on the second release train and on a316c6b, where it behaves the same. Setup: tap the fixture, then rm -rf ~/.boost/cache; chmod 500 ~/.boost. search, info and browse now exit 0 (cache-writers-that-still-crash-on-a-read-only-cache), and reindex exits 1 naming ~/.boost. But update, heal and doctor exit 70 with crash reports. update dies in journal.log's ensure_dirs. The command whose job is to name the problem, doctor, is one that crashes. heal --dry-run exits 0 in that state ("would rebuild catalog cache …") while heal exits 70. The cache-dir check heal and doctor gained in the first release train asks whether an existing directory refuses writes, deliberately, so that a missing one is not called unwritable. A missing cache dir under a parent that refuses the mkdir is a third case that neither branch names. Likely fix: paths.ensure_dirs callers on the doctor, heal, update and journal paths tolerate a refused mkdir, like rebuild_tap now does. Doctor and heal then name the nearest existing directory that refuses writes, which is what rag._unsaved already does. The preview and the run must agree.

Complexity S Impact Low Wow ★★ With ~/.boost read-only and no cache dir, update, heal and doctor exit 70; heal --dry-run exits 0 while heal exits 70…
PlannedConsistency · Bug

With an API key exported and no vector store yet, doctor and search send the user to a paid build while quickstart offers the free download

Found while verifying the quickstart shard-status fix. After that fix, shards.remedy() tells a keyed machine with no store to unset VOYAGE_API_KEY and run boost update --shards, which downloads the published keyless vectors for free. boost doctor and boost search read dense.fix_hint("no-store") instead, which says build it: boost reindex --dense. With a key in force, that embeds the whole catalogue through the paid API. That leaves two commands giving the same user contradictory advice, which is the thing fix_hint exists to prevent. Fix direction: when the reason is no-store and a key outranks the local model, have fix_hint answer with shards.remedy() (or the same table row), so doctor, search, quickstart and update --shards give one answer. Keep the manifest fetch off the search path: search must not touch the network to word a hint.

Complexity S Impact Low Wow ★★ On a keyed machine with no vector store, doctor and search say "build it" (paid), while quickstart and update --shards say to unset the key and download free…
PlannedRobustness · Bug

A rule or workflow row for a disabled agent makes boost sync claim the same repair on every run, and doctor never goes healthy

Found while verifying unwritable-rule-or-workflow-dir-crashes-install. A rule or workflow keeps one materialization row per agent. sync_plan reads every row, but the repair it runs is an install, and an install writes only to agents.materializing_agents(). A row for an agent that is no longer enabled is never written and never cleared, so it is "missing" again on the next run. Measured in a disposable HOME. Install a rule with ~/.cursor/rules at mode 500, so the Cursor row is recorded as refused, then set agents.cursor.enabled to false. That is a natural response when the dir was locked on purpose. boost sync then prints re-materialized rule team-conventions on every run, and boost doctor stays at rc 1 with "rule team-conventions was not written for cursor … boost sync writes it once it is", a remedy that cannot work. This is not new with that change. On main, install the rule, delete the Cursor file, disable Cursor, and run sync twice: it loops the same way for any missing file of a disabled agent. Fix sketch: have sync_plan and doctor skip rows for agents outside materializing_agents() for that row's scope (and say once that the row belongs to a disabled agent, with boost uninstall or re-enabling as the next step), or have sync_apply claim a repair only when the rows it meant to fix are actually clean afterwards. A test should run sync twice and assert the second run is "everything in sync". The same fix must also cover store.unwritable_agent_dirs(), which reads the same rows: a refused row of a disabled agent keeps its locked dir in doctor's issues and in sync's warnings, with a boost sync remedy that never writes there.

Complexity S Impact Low Wow ★ a rule or workflow row for an agent later disabled in config makes sync print "re-materialized" on every run and keeps doctor at rc 1

Engine & command internals

// concrete file:line findings from the code scan
ShippedRetrieval · Architecture

Semantic search is gated behind an API key it does not need

Write-up · keyless-semantic-search-for-everyone.md

Complexity L Impact High Wow ★★★★★ the vector store was never the problem — only turning text into vectors needs a key
ShippedSearch · Performance

Every dense search re-scanned all 3.08 GB of vectors — vec0 has no ANN index

Write-up · dense-search-rescans-every-vector.md

Complexity M Impact High Wow ★★★★★ 33.9s cold search; 28.2s of it was one brute-force scan of 3.08 GB
ShippedInstall engine · Scope

Workspace scope — boost install --local into the project

Write-up · workspace-scope-install.md

Complexity L Impact High Wow ★★★★ --local, committable repo lock
ShippedInstall engine · UX

boost list shows installed rules and workflows

Write-up · list-rules-and-workflows.md

Complexity S Impact Med Wow ★★ list was skill-only after rule/workflow install landed
ShippedEval · Correctness

The golden set grades by name, and 35 of 53 names are ambiguous

Write-up · golden-set-grades-by-name-not-by-skill.md

Complexity M Impact High Wow ★★★★ all 50 rows pinned; the 22 judgment calls were measured to move the published number by zero
ShippedEval · Correctness

The “pinned” eval corpus pinned names, not commits

Write-up · eval-corpus-was-not-actually-pinned.md

Complexity M Impact High Wow ★★★★ 62% of the required gate's corpus was one unpinned third-party repo, against a 1.15-query margin
ShippedEval · Correctness

The eval de-duplicated its ranked list by name, so homonyms shared a rank

Write-up · eval-deduped-ranked-lists-by-name.md

Complexity M Impact High Wow ★★★★ 13 different skills named code-reviewer shared one rank slot — where "recall is 1.000" came from
ShippedEval · Correctness

62% of the required gate's corpus is a single third-party repository

Write-up · eval-corpus-is-one-strangers-repo.md

Complexity S Impact High Wow ★★★ one third-party repo is 62% of the gate's corpus, and if it disappears every PR goes red
ShippedEval · Correctness

Nothing refreshes the eval corpus pins, so the gate measures one frozen day

Write-up · eval-corpus-pins-have-no-refresh-path.md

Complexity S Impact Medium Wow ★★★ the corpus is now frozen at one August 2026 snapshot, and nothing will ever move it
ShippedCompat · Python

the Python floor moves from 3.9 to 3.12

Write-up · python-floor-moves-to-312.md

Complexity M Impact High Wow ★★★ the floor was blocking a CVE fix and a dependency major at once
ShippedTech-debt

the typing.List → list sweep the floor now allows

Write-up · pep585-604-modernization-sweep.md

Complexity M Impact Low Wow ★ unblocked by the 3.12 floor; deferred only because the diff touches ~60 modules
ShippedCorrectness

audit the 16 zip() calls the 3.12 floor made checkable

Write-up · zip-strict-audit.md

Complexity S Impact Med Wow ★★ 16 call sites, each a judgement — a silent truncation or a new raise, never a mechanical fix
ShippedBug

boost search never noticed a tap added after the first search

Write-up · search-never-notices-a-new-tap.md

Complexity S Impact High Wow ★★★★ search said "no matches" for a skill `boost info` described from the same machine
ShippedBug

one deleted upstream stopped boost update for every other tap

Write-up · one-dead-tap-broke-every-update.md

Complexity S Impact High Wow ★★★★ found on the maintainer's own machine — 80+ taps, one deleted upstream, no updates for any of them
ShippedSecurity · Correctness

The update path skipped the scan the install path runs

Write-up · update-refreshed-rules-without-a-gate.md

Complexity M Impact High Wow ★★★★★ install scanned rule content; update did not — and the scanner could not see concealment anyway
ShippedTech-debt

bring scripts/ under the ruff gate

Write-up · scripts-under-the-lint-gate.md

Complexity M Impact Med Wow ★ 291 findings across 28 files, measured — most are the same UP sweep the package just finished
ShippedCorrectness · MCP

boost_search never said which ranking produced its answer

Write-up · mcp-search-hid-which-ranking-ran.md

Complexity S Impact Medium Wow ★★★★ the degraded order was byte-for-byte the shape of the promised one
ShippedCatalog · Correctness

est_items counted one skill fourteen times once registries went multi-agent

Write-up · est-items-counts-agent-mirrors-as-items.md

Complexity S Impact Med Wow ★★★ registries now ship one rendered copy per agent, so a raw walk credits pbakaus/impeccable with 40 items for the 9 it has
ShippedCatalog · Curation

The catalog was missing the two most-starred token-efficiency registries

Write-up · efficiency-registries-ponytail-caveman.md

Complexity S Impact Med Wow ★★★ two ~97k-star repos the catalog was missing; both advertise savings their own benchmarks contradict
ShippedBuild · Bug

The shards workflow has never once produced a shard

Write-up · shards-pipeline-never-produced-a-shard.md

Complexity M Impact High Wow ★★★★ two scheduled runs, 0 artifacts — 40 of 60 jobs tapped a bare SHA, and the other 20 hit an export that cannot work
ShippedBug

The repair command could not repair the thing two commands sent you to it for

Write-up · sync-could-not-repair-a-gutted-skill.md

Complexity S Impact Med Wow ★★★★ two commands named `boost sync` as the repair; sync answered "everything in sync" and changed nothing
ShippedBug

Path.exists() looks total, and is not

Write-up · pathlib-exists-is-not-total.md

Complexity S Impact Medium Wow ★★★★ the weekly fuzzer has now found two crashes in one function, three weeks apart, and been ignored both times
ShippedCI · Correctness

a publisher that could not publish, and an alert that could not stand down

Write-up · two-ci-controls-that-could-not-act.md

Complexity S Impact Medium Wow ★★★★ one publisher could never publish and one alert could never stand down — both reported success
ShippedCI · Performance

the shard job that had never once finished, and the timeout that could not be raised

Write-up · the-shard-job-that-could-not-finish.md

Complexity S Impact High Wow ★★★★★ 5h30m cancelled at the ceiling, then 2h07m green — the same job, with duplicate embeddings collapsed
ShippedCI · Evaluation

the scheduled re-pin that refreshed the twenty rows it must not touch, and none of the hundred and sixty-five it existed to pin

Write-up · the-scale-corpus-refreshed-the-wrong-rows.md

Complexity S Impact High Wow ★★★★★ 185 rows, 20 pinned — the monthly job moved exactly the 20 it did not own and pinned none of the 165 it existed to pin
ShippedCatalog · Curation

--category marketing matched nothing, while four marketing registries sat in the catalog under other names

Write-up · marketing-crm-domain-had-no-category.md

Complexity M Impact High Wow ★★★ the four marketing registries already carried were filed under writing and general, so the category returned nothing
ShippedSearch · Performance

Dense reuse is per tap, so one changed file re-embeds the whole registry

Write-up · dense-reindex-reuses-whole-taps.md

Complexity M Impact High Wow ★★★ one changed file re-embeds 29 min for 1.9 s of new text
ShippedCLI · Bug

boost create: CLI audit findings (2026-08)

Write-up · audit-create-findings.md

Complexity M Impact Med Wow ★★ parse(dump(meta)) != meta — evolve rewrites untouched lines, create writes broken YAML
ShippedCLI · Bug

boost reindex: CLI audit findings (2026-08)

Write-up · audit-reindex-findings.md

Complexity S Impact Med Wow ★★ reindex --json names taps two ways in one object (reused vs reindexed)
ShippedSearch · Performance

quickstart reruns re-download every shard; shards.sync never asks what is built

Write-up · shard-refresh-skips-processed-commits.md

Complexity S Impact Med Wow ★★ a quickstart rerun re-downloads every shard the store already holds at that commit
ShippedReliability · Bug

Two concurrent rag.build() runs delete each other's temp index

Write-up · concurrent-rag-builds-delete-each-others-temp-index.md

Complexity S Impact High Wow ★★★ two overlapping rag.build() runs share one .tmp name; the second unlinks the first's finished 644 MB file and the first dies on replace()

Code health & security

// planned · free tooling to catch vulns, smells & bugs
ShippedSecurity · Posture

OpenSSF Best Practices — all 67 passing criteria answered

Write-up · openssf-best-practices-badge.md

Complexity M Impact Med Wow ★★★ 66 Met/N-A · 1 justified Unmet · registration is the only step left
ShippedSecurity · Posture

OSPS Baseline — three levels, audited rather than assumed

Write-up · osps-baseline-levels.md

Complexity M Impact Med Wow ★★★ L1 + L2 earned · L3 blocked · audit found a merged regression
ShippedSecurity · Posture

OpenSSF silver — reachable solo, and mostly already true

Write-up · openssf-silver.md

Complexity S Impact Med Wow ★★★ 55 criteria, 31% → the gap was two documents, not fifty
ShippedSecurity · Posture

OpenSSF gold — how far it goes without a second human

Write-up · openssf-gold-ceiling.md

Complexity M Impact Med Wow ★★★ 26% → 61% live; the parser that said "21 criteria" had missed two
ShippedDocs · Onboarding

The README read like a machine wrote it — measurably

Write-up · readme-prune-and-search-guide.md

Complexity S Impact Med Wow ★★ 609 → 399 lines; 75 em dashes → 2; a real semantic-search guide
ShippedCI · Release safety

main went red because "require branches up to date" was never actually on

Write-up · branch-current-gate.md

Complexity S Impact High Wow ★★★★ two protection mechanisms disagreed; the binding one had the safety off
ShippedInterop · Bug

Gemini logged a skill conflict every session and boost doctor called the machine healthy

Write-up · gemini-duplicate-skill-discovery.md

Complexity S Impact Med Wow ★★★ Gemini warned once per skill per session and no boost surface could see it
ShippedSecurity · Posture

A machine-readable VEX feed, sourced from findings that already existed

Write-up · vex-openvex-feed.md

Complexity S Impact Low Wow ★★ OSPS-VM-04.02 — 4 statements, zero fabricated CVEs
ShippedSafety · Bug

boost audit: CLI audit findings (2026-08)

Write-up · audit-audit-findings.md

Complexity M Impact Med Wow ★★ rm -rf ~/ passes clean, hidden.js is never scanned, and a missing dir counts as scanned
ShippedCLI · UX

boost doctor: CLI audit findings (2026-08)

Write-up · audit-doctor-findings.md

Complexity S Impact Low Wow ★ crash line wears "!" but verdicts "● healthy" exit 0; and "1 issue need attention"
ShippedCLI · Bug

boost fingerprint: CLI audit findings (2026-08)

Write-up · audit-fingerprint-findings.md

Complexity S Impact Low Wow ★ quarantine doesn't change the fingerprint; an uncloned tap hashes as empty, silently
ShippedCLI · Bug

boost health: CLI audit findings (2026-08)

Write-up · audit-health-findings.md

Complexity S Impact Med Wow ★ health calls a drifted machine "● healthy" and scores gemini 1/1 with the store dir gone
ShippedCLI · Bug

boost lint: CLI audit findings (2026-08)

Write-up · audit-lint-findings.md

Complexity M Impact Med Wow ★★ lint can't lint a path on disk, counts 6307 "skills" for 1997 names, misreads one missing ---
ShippedCLI · Bug

boost policy: CLI audit findings (2026-08)

Write-up · audit-policy-findings.md

Complexity M Impact Med Wow ★★ policy check evaluates 4 of the 7 rules install enforces — a refused env "passes"
ShippedSafety · Bug

boost quarantine --release: CLI audit findings (2026-08)

Write-up · audit-quarantine-findings.md

Complexity S Impact Med Wow ★★ a quarantine/release round trip widens 1 agent to 4 and turns doctor red
ShippedCLI · Bug

boost test: CLI audit findings (2026-08)

Write-up · audit-test-findings.md

Complexity S Impact Med Wow ★★ a skill boost lint fails (exit 1) passes boost test outright (exit 0)
ShippedSafety · Bug

boost verify: CLI audit findings (2026-08)

Write-up · audit-verify-findings.md

Complexity S Impact Low Wow ★ a row counted among the N failed still renders the green ok token
ShippedBug

Three commands that denied what was on the disk in front of them

Write-up · because-that-was-not-the-reason.md

Complexity M Impact Med Wow ★★★★★ A `.github/workflows/` directory made `boost recommend` suggest a pasta-recipe skill "because: ci"

Pipeline & supply-chain integrity

// planned · free tooling to secure the CI/CD path itself
ShippedRetrieval · Onboarding

Prebuilt vectors are published where no new user can reach them

Write-up · published-shards-have-no-consumer.md

Complexity L Impact High Wow ★★★★★ the vectors were being built and then nobody could get them
ShippedSupply chain

OpenSSF Scorecard's findings, triaged into three piles

Write-up · scorecard-findings-triage.md

Complexity M Impact Med Wow ★★ 7 open · 3 now hash-pinned, 3 decided, 1 clears itself
ShippedRelease safety

main has no branch protection, so the release rules are honour-system

Write-up · main-has-no-branch-protection.md

Complexity S Impact High Wow ★★★ decided, applied, and now gated against deadlock
ShippedBug

markdownlint linted the fuzzer's corpus, so shipping a crash reproducer would redden a prose gate

Write-up · markdownlint-lints-the-fuzz-corpus.md

Complexity S Impact Med Wow ★★★ 12 of 21 linted files were malformed on purpose
ShippedCI speed

The mutation gate's floor is a single file — shard 0 is store.py

Write-up · mutation-shard-floor-is-one-file.md

Complexity M Impact Med Wow ★★★ shard 0 is store.py alone — 9.3-11.7 min against a 6.1 even split
ShippedCI speed

A third of CI job time is spent waiting for a runner, not running

Write-up · ci-time-is-now-runner-queueing.md

Complexity M Impact High Wow ★★★★ 104 job-min queued vs 236 executing — 31% of CI is waiting for a runner
ShippedSecurity · CI/CD

The required lint job pins zizmor==1.27.0 — a yanked release

Write-up · zizmor-pin-is-a-yanked-release.md

Complexity S Impact Med Wow ★★★★ already fixed by a4450f76 twelve hours before this card was filed
ShippedCI reporting

demo.yml still fails on every push — and the fix is not in the workflow

Write-up · demo-cannot-open-its-own-pr.md

Complexity S Impact Med Wow ★★★★ a repo setting blocks it — no edit to demo.yml can fix this one
ShippedRelease safety

One commit can cut two releases, and the naive guard against it breaks retries

Write-up · one-commit-can-cut-two-releases.md

Complexity M Impact Med Wow ★★★ guarded on PyPI, not on the tag — so retries still work
ShippedSearch · Index

The BM25 index is one JSON blob, and it stops working between 10k and 50k items

Write-up · bm25-index-is-one-json-blob.md

Complexity L Impact High Wow ★★★★★ 2.5 GB RSS per search at 50k items
DeclinedSearch · Retrieval

Semantic search for users who will never set an API key

Dense retrieval today needs the [rag] extra and a VOYAGE_API_KEY/OPENAI_API_KEY and a built store. Most users will do none of that, so the default experience is BM25 forever. The keyless path is a local static embedding model — potion-retrieval-32M class, MIT, model2vec family — which is not a transformer: the entire weight file is one lookup table, so inference is tokenize → gather rows → mean-pool → L2-normalize. Measured locally, pure stdlib: ~1 ms to embed a query (mmap + bisect over sorted keys), 12.8 MB of int8 vectors for 50k items at 256-d, and ~20 ms to rerank BM25's top-200. No numpy, no sqlite-vec, no ANN index, no new runtime dependency. import numpy alone costs 180–390 ms cold, which disqualifies it from a one-shot CLI query path; the BM25 prefilter is what makes the stdlib version viable, since a full 50k brute-force scan is 1.5 s in pure Python. Pool depth is justified by measurement: BM25 recall saturates at 0.890 by depth 200 and gains nothing at 400, so reranking the top-200 gives up essentially nothing versus scanning everything. Why this and not a shipped Voyage index. A precomputed Voyage index is inert without a Voyage query vector, and the only keyless way to get one is a maintainer-run anonymous embedding endpoint — an unauthenticated free embeddings API backed by the maintainer's card, which also ships every user query off-machine and breaks offline. A local model is deterministic, so the artifact becomes a cache rather than a correctness dependency: a newly tapped repo can be embedded on the user's own machine. Doc-side is the asymmetry worth shipping for — 29 ms/doc in pure Python is ~24 min for 50k single-core, versus seconds in CI with numpy. Do not ship this before the eval and dedup items. The headline claim (+11.0 recall / +15.9 hit@1) did not survive verification: its baseline used the kind oracle the real search path lacks, and both the blend weight (w_dense=0.7) and the pool depth were argmax'd on the same 82 queries they were reported on, by 2-query margins. On a binary metric at n=82 the smallest net win reaching p<0.05 is 6 queries; hit@1 (+13 net queries) holds up, recall (+9) sits at the resolution floor. And the structural risk is real: the name is only ~10.5% of a mean-pooled surface vector while 106 description clusters are shared across 270 distinct names, so the lift may shrink toward 50k rather than hold. Sequence: fix the gate, dedup, fix the index format, then re-measure with McNemar and a held-out blend weight, leading with hit@1. Test entry-level dense alone before the blend, ship 512-d not 256-d (the whole case for 256-d was one query), keep Voyage/OpenAI as the opt-in ceiling, and keep BM25 as the floor. The model table cannot go in the default wheel — the shipped runtime is 0.79 MB and every merge to main cuts a release, so +17.4 MB × ~24 releases/day exhausts PyPI's 10 GB project quota in under a month; it needs a separate, rarely-released data package behind an extra. Related, and partly overtaken: [[keyless-semantic-search-for-everyone]] shipped a keyless path using a transformer (BGE via ONNX Runtime, in the [rag] extra) while this item was open. That does not settle the question this card asks — a static lookup table is still far cheaper, and this card's discipline about not shipping a retrieval claim before the eval still stands. What it does change is the baseline: "keyless" is no longer the differentiator, so the case for a static model now rests on cost (~1 ms and no runtime dependency, against a measured 233 ms cold and 34 MB of wheels) rather than on availability. Status left alone deliberately — this is another loop's item to own. Unblocked, and the case for it got stronger. This card says “do not ship this before the eval and dedup items”. Both have now landed: the eval gate floors four metrics over a realistic-sized corpus with baselines keyed to their query set, and content-hash dedup has merged. The new evidence is a timing measurement. Building the shipped ONNX keyless store over 743 entries (3,740 chunks, bge-small-en-v1.5 on CPU) took 4,431 s — 74 minutes, about 1.2 s per chunk. This card's static-embedding proposal claims ~29 ms/doc in pure Python. If that holds it is a difference of more than an order of magnitude on the doc side, which is exactly the cost that makes prebuilt shards mandatory today. Worth measuring the model2vec path directly before committing — but the gap it claims to close is now a measured number rather than an estimate. Spike done — the prerequisites this card set have all shipped, so the measurement it asked for was finally runnable. It says “do not ship before the eval and dedup items”; dedup landed in #370, the index format in #367/#371, the published eval in #373. What follows is potion-retrieval-32M (MIT, 63,091 × 512 F32 lookup table — confirmed a single tensor, no transformer) driven by a hand-written pure-stdlib loader: WordPiece → gather rows → mean-pool → L2, mmap'd, no numpy. The two unverified performance claims were not just right, they were conservative. Query embedding measured 0.16 ms against the card's ~1 ms. Document embedding measured 1.34 ms on a synthetic 105-token doc and 3.27 ms on 300 real catalogue entries (median 61 tokens), against the card's ~29 ms. That reverses one of this card's design arguments. The doc-side cost was the reason prebuilt artifacts looked mandatory: “29 ms/doc in pure Python is ~24 min for 50k single-core”. At the measured 3.27 ms it is 2.7 min — roughly the time a first boost tap --defaults already takes. Local embedding is therefore viable on the user's own machine, and shipped shards become a genuine optimisation rather than a requirement. (Not to be confused with the ONNX bge-small path measured at ~1.2 s/chunk in keyless-semantic-search-for-everyone; that number stands, and the gap between them is the case for the static model.) But reranking bought nothing at real scale, which is the result that matters. Over the 50 natural-language golden queries against a real 71,655-entry catalogue, reranking BM25's top-200 by cosine scored hit@1 2/50 — identical to BM25's own 2/50, a net change of +0 queries where this card's own statistics note says 6 net queries is the smallest win reaching p<0.05. On two hand-checked pairs the ordering was right but the margin was thin (related 0.154 vs unrelated 0.097). Stated limits, because this does not settle the question. The document vector was built from name + description truncated to 1,500 characters, not the full body the real dense path indexes, so this measures a weaker representation than the one being proposed. No blend was tried — pure rerank, no w_dense — and this card explicitly asks for a held-out blend weight and McNemar. What it does establish is that the cheap version of the idea does not pay for itself, so the remaining work is representation and blending, not inference speed. An unrelated finding fell out of it, and it is the more important one. BM25 scored hit@1 0.040 here against the 0.340 published in #373. Both are correct: the published figure is measured over the pinned 6-tap eval corpus of 743 entries, and this run used a real 77-tap install — 96× larger. Golden targets are all present and rank 7th, 8th, 38th, 163rd rather than 1st. The eval corpus is not a scale model of a real install, and the gate's floors describe a catalogue two orders of magnitude smaller than the one users have. Tracked separately in [[eval-corpus-is-96x-smaller-than-a-real-install]]. Declined on measurement, after a second model was tried specifically to avoid declining on one data point. The card's premise is that a local static model buys keyless semantic search. Tested against the 50 natural-language golden queries over the pinned 20-tap corpus (3,843 entries as those registries stand today), with BM25 at hit@1 0.260 as the baseline in every run: potion-retrieval-32M (retrieval-tuned, 63,091×512 F32) scored dense 0.220, hybrid RRF 0.260. potion-code-16M-v2 (code-domain, 63,457×256 F16 — chosen because this catalogue is coding-agent skills, which is the strongest hypothesis for why a general retrieval model would underperform here) scored dense 0.240, hybrid 0.260. A third representation — name+description only, over the full 71,655-entry catalogue — reranked BM25's top-200 to +0 net queries. So: two models, three representations, no measurable gain, and fusion never beats BM25 alone. The code model is one query better than the retrieval model, which at n=50 is inside the noise (±0.02 per query) and should not be read as a trend. The contrast is what makes this a decline rather than a shrug. The published eval measures a real embedding model at hybrid 0.440 against BM25 0.340 — a genuine +0.100. Static embeddings reproduce the cost profile that made the keyless tier attractive (0.16 ms/query, 3.3 ms/doc, no dependency) but not the quality that made it worth having. Cheap and no better than what ships today is not a tier; it is a second code path to maintain for nothing. What survives. The performance findings stand on their own and are already recorded above: doc-side embedding is ~24× faster than the card assumed, which is why prebuilt shards are an optimisation rather than a requirement for the real models in [[keyless-semantic-search-for-everyone]]. The pure-stdlib loader (WordPiece → gather → mean-pool → L2, mmap'd, F32 and F16) is proven workable if a future model justifies it. What would reopen this. A static model that actually separates on this task — the bar is beating 0.260 as a reranker, not merely producing plausible cosines. The two hand pairs looked fine for both models (0.224 related vs -0.001 unrelated for the code model), which is exactly why plausible similarity was not accepted as evidence.

Complexity L Impact High Wow ★★★★★ two models, three representations — no measurable gain over BM25, and fusion only ties
ShippedBuild · Bug

One global concurrency group let any PR cancel any other PR's check

Write-up · demo-workflow-cancels-other-prs.md

Complexity S Impact High Wow ★★★★ two PRs deadlocked each other for hours — re-running the job just moved the cancellation to the other one
ShippedBuild · Bug

The fuzzer found a real crash and nobody was listening

Write-up · unwatched-crons-hid-a-real-fuzz-crash.md

Complexity M Impact High Wow ★★★★★ the fuzzer found a real crash and was right for three weeks — 24 unattended workflows, 2 watched
ShippedBuild · Bug

The published metrics could never be published

Write-up · eval-stats-could-never-publish.md

Complexity S Impact Med Wow ★★★★ the published-metrics file has ONE commit in its whole history — the one that created it
ShippedPerformance · Search

A cold search materialises 71,600 entries to print five

Write-up · cold-search-reads-the-whole-catalogue.md

Complexity M Impact High Wow ★★★★ 0.94 s cold search at 71.6k entries — ~0.5 s spent materialising data the top hits never use
ShippedSearch · Retrieval

boost search brainstorm finds nothing, and brainstorming finds it

Write-up · bm25-has-no-stemming.md

Complexity M Impact High Wow ★★★★ a term with no postings is now replaced by the commonest term it prefixes; a term that has postings is never touched, which is what keeps the eval floors still
ShippedSecurity · Reproducibility

Reproducible release builds — the sdist half nobody's setuptools does for you

Write-up · reproducible-release-artifacts.md

Complexity M Impact Med Wow ★★★ wheel + sdist now bit-identical; setuptools#2133 has no native fix
ShippedRetrieval · Onboarding

The weekly republish reached the machines that had never been set up

Write-up · weekly-vectors-had-no-ingestion-path.md

Complexity M Impact High Wow ★★★★ the vectors were republished weekly and no command could take delivery
ShippedQuality · Retrieval eval

A query made only of characters tokenize drops returns zero results, and the documented catalog.search fallback is unreachable — boost search "C++" finds nothing on a machine holding …

Write-up · bm25-empty-tokenization-kills-catalog-fallback.md

Complexity L Impact High Wow ★★★★ the four symbol-bearing language names now tokenize, and a query that still tokenizes to nothing says so instead of reporting a miss; the catalog.search fallback stays unreachable on purpose — substring 'R' returns 10,092 of 10,152 entries
ShippedUX · Bug

boost search 'C++' returns zero and blames the catalogue: tokenize drops every 1-char token, and nothing ever says a term was discarded

Write-up · search-drops-symbol-language-queries-silently.md

Complexity L Impact High Wow ★★★★ c++/c#/f#/objective-c now alias to indexable tokens on BOTH the index and query side (INDEX_VERSION 7 -> 8); a term that is still dropped is named on stdout, on stderr under --json, and in the MCP reply
ShippedCI · Bug

The sweep gate died at a page, not at launch — and took six unrun checks with it

Write-up · sweep-died-at-a-page-not-at-launch.md

Complexity S Impact High Wow ★★★★ the gate kept reporting failure after it had stopped checking a page and a half

Developer experience & maintainability

// planned · free tooling to catch issues earlier & keep the code legible
ShippedCI · Reproducibility

Pin the lint toolchain so a release can't redden the gate

Write-up · pin-the-lint-toolchain.md

Complexity S Impact High Wow ★★★ unpinned ruff 0.16 reddened every PR
ShippedFeature

make boost mcp the whole setup, and put all three kinds behind it

Write-up · mcp-zero-setup-and-three-kinds.md

Complexity M Impact High Wow ★★★★ measured — a fresh install answers every MCP search with "no skills match", which reads as "boost is empty" rather than "nothing is tapped"
ShippedBug · MCP

A project-scoped install registers its MCP servers machine-wide

Write-up · mcp-servers-ignore-install-scope.md

Complexity M Impact High Wow ★★★★ --scope project installs the skill into the repo and its MCP server machine-wide
ShippedInterop · MCP

MCP-aware skills — declare and wire an .mcp.json on install

Write-up · mcp-aware-skills.md

Complexity L Impact Med Wow ★★★★ skills that need a server
ShippedDX

roadmap.html goes stale on every rebase, so a card and a merge race redden the whole matrix

Write-up · roadmap-html-goes-stale-on-every-rebase.md

Complexity M Impact Med Wow ★★★ two of the three fixes measured dead; one survivor, and it needs a Pages change
ShippedTesting · Bug

make lint reports success when actionlint fails — and says it wasn't installed

Write-up · make-lint-masks-actionlint-failures.md

Complexity S Impact Med Wow ★★★★ fixed — a failing actionlint now fails make, in all three targets
ShippedTesting · Bug

The second silent skip — actionlint runs, and checks no run: block at all

Write-up · actionlint-skips-shellcheck-silently.md

Complexity S Impact Med Wow ★★★★ fixed — shellcheck is pinned in lint-tools and both gates now assert it is there
ShippedInterop · Adoption

MCP — answer the veto that overruled the trigger ("a skill already matched")

Write-up · mcp-already-covered-defeater.md

Complexity M Impact High Wow ★★★★ every trigger was a predicate over the request; the veto was a predicate over the agent's own context
ShippedInterop · Adoption

boost-first — the one rule boost authors, offered opt-in at boost mcp register

Write-up · boost-first-rule.md

Complexity M Impact High Wow ★★★★ the tool descriptions only help on hosts that deliver them — this is the surface that survives when none do
ShippedCatalog · DX

install dead-ends on a registry that vendors its own skills

Write-up · resolve-vendored-duplicate-copies.md

Complexity S Impact Med Wow ★★★★ found by dogfooding — the fix hint re-raised the error it was fixing
ShippedCLI · Install

install refused an ambiguous name and offered no way to answer it

Write-up · install-path-disambiguation.md

Complexity S Impact High Wow ★★★★ the ambiguity error named the paths and no flag could act on them — a dead end
ShippedDX · Feature

boost discover <query> asks GitHub, instead of filtering whatever boost index happened to sample

Write-up · discover-searches-github-not-a-stale-sample.md

Complexity M Impact High Wow ★★★★ an adversarial review of the first draft confirmed 21 defects, two of which would have reddened CI
ShippedFeature

Share the catalogue instead of making everyone re-tap it

Write-up · shareable-catalogue-bundle.md

Complexity M Impact High Wow ★★★★★ 10.9 MB replaces a 12 GB clone — a fresh machine reaches 59,972 searchable items in 4 seconds
ShippedInterop · Adoption

boost-first carried the trigger that had already fired and lost — and could never be updated

Write-up · boost-first-carried-the-losing-trigger.md

Complexity M Impact High Wow ★★★★ the rule shipped the one trigger boost had already measured as losing, and no revision of it could ever reach a machine that had installed it
ShippedDX · Feature

boost serve becomes a searchable, faceted catalogue with a graph of the taps

Write-up · serve-is-a-searchable-catalogue.md

Complexity L Impact High Wow ★★★★★ the old page listed only what was installed — 147 rows out of 71,695 — with no search, no tags and no way to see what a tap actually is
ShippedBug · UX

browse could not search for two words, and the fix reshaped the whole browser

Write-up · browse-could-not-search-two-words.md

Complexity M Impact High Wow ★★★★ space was bound to select, so two words could never be searched for
ShippedUX · Design

One design system across search and browse

Write-up · search-and-browse-visual-refresh.md

Complexity M Impact High Wow ★★★★ search rows learned kind/tap/installed with a stated drop order; browse got its gradient, an empty state, a badge rail, a list scrollbar and a session chip
ShippedPerformance · MCP

The smart rerank pays the LLM again for a search it already answered

Write-up · smart-rerank-pays-per-novel-query.md

Complexity S Impact High Wow ★★★ MCP boost_search measured 11.7-17 s per call — every call, even a repeat of the last one
ShippedCLI · Output

The box drew 108 columns into an 80-column pane, and --help never asked how wide the pane was

Write-up · cli-output-ignored-the-terminal.md

Complexity S Impact Med Wow ★★★ an 80-command sweep found a box that drew wider than the pane, a help screen that never measured it, and a literal %% on screen
ShippedCLI · Output

The hints still run past the pane, and the worst one is pinned by six test files

Write-up · long-hints-overflow-narrow-panes.md

Complexity M Impact Low Wow ★★ a code span is one atomic token, so a hint folds to the pane without splitting the command it tells you to run
ShippedCLI · Bug

install --dry-run promises agents the real install never writes (antigravity-cli copy, antigravity materialize) and omits the MCP plan

Write-up · audit-install-dry-run-promises-agents-the-real-install-never-write.md

Complexity S Impact High Wow ★★ fixed — dry run now reads agents_for_scope/materializing_agents and plans the MCP action; PR awaiting CI (PyPI unreachable in the authoring sandbox, so make check needs CI's confirmation)
ShippedCLI · UX

AI degrade note blames PATH/API keys regardless of cause; several commands fall back with no note at all

Write-up · audit-ai-degrade-note-blames-path-api-keys-regardless-of-cause-sev.md

Complexity M Impact Med Wow ★★ fix implemented and unit/functional/smoke-tested locally; mutation/eval/full-lint gates unverified — this session had no PyPI/network egress to run them, PR relies on CI
ShippedCLI · Bug

Dry-runs disagree with the real run: compact, heal and onboard previews mispredict

Write-up · audit-dry-runs-disagree-with-the-real-run-compact-counts-bytes-it.md

Complexity M Impact Med Wow ★★ fixed — compact counts only what reapply removes and previews a reclone as one; heal words the branch sync takes and names its directories; onboard marks its cut and checks --pr first
ShippedCLI · Bug

--json accepted but ignored: cohort/config/policy set, focus, profile, replay rollback, who empty state

Write-up · audit-json-accepted-but-ignored-on-many-branches-cohort-config-pol.md

Complexity M Impact Med Wow ★ shipped in PR 804 — all seven sites now emit JSON (or reject --json as a usage error, for `update` without `--shards`); CI's full check suite (lint, evals, tests, smoke, mutation) is green on the merge-ready head
PlannedCLI · Bug

Project scope seams: uninstall/verify/list/info/reinstall disagree with what install --local wrote

The project-scope-across-every-command item shipped, and the 2026-08 CLI audit found its seams: the writers and readers resolve "the project" differently. install --local uses scopes.resolve_base, which falls back to the cwd (scopes.py:83-104), while uninstall's project fallback (store.py:1249) and verify/doctor/list all go through scopes.project_root, which requires a VCS marker (scopes.py:45). So from a plain directory, install anthropics/skills:pdf --local writes .boost/skill-lock.json and .claude/skills/pdf and reports success — then verify pdf answers “Error: not installed: pdf”, plain uninstall answers “brainstorming is not installed”, and doctor/list show no project row. After mkdir .git the same commands find everything. All six findings reproduced. Four more seams, each confirmed in source. The already-installed error hints “boost reinstall brainstorming --local to force” — a flag reinstall does not have; following the hint exits 2 with “unrecognized arguments: --local” (store.py:615-617). verify <project-only name> ignores the filter and grades every user-scope item — cmd_verify already passes [] for project-only names (safety.py:355-356) but _iter_installed_all treats [] as “everything” (_common.py:66, if names:), so the run can fail on a rule the user never named. list --local --kind rule prints “○ no rules installed” although project scope holds skills only. And info on a project-scoped skill shows the not-installed card — no version, installed date, commit or agents rows — though the project lock records them all and --json returns them under project. The verified fix, one follow-up card: make _iter_installed_all treat [] as nothing (_common.py:66) · change the hint to boost install NAME --local --force (matching README ~328) or add --local to reinstall · unify the uninstall fallback on resolve_base or hint --local (store.py:1249) · have install --local warn outside a VCS root, or teach project_root to accept .boost/skill-lock.json as a marker · refuse list --local --kind rule|workflow the way the existing --tag guard does (info.py:274-281) · render the plock identity rows in cmd_info. Docs: README ~304–328 (uninstall/reinstall routes), a follow-up note on docs/roadmap/items/project-scope-across-every-command.md, and regenerate docs/commands.html if reinstall gains --local. Found by the 2026-08 CLI audit (cluster project-scope-readers); repro in the audit log.

Complexity M Impact Med Wow ★★ install --local writes a lock that uninstall, verify, doctor and list then cannot find
PlannedCLI · UX

Sweep: positionals/--json lack help strings and no command help shows examples (~30 cmds)

Across roughly thirty commands the help screens end at the options table with undocumented arguments. Observed verbatim: help cohort prints {list,create,delete,status,apply} with no help string for the action; test --help shows positional arguments: then NAME with nothing after it; conflict --help and attest --help each list --json with an empty help line; edit/explain/home give name no text (so docs/commands.html renders <code>name</code><span></span>). No help screen in the audit shows an Examples block. The gaps hide real contracts: run never mentions its SDK/key prerequisites (only the runtime error does), discover --help never says a query hits GitHub live while bare/--local read the cache, conflict's exit-1-on-findings is undocumented, cohort status is an unadvertised alias of list, and policy's 11 valid keys appear only in the error hint after a wrong set. Verification confirmed this is omission, not style: the mechanism works and is used exactly once — the only epilog= in all of boost_cli is cohort's membership-hash paragraph (team.py:74), and team.py:77 adds the action positional with no help=. Nothing in CLAUDE.md declares terse help deliberate, and the content gap is unchanged at COLUMNS=60 under a TTY, so it is not a rendering artifact. Fix as one sweep PR: add help= to every bare positional (action choices, NAME, --json) and an Examples epilog per parser — cliparse.parser forwards **kwargs to argparse, so no plumbing is needed. Extend scripts/build_command_reference.py to render parser.epilog (lines 119/137 render description and per-arg help but currently drop the epilog) and fail --check on empty help strings, then regenerate docs/commands.html. Document the defaults (cohort/profile default action = list) and the cohort status alias while there; the list summary in cli.py COMMANDS should also say "skills, rules and workflows" to match its own description. Found by the 2026-08 CLI audit (cluster help-examples-sweep); repro in the audit log.

Complexity M Impact Med Wow ★ exactly one epilog= exists in all of boost_cli; ~30 commands ship bare positionals
ShippedCLI · Bug

boost ROOT: CLI audit findings (2026-08)

Write-up · audit-root-findings.md

Complexity S Impact Low Wow ★ EPIPE exits 120 with stderr noise; `help version` suggests verify; launcher gates at 3.9
ShippedCLI · Consistency

boost absorb: CLI audit findings (2026-08)

Write-up · audit-absorb-findings.md

Complexity S Impact Low Wow ★ absorb is the only generated-skill command with no journal.log call at all
In flightCLI · Bug

boost adapt: CLI audit findings (2026-08)

A subagent named after a declared tool renders modules that compile but cannot run. With a subagent grep and tool Grep: crewai emits @tool("grep") def grep then grep = Agent(...), so reviewer_1 = Agent(tools=[read, grep]) hands the Agent where a tool belongs (stub run: TypeError); langgraph assigns grep = create_react_agent(...) inside build_mycrew, making it local, so the earlier tools=[read, grep] raises UnboundLocalError. _unique_idents (core/adapters.py:188-202) dedups only among agent specs and _unique_tools (:205-212) allocates stub names independently. Fix: pass the tool ident set into _unique_idents as pre-reserved names (or prefix stubs tool_<name>) in render_crew/render_graph, plus a golden test that executes a colliding render against stubs.

docs/commands.html brackets required options as optional. Line 369 shows boost adapt [--to FRAMEWORK] … while adapt --help prints an unbracketed --to FRAMEWORK and omitting it exits 2 — and the verify pass found it is broader than adapt: evolve's required --feedback and catalog's required mutually-exclusive group render all-optional too. Pure generator bug: scripts/build_command_reference.py:122-126 brackets every option unconditionally. Emit required options unbracketed (a required group as (--a | --b)), prefer the short flag like argparse, then make generate — the --check gate holds it after that.

Colon-form model ids get double-prefixed for the LiteLLM targets. --model anthropic:claude-x — the form langgraph accepts and emits — renders llm=LLM(model="anthropic/anthropic:claude-x") for crewai and the same for agents-sdk; multi-agent crews inherit it via adapters.py:376. _litellm_model's docstring says a provider-qualified value passes through, but the code checks only /. Fix: replace the first : with / before deciding to prefix (mirror of _langchain_model); document accepted syntaxes in docs/adapters.html's --model paragraph (~line 344, slash form only today), and regenerate docs/commands.html only if the argparse help changes.

adapt -o and run --print -o write generated source mode 0600 — and re-rendering over an existing 0644 file silently downgrades it, unlike a shell redirect (-rw------- vs -rw-r--r-- under umask 022). util.atomic_write_text (core/util.py:91-116) inherits mkstemp's 0600, right for the lock/config it was written for, wrong for source the user asked boost to write. Fix: add an optional mode parameter (fchmod the temp fd before os.replace), keep 0600 the default, and have cmd_adapt (pkg.py:1753-1761) and cmd_run (run.py:62) pass the umask default. Still open — an implementation of exactly this shape (path-based os.chmod, not os.fchmod, learned the hard way: the latter raises on Windows) shipped and then was reverted from PR #728 after three rounds of Windows-only windows-latest CI failures in the "unit + functional (90% coverage gate)" step that neither pytest-cov nor a temporary diagnostic artifact-upload commit could surface a cause for — the session driving that PR could not read Windows job logs (capped and consumed by harden-runner's own diagnostic noise) or download the diagnostic artifact (productionresultssa*.blob.core.windows.net blocked by that session's network egress policy) to see the actual failure. The other three findings landed clean on every platform. Whoever picks this back up needs either a session with working Windows CI log access, or to reproduce locally on a real Windows box.

Found by the 2026-08 CLI audit (clusters adapt-ident-collision, docs-required-flag-synopsis, adapt-model-id-syntax, generated-file-mode); repro in the audit log.

Complexity M Impact Med Wow ★★ PR
ShippedCLI · UX

boost bmad: CLI audit findings (2026-08)

Write-up · audit-bmad-findings.md

Complexity M Impact Med Wow ★ an edited persona reads "not installed"; each no-op bmad on burns 4 of 50 history slots
ShippedCLI · Bug

boost browse: CLI audit findings (2026-08)

Write-up · audit-browse-findings.md

Complexity S Impact Med Wow ★ both findings fixed in PR; make check unrunnable in the sandbox (no PyPI/apt egress) — CI is the real gate
ShippedCLI · UX

boost bundle: CLI audit findings (2026-08)

Write-up · audit-bundle-findings.md

Complexity M Impact Med Wow ★ mismatched tap/version lines count "already present" — the Boostfile stops being reproducible
ShippedCLI · Bug

boost catalog: CLI audit findings (2026-08)

Write-up · audit-catalog-findings.md

Complexity S Impact Med Wow ★ --show says "22 taps" then tables exactly 20; --export ships sender-machine paths as URLs
ShippedCLI · Bug

boost changelog: CLI audit findings (2026-08)

Write-up · audit-changelog-findings.md

Complexity M Impact Med Wow ★ fetch --unshallow advised on complete clones; rules/workflows logged at directory granularity
ShippedCLI · Bug

boost chat: CLI audit findings (2026-08)

Write-up · audit-chat-findings.md

Complexity M Impact Med Wow ★★ chat follow-ups like "which of these" lead with the previous turn; no "> " prompt on piped stdin; search takes -k
ShippedCLI · Bug

boost cohort: CLI audit findings (2026-08)

Write-up · audit-cohort-findings.md

Complexity S Impact Low Wow ★ fixed in PR #767 — all 41 CI checks green, including mutation, evals, and the full test matrix
ShippedCLI · UX

boost completions: CLI audit findings (2026-08)

Write-up · audit-completions-findings.md

Complexity M Impact Low Wow ★ CI ran the full pinned toolchain (lint, all mutation shards + gate, evals, full OS/version test matrix, DCO) and came back green with no merge conflict and no open review threads — the local sandbox couldn't run mutmut/vulture/etc. (PyPI blocked), so CI is what actually confirms this landed clean
ShippedCLI · Bug

boost config: CLI audit findings (2026-08)

Write-up · audit-config-findings.md

Complexity S Impact Low Wow ★★ `config unset` on a pristine HOME creates config.json and freezes all defaults into it
ShippedCLI · Bug

boost context: CLI audit findings (2026-08)

Write-up · audit-context-findings.md

Complexity S Impact Med Wow ★ git missing from PATH is reported as "not in a git repository", even inside a repo
ShippedCLI · Bug

boost deps: CLI audit findings (2026-08)

Write-up · audit-deps-findings.md

Complexity S Impact Med Wow ★★ deps shows ✗ not installed yet exits 0; the only real-world requires: shape reads as (none)
ShippedCLI · UX

boost discover: CLI audit findings (2026-08)

Write-up · audit-discover-findings.md

Complexity S Impact Low Wow ★ footer blames the network when gh is missing; --json prints [] silently with no index
ShippedCLI · Bug

boost edit: CLI audit findings (2026-08)

Write-up · audit-edit-findings.md

Complexity S Impact Med Wow ★ editor fails → green "✓ no changes", exit 0; drift calls a local edit upstream-moved
PlannedCLI · Bug

boost evolve: CLI audit findings (2026-08)

evolve accepts empty --feedback and has no stdin/file form. evolve brainstorming --feedback "" exits 0 and diffs in +## Feedback (2026-08-31) followed by nothing, plus a bump to version: 0.0.1 — with --apply that empty section lands in the store and the lock. --feedback - becomes the literal bullet +- -. and --feedback @/dev/null becomes +- @/dev/null.. In cmd_evolve (intelligence.py:663-713) raise BoostError when args.feedback.strip() is empty before calling the AI or heuristic; treat - as read-from-stdin and @path as read-from-file, documented in --help (then regenerate docs/commands.html).

evolve --apply leaves the revision unpinned, so a later boost update can silently overwrite it. After --apply the lock holds the evolved sha and pinned: false, and evolve prints only “✓ evolved brainstorming”; pkg.py's update loop (pkg.py:1035-1039) skips only pinned/quarantined/local entries, so once the tap moves, store.install(entry, force=True) (pkg.py:1063-1067) replaces the revision with no warning. On --apply set entry["pinned"] = True (or a local_revision flag the update loop honours) — at minimum print “boost pin <name> to keep this across boost update” after the success line.

After evolve, info claims an update to a lower version while outdated says up to date. With the lock at 0.0.1 and the tap at 0.0.0, info prints “[update available] … version 0.0.1 / latest 0.0.0 (update available)” — info.py:477-479 and 498-501 test latest != inst_v (string inequality) where cmd_outdated (taps.py:285, 340) correctly uses util.semver_gt for the same decision. Replace both checks with util.semver_gt(latest, inst_v) and label the locally-ahead case (e.g. local revision, tap has an older 0.0.0).

Found by the 2026-08 CLI audit (clusters evolve-feedback-input, evolve-revision-unpinned, naive-version-comparison); repro in the audit log.

Complexity M Impact Med Wow ★ empty --feedback writes an empty section + version bump; the revision is left unpinned
ShippedSafety · Bug

boost explain: CLI audit findings (2026-08)

Write-up · audit-explain-findings.md

Complexity M Impact Med Wow ★★ fabricated Kubernetes/Flyway summary scores faithfulness 1.0 and prints verbatim
ShippedCLI · Bug

boost export: CLI audit findings (2026-08)

Write-up · audit-export-findings.md

Complexity M Impact Med Wow ★ -o x.zip writes a gzip tarball; the repair hint drops the skill from the lock instead
ShippedCLI · Bug

boost import: CLI audit findings (2026-08)

Write-up · audit-import-findings.md

Complexity M Impact Med Wow ★ import turns a tap install into "local" silently; a URL import records a deleted temp path
ShippedCLI · Bug

boost index: CLI audit findings (2026-08)

Write-up · audit-index-findings.md

Complexity S Impact Med Wow ★ PR #802: progress bar clears before raise/warn, 0 results keeps the previous index, gh rate-limit/403 gets a native hint — CI green (lint/eval/test/smoke/mutation), awaiting human merge
In flightCLI · Bug

boost install: CLI audit findings (2026-08)

Rule/workflow lock entries are name-keyed across scopes (med). With the benchmarking rule at user scope, install benchmarking --local in a project fails “Error: benchmarking is already installed / hint: boost reinstall benchmarking to force” — the project has no copy, and the hint would reinstall the user one. The reverse direction blocks too, and skills coexist fine (separate project lock). Worse, the --force escape overwrites the user-scope lock entry with the project one, orphaning the user materializations so uninstall can no longer clean them. _install_rule (store.py:836-839) and _install_workflow (store.py:1074) gate on a name-only lookup with no scope/base comparison. Fix: key entries by scope (or compare existing scope/base before raising), word the error “already installed at user scope”, and refuse a cross-scope --force overwrite without cleanup. Docs: README's install-scope section (~301-328) and docs/roadmap/items/install-scope-user-or-project.md. (Cluster cross-scope-name-block.)

--path says “under path” but matches suffix-only (low). --path plugins/tdd/skills is refused while the error's own hint lists plugins/tdd/skills/test-driven-development — a path that is under it. Suffix matching is the shipped design (install-path-disambiguation, PR 483); the wording is the defect. Reword the raise in catalog.py:~502 to “no copy of X whose path ends with Y” and hint “pass a trailing segment of one of: …”. (Cluster install-path-prefix-match.)

The MCP offer never shows the runnable command (low). The server row prints only demo-echo  npx though the sidecar declares npx -y @example/demo-echo-mcp plus env, and on decline the hint is a literal elided claude mcp add …; the full argv only prints when the host CLI is missing. _offer_mcp renders how from spec['command'] alone (pkg.py:161-164) and mcpdecl.register_argv already exists (pkg.py:201) — render command+args, print the joined argv on decline, and indent the confirm prompt to match its neighbours. (Cluster mcp-offer-command-detail.)

The typosquat warning prints three times (low). install NeoLabHQ/context-engineering-kit:test-driven-development --dry-run prints the identical “closely resembles test-driven-development (sickn33/antigravity-awesome-skills)” warning 3×, one per mirror copy in the look-alike tap. De-duplicate find_confusions on (name.lower(), tap) (typosquat.py:79-87) so the [:3] slice in _warn_confusions covers three distinct look-alikes. Found by the 2026-08 CLI audit (cluster typosquat-warning-dupes); repro in the audit log. Status (2026-09). Three of the four clusters shipped as described above: install-path-prefix-match (catalog.py wording), mcp-offer-command-detail (mcpdecl.command_line renders the full command+args, the decline path prints the real argv), and typosquat-warning-dupes (find_confusions dedupes on (name.lower(), tap)). cross-scope-name-block got the narrower of the fix's own two options: store._check_scope_conflict now refuses a rule/workflow install whose name collides with an existing lock entry recorded under a different scope/base — naming the real location ("already installed at user scope") and refusing even under --force, which closes the silent-corruption half of the bug (a forced cross-scope install used to overwrite the other scope's lock entry, orphaning its materializations). What is still missing is the other half: rules and workflows still cannot coexist across scopes the way skills do, because they share one lock keyed by bare name with no per-location table — skills got a separate projectlock.py when project scope was added, rules/workflows never did. Giving them the same treatment (a rules/workflows section in projectlock.py, wiring _install_rule/_install_workflow and their uninstall/sync counterparts through it for project scope) is real coexistence but is its own, larger change, and belongs in its own card rather than folded into a bugfix PR.

Complexity M Impact Med Wow ★ 3 of 4 clusters landed; cross-scope-name-block gets a safe refusal, not true coexistence
ShippedCLI · Bug

boost log: CLI audit findings (2026-08)

Write-up · audit-log-findings.md

Complexity S Impact Med Wow ★ fixed — cli.main() now catches SystemExit and journals the real rc for --help/usage exits
ShippedCLI · Bug

boost mcp: CLI audit findings (2026-08)

Write-up · audit-mcp-findings.md

Complexity M Impact Med Wow ★ no --dry-run; a named missing host exits 0; unregister claims success Gemini denies
ShippedSafety · Bug

boost onboard: CLI audit findings (2026-08)

Write-up · audit-onboard-findings.md

Complexity S Impact Med Wow ★ onboard --pr pushes absolute /Users/… paths from the global lock file to GitHub
ShippedCLI · Bug

boost outdated: CLI audit findings (2026-08)

Write-up · audit-outdated-findings.md

Complexity S Impact Med Wow ★ an untapped skill vanishes from outdated; the same untapped rule shows "source missing"
ShippedCLI · UX

boost preview: CLI audit findings (2026-08)

Write-up · audit-preview-findings.md

Complexity S Impact Med Wow ★ piped preview strips ** with no substitute; at 60 cols 10 lines leak raw markers
ShippedCLI · Bug

boost profile use: CLI audit findings (2026-08)

Write-up · audit-profile-findings.md

Complexity S Impact Low Wow ★ declined --prune leaves extras fully linked yet still prints "✓ switched"
ShippedCLI · Bug

boost protocol: CLI audit findings (2026-08)

Write-up · audit-protocol-findings.md

Complexity S Impact Med Wow ★ macOS status reads "registered" though register never calls Launch Services
ShippedCLI · UX

boost pulse: CLI audit findings (2026-08)

Write-up · audit-pulse-findings.md

Complexity S Impact Med Wow ★ fix implemented + tested; make check's eval/mutation/smoke gates could not run locally (no PyPI egress) — see PR for what did run
PlannedCLI · Bug

boost quickstart: CLI audit findings (2026-08)

Without the [rag] extra, quickstart taps unpinned at HEAD — and the rerun it promises cannot fix it. cmd_quickstart only fetches the manifest (the source of pins) when want_vectors is true (boost_cli/commands/quickstart.py:145-152), so on a machine without a dense backend the six new taps land with pin: null while the output ends “…install the extra…, then boost quickstart again”. The second run prints <tap> already tapped for all seven (registry.add_many skips existing taps, never re-pins), and once the extra is present shards.sync refuses every mismatched commit: refused (tap is at X, shard is for Y). That contradicts the module's own docstring — “Pinning is the whole point”. Fix: fetch the manifest and pin regardless of dense.have_backend() (pinning is a network-and-config operation, not an embedding one), and on rerun retarget already-tapped registries via shards.ingest instead of skipping them. Update README.md (quickstart section, ~line 145) and docs/semantic-search.md (~line 63). The [rag] install hint has three different wordings. quickstart says pipx inject boost-skill-cli "boost-skill-cli[rag]" (hard-coded at quickstart.py:175-177 and 202-204); reindex's embed.fallback_note (boost_cli/core/embed.py:170-179) says unquoted pip install boost-skill-cli[rag], which fails in zsh (no matches found); doctor and search say quoted pip install 'boost-skill-cli[rag]' via dense.fix_hint(). CLAUDE.md's rule is that doctor and search read one table so they cannot contradict — these two surfaces bypass it. Fix: have embed.fallback_note() and both quickstart paths call dense.fix_hint(); if pipx wording is wanted, put install-method detection inside fix_hint so every caller inherits it. docs/semantic-search.md is already quoted — keep it as the reference. Found by the 2026-08 CLI audit (clusters quickstart-pinning, rag-hint-drift); repro in the audit log.

Complexity M Impact Med Wow ★★ without [rag] quickstart taps unpinned at HEAD, and a rerun can never pin them
ShippedCLI · Bug

boost recommend: CLI audit findings (2026-08)

Write-up · audit-recommend-findings.md

Complexity M Impact Med Wow ★★ curated picks repeat one name 6 of 8 rows; --json returns [] while text prints them
ShippedCLI · Bug

boost replay: CLI audit findings (2026-08)

Write-up · audit-replay-findings.md

Complexity S Impact Low Wow ★ rollback says "complete" (exit 0) with a skill unrestored, and replans it forever
ShippedCLI · Bug

boost run: CLI audit findings (2026-08)

Write-up · audit-run-findings.md

Complexity S Impact Low Wow ★ the 55-line runner's banner names a command that produces a 9-line file
ShippedCLI · Bug

boost schedule: CLI audit findings (2026-08)

Write-up · audit-schedule-findings.md

Complexity S Impact Med Wow ★★ 'interval every None' without StartInterval — and StartInterval 0 hangs status forever
ShippedCLI · Bug

boost search: CLI audit findings (2026-08)

Write-up · audit-search-findings.md

Complexity M Impact Med Wow ★★ CJK rows 72 cells in a 60 pane; --json drops --smart; '60 matches' is the cap, not the count
ShippedCLI · UX

boost simulate: CLI audit findings (2026-08)

Write-up · audit-simulate-findings.md

Complexity S Impact Low Wow ★ fixed — norm_rule lowercases the whole modal; trigger desc clips on a word boundary
ShippedCLI · Bug

boost sync: CLI audit findings (2026-08)

Write-up · audit-sync-findings.md

Complexity M Impact Med Wow ★★ First sync run hides a blocked link; --diff prints it as a raw Python tuple
In flightCLI · Bug

boost tag: CLI audit findings (2026-08)

boost tag swallows unknown flags and misreads them as operands. tag brainstorming --verbose prints the current tags and exits 0 — the flag is consumed as a removal of the tag -verbose; tag --verbose gives "Error: --verbose is not installed" (the flag becomes a skill name); verification found a third hole: tag brainstorming --list silently discards the skill-name operand and lists all tags. Cause: cmd_tag's manual split (boost_cli/commands/info.py:988-993) whitelists only --list/--json/-h/--help; every other --x token falls through as an operand. Every sibling command rejects unknown options with "unrecognized arguments" exit 2. And the mutation path has no before/after check. tag brainstorming -nosuch removes a tag that was never present — silent, exit 0; tag brainstorming +x -x prints ✓ and writes the lock plus a journal event for a net no-op (changed is set per-token at info.py:1027-1041, never compared to the before set); "+with space" is accepted as #with space; +Design and #design coexist. The shipped roadmap item robust-tag-argument-parsing (PR 94) built this manual split — these are residual holes in it, not a duplicate. Fix in cmd_tag: hand any token starting with -- (or -letter that is not a tag operand) to argparse so it errors; compute changed = sorted(tags) != sorted(before); print a one-line notice for removing an absent tag; reject whitespace in tags; document or fold case; error when a name is given with --list. Regenerate docs/commands.html if the help text gains the tag grammar. Found by the 2026-08 CLI audit (cluster tag-arg-parsing); repro in the audit log. Partly landed — PR 735. The correctness half shipped: any unrecognized -- token now reaches argparse (unrecognized arguments, exit 2), --list with a skill name is a named error, whitespace in a tag is rejected, and changed is a before/after set comparison in the new lockfile.apply_tag_mods, so +x -x no longer writes the lock and a journal event for a net no-op. Still open, and why this card stays inflight: the one-line notice when -tag removes a tag that was never present (the remove branch is still a silent no-op), and documenting or folding tag case (+Design and #design still coexist). Both are UX asks rather than correctness bugs, which is why the PR left them.

Complexity S Impact Med Wow ★★ tag brainstorming --verbose exits 0 as a remove of '-verbose'; +x -x writes lock + journal
ShippedCLI · Bug

boost tap: CLI audit findings (2026-08)

Write-up · audit-tap-findings.md

Complexity S Impact Med Wow ★ a missing local dir is cloned as https://github.com//private/tmp/… before any check
ShippedCLI · UX

boost taps: CLI audit findings (2026-08)

Write-up · audit-taps-findings.md

Complexity S Impact Low Wow ★ one UPDATED column shows "@b29e7cf", "2026-07-24" and "11h ago" with no legend
ShippedCLI · UX

boost unpin: CLI audit findings (2026-08)

Write-up · audit-unpin-findings.md

Complexity S Impact Low Wow ★ unpin prints 'released the commit pin too' before the unpinned line it qualifies
ShippedCLI · UX

boost untap: CLI audit findings (2026-08)

Write-up · audit-untap-findings.md

Complexity S Impact Low Wow ★ tap takes several SPECs in parallel; untap still errors on a second name
ShippedCLI · Performance

boost update: CLI audit findings (2026-08)

Write-up · audit-update-findings.md

Complexity M Impact Med Wow ★ a no-op update over 20 taps takes ~14 s serial; --force drops 20 pins without a word
ShippedCLI · Bug

boost who: CLI audit findings (2026-08)

Write-up · audit-who-findings.md

Complexity S Impact Med Wow ★ 29 "skills" = 20 tap names + "10152 passages" + 5 cohort names + 3 real items
ShippedCLI · Audit

August 2026 full-CLI audit: every one of the 81 boost commands exercised and verified

Write-up · cli-audit-2026-08-full-sweep.md

Complexity L Impact High Wow ★★ 81 commands, ~2,000+ invocations, 367 findings, 161 clusters, 107 cards on the board
In flightCatalog · UX

Per-item categories in search/browse/info — not just a ★ curated bool

From a user request: “proper categories for skills (can't have all of them listed as just curated)”. They are right about the item level: the only taxonomy a catalog entry carries is a boolean. A boost search row shows name, kind, tap, description and at most a ★; boost recommend's no-match fallback is literally headed “curated picks”; boost info prints no category at all. Across a real install of tens of thousands of items, “starred or not” is the entire classification a user can see or filter by. What the code confirms. catalog._make_entry stamps "curated": curated onto every entry (boost_cli/core/catalog.py:119, signature at 105–106) — and that bool is per-tap, from Tap.curated (core/registry.py:23), set by tap --defaults or by anyone passing --curated (commands/taps.py:126) — a trust star, not a classification. Category-like data does exist, but only per tap: data/registries.json rows carry one (487 registries, 21 values; general alone covers 127), and exactly two surfaces read it — browse's row badge via _tap_categories (commands/discovery.py:936–941, whose own docstring says “catalog entries themselves carry no category, only their tap does”; badge appended last in _row_badges, discovery.py:961–963, so narrow panes drop it first, and taps outside the bundled 487 get none) — and boost serve's web facets (core/serve.py:65). cmd_search renders only the star (discovery.py:179) and takes no filter flag; info shows frontmatter tags when present (commands/info.py, the meta.get("tags") kv) but no category, and its --json has no such field. An item's own frontmatter category/tags ride along invisibly in entry["meta"] and the substring search_blob (catalog.py:131, 621–627), so they can match a query yet can never be displayed or filtered. Proposed fix. Stamp a first-class category on each entry at scan time in _make_entry (catalog.py:105–132): the item's frontmatter category (or first tag) when declared, else inherited from its tap's registry category — and bump catalog.CACHE_FORMAT so hundreds of existing tap caches backfill without a re-tap, per the versioned-cache rule. Then surface it where a category would live: a badge in search rows and a --category filter on search/browse/recommend, a kv row plus JSON field in info, and browse's existing badge switched from tap-level to the entry field (which also gives un-bundled taps' items a label for the first time). ★ keeps meaning curation/trust only. Consumers must degrade cleanly when category is absent (old caches, synthesised entries), same as the content digest rule. Docs: regenerate docs/commands.html for the new flags; no other doc names categories. Found by the 2026-08 CLI audit (cluster catalog-categories-beyond-curated, filed from the user's request); repro in the audit log. Verified against source 2026-08-31. Status (2026-09-01). Landed: the category stamp at scan time (catalog._entry_category, own frontmatter category → first tags entry → tap's registry category), CACHE_FORMAT bumped to 2 so existing caches backfill on next scan, a --category filter on search/browse/recommend (catalog.matches_category/filter_by_category), info's kv row and --json field, and browse's row badge switched from the tap-level lookup to the entry's own field (falling back to the tap lookup for a cache not yet rescanned). Not done: the badge in plain boost search rows. That row's column widths (out.search_layout/format_search_row) are a tuned, heavily-pinned budget system (drop order, per-cap name shrinking, a reserved curated tail) — working it out safely needs its own pass rather than a bolt-on inside this PR. Left inflight rather than shipped for that reason; the next claim on this item is scoped to exactly that piece.

Complexity M Impact Med Wow ★★ landed everywhere except the search-row badge — see PR for what remains
ShippedBug

The last command that blamed boost for the user's typo

Write-up · typos-reported-as-boost-bugs.md

Complexity S Impact Med Wow ★★★★ `boost create --dir` answered a mistyped path with "file it at github.com/.../issues"
ShippedAgents · BMAD

The BMAD router reads each prompt alone, so a pasted log, an “ok update both and rerun” and a yes/no question all get a banner

Write-up · bmad-router-reads-each-prompt-alone.md

Complexity M Impact High Wow ★★★ pastes, follow-up replies and yes/no questions get delegation banners
ShippedAgents · BMAD

boost bmad gives every track the build contract, so a review is told to finish a change

Write-up · bmad-done-contract-ignores-the-track.md

Complexity M Impact High Wow ★★ a review or research ask is told to add tests and finish a change
ShippedAgents · BMAD

The autopilot routes docs at a skill BMAD 6.12 no longer installs, and no test would notice

Write-up · bmad-roster-drifted-from-bmad-6-12.md

Complexity M Impact Med Wow ★★ docs routes at a skill no default 6.12 install contains
PlannedCLI · Bug

out.err() judges colour by stdout while writing to stderr

Every Error: line asks stdout whether to colour a line it writes to stderr. out.err (boost_cli/core/output.py:267 and :271) paints with c(), which takes no stream and calls use_color(), so the answer is sys.stdout.isatty(). Every BoostError reaches the user through it (cli.py:398), as do the unknown-command and unknown-option errors.

Measured through a real pty (Python pty.fork, BOOST_COLOR / NO_COLOR / CLICOLOR_FORCE unset), boost bundle install nosuch on loop/bundle-audit (err() is unchanged from origin/main). With stdout on the terminal and 2>log, the log holds \x1b[31m\x1b[1mError: \x1b[0mno Boostfile at …/nosuch\n\x1b[2m hint: create one with `boost bundle dump Boostfile`\x1b[0m\n — five escape sequences written into a file. With stdout to a file and stderr on the terminal (>out), the terminal gets Error: no Boostfile at …/nosuch with no colour at all. Each case gets what the other one should. out.warn(stream=…) had the same bug; the bundle audit fixed it by passing the stream through to role(), and left err out of scope. A grep for c( on a file=sys.stderr line in boost_cli finds only these two.

Fix: give c() a stream= keyword forwarded to use_color, and pass sys.stderr from both calls in err. Test it the way tests/unit/test_output.py::TestWarnColourFollowsItsStream tests warn: with stdout a TTY and stderr a plain buffer, no \x1b[ in stderr; with the two swapped, the Error: prefix is coloured.

Complexity S Impact Low Wow ★ boost … 2>log on a terminal writes escape codes into the log, while boost … >out prints the error line plain

Docs-site & content quality

// planned · free tooling for the Pages site, README & prose
ShippedDocs

The OpenSSF badge playbook

Write-up · openssf-playbook.md

Complexity S Impact Med Wow ★★★ the method, not the answers — so another repo can repeat it
ShippedDocs · Onboarding

The engine had no architecture diagram — and the one written rule was documented backwards

Write-up · internal-architecture-diagrams.md

Complexity M Impact Med Wow ★★★ 45 core modules and an enforced layering rule, with no diagram of either
ShippedDocs · Design system

A GIF carousel touring one flagship command per group

Write-up · command-carousel-page.md

Complexity M Impact Med Wow ★★★★ 8 GIFs · one flagship per group
ShippedDocs · Performance

The Lighthouse budget passes on noise, not on margin

Write-up · roadmap-perf-budget-has-no-local-guard.md

Complexity S Impact Medium Wow ★★★★ main passes this budget on run-to-run luck — its own three runs are 0.810, 0.840, 0.850
ShippedDocs · Interop

an explainer page for the LangChain / LangGraph / LangSmith integration

Write-up · langchain-integration-explainer-page.md

Complexity M Impact Med Wow ★★★★ the eval.html genre, pointed at the LangChain stack — and it can only document what has shipped
ShippedDocsite · Bug

Expanded card bodies overflow the roadmap board sideways

Write-up · expanded-card-bodies-overflow.md

Complexity S Impact Med Wow ★★★ the closed <details> was added for paint cost and is quietly also the only thing keeping long code tokens on the page
ShippedCI · Bug

The performance gate flips on byte-identical input

Write-up · lighthouse-flips-on-byte-identical-input.md

Complexity M Impact High Wow ★★★★ the same roadmap.html scored 0.78 and passed, then 0.78 and failed — the gate now decides on runner noise

Compatibility & install integrity

// planned · free tooling to prove boost installs & runs everywhere it claims
ShippedCompat

What Gemini actually receives from boost, audited

Write-up · gemini-mcp-parity.md

Complexity M Impact Med Wow ★★★ audited what Gemini actually receives — two of three findings were our own wrong claims
ShippedCompat

boost hooks learns a second host — and finds two bugs upstream

Write-up · gemini-hooks-host-aware.md

Complexity M Impact Med Wow ★★★★ two upstream Gemini bugs found while establishing the schema
ShippedRelease

ship the LangChain integration inside the wheel, behind a [langchain] extra

Write-up · langchain-in-the-wheel.md

Complexity M Impact High Wow ★★★ same import, same tests, zero new infrastructure — the wheel that already ships on every merge carries the integration too
ShippedBug

sanitize agent frontmatter for Gemini instead of copying it verbatim

Write-up · gemini-agent-frontmatter-sanitizer.md

Complexity S Impact Med Wow ★★ measured on Gemini CLI 0.53.1 — a boost-installed agent fails Zod validation at startup, and hand-fixes regress on the next sync
ShippedInterop · Registry

garrytan/gstack — tap it first, then learn to coexist with it

Write-up · gstack-tap-first-then-coexistence.md

Complexity M Impact Med Wow ★★★ 130k stars of SKILL.md that boost can index today, and a second installer writing into the same dotdirs

Skill-content trust & safety

// planned · boost's core threat model — the third-party skills it installs run inside an agent
ShippedSecurity · Content

The MCP boost_install tool skipped the injection scan the CLI runs

Write-up · mcp-install-skips-the-injection-scan.md

Complexity S Impact High Wow ★★★★ the one install path with no human watching was the one not scanning
ShippedSupply chain

sbom.yml has never run — it waits for an event GITHUB_TOKEN cannot emit

Write-up · sbom-release-event-never-fires.md

Complexity S Impact High Wow ★★★★ 253 releases shipped with no SBOM; fixed with workflow_run
ShippedRelease safety

The code_scanning ruleset rule can go back on — but only scoped to CodeQL

Write-up · code-scanning-rule-can-be-restored.md

Complexity S Impact Med Wow ★★★★ safe scoped to CodeQL (0 open) — adding Scorecard deadlocks every merge
ShippedSupply chain · Bug

The SBOM can declare a different version than the release it is attached to

Write-up · sbom-declares-the-wrong-version.md

Complexity S Impact High Wow ★★★★★ fixed — release v1.0.278's SBOM declared 1.0.277; the version is now pinned to the tag
ShippedSecurity · Correctness

rules and workflows install, then cannot be governed

Write-up · rules-install-but-cannot-be-governed.md

Complexity L Impact High Wow ★★★★★ an upstream push rewrote CLAUDE.md silently; pin and quarantine both answered "not installed"
ShippedSecurity · Bug

boost serve echoed the request path back into its 404 body

Write-up · serve-404-reflected-the-request.md

Complexity S Impact Med Wow ★★★ the test suite pinned the echo in place as if it were the contract
ShippedTrust · UX

boost attest: CLI audit findings (2026-08)

Write-up · audit-attest-findings.md

Complexity S Impact Low Wow ★ a deleted store dir is reported as a sha mismatch; drift names the same state correctly
ShippedCLI · Bug

boost trust: CLI audit findings (2026-08)

Write-up · audit-trust-findings.md

Complexity S Impact Med Wow ★ trust add of a missing .pub path blames "invalid base64 in minisign data"