Performance field notebook
What we measured, changed, declined and learned making Bristlenose open faster. Two days of work (4 to 5 November 2026), written so it can be distilled later into a long-form piece on how to do performance work without fooling yourself.
Summary
−0.6 soff every study opened from the Welcome screen: standby sidecar, measured in the app (1,824 → 1,238 ms click to report)
−30%sidecar start from one lazy import (
inflect): 3.19 → 2.24 s cold, in the app−30%spawn to first health check from loading the MCP endpoint on first use: 1,454 → 1,005 ms
75%off the first click on Sessions: 361 → 86 ms (preload the chunk when idle)
The shape of the result: the big wins were not tuning. They were finding work that did not depend on the thing the researcher had just asked for (importing a plural library, building an agent endpoint nobody had called, starting a Python process that could have been started earlier) and moving it out of the click. What is left is smaller, harder, and mostly in two places we have deliberately not touched yet: the server's bind step and the web view's own startup.
Two things also went wrong in instructive ways. A fix to return the window to its lens after an analysis passed its unit tests and did nothing in the real app (a run replaces the page and records the wrong lens). And a "regression" in the standby turned out to be a changed macOS setting under the test harness. Both are in What we learned.
Method
The loop, every time: instrument, change one thing, re-measure, stop at a number you wrote down before you started. The rest is discipline about not fooling ourselves.
Rules we held to
- Label every number: measured a repeatable run; estimate reasoned from a measured part, with the reasoning shown; unmeasured no number, and none guessed.
- Interleave A/B. Alternate the arms (off, on, on, off…) so drift in machine load hits both. Report the spread, not just the median.
- Same binary, one switch. The standby was toggled by an environment variable on one app build, never by comparing two builds.
- Measure in the real thing. A dev-server number is a hypothesis. The sandboxed app's own signposts decide.
- Write the stop number first. "Good enough" is a figure, so a pass can end by succeeding or by declining.
- Declined is a result. A change that does not pay is recorded with its number, so nobody re-proposes it.
Instruments we built
- Signposts in the Mac app (
os_signpost): launch, project select, serve ready, and one named step per stage between the click and the report being ready. - Server stamps:
BRISTLENOSE_STARTUP_TRACEandServer-Timingon routes. - Harness scripts: a launcher that drives the real app through Accessibility, an interleaved A/B runner, a bare
WKWebViewloader, a Chromium timeline script. - A spend guard: the harness aborts if the study's LLM call log grows, because some trial launches start a paid job.
- Isolated Swift verification: build in a throwaway worktree with only your own files, so a concurrent session's half-saved edit cannot flip your result.
- Mutation proof: revert the fix and see that exactly the new test fails. A test you have not seen fail is a guess.
What we did, in order
| When | What | Outcome |
|---|---|---|
| 4 Nov | Window resize, sidebar hold, selector fix | 10 → 59 window updates per second; web content main thread 4,145 → 1,705 ms (−58%). |
| 4 Nov | Ledger opened; lazy inflect, libyaml codebook, Keychain cache, export skip, gzip level | Cold sidecar start −30% from the plural library alone; codebook import 130 → 40 ms; gzip changed nothing at real size, and was kept only because it is free. |
| 4 Nov | Sessions grid chunk preloaded when idle; WebKit has no idle callback, so a 250 ms timer fallback | First click on Sessions 361 → 86 ms in Chromium; 366 → 101 ms in WebKit. |
| 4 Nov | MCP endpoint built on first request; Screens availability check off the event loop | Spawn to first health 1,454 → 1,005 ms; with Screens on, 1,450 → 1,170 ms. A code review caught that the health route still imported the SDK. |
| 5 Nov | Standby sidecar: plan, independent review, build, second review, fix | Spare Python process booted ahead and claimed over stdin. Review found a blocker no unit test saw: nothing owned the adopted process, so it was freed after the claim. |
| 5 Nov | Measured the standby in the sandboxed app, interleaved | serve-ready 1,140 → 516 ms cold; click to report 1,824 → 1,238 ms. Sandbox lease inheritance works. |
| 5 Nov | Restored-launch trial | Found a correctness bug (restored windows came back empty on macOS 27) and declined the "earlier spare" idea: the window asks for its sidecar 0.13 s after launch. |
| 5 Nov | Web view pre-warm | Measured, declined. |
| 5 Nov | Busy-aware reap; lens restore across relaunch and across a finished analysis | A study with paid work in flight is no longer reaped for lack of a window. The lens fix first passed tests and failed live, then was corrected and verified on two lenses. |
| 5 Nov | Attributed the time after the port answers; bundle and mount pass | Five new signposts, two page-load harnesses; no cheap win in the bundle (see below). |
The numbers
Before and after, per change
beforeafter
Where the time goes between click and report
sidecar boot and bindport poll
web view startpage loadReact mount to readyother / native
The law behind the standby
The web side: our app against bare WebKit
load call to navigation startnavigation start to did-finishdid-finish to ready
A restored launch
Why the standby cannot help a relaunch. macOS restores windows by default, so most relaunches skip the Welcome screen. The standby earns its 0.6 s on the first study opened from Welcome and on every later open in the session (a replacement spare is started after each claim).
The thing that looked like a regression was this: the harness had been written while the system switch for restoring windows was off, so every launch landed on Welcome and then clicked. With the switch back at its factory default, every launch restored the study at once and no spare existed in time. The build was fine. The test had changed under us.
Declined, tried and not kept
| Idea | Result | Why we stopped |
|---|---|---|
| Pre-bind the last-opened study | Not built | The server's app construction is welded to the study (database, watchers, media, paid background jobs). A quiet bind is a core refactor, and risks starting paid work on a study nobody chose. |
| Claim the spare while it is still booting | Dropped before building | Measured law: finishes at the same moment as waiting for ready. |
| Spawn the spare earlier, before the window asks | Declined | Window asks at 0.13 s; the best case saves 0.1 to 0.2 s for a hook before window creation. |
| Pre-warm a web content process | Declined, in the app | Did not pay when measured. Worth one re-test with the bare numbers (see backlog). |
| Delay the Screens job by 1.5 s | Dropped | 1,223 vs 1,191 ms: nothing. |
| Bulk-validate transcript words | Byte-identical output, a 300 → 266 ms first read | Not kept: the interleaved numbers did not hold the gain. |
| Startup importer skip-when-unchanged | About 70 ms at real size | Declined at real size. |
| Further import work on Python start | About 0 gain left | 740 ms of diffuse imports; no single item over 35 ms except the MCP one, already done. |
| Playback follow paint cost | No difference in WebKit | The layout-pass fix landed; the rest was headless software rendering. |
| Scene strips off to speed the first load | ready 357 / 354 / 367 ms with them on, 386 / 399 / 397 ms off (bare view right after bind, 3 launches per arm) | Turning them off did not help (off was slower; n = 3, nothing more claimed). The first-load penalty is not their contention. |
What we learned
About the product
- Find work that does not depend on the click. 47% of a cold start was Python importing things unrelated to which study was opened. The three largest wins were all of this kind.
- The platform default decides who benefits. Window restoration is on by default, so the thing we optimised (open from Welcome) is the minority path on a relaunch and the common path within a session.
- A restored window is a different product moment, and it hid a real bug: on macOS 27 the restored scene value arrives after the view appears, so every relaunch used to come back empty.
- Bind is the floor we have not touched. About 440 ms of the 1.2 s after a click is the server building the study's app. Lowering it means a per-study warm worker, which is an architecture decision, not a patch.
About doing the work
- The first measurement is usually of the wrong scenario. Three times in two days a harness measured something other than what we thought: a changed system setting, a
pgreppattern that matched nothing, and a clock that started a second early. Check what the run actually did before reading its numbers. - Unit tests do not see the product. The lens-after-analysis fix held the lens the window was on at reload time. In the real app that lens was always Project, because a run replaces the report with its own page. Only running an analysis, with a harness reading the window, showed it. The corrected version captures the lens when the run starts, and was checked live on two lenses.
- Reviews find a different class of bug. An independent review found the standby process was freed immediately after being claimed (nothing owned it). Every unit test had passed.
- An honest "not measured" beats a confident guess. Several rows in the ledger say so, and two of our own earlier claims were corrected after a review: a lazy-loading change that still loaded the SDK on its first health call, and a standby "cost" that did not reproduce.
- Verification costs something. Trial launches spent real LLM calls (five early, then about seventy on a scratch study for the lens check). The spend guard exists because of the first five.
- Shared working trees bite. One commit swept in another session's uncommitted edit. Stage by explicit path, read
git difffor hunks you did not write, and never rewrite another session's work to tidy up.
The biggest caveat, and the checks we then ran
Every app number above is from a Debug build (apart from the checks below), and the in-app web side runs about 230 ms slower than bare WebKit. We tested the obvious suspects in turn.
1. Release build. Same signing and sandbox entitlements as Debug, so optimisation was the only change. View made to ready was 596 ms (standby off) and 567 ms (on), against 604 ms in Debug; the mount phase may be about 40 ms faster (inside the spread). Debug is not the cause.
2. Settling the sidecar and warming the process (a busy Mac, load 4 to 13). After the sidecar had settled, loads made by switching away and back took 460 ms median (383 to 528), against 671 ms for the first load: about 210 ms of the first-load premium is the cold process and the sidecar's own startup work. But a floor stays: made to navigation start is still 200 to 230 ms in the app against 57 ms bare.
3. A main-thread sample. One sample of the app's main thread during a project switch: of 746 samples, about 460 are in the Core Animation transaction flush, about 330 of them AppKit layout (the sidebar creating row views, the toolbar's layout, NSHostingView layout) and about 90 to 150 SwiftUI view-graph updates. The web view cannot start navigating while the main thread is in that. The cause is most likely our own UI work at a project switch, not the sandbox. One sample, not repeated; the bare view was not run sandboxed, so the sandbox is not ruled out.
The Release app is a measurement build only: Debug entitlements, no hardened runtime. It says nothing about an App Store build's signing cost.
Backlog: opportunities, costs, pros and cons
Gains are per occurrence, with the label that says how much to trust them. Cost is engineering effort: S under a day, M a few days, L weeks. "Frequency" is how often a researcher meets the moment.
| # | Opportunity | Moment, frequency | Gain | Cost | Pros | Cons and risks |
|---|---|---|---|---|---|---|
| 1 | Re-run the app harness on a Release build done | Decided items 3 and 4 | measured no change: 596 / 567 ms against 604 in Debug; about 40 ms at most in the mount phase | S (an hour) | Removed the biggest caveat. | Measurement build only (Debug entitlements, no hardened runtime). |
| 2 | Idle CPU and energy measurement (powermetrics) | App open all day, continuous | unmeasured polls: agent pulse 1.5 s, screens 3 s, run 1 s, plus the web content process | S (needs sudo) | For an app that stays open, battery is felt more than a 100 ms open. | Needs an idle machine and an elevated run; may find nothing to fix. |
| 3 | Web view start-up: re-test pre-warm with the bare numbers (about 110 ms is content-process launch) | First study open of a session, daily | estimate ≤ 110 ms, first open only | S (a day) | Small and contained; harness exists. | Earlier in-app test said it does not pay. Hidden pre-emptive work must be invisible and failure our problem, never the user's. |
| 4 | Free the main thread at a project switch: the web view cannot start navigating while the app is busy laying out the sidebar, toolbar and SwiftUI graph | Every study open and every study switch, daily | estimate up to about 170 ms of the made-to-start phase (228 ms in app against 57 bare); one main-thread sample, not repeated | M (profile, then change) | Our own code, so it is actionable; helps every switch, not only the first open. | One sample on a busy Mac. Needs a Time Profiler pass to find what rebuilds the sidebar rows and re-lays-out the toolbar; the cure may touch the AppKit sidebar, which is risky ground. |
| 5 | QuoteCard memoisation and selector subscriptions | Search, tag, star, keyboard moves; many a day | estimate search echo 170 → under 16 ms at 1,500 quotes. At real size the Quotes revisit render is only 15 ms of 117 (fetch is 92) | S to M | Protects large studies; low blast radius. | At typical size the gain is small. Ownership overlapped another session's display-preferences work. |
| 6 | Lens revisit from last-known data (stale while revalidate), with per-lens scroll position | Switching lenses, many a day | estimate Quotes revisit about 117 ms → near 0 to 20 ms perceived | M | The most felt interaction gain; pairs with remembering scroll position. | Staleness and invalidation bugs are the classic failure; owner's call, parked. |
| 7 | Per-study warm worker / split bind from the study | Every study open, daily | measured bind is about 440 ms; and would let a restored launch benefit from a spare | L (weeks) | The biggest remaining lever; also fixes the restored-launch case. | Core server refactor; paid-job risk; changes lifecycle and the sandbox lease story. Do as a designed project, not a patch. |
| 8 | Performance regression gate (a ratchet on a few numbers) | Protects every gain | Prevents loss, not a gain | M | Gains rot silently without it; the ratchet pattern already exists for other gates. | Timing is noisy on shared CI; use ratios and wide margins, or run on a pinned machine. |
| 9 | Startup importer skip-when-unchanged | Study start, daily | measured about 70 ms at real size | M | Cheap per study. | Declined at real size; revisit only if studies grow. |
Value against cost
Sequencing
- Now (measure only): the Release-build re-run (#1) is done and ruled Debug out. Idle energy (#2) is next: it changes nothing in the product and can kill or confirm other items.
- Next (small, contained): web view start-up re-test (#3). The in-app gap (#4) is reopened with a better cause: the main thread is busy with our own layout at a project switch. It needs a profiler pass before any change. Memoisation and selectors (#5) when the display-preferences work has landed.
- After launch: lens revisit from last-known data together with per-lens scroll position (#6), since they share the same state model; then the regression gate (#8) once the numbers have stopped moving.
- As a designed project: the warm worker (#7). It deserves its own design note first, and its decision rule is the standby's law: it must beat bind (440 ms) by more than it costs in memory and lifecycle complexity.
Notes for the essay
Angles that came out of the work and could carry a piece on their own:
- "Find the work that does not depend on the click." The three biggest wins, with the 47% figure for how much of a cold start was independent of the study.
- "The test changed under us." How a macOS default turned a working optimisation into an apparent regression, and what a harness should assert before it measures.
- "It passed its tests and did nothing." The lens fix: why a live run found what unit tests could not, and why a review found what the live run could not (the freed standby).
- "The stop number." Writing the good-enough figure first, and treating declined changes as results with a recorded number.
- "Label your numbers." Measured, estimate, unmeasured, and the discipline of saying which.
- "Everything here is a Debug number." The honest caveat, and then the check that showed it did not matter (Release changed nothing). A caveat is only worth stating if you then go and test it.
Appendix: the raw figures used above
| Figure | Value | Source |
|---|---|---|
| Standby off, serve-ready cold / warm | 1,140 / 1,106 ms | app A/B, 6 reps |
| Standby on, serve-ready cold / warm | 516 / 508 ms | app A/B, 6 reps |
| Click to report ready, off, cold / warm | 1,824 / 1,752 ms | app A/B |
| Click to report ready, on, cold / warm | 1,238 / 1,144 ms | app A/B |
| Restored launch, launch to report, off / on | 1,915 / 1,920 ms | 5 launches per arm, interleaved |
| Standby spare boot in the app | 595 to 629 ms | 12 of 12 claimed in the A/B |
| In-app web side: setup / load to start / start to finish / finish to ready | 23 / 168 / 272 / 148 ms | 14 launches, medians |
| Port line to port open | 66 ms | 6 launches |
| Frontend bundle, entry | 52 to 55 JS files, 331 KB transferred (829 KB raw) | Chromium resource timing |
| Bare WKWebView, warm: start / finish / ready | 57 / 124 / 170 ms | real-size study copy, 7 runs |
| Bare WKWebView, first of process, right after bind | 166 / 300 / 357 ms | 3 runs, scene strips on |