LLM availability — how to expose it

An exposure sketch, not a design. The plumbing landed first (an all-batches-failed job is now failed and classified; the refusal mapper is wired back up; the Mac out-of-credit pill hears about sidecar jobs). What is still open is where availability should show, and how hard it should push. The argument below is that the honest answer is three tiers, because the three states are known at three different moments and with three different confidences — one global “LLM unavailable” switch would have to pick one and be wrong about the other two.

---

1  Three states, one control

The distinction that matters is when the answer is knowable and how long it stays true. Configuration is a local fact and never stale. Credit is a fact about someone else’s server, and is stale the instant it is read. What actually happened is only knowable after spending. None of the three earns a change to the control, and the table’s last column is why: each argument for one turns out to restate a global condition locally, on every card on screen, to save a click that is either free (the pre-flight refuses without spending) or is itself the only accurate test.

State	Known when	Stays true?	Treatment	Why not the obvious thing
No provider no key · local-only · ambiguous	Before the click, free	Yes — local config	Nothing on the control. The refusal costs no tokens and names the fix in the toast.	Not a replaced control (“Set up AI…”) and not a disabled one. Both restate a global condition on every card that happens to be on screen, to buy back a click that costs nothing — the pre-flight refuses before spending.
Out of credit last verdict said dry	Only by spending (cached ≤24h)	No — a top-up happens in a browser tab we cannot see	Nothing on the control. The pill carries it, once. The click is the probe; the toast is the report.	Not a block, and not a warning either. OutOfCreditModel documents that its cache lingers after a top-up, so both would fire at a researcher who has already paid — one stopping them, one talking them out of it.
Refused now the call actually failed	After spending	It is the ground truth	Fail loudly, name it, record it.	Not “partial success”. This is the tier that produced the Tagged 0 of 33 screenshot.plumbed

2  The control never changes

Not two ways and not three — one. Install stays Install in every state. It is not replaced by “Set up AI…”, it is not disabled, and it grows no warning. The researcher clicks, and the toast says what happened. Everything below is the same card and the same button; only the outcome differs.

The reasoning is one line: this is a global condition, and a codebook card is a local surface. A screen of twenty cards would say the same sentence twenty times, in the place least able to act on it, built on a cached verdict that is wrong the moment the researcher pays. It is already stated once in the titlebar, where a standing condition belongs, and it will be stated again — accurately, and within seconds — by the toast.

What this leaves behind

Outcomes B and C both end with the codebook installed and uncoded: importCodebookTemplate runs before startAutoCode, so the groups are linked whatever the job then does. That state is not new — it is the consequence of never gating, accepted deliberately — but it is currently invisible and unrecoverable, and that is the real work. Section 5.

3  The failure report, before and after

Before and after the plumbing. The left chip is the 3 Sep screenshot; the right is what the same failure produces now. Nothing here needs designing — it is shown because it is the load-bearing surface: once the control never changes, the toast is the only thing that tells a researcher what happened, so it has to be right.

And the app hears about it

The pill is settled and is not redrawn here — it shipped from out-of-credit-ux.html, which is the design of record. Reproduced below only so tier 2 has a referent. What changed is upstream of it: OutOfCreditModel was fed from PipelineRunner.deriveFailureState alone, so a sidecar AutoCode job that emptied the account lit nothing. It now hears via the bridge. The pill is the whole of tier 2’s ambient presence — one place, and its popover is the only resolve surface.

4  Which surfaces can name a failure

Every server route that spends tokens, and what it does today. With the control never changing, each of these needs exactly two things and no more: a pre-flight that refuses before spending where it cheaply can, and a failure that arrives named. No per-surface availability state, nothing to keep fresh. Three of the five have neither.

Surface	Route	Tier 1 gate	Tier 3 message	Note
Install / AutoCode	routes/autocode.py	yes — 3 refusals	namedfixed	Refusal mapper had no caller since the v1 lens was deleted; now wired.
Chat lens	routes/chat_lens.py	none	raw	502 carries f"{type(exc).__name__}: {exc}" — the raw SDK text.
Codebook builder	routes/codebook_builder.py	none	unhandled	Calls the LLM directly from the route.
Signal elaboration	server/elaboration.py	silent skip	silent	Logs a warning, returns cached only. The researcher sees fewer elaborations and is told nothing.
Analyse / re-analyse	routes/analysis.py	none in serve	events	The CLI path has preflight_api_key; serve does not.

The shared helper the table implies

Two hand-rolled copies of _has_api_key exist (routes/autocode.py, elaboration.py), neither reading the provider registry — a third provider-name switch to forget when a provider is added. One llm_availability(settings) → (ok, RefusalReason) collapses both and gives the four ungated surfaces the same vocabulary AutoCode already speaks.not done

5  The live question: getting back out of it

Never gating means outcomes B and C leave a codebook installed and uncoded. That is accepted. What is not yet decided is how the researcher gets out of it once they fix the provider — by switching to one that works, or by topping up the one that is stuck. The expectation to design against: autotagging should come back to life on its own.

What the code already does

Piece	State	Detail
Retry is possible	works	start_autocode_job 409s already_applied on a completed job but discards a failed one and re-runs. Calling the all-failed case completed was what walled the framework off for good.fixed
A retry codes everything	works	completed_at is the applied watermark, not a stop time. A job that coded nothing no longer stamps one, so it stays out of reapply_active_frameworks’ maintained set and a retry sees the whole corpus rather than the delta since the failure.fixed
The state is visible	no	installed means “has groups in this project”, and the template is imported before the job runs — so the card reads Uninstall and the navigator draws an uncoded codebook exactly like a coded one. Recovery today is Uninstall → Install, undiscoverable.
Something triggers the retry	no	Nothing re-runs a failed job. This is the design question.

The signals that exist

Event	What we get	Use
Switch provider, or paste a new key	.bristlenosePrefsChanged → ServeManager.restartIfRunning()	Strong — the sidecar restarts, so there is a clean moment to act.
A validation records .online after .outOfCredit	verdict-cache transition	The truest trigger — it fires on the change itself. But only when something actually revalidates.
Adding credit	nothing, ever	It happens in a browser tab. OutOfCreditModel’s own comment records that refocus re-reads the cache without revalidating — which is also why the pill lingers after a top-up.

Proposed

1. Revalidate on return from “Add funds…”, and retry on the transition. The popover button opens the billing console; when the app comes back, revalidate that provider. On .outOfCredit → .online, re-run whatever failed — sequentially, as reapply_active_frameworks already does “so spend stays legible”, with the ordinary progress chips. This is the jump-back-into-life, driven by the real event rather than a timer, and it fixes the pill’s lingering as a side effect.
2. Put the state on the row. Not chrome for a global condition — this one is per-framework, and the navigator currently asserts the opposite of the truth. It is also the only recovery for the case we can never detect.
3. Not on serve restart. Tempting, since a provider switch is a restart — but serve restarts for many reasons, and spending at process start, unattended, is a different act from spending because someone clicked.

Still to decide

1. Retry silently, or offer? Install already spends without confirmation because the researcher asked by clicking — and that ask is still standing and unfulfilled, so re-running fulfils it rather than making a new one. Against: if five codebooks failed, five jobs start at once. They are visible and cancellable in the chip stack, but nobody clicked five times.
2. Rate limits are the same shape and need no provider event. The work was asked for and never done, and the condition clears itself. There is currently no retry or backoff anywhere in the batch loop, so a single 429 kills a batch outright — a cheaper, independent win that does not wait on any of the above.
3. Azure and Gemini have no out-of-credit at all. Per the classifier’s own honesty note it is only isolable on Anthropic and OpenAI; Azure has no per-account credit and Gemini folds credit, quota and free-tier cap into one signal. So the transition trigger cannot fire for half the cloud providers, and the visible-state recovery has to carry them alone.

Settled, kept as reasoning

1. The control never changes. Two treatments were drawn and rejected: Install becoming “Set up AI…” when no provider is configured, and a per-card amber warning when the account is believed empty. Both restate a global condition on every card on screen — twenty codebooks, twenty copies — to save a click that is either free (the pre-flight refuses before spending) or is itself the only accurate test. The titlebar says it once; the toast says it accurately, within seconds.
2. No credit probe in serve mode. preflight/api_key.py already spends ~$0.0001 to answer this for run/analyze. Its only use here was the pre-emptive warning, which is rejected — so it would charge the researcher for a question nobody asks, on a surface they may only be reading, and still not answer for two providers. The run is the probe.