Voice and the assistant
Two independent halves. Read aloud speaks what an agent said. Talk listens, and turns speech into navigation, commands, dictation, or an assistant turn. Neither one needs the other, and both are off until you turn them on.
Read aloud, in three ordered layers
Getting this wrong is what makes a talking application unbearable, so the policy is three explicit layers rather than one switch:
- The master switch. Off means nothing is generated or played, anywhere, by any path. Settings, Voice owns it.
- Per session. Does this session generate speech at all.
- Per device, plus one global rule. The focused session plays here; every other session holds its clip and shows it as held on that pane's strip rather than talking over you.
A reply can be spoken as a summary, which costs a call to the cheap model you configured, or verbatim, which costs nothing and calls no model at all.
Which voice engine speaks
| Engine | Where it runs | What it costs |
|---|---|---|
| The operating system's voice | On your machine, through the OS speech engine. | Nothing. The default, and the fallback. |
| A local neural model | On your machine, from the voice-local extra. The Windows desktop bundle always carries it. | Nothing per call. A one-time model download. |
| External Edge TTS | A Microsoft endpoint, and text leaves the machine. | Explicitly experimental. Requires a versioned acknowledgement before anything is sent, and is never bundled. |
A word the local model can only spell out letter by letter is fixable with a respelling in Settings, Voice, applied without a restart. Each spell-out is recorded, so the list of words that need one is a list rather than something you have to notice.
Talk: hands-free conversation
Browser microphone capture, on-device voice activity detection, and local transcription. Nothing is written to disk, and the audio is decoded from memory.
- Wake words and commandsBoth are yours to configure. The set of actions a phrase can map to is fixed - send, append, cancel, undo, mute, read, interrupt, help, standby, resume, hold, proceed, stop and a few more - and the phrases that reach them are not.
- Spoken navigationProjects and sessions are addressed by number, following the order the sidebar draws them, so "Project 2, Session 3" resolves both coordinates at once without needing a list read out first.
- Standby and stopStandby keeps the microphone on and ignores everything except a resume or a stop. Stop releases the microphone.
- Hold and proceedThe brainstorm pair. Plain speech buffers instead of becoming turns until "go ahead" releases it as one consolidated turn.
- Push to talkHold Ctrl+Alt+Space for capture with no endpoint detection at all.
Playback leaves capture open, so you can interrupt. Speech immediately ducks the audio, waits for the echo to drain, and stops the stream if what it heard was really you - and restores playback if it was the echo.
The microphone needs a secure context. On a phone that means HTTPS, which is what the Tailscale Serve step on the phone page is for.
The Mux assistant
Ask for something in words and have it happen. The division of labour is the whole design: the model proposes names, deterministic code resolves and executes them through the paths a button would have used. The model does not get a shell, and it does not get an API of its own.
The confirmation floor for a consequential action is not configurable. There is no setting that turns it off, because a setting that turns off confirmation is the setting somebody turns off once and then forgets about.
A turn has a round budget it is told about and asked to work within, and running out of rounds is announced rather than silent. Anything you say while a turn is running is queued and merged into the next fragment rather than refused, because a refusal had nowhere to go and simply lost what you said.
Where the routing goes
Three tiers, in order, and the assistant is the last resort rather than the first: a deterministic grammar for help, fleet queries, navigation, and reply reading; then a conservative fuzzy pass; then - only on no match, and only when it is enabled - the assistant.