Metadata-Version: 2.5
Name: gauntlet-spec
Version: 1.4.0
Summary: Adversarial multi-agent development harness: PRD -> plan -> phased implementation, every artifact reviewed adversarially
Project-URL: Homepage, https://github.com/johnpletka/gauntlet
Project-URL: Repository, https://github.com/johnpletka/gauntlet
Project-URL: Issues, https://github.com/johnpletka/gauntlet/issues
Author-email: John Pletka <john.pletka@gmail.com>
License-Expression: MIT
License-File: LICENSE
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.10
Requires-Dist: fastapi>=0.111
Requires-Dist: httpx>=0.27
Requires-Dist: jinja2>=3.1
Requires-Dist: jsonschema>=4.21
Requires-Dist: litellm<1.89,>=1.40
Requires-Dist: pydantic>=2.7
Requires-Dist: pytest>=8.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: typer>=0.12
Requires-Dist: uvicorn>=0.30
Description-Content-Type: text/markdown

# Gauntlet

Adversarial multi-agent development harness. Every artifact — PRD, plan, and
each implementation phase — runs the gauntlet of adversarial review before it
ships: a **builder** agent implements, an independent **reviewer** agent
attacks the result, a cheap **triage** model sorts the findings, the builder
fixes, and the reviewer confirms the fix against the diff. A localhost
**judge** service gates every tool call the agents make, failing closed.

A local-first, loopback-only **console** (`gauntlet serve`) makes every run
visible, answerable, and recoverable from the browser, and the CLI exposes the
same observability — live log tailing, machine-readable status, and guarded
recovery — for headless use.

The canonical spec is [`PRD-gauntlet.md`](PRD-gauntlet.md). The bootstrap plan
is [`runs/gauntlet/plan.md`](runs/gauntlet/plan.md).

> **Status:** the bootstrap is complete — Gauntlet was built by running its own
> pipeline against itself (phases P1–P7, each adversarially reviewed and
> human-ratified). It is usable on other repositories via the steps below.

---

## Table of contents

- [How it works](#how-it-works)
- [Prerequisites](#prerequisites)
- [Install](#install)
  - [macOS / Linux](#macos--linux)
  - [Windows](#windows)
- [Configure credentials](#configure-credentials)
- [Quick start (≤ 3 commands)](#quick-start--3-commands)
- [Authoring a PRD (the repo teaches you how)](#authoring-a-prd-the-repo-teaches-you-how)
- [The run lifecycle](#the-run-lifecycle)
- [Watching a run (console + observability)](#watching-a-run-console--observability)
- [Command reference](#command-reference)
- [Configuration](#configuration)
- [Safety model](#safety-model)
- [Development](#development)
- [Troubleshooting](#troubleshooting)

---

## How it works

A *pipeline* (YAML) is a sequence of stages; each stage is built from a few
step types:

| Step type | What it does |
|---|---|
| `agent_task` | The builder implements a phase in the working tree. |
| `shell` | Runs a command (e.g. the test suite) as a hard gate. |
| `commit` | Commits the phase with an enforced message format. |
| `adversarial_cycle` | review → triage → fix → confirm, looped to convergence. |
| `human_gate` | Pauses the run for a human to `approve` / `reject`. |

The **central invariant** is that the working tree is clean and committed at
every point where control passes to the reviewer — this is what makes review
diffs meaningful and `kill -9` resume safe.

Two pipelines ship by default: `standard` (for real work) and `bootstrap` (the
self-hosting pipeline used to build Gauntlet itself).

---

## Prerequisites

Gauntlet is a thin orchestrator that drives external agent CLIs and model APIs.
You need:

| Requirement | Why | Notes |
|---|---|---|
| **Python ≥ 3.10** | runtime | Managed for you by `uv`. |
| **[`uv`](https://docs.astral.sh/uv/)** | install + run | The only build/run tool you install by hand. |
| **`claude` CLI** ([Claude Code](https://docs.claude.com/en/docs/claude-code)) | the **builder** agent | Must be installed and authenticated. |
| **`codex` CLI** ([Codex CLI](https://github.com/openai/codex)) | the **reviewer** agent | Must be installed and authenticated. |
| **`OPENAI_API_KEY`** | triage / judge / escalation tiers | Default config uses `gpt-5-mini` (triage, judge) and `gpt-5` (escalation) via LiteLLM. |

The default agent profiles are: builder = `claude` (model `opus`), reviewer =
`codex` (model `gpt-5.5`), triage/judge = `gpt-5-mini`, escalation = `gpt-5`.
You can repoint any tier to a different provider in config (see
[Configuration](#configuration)). The default review panel runs the `reviewer`
profile under both the correctness and spec-coverage lenses. The scaffolded
config includes a commented Gemini profile for using a distinct provider for
the latter; enabling it requires `GEMINI_API_KEY`. `ANTHROPIC_API_KEY` is only
needed if you switch an API-backed tier to Anthropic.

---

## Install

### macOS / Linux

**1. Install `uv`** (if you don't have it):

```sh
curl -LsSf https://astral.sh/uv/install.sh | sh
```

**2. Install the agent CLIs** and sign in to each (follow each tool's own docs):

```sh
# Claude Code (builder) — see https://docs.claude.com/en/docs/claude-code
claude --version        # confirm it's on PATH
claude /login           # or however your install authenticates

# Codex CLI (reviewer) — see https://github.com/openai/codex
codex --version
codex login
```

**3. Install Gauntlet** as a global tool:

```sh
uv tool install gauntlet-spec       # from PyPI; or the git URL below for HEAD
# uv tool install git+https://github.com/johnpletka/gauntlet.git
gauntlet version
```

> **The PyPI package is `gauntlet-spec`, not `gauntlet`.** The bare name
> `gauntlet` on PyPI is an unrelated (and broken) project. The installed command
> is still `gauntlet` — only the install name differs.

> **Python 3.10+ is required.** If your default interpreter is older, `uv` will
> refuse with `does not satisfy Python>=3.10`. Add `--python 3.10` (or newer) to
> the command and `uv` will fetch a suitable interpreter automatically.

This puts two console scripts on your PATH: `gauntlet` (the CLI) and
`gauntlet-judge-hook` (the per-tool-call safety hook, wired automatically by
`gauntlet init`).

### Windows

Gauntlet itself is pure Python and runs natively on Windows via `uv`. Use
**PowerShell**.

**1. Install `uv`:**

```powershell
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
```

**2. Install and authenticate the agent CLIs.** Install `claude` (Claude Code)
and `codex` per their official docs and confirm each is on your `PATH`:

```powershell
claude --version
codex --version
```

> **Note on the agent CLIs:** if a given CLI does not yet ship a native Windows
> build, install Gauntlet and that CLI inside **WSL2** (Ubuntu) and follow the
> macOS / Linux steps there instead. The orchestrator, judge service (loopback
> HTTP on `127.0.0.1`), and hooks are all cross-platform; the only
> platform-sensitive dependency is the agent CLIs themselves.

**3. Install Gauntlet:**

```powershell
uv tool install gauntlet-spec
# or, for HEAD: uv tool install "git+https://github.com/johnpletka/gauntlet.git"
gauntlet version
```

> **The PyPI package is `gauntlet-spec`, not `gauntlet`** — the bare name is an
> unrelated, broken project. The command is still `gauntlet`. If `uv` reports
> `does not satisfy Python>=3.10`, append `--python 3.10` (or newer) and it will
> fetch a compatible interpreter.

---

## Configure credentials

The API tiers (triage, judge, escalation) read credentials **from the
environment only** — never from repo config (so keys never get committed).

**macOS / Linux** (add to `~/.zshrc` / `~/.bashrc` to persist):

```sh
export OPENAI_API_KEY="sk-..."
```

**Windows — PowerShell** (current session):

```powershell
$env:OPENAI_API_KEY = "sk-..."
```

**Windows — persist across sessions:**

```powershell
setx OPENAI_API_KEY "sk-..."
# then open a new terminal
```

Run `gauntlet doctor` (below) to verify everything resolves before your first
run.

**macOS — the sandboxed verifier and your `claude` login.** The adversarial
verifier runs `claude` in an isolated `HOME` (it hides `~/.aws`/`~/.ssh` from
un-hooked subprocesses). On macOS the `claude` login lives in the **Keychain**
(no `~/.claude/.credentials.json`), and that isolation breaks the CLI's Keychain
lookup. The verifier handles this for you: it reads your **existing** login
session from the Keychain and hands it to the sandboxed turn, so a normal
`claude /login` just works — no extra setup.

If you'd rather not depend on the Keychain session (it holds a short-lived token
the sandbox can't refresh, and CI has no interactive login), set an explicit
long-lived token, which takes precedence:

```sh
claude setup-token                       # mints a long-lived OAuth token
export CLAUDE_CODE_OAUTH_TOKEN="sk-ant-oat-..."   # add to ~/.zshenv to persist
```

Either way this is the one claude credential the verifier is allowed to carry
(the same class as the run's judge token); every other secret stays stripped from
the sandbox. Linux hosts with file-based `~/.claude/.credentials.json` are
unaffected. If neither the session nor a token is available, the verifier parks
the review closed with an actionable hook-probe message rather than running
unauthenticated.

---

## Quick start (≤ 3 commands)

From the repository you want Gauntlet to work on:

```sh
gauntlet init        # 1. scaffold config, pipeline, prompts, policy + wire hooks (idempotent)
gauntlet doctor      # 2. validate CLIs, auth, hook wiring, judge, API keys
gauntlet new myfeat  # 3a. scaffold .gauntlet/runs/myfeat/ with a PRD stub
#    ...author .gauntlet/runs/myfeat/prd.md...
gauntlet run myfeat  # 3b. start the pipeline
```

If the repository already carries committed Gauntlet assets (a teammate ran
`init` before you), you only need to wire **this machine's** hooks:

```sh
gauntlet init --from-repo
```

`gauntlet doctor` reports actionable, per-check status — installed CLI versions
vs. the verified pin file (`.gauntlet/pins.yaml`), authentication, hook wiring,
judge startability, and ApiAdapter keys — and exits non-zero on any blocker.

---

## Authoring a PRD (the repo teaches you how)

A Gauntlet run starts from a human-authored PRD. `gauntlet init` installs two
committable aids so you don't have to carry the conventions in your head — and a
teammate who clones the repo inherits both automatically:

- **A Claude Code skill** at `.claude/skills/gauntlet-prd-author/SKILL.md`. In a
  Claude session, a natural-language request like *"help me write a PRD"* or
  *"start a Gauntlet run"* triggers it; it routes you to this repo's authoring
  playbook (`prompts/prd-author.md`, under your `asset_root`) and the conventions
  for where the PRD lives and how to scaffold and launch it. It's a thin pointer
  to the playbook, not a copy, so there's one source of truth.
- **A structured stub.** `gauntlet new <slug>` writes a PRD stub with the
  playbook's full section skeleton and a one-line hint per section, so you start
  from the right shape. The stub is the committable template
  `<asset_root>/prd-stub.md` — edit it to change the house style for every future
  PRD.

The skill *teaches and routes*; it never authors the PRD for you. A human writes
and ratifies it (FR-10.1): `gauntlet run` refuses to start while the file is
still the stub (marker present, or no substantive content added), so an unfilled
skeleton can't become a runnable non-PRD.

Both aids are idempotent and never-clobber: re-running `gauntlet init` leaves any
customization byte-for-byte intact (only an *unmodified* generated file is ever
refreshed, and only after a template version bump). `gauntlet doctor` includes a
warn-only check that the skill is installed and well-formed — it never blocks a
run, since the skill gates nothing.

---

## The run lifecycle

A run advances automatically until it hits a `human_gate`, then **parks** for
your decision:

```sh
gauntlet run myfeat              # start (parks at the first gate)
gauntlet status myfeat           # see current step + every step's state
gauntlet approve myfeat          # accept the parked gate; drive to the next one
gauntlet reject myfeat --notes "…"   # send the phase back for another fix round
gauntlet resume myfeat           # resume after an interruption (kill -9 safe)
gauntlet resume myfeat --response "…"   # decide an upstream conflict (see below)
gauntlet report myfeat           # per-step / per-agent cost, tokens + clock time
```

- **Interrupted runs are resumable.** The local journal is authoritative;
  `manifest.json` is its rebuildable projection. Both CLI status and the console
  read the journal when that projection is stale, missing, or corrupt.
  `gauntlet resume` re-enters at the last incomplete step. A step that wrote a
  dirty tree before dying is parked or reset rather than re-run blindly.
- **Completion travels through Git.** After a run finishes, the engine commits
  `runs/<slug>/<run-id>/completion.json` on the run branch — without moving
  your checkout, even when a same-tree run completes while you are on another
  branch. A checkout without the local journal reads this validated terminal
  snapshot; its first mutating command imports the same state into a journal.
  A journal this checkout has driven always wins, including after rollback. A
  journal holding only an imported snapshot follows the branch: if the branch
  is later rewound upstream and the snapshot disappears, the checkout
  bootstraps again from what Git carries. The completion commit stages only
  this file. If publication fails, resume the completed run to retry without
  rerunning its agents. Older runs without this snapshot retain their legacy
  behavior; reconcile their manifests once before sharing them through Git.
- **Provider usage limits pause, they don't destroy.** A quota/429/usage-limit
  hit mid-step — including inside a review cycle's sub-agents — **parks** the run
  (`parked_usage_limit`) with the worktree untouched and the agent session
  preserved; `gauntlet resume` continues the *same session* with a short
  continuation prompt instead of re-running the step. Cycle sub-steps checkpoint
  as they complete, so a resumed cycle re-enters at the first incomplete
  sub-step. Builders also commit `P<N> wip:` milestones inside a phase, bounding
  worst-case lost work to one milestone. Opt-in `resume_on_quota: auto`
  uses a future structured provider reset when available (never closer than
  60 s), otherwise retrying on `quota_retry_interval_s`; recognized quota
  denials keep retrying until the policy is disabled or the run is aborted.
  After `quota_denials_before_escalation` denials in a row the run is flagged —
  once, distinctly — as a possibly persistent restriction (billing / plan),
  since the classifier cannot tell that apart from a session window.
- **Laptop sleep is survivable.** A driver heartbeat detects host suspension and
  credits the slept time back to the running step's deadline (capped), so
  closing the lid neither silently stalls the run nor spuriously kills a healthy
  step; `status` reports detected suspensions. `keep_awake` (default `true`)
  wraps the driver in `caffeinate -i` on macOS so the host does not sleep
  mid-run; set it to `false` to opt out.
- **Malformed structured artifacts self-repair.** Agent-authored artifacts (like
  the plan's `gauntlet-phases` block) are validated in-step; the agent gets its
  own parse error back for a bounded repair loop, and if that fails the run
  parks (`parked_artifact_invalid`) for a sanctioned hand-edit — `resume`
  re-runs only the validator and audits the edit via content hashes.
- **Approved artifacts are immutable.** A later phase that finds an approved
  PRD/plan incomplete *halts and surfaces the conflict* rather than amending it.
  You resolve that conflict with `gauntlet resume <slug> --response "…"` (see
  **Resolving an upstream conflict** below), which routes any artifact change
  back through its own gate rather than letting the builder amend it in place.
- At the final gate a **`PR.md` draft** is written under `.gauntlet/runs/<slug>/`
  (it is **not** opened or pushed — that stays a human action).
- After a run, `gauntlet feedback <slug>` captures your retrospective notes and
  triage corrections to feed the self-improvement loop.

---

## Watching a run (console + observability)

A run advances on its own between gates, so the question is usually *"where is it
now, and does it need me?"* Gauntlet answers that two ways — a browser console
and CLI primitives that expose the same state for headless/CI use.

### The console (`gauntlet serve`)

```sh
gauntlet serve                 # loopback-only, token-authenticated console
gauntlet serve --resume        # reuse/boot the console, open the browser, return
gauntlet run myfeat --watch    # boot/reuse the console, open the browser, then run
```

`gauntlet serve` starts a **loopback-only, token-authenticated** web console that
runs strictly *above* the orchestrator: every control action it offers launches
the same sanctioned `gauntlet` CLI verb you would type, so it inherits every
safety invariant rather than being able to weaken one. It lists every run across
all slugs with live status / current step / cost, drills into each step's
`prompt.md`, rendered `transcript.md`, and `events.jsonl` (with live tailing for
running steps), assembles the evidence behind a parked gate and offers
**Approve / Reject** in one place, and classifies a failed/parked run into the
action that actually applies. It can also launch and abort runs as supervised
children and survive its own restart by re-attaching to live PIDs, and adds an
in-tab notification channel on top of the driver's own push (below).

**Notifications come from the driver itself.** Every park (gate, escalation,
decision, usage limit, provider outage, usage window, invalid artifact), halt,
failure and completion is pushed the instant the driver persists it — to macOS
desktop, a Slack incoming webhook, and/or a generic JSON webhook — whether or
not a console is running, so detection latency no longer depends on someone
being resident (#134). A gate notification carries a pre-built review bundle:
`git diff --stat` of the reviewed range, finding/triage counts, spend and
elapsed time, and the exact next command. Emissions are recorded in the run's
`notifications.jsonl` ledger, which the console consults so the two never
double-fire. Configure it once in `.gauntlet/config.yaml`:

```yaml
notify:                      # driver-side push (defaults shown; all opt-out)
  desktop: true              # terminal-notifier / osascript on macOS
  slack: true                # fires only when a webhook resolves
  slack_webhook: null        # or the GAUNTLET_SLACK_WEBHOOK env var
  webhook: true              # generic JSON POST; fires only when a URL resolves
  webhook_url: null          # or the GAUNTLET_NOTIFY_WEBHOOK env var
  # kinds: [gate-reached, escalation-parked, run-failed]   # allowlist; absent = all
```

Kinds: `gate-reached`, `escalation-parked`, `parked-for-response`,
`parked-usage-limit`, `parked-provider-unavailable`, `parked-usage-window`,
`parked-artifact-invalid`, `run-halted`, `run-failed`, `run-completed` (plus the
console-only advisories `usage-window-warning`, `gate-auto-approved`, and
`run-orphaned`). `GAUNTLET_NOTIFY_DISABLED=1` silences the driver for one
invocation. An absent `web.notify` block inherits `notify:`.

`gauntlet run --watch` ensures the console is up (booting or reusing it), prints
its URL, and **opens the authenticated console in your browser** before running
in the foreground; pass `--no-browser` (on either command) to skip the launch.
`--console-host` / `--console-port` override the bind (default `127.0.0.1:8765`). `gauntlet serve --resume` does the same boot-or-reuse-and-open without holding the foreground.

### Unattended recovery: `gauntlet sweep`

A dead driver cannot self-resume, and a stale drive lock is only ever reclaimed
by the next driving verb someone types. `gauntlet sweep` is the idempotent,
judgment-free sweep a resident process runs instead of a human (#134):

```sh
gauntlet sweep myfeat          # one run: act only on a no-decision rule
gauntlet sweep --all           # every run under run_root, each resume detached
gauntlet sweep --all --json    # the same, one object per run
```

It takes exactly three actions: **reclaim an orphaned run** whose drive lock
proves the driver dead or PID-reused, **arm a fallback schedule on a legacy
recognized quota park**, and **fire a due `scheduled_resume`** on a usage-limit
/ provider-unavailable park under the knob that armed it
(`resume_on_quota: auto` / `resume_on_provider_unavailable: auto`). Everything
else — gates, response parks, failures, indeterminate liveness, malformed
locks, live drivers, terminal runs — is skipped with a one-line reason. Exit 0
whether or not anything was resumed. Every action stamps
`unattended sweep resumed (<reason>) at <iso>` into the manifest, and a
`--all` resume appends its output to `<run_dir>/sweep-resume.log`.

`gauntlet serve` runs the same sweep on a timer (`web.sweep_interval_s`,
default 120; 0 disables), launching each resume as a console-owned driver.
Without a console, schedule it yourself and set `external_scheduler: true` so
the config lint knows the wait is covered:

```sh
# cron: every 5 minutes
*/5 * * * * cd /path/to/repo && /path/to/gauntlet sweep --all >> ~/.gauntlet/sweep.log 2>&1
```

```xml
<!-- ~/Library/LaunchAgents/com.gauntlet.sweep.plist (macOS) -->
<plist version="1.0"><dict>
  <key>Label</key><string>com.gauntlet.sweep</string>
  <key>ProgramArguments</key>
  <array><string>/path/to/gauntlet</string><string>sweep</string><string>--all</string></array>
  <key>WorkingDirectory</key><string>/path/to/repo</string>
  <key>StartInterval</key><integer>300</integer>
  <key>StandardOutPath</key><string>/Users/you/.gauntlet/sweep.log</string>
  <key>StandardErrorPath</key><string>/Users/you/.gauntlet/sweep.log</string>
</dict></plist>
```

Mutual exclusion between a cron sweep, the console's sweep and your own
`resume` rides on the drive lock: a resume that loses the race fails closed
inside the engine and the sweep reports it as `refused`, never retried.
### Plan preconditions

A plan's `gauntlet-phases` block may declare the environmental things its phases
depend on and no agent can create — staged data files and environment variables
(#134). Provision separately before approval; command items are rejected. Per phase or for the whole plan (mapping form):

```markdown
```gauntlet-phases
preconditions:                       # whole-plan items
  - {env: OPENAI_API_KEY, description: "scoring calls"}
phases:
  - id: P1
    title: Build the feature table
    goal: …
    frs: [FR-1.1]
    acceptance: [{id: P1-A1, clause: "…"}]
    preconditions:                   # this phase's own items
      - {path: data/restricted/bundle.parquet, description: "staged by ops"}
```
```

`plan-lint` fails closed on a malformed item. `gauntlet approve` on the plan
gate (`preflight: plan_preconditions`) checks every item without executing plan
text, records the checklist under `<run_dir>/preflight/`, and
**refuses while any is unmet**, listing each; `--skip-preflight` approves anyway
with an audited manifest warning. Each implement phase (`preconditions_from:
plan`) re-resolves the plan-level items plus its own before the builder
launches; an unmet item fails the step as a re-runnable precondition (nothing
invoked, no tokens spent) and a plain `gauntlet resume` re-checks. `gauntlet
status` on a parked plan gate lists unmet `path`/`env` items read-only. An `env` value is only ever tested for presence and
never written anywhere.

### CLI observability

```sh
gauntlet status myfeat              # driver liveness, run-state, next action
gauntlet status myfeat --json       # the same state as one machine-readable object
gauntlet logs myfeat                # a step's dir + transcript tail (read-only)
gauntlet logs myfeat --follow       # tail a running step's events.jsonl live
gauntlet recover myfeat             # terminate a verified-wedged driver (guarded)
gauntlet run myfeat --interactive   # detach the run, foreground a monitor agent
```

- **`status`** reports driver liveness, the computed run-state, and the next
  action / recovery hint; `--json` emits the same payload (schema
  `schemas/status.json`) for scripts and CI. The payload carries run elapsed
  time, token/cost totals (run-level and per agent profile), per-step
  `duration_s`/`notes` and engine-stamped `halt_reason`/`parked_reason` enums,
  heartbeat age with detected suspension intervals, and the quota reset time on
  a usage-limit park — every parked/halted/failed state is explainable from
  `status` alone, no transcript required. Additions are strictly additive
  (`schema_version` stays 1); a consumer pinning an older strict schema copy
  must re-pin on upgrade.
- **`logs`** is strictly read-only evidence-on-demand; `--follow` streams a
  step's events as they're written (paired with opt-in live step streaming).
- **`recover`** terminates a driver only after verifying it is genuinely wedged,
  then marks its step `INTERRUPTED` so a plain `resume` re-enters cleanly — it
  never kills a healthy run.
- **`run --interactive[=claude|codex]`** launches the run detached and hands the
  terminal to an interactive monitoring agent (wired to the run's judge as the
  operator's own session); `status --interactive` attaches the same monitor to an
  already-running run. An installed **`gauntlet-operator`** Claude Code skill
  routes a supervising session to this repo's recovery playbook.

---

## Resolving an upstream conflict

When a builder finds that the approved PRD or plan is wrong or under-specified,
it **halts with an `UPSTREAM CONFLICT`** instead of working around the approved
artifact (FR-10.4). The step parks; the run is stuck until you decide. The
standard, audited way to decide is:

```sh
gauntlet resume <slug> --response "<your decision, in plain text>"
```

The decision is recorded verbatim in the manifest (timestamped, attributed to
your operator identity) and injected into a fresh builder run, which
**re-evaluates** the conflict in light of it rather than re-surfacing it. The
builder then emits one of three outcomes:

- **Proceeds** — the decision resolves the conflict within what the approved
  artifacts already allow (e.g. ratifying one of the options they leave open, or
  deferring out-of-scope follow-up to `FUTURE.md`). The run un-sticks and
  continues.
- **Re-parks for an artifact amendment** — the decision would require changing
  approved PRD/plan text (**including** "proceed even though this contradicts the
  plan"). There is **no** proceed-now-amend-later path: amend that artifact on
  its **own** branch, take it through **its own** review-and-gate cycle
  (FR-10.4), then resume again with a decision that no longer contradicts it.
- **Re-parks for clarification** — the decision was ambiguous; the builder names
  what it still needs. Supply another `--response`.

**Ratifying the artifacts as they stand.** When your decision is simply "the
PRD/plan as written (including any sanctioned hand-edit) are the approved
artifacts — proceed", use the structured form instead of prose (#134):

```sh
gauntlet resume <slug> --accept-artifacts
```

It records the sha256 of each governed artifact on the authoring surface
(`<run_root>/<slug>/prd.md`, `plan.md`) as ratified, appends an engine-generated
response naming those digests, and re-drives with `proceed_in_place` — no prose
is classified and no disposition model runs, so acceptance wording that happens
to contain imperative verbs can never be re-parked as an amendment request. A
digest that differs from the run's last-known approved one (a prior
ratification, else the bytes committed on the run branch) is recorded and
printed **loudly**, never refused: manual governed-artifact edits are a
sanctioned recovery workflow. Mutually exclusive with `--response`; only valid
for a `parked_for_response` park.

Notes:

- `--response` is **required** to resume a step parked on an upstream conflict.
  Other parks (e.g. a `human_gate`) are unaffected — use `approve` / `reject`
  for those, and a plain `gauntlet resume` for a non-conflict agent park.
- **Conflicts do not consume the retry budget** — only genuine failures do. You
  can supply as many `--response` cycles as it takes; you decide when to stop or
  abort.
- The whole history of your decisions is preserved in the manifest
  (`steps[N].human_responses`, append-only) and reaches git history, so the audit
  trail of who decided what, and when, is never lost.

---

## Command reference

| Command | Purpose |
|---|---|
| `gauntlet init [--from-repo]` | Scaffold config/pipeline/prompts/policy + wire hooks (idempotent). |
| `gauntlet doctor` | Validate environment: CLIs, auth, hooks, judge, keys. |
| `gauntlet new <slug>` | Scaffold `.gauntlet/runs/<slug>/` with a PRD stub. |
| `gauntlet run <slug> [--pipeline standard\|bootstrap] [--no-judge] [--watch] [--interactive[=claude\|codex]]` | Start a run on branch `gauntlet/<slug>`. `--watch` boots/reuses the console; `--interactive` detaches the run and foregrounds a monitor agent. |
| `gauntlet status <slug> [--json] [--interactive[=claude\|codex]]` | Show run status, driver liveness, and the next action; `--json` for a machine-readable payload; `--interactive` attaches a monitor. |
| `gauntlet logs <slug> [--follow]` | Surface a step's dir + transcript (read-only); `--follow` tails its `events.jsonl` live. |
| `gauntlet serve [--host …] [--port 8765]` | Run the loopback-only supervisory console (FR-11). |
| `gauntlet approve <slug> [--gate ID] [--notes …]` | Approve a parked gate, continue the run. |
| `gauntlet reject <slug> --notes … [--gate ID] [--terminal]` | Reject a parked gate: the note re-runs the gate's upstream review cycle as a new fix round. A gate with no upstream cycle would fail the run terminally — that requires the explicit `--terminal` flag. |
| `gauntlet resume <slug>` | Resume an interrupted run at its last incomplete step. |
| `gauntlet resume <slug> --response "…"` | Decide a step parked on an upstream conflict (FR-10.4); records the decision and re-runs the builder with it. Required for conflict parks. |
| `gauntlet recover <slug>` | Terminate a verified-wedged live driver and mark its step `INTERRUPTED` (guarded; FR-5). |
| `gauntlet abort <slug>` | Abort a run. |
| `gauntlet finish <slug>` | Merge a completed run into its base, then delete the branch + pointer. |
| `gauntlet clean <slug>` | Delete a merged run branch + clear its pointer; keep the run record. |
| `gauntlet report <slug>` | Per-step / per-agent-profile cost breakdown, incl. cache-read share per step type/profile, plus a clock-time section: the run's wall-clock span partitioned into disjoint agent time (the union of call intervals) / parked (by reason) / host-suspended / other, and agent-seconds per step, per agent profile (→ model) and per activity (review, triage, fix, confirm, verify pooled across cycles; other steps by id). Measured by the engine around every adapter call, so it is identical for Claude Code and Codex; each call also freezes the adapter/model/effort that ran, and the cost tables carry the raw cache-write and reasoning token counters. |
| `gauntlet ledger backfill` | One-shot, idempotent import of existing run manifests into the machine-global usage ledger (`~/.gauntlet/usage-ledger.jsonl`) so window-admission estimates have history. |
| `gauntlet feedback <slug>` | Capture human feedback + triage corrections (FR-6.1). |
| `gauntlet rollback <slug> --phase N` | Reset the branch + manifest to a phase boundary (guarded). |
| `gauntlet judge serve [...]` | Run the localhost judge service (normally engine-managed). |
| `gauntlet version` | Print the installed version. |

`--no-judge` disables the safety judge and is for **testing only** — it leaves
agent tool calls ungated. Don't use it on real work.

---

## Configuration

`gauntlet init` writes a `.gauntlet/` directory in your repo:

- **`.gauntlet/config.yaml`** — agent profiles (adapter + model + flags),
  per-agent commit identities, run timeouts and budgets. References models, not
  credentials.
- **`.gauntlet/pins.yaml`** — the CLI versions and exact flags verified by the
  contract suite; `doctor` checks the installed CLIs against it.

Pipelines, prompt templates (versioned data, not code), structured-output
schemas, and the judge fast-path `policy.yaml` all live under `.gauntlet/` too
— `.gauntlet/pipelines/*.yaml`, `.gauntlet/prompts/`, `.gauntlet/schemas/`,
`.gauntlet/policy.yaml`. The config's `asset_root` (default `.gauntlet` in a
scaffolded repo) is where the engine resolves them; everything is committable,
so a teammate who clones the repo gets the identical workflow. (Gauntlet's own
source repo sets `asset_root: "."` to keep these assets at the repo root as
first-class source rather than tucked into a dotfile dir.)

To repoint a tier at a different provider, edit the agent profile's `adapter`
and `model` in `.gauntlet/config.yaml` and set that provider's key in your
environment (e.g. `ANTHROPIC_API_KEY` for an `anthropic/*` model). LiteLLM
model naming applies to `api` adapter profiles.

**Per-agent reasoning effort.** Any profile (and any pipeline step, which wins
over its profile) accepts an optional `effort` drawn from the **canonical enum
`minimal` / `low` / `medium` / `high`**. The engine maps the canonical value to
each adapter's real surface: `claude-code` → `--effort` (which accepts
`low`/`medium`/`high`; canonical `minimal` remaps to `low` with a load-time
warning), `codex` → `-c model_reasoning_effort=…`, `api` → the
`reasoning_effort` param. A value an adapter/model cannot accept is a
**config-load error**, never a silent drop. Optional and no-op when absent. A
natural use is a cheaper fixer role for review-fix rounds while the initial
builder runs at higher effort:

```yaml
agents:
  builder:   { adapter: claude-code, model: opus,   effort: high }
  impl_fixer:{ adapter: claude-code, model: sonnet, effort: medium }
  reviewer:  { adapter: codex,       model: gpt-5.5, effort: high }
```

The `judge_llm` profile uses this same validated `effort` value. Its
backward-compatible default is `minimal`; models that reject that tier can set
`effort: low` (or another supported canonical tier). `gauntlet doctor` executes
one live probe through the judge's actual classifier schema, timeout, and effort
path, so an incompatible model/effort pair fails preflight instead of denying
every agent tool call at runtime.

Mechanical emissions — commit-message drafting and resume-disposition output —
run on a designated cheap `mechanic:` profile in the shipped config, so the
builder's constrained provider window is spent on building.

**Resilience & window knobs** (all default to today's behavior; opt in per
knob):

```yaml
resume_on_quota: notify      # notify (default) | auto — self-resume a
                             #   usage-limit park at a structured reset deadline,
                             #   else on the fallback cadence below
quota_retry_interval_s: 1800 # fallback cadence; prose reset hints are not parsed
quota_denials_before_escalation: 6
                             # consecutive quota denials before `status` and the
                             #   notifier flag a possibly persistent restriction
                             #   (retries continue; you decide to stop or abort)
keep_awake: true             # default; wraps the driver in `caffeinate -i`
                             #   (darwin) — false lets the host sleep mid-run
resume_on_provider_unavailable: notify
                             # notify (default) | auto — self-resume a
                             #   provider_unavailable park (bounded dependency
                             #   retries exhausted) at its recorded backoff /
                             #   Retry-After deadline; the same in-process wait,
                             #   survival requirement (no provider health probe —
                             #   the deadline is the only signal)
max_auto_resume_attempts: 3  # provider_unavailable ceiling; recognized quota
                             #   retries continue until disabled or aborted
heartbeat_interval_s: 15     # driver heartbeat cadence (suspend detection)
suspend_credit_cap_s: 43200  # max slept time credited back to a step deadline
checkpoint_commits: keep     # keep | squash — builders' intra-phase `PN wip:`
                             #   milestone commits; the phase always ends in a
                             #   `PN:` commit and reviewers always see the
                             #   cumulative range diff either way
triage_concurrency: 4        # bounded pool for per-finding triage calls;
                             #   final triage.json is byte-identical to a
                             #   sequential run on all-success rounds
providers:                   # pre-step window admission (FR-10); absent = off
  anthropic:
    window_hours: 5
    window_budget: 1500000   # in budget_unit
    budget_unit: tokens      # tokens | cost
    enforce: false           # false = advisory warning; true = park pre-step
                             #   (`parked_usage_window`) with zero work in flight
    # fallback_estimate: 50000   # used when the ledger has no history yet
```

Admission estimates come from the machine-global usage ledger
(`~/.gauntlet/usage-ledger.jsonl`, content-free counts only) that every run
appends to; seed it from past runs with `gauntlet ledger backfill`. The ledger
cannot see non-gauntlet usage, so admission is advisory by design — a wrong
*continue* is survivable via the reactive usage-limit park.

**Scoped context (pipeline-level).** `agent_task` inputs accept a per-input
mode so large artifacts travel by reference instead of being inlined into every
prompt — the CLI agents read them in-session, where subsequent turns hit the
provider prompt cache:

```yaml
- id: implement
  type: agent_task
  agent: builder
  inputs:
    - { name: prd.md,  mode: reference }   # inject the path, agent reads it
    - { name: plan.md, mode: phase }       # inject only the current phase's
                                           #   plan section + the full-doc path
  # (bare `- prd.md` still means mode: inline, today's behavior)
```

`reference`/`phase` require a profile whose adapter can read the repo (`api`
profiles can't; pipeline load fails closed, and `doctor` probes that a
reference-capable profile's sandbox can actually read a repo file). Agent-task
steps also accept `validate: <name>` (e.g. `plan_phases`) to check their output
artifact in-step with a bounded self-repair loop.

---

## Safety model

- Agent tool calls (e.g. the builder's shell commands and file writes) pass
  through a **PreToolUse hook → localhost judge service**. The judge decides via
  a deterministic policy fast-path, then an LLM classifier rung, and **fails
  closed** (deny) on timeout, parse error, or any unexpected outcome.
- The judge binds `127.0.0.1` only and rejects callers lacking the per-run
  token. Every decision is written to an audit log.
- The reviewer runs **read-only** (codex sandbox `read-only`); any worktree
  mutation by a reviewer is a detected process violation.
- Permission-bypass flags (e.g. `--dangerously-skip-permissions`) are rejected
  by config lint — they would disable the hook layer.

---

## Development

Working on Gauntlet itself:

```sh
uv sync                       # create the venv, install deps + package (editable)
uv run pytest                 # unit suite (no credentials required)
uv run pytest -m integration  # contract tests against live CLIs/APIs (needs creds)
uv run gauntlet doctor        # validate your dev environment
```

`uv run pytest` runs unit tests only; the `integration` marker selects the live
contract suite, which requires authenticated CLIs and API keys.

---

## Troubleshooting

- **`gauntlet` errors with `ModuleNotFoundError: No module named 'gauntlet'`**
  (or `gauntlet.main`) — you installed the unrelated PyPI package via
  `uv tool install gauntlet`. Run `uv tool uninstall gauntlet`, then reinstall
  the correct package: `uv tool install gauntlet-spec` (add `--python 3.10` if
  your default interpreter is older).
- **A teammate who hasn't installed Gauntlet** sees no hook errors. The wired
  PreToolUse `command` is an install-tolerant launcher: when `gauntlet-judge-hook`
  isn't on PATH it stands aside silently (exit 0) rather than emitting a per-call
  `command not found` notice — *unless* a gauntlet run is active. A shared repo can
  mix Gauntlet and non-Gauntlet developers freely.
- **A run halts with `gauntlet-judge-hook not on PATH during an active gauntlet
  run; failing closed`** — the hook console script isn't on the PATH the agent CLI
  sees *inside a run*, so the launcher fails closed (exit 2) rather than letting the
  run proceed ungated. Re-run `gauntlet init` (or `gauntlet init --from-repo`) and
  confirm `uv tool`'s bin directory is on your PATH (`uv tool update-shell`, then
  open a new terminal). On native Windows, run inside WSL2 — the launcher is POSIX
  sh (see the install note above).
- **`doctor` reports a stale CLI version** — your installed `claude` / `codex`
  differs from `.gauntlet/pins.yaml`. Re-verify with the integration suite, or
  update the pin file if the new version is intended.
- **`doctor` warns about `codex-cache`** — Codex and ChatGPT Desktop may be
  sharing `models_cache.json` while running different CLI versions or cache
  schemas. Align the PATH CLI with `.gauntlet/pins.yaml` and the reported cache
  writer, or give Gauntlet a separate `CODEX_HOME`. If the cache is reported
  mid-rewrite, retry after the Desktop/CLI refresh completes; move it aside for
  the PATH CLI to rebuild only if the warning persists.
- **A run parks unexpectedly / a step is `failed`** — `gauntlet status <slug>`
  shows where; the step's transcript under `.gauntlet/runs/<slug>/<run>/steps/` has the
  detail. `gauntlet resume <slug>` re-enters safely once the cause is cleared.
  A failed `shell` step with an `on_fail` route (the standard pipeline's
  `tests` / `tests-recheck`) whose retry budget is spent needs no git surgery:
  each plain `gauntlet resume <slug>` re-arms exactly one more route, audited
  as a manifest warning.
- **An agent hits a provider session/usage limit mid-step** — the engine fails
  the step closed (it does not fake success). Wait for the limit to reset, then
  `gauntlet resume <slug>`.
- **An ensemble member hits provider capacity or a transient startup fault** —
  Gauntlet spends the bounded persisted dependency-retry budget without reducing
  the panel. On exhaustion it parks `provider_unavailable`; a plain
  `gauntlet resume <slug>` retries only the incomplete member, with no
  `--response` decision required.

### Phase-scoped test commands

For large adopter suites, opt in to testing the changes in each implementation phase (#160):

```yaml
# .gauntlet/config.yaml: retain the authoritative FULL validation command.
test_command: "pnpm run typecheck && pnpm exec vitest run"
# Supply this repository-owned script before enabling the option.
phase_test_command: "pnpm run typecheck && node scripts/test-changed.cjs"
```

Gauntlet does not infer a repository's test graph. The phase command owns dependency selection,
explicit file-read/data guards, and full-suite fallback for unmapped inputs or test/toolchain
configuration changes. For example, Vitest's import graph alone cannot discover tests that read
YAML, SQL or JSON with `readFileSync`. Always union those guards with affected tests. Keep full
static typechecking; require an explicit, justified empty-selection outcome, and exit nonzero on
selection or test failures. Log the selected test IDs and reasons to stdout so the step transcript
records what ran. A failing phase command is a failed step, never silently replaced by a green full run.

The shipped standard pipeline marks `tests` and `tests-recheck` with `test_scope: phase`, and runs
`full-tests` once after all phases when this option is configured. Existing installations must update
their copied pipeline to include these annotations **and** the final full step. Configuration alone
never rewrites a pinned pipeline or a live run's config snapshot. In custom pipelines, use:

```yaml
- {id: tests, type: shell, test_scope: phase, run: "{{config.test_command}}"}
# At the final validation boundary (and whenever full validation is required):
- {id: full-tests, type: shell, test_scope: full, run: "{{config.test_command}}"}
```

`test_scope` accepts only `phase` or `full` on a shell step whose entire `run` is the trusted
`{{config.test_command}}` token. Other shell commands retain their existing behavior. No new
artifact-to-shell interpolation is permitted.

For an opted-in phase, the engine validates the immutable `phase_start_sha`, rather than the current
step's retry boundary or `HEAD~1`. Its inventory includes committed phase changes and review fixes,
staged/unstaged paths and nonignored untracked files. Renames include both old and new names.
A missing/invalid/non-ancestor baseline, Git failure, deleted path, unsupported changed entry, empty
change set or oversized context falls back to the full command. Project-specific uncertainty must
also fall back in the repository's selector. Recovery/rechecks reuse the persisted phase start;
rollback that resets the phase follows the existing phase-start reset contract.

The child receives engine-owned variables (ambient `GAUNTLET_TEST_*` values are removed):

- `GAUNTLET_TEST_MODE`: `phase` or `full`.
- `GAUNTLET_TEST_BASE_SHA`: validated phase-start commit, only in phase mode.
- `GAUNTLET_TEST_CONTEXT`: JSON, only in phase mode, with `version: 1`, `base_sha`, `head_sha`,
  `head_tree`, `changed_paths`, `deleted_paths`, `worktree_fingerprint`, `mode`, `reason` and `command`.
  Parse JSON instead of splitting filenames on whitespace. Paths are relative to the subprocess cwd.

The engine records `test-selection.json` next to each scoped shell step's output, including the
chosen command and fallback reason, plus exit status or timeout after execution. The fingerprint
identifies HEAD plus the current bytes/modes of changed files; it is evidence, not a test-result cache.
The verifier independently prepares the same contract inside its disposable Git worktree, adds only
these generated variables after secret stripping, and receives the chosen command in its prompt.
Its transcript remains the evidence of commands actually executed; the context file alone does not
claim that a verifier ran or passed tests. Existing verifier behavior remains unchanged when the
option is absent. A selector started outside Gauntlet must default to full validation when context
is absent. Keep normal CI validation as well as the final full-suite boundary.
