Cascade Guard

checking server…

In a chain of AI agents, one agent's mistake gets accepted by the next and builds up until it reaches the final answer. Cascade Guard checks every step, re-runs a failing step, and stops the order if it can't be fixed, so a wrong answer is never delivered silently.

API key needed

This server is protected. Enter your key: an admin key (API_KEY), a reviewer key or a viewer key. It is kept only in this browser tab.

1. Live demo: the same order, with and without the guard

Faults are injected into the agents' outputs on purpose. Both sides get exactly the same faults.

Without guard

Press “Run demo”.

With guard

Press “Run demo”.

2. Run it on a real LLM

Now the analyze and reason agents are a language model, and the guard is a second model call that checks each answer before the next agent uses it.

checking model…
No model connected yet.
  1. Run cascade-guard init in the working folder (or copy .env.example to .env).
  2. Paste your API key into it (Anthropic, OpenAI, Groq, Gemini, OpenRouter or a local Ollama all work).
  3. Stop the server (Ctrl+C) and start it again, then reload this page.
Simulated model (LLM_PROVIDER=fake): the answers are made up offline and cost nothing. Use it to try the page, not to judge a model.

Without guard

Press “Run on LLM”. Each run makes a few model calls.

With guard

Both sides start from the same first answer of the model.

3. Pilot: invoice checking

A real-world workflow on LangGraph: read the invoice, price it, approve it, write the summary. Every step is checked before the next one uses it. Paste your own invoice in the same layout, or use a sample.

Injected errors imitate real ones: a price with the decimal point moved, a wrong quantity, a dropped or duplicated line, a wrong tax rate, a flipped approval. Some mistakes are systematic and come back on a blind retry; with guided repair the agent is told exactly what was wrong (simulated for the rule agents, real for LLM agents).

Without guard

Press “Check this invoice”.

With guard

Same invoice, same injected errors.

4. Measured results over many orders

Runs hundreds of random orders through the chain three ways and compares them.

5. Check any AI output

Paste text from any agent. It flags empty output and known failure markers (CORRUPTED_OUTPUT, MISSING_DATA, INVALID_REASONING, TRUNCATED_OUTPUT).

With the LLM validator: check an output against its source

Works for any agent, not just this demo: describe the task, paste what the agent was given and what it produced. JSON or plain text.

6. Review queue and history

Every LLM run and LLM check is saved in a local database. Jobs the guard stopped, and checks that did not pass, wait here for a person to decide.

loading…

Waiting for review

Nothing yet.

Recent runs

Nothing yet.

7. Use it from your own system

Any program can call the same API the dashboard uses. Interactive docs are at /docs.


8. Evaluate a model: measured error rates

The full evaluation, the same as cascade-guard eval: how often each agent is right, how many errors the checker catches and how often it wrongly rejects, and what the guard does end to end. Every rate comes with a 95% interval. Each seed is one independent repeat; results are pooled. Results are also saved in results/.

Sections 1 and 3 use rule-based agents, so they show how the chain's design lets errors through. Sections 2, 5 and 8 use your configured model (or the simulator); section 8 gives the measured error rates.