Metadata-Version: 2.4
Name: fast-mode-kit
Version: 0.2.0
Summary: Starter kit for adding a fast, data-only answer mode to any chat agent
License: MIT
Keywords: llm,agents,latency,tool-calling,evals
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: httpx>=0.27
Requires-Dist: pydantic>=2.6
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Dynamic: license-file

# fast-mode-kit

A starter kit for adding a **Fast mode** to any chat agent. Fast mode answers exactly what was asked, from your data, right the first time, in a few seconds.

It comes from shipping Fast mode in a live product. Every rule here traces back to a measured result or a real failure, and the [failure catalogue](docs/08-failure-catalogue.md) lists them.

It works with **frontier APIs** (any OpenAI-compatible endpoint) and with **models you host yourself** (mlx, vLLM, Ollama, llama.cpp).

## Quick start (offline, no key, no cost)

```bash
pip install fast-mode-kit

fastkit demo "rank products by revenue"     # the example shop agent, offline
fastkit eval                                # accuracy, latency, cost and fallback report
fastkit init my-agent                       # your own Fast mode project
cd my-agent && python agent.py "how many widgets in stock?"
```

The offline stand-in model picks tools by word overlap and has no understanding, so the example scores about 40% on `fastkit eval`. That is expected: it shows what keyword routing gets you. A real model fills the gap.

## With a real model

```bash
export FASTKIT_API_KEY=...          # leave unset for local servers (Ollama, vLLM, mlx, llama.cpp)

# 1. One tiny request: is this model fast enough, and does it call tools?
fastkit check --base-url https://openrouter.ai/api/v1 --model google/gemini-2.5-flash

# 2. Measure your agent on your questions, with a spend cap
fastkit eval --agent agent.py --gold gold.jsonl \
    --base-url https://openrouter.ai/api/v1 --model google/gemini-2.5-flash \
    --price-in 0.30 --price-out 2.50 --spend-cap 0.50
```

`fastkit check` prints PASS, SLOW or FAIL against the 4.5 s first-words budget and tells you whether the model called the tool. A model that answers without calling it can't run the one-call loop.

`fastkit.checks.safe_filter` runs after the answer has finished. Use it in evals and logs, or buffer the output if you want to filter before the user sees it.

## Develop

```bash
git clone <this repo> && cd fast-mode-kit
pip install -e ".[dev]" && pytest -q        # every test runs against a mock model
```

## What's inside

| Path | What it is |
|---|---|
| `docs/` | the guide, chapters 1-9: what Fast is, architecture, deadlines, prompts, checks, model choice, self-hosting, failure catalogue, measuring |
| `fastkit/templates/` | prompt templates: Fast system, reader, writer, server-owned lines |
| `fastkit/client.py` | streaming client for any OpenAI-compatible server |
| `fastkit/loop.py` | the one-call loop: read, call tools and write in one conversation, with a fallback |
| `fastkit/pipeline.py` | the fallback: a reader plans, the server fetches, a writer answers |
| `fastkit/hedge.py` | hedged routes; a stream that has started is never cut |
| `fastkit/tools.py` | tool registry, `find_tools`, `note_turn`, propose/confirm writes |
| `fastkit/checks.py` | do-no-harm figure checks |
| `fastkit/modes.py` | Fast-only gate |
| `fastkit/mock.py` | scripted model for offline tests |
| `fastkit/examples/shop/` | a complete agent over a toy shop database |
| `fastkit/evals/` | gold-set runner, report and spend cap |
| `fastkit/cli.py` | the `fastkit` command: demo, init, check, eval |
| `fastkit/scaffold/` | the starter project `fastkit init` writes |

## Adapt it to your domain in 5 steps

1. **Tools:** write read tools that return rows with every derived figure already computed (totals, ranks, percentages), plus propose tools for writes. See `fastkit/examples/shop/tools.py`.
2. **Prompt:** render `fastkit/templates/fast_system.md` with your domain and tool list.
3. **Loop:** call `fast_answer(...)` with your model and a pipeline fallback. See `fastkit/examples/shop/agent.py`.
4. **Gold set:** write 40-60 real questions with their expected figures, across the languages your users write in.
5. **Measure:** run `fastkit eval` on two or three models and pick by tail latency and per-language accuracy, not by the median.

## The rules in one screen

- No second chance in Fast, so the first answer must be right.
- One line per asked part, figure first, data only.
- The server computes every figure. The model copies.
- Fall back to another model path, never to keyword guesses.
- A deadline never cuts an answer that has started streaming.
- A check that edits a correct line is a bug in the check.
- Keep Fast changes behind one gate, and prove the other modes unchanged.
- Never judge a reading model with thinking switched off.

## License

MIT
