Metadata-Version: 2.5
Name: watch-skill
Version: 1.3.0rc2
Summary: Give any agent a video input: watch, index, ask, and iterate on video via MCP, CLI, REST, and Python.
Project-URL: Homepage, https://github.com/oxbshw/watch-skill
Project-URL: Documentation, https://github.com/oxbshw/watch-skill/tree/main/docs
Project-URL: Changelog, https://github.com/oxbshw/watch-skill/blob/main/CHANGELOG.md
Project-URL: Issues, https://github.com/oxbshw/watch-skill/issues
Project-URL: Source, https://github.com/oxbshw/watch-skill
Author: oxbshw
License: MIT
License-File: LICENSE
Keywords: agents,ffmpeg,mcp,transcription,video,vision,yt-dlp
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Multimedia :: Video
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Image Recognition
Requires-Python: >=3.11
Requires-Dist: httpx<1,>=0.28
Requires-Dist: pydantic-settings<3,>=2.14
Requires-Dist: pydantic<3,>=2.13
Requires-Dist: rich<16,>=13.7
Requires-Dist: typer<1,>=0.26
Provides-Extra: all
Requires-Dist: fastapi<1,>=0.139; extra == 'all'
Requires-Dist: fastembed<0.9,>=0.8; extra == 'all'
Requires-Dist: faster-whisper<2,>=1.2; extra == 'all'
Requires-Dist: fastmcp<4,>=3.4; extra == 'all'
Requires-Dist: imagehash<5,>=4.3; extra == 'all'
Requires-Dist: numpy<3,>=2.4; extra == 'all'
Requires-Dist: onnxruntime<2,>=1.27; extra == 'all'
Requires-Dist: opencv-python-headless<6,>=5.0; extra == 'all'
Requires-Dist: pillow<13,>=12.3; extra == 'all'
Requires-Dist: playwright<2,>=1.61; extra == 'all'
Requires-Dist: python-bidi<1,>=0.6; extra == 'all'
Requires-Dist: rapidocr<4,>=3.9; extra == 'all'
Requires-Dist: scenedetect<0.8,>=0.7; extra == 'all'
Requires-Dist: uvicorn<1,>=0.50; extra == 'all'
Provides-Extra: api
Requires-Dist: fastapi<1,>=0.139; extra == 'api'
Requires-Dist: uvicorn<1,>=0.50; extra == 'api'
Provides-Extra: autogen
Requires-Dist: autogen-core<0.8,>=0.4; extra == 'autogen'
Provides-Extra: crewai
Requires-Dist: crewai>=0.80; extra == 'crewai'
Provides-Extra: diarize
Requires-Dist: pyannote-audio<5,>=4.0; extra == 'diarize'
Provides-Extra: index
Requires-Dist: fastembed<0.9,>=0.8; extra == 'index'
Requires-Dist: numpy<3,>=2.4; extra == 'index'
Provides-Extra: langchain
Requires-Dist: langchain-core<2,>=0.3; extra == 'langchain'
Provides-Extra: llamaindex
Requires-Dist: llama-index-core<0.16,>=0.12; extra == 'llamaindex'
Provides-Extra: loop
Requires-Dist: playwright<2,>=1.61; extra == 'loop'
Provides-Extra: mcp
Requires-Dist: fastmcp<4,>=3.4; extra == 'mcp'
Provides-Extra: ocr
Requires-Dist: onnxruntime<2,>=1.27; extra == 'ocr'
Requires-Dist: python-bidi<1,>=0.6; extra == 'ocr'
Requires-Dist: rapidocr<4,>=3.9; extra == 'ocr'
Provides-Extra: openai-agents
Requires-Dist: openai-agents<1,>=0.1; extra == 'openai-agents'
Provides-Extra: perceive
Requires-Dist: imagehash<5,>=4.3; extra == 'perceive'
Requires-Dist: opencv-python-headless<6,>=5.0; extra == 'perceive'
Requires-Dist: pillow<13,>=12.3; extra == 'perceive'
Requires-Dist: scenedetect<0.8,>=0.7; extra == 'perceive'
Provides-Extra: standard
Requires-Dist: fastembed<0.9,>=0.8; extra == 'standard'
Requires-Dist: fastmcp<4,>=3.4; extra == 'standard'
Requires-Dist: imagehash<5,>=4.3; extra == 'standard'
Requires-Dist: numpy<3,>=2.4; extra == 'standard'
Requires-Dist: opencv-python-headless<6,>=5.0; extra == 'standard'
Requires-Dist: pillow<13,>=12.3; extra == 'standard'
Requires-Dist: scenedetect<0.8,>=0.7; extra == 'standard'
Provides-Extra: whisper
Requires-Dist: faster-whisper<2,>=1.2; extra == 'whisper'
Description-Content-Type: text/markdown

<!-- mcp-name: io.github.oxbshw/watch-skill -->
<div align="center">

<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/watch-skill-hero.webp" alt="Watch Skill: watch a video, remember the evidence with timestamps, and verify an agent's work through THE LOOP" width="88%">

# Watch Skill

**Give every AI agent eyes for video — and a way to check its own work.**

[![CI](https://github.com/oxbshw/watch-skill/actions/workflows/ci.yml/badge.svg)](https://github.com/oxbshw/watch-skill/actions/workflows/ci.yml)
[![Install](https://github.com/oxbshw/watch-skill/actions/workflows/install.yml/badge.svg)](https://github.com/oxbshw/watch-skill/actions/workflows/install.yml)
[![PyPI](https://img.shields.io/pypi/v/watch-skill)](https://pypi.org/project/watch-skill/)
[![Downloads](https://img.shields.io/pypi/dm/watch-skill)](https://pypi.org/project/watch-skill/)
[![Agent Skills](https://www.skills.sh/b/oxbshw/watch-skill)](https://www.skills.sh/oxbshw/watch-skill)
[![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-3776AB)](https://github.com/oxbshw/watch-skill/blob/main/pyproject.toml)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/oxbshw/watch-skill/blob/main/LICENSE)

[Install](#install) · [Documentation](https://github.com/oxbshw/watch-skill/blob/main/docs/README.md) · [Examples](https://github.com/oxbshw/watch-skill/blob/main/examples/README.md) · [Comparison](https://github.com/oxbshw/watch-skill/blob/main/docs/comparison.md) · [Roadmap](https://github.com/oxbshw/watch-skill/blob/main/docs/ROADMAP.md)

</div>

Watch Skill turns videos, live streams, meetings, and screen recordings into a searchable,
timestamped index. An agent can ask what happened, get an answer that cites the exact
moment behind it, and ask again tomorrow without processing the video a second time.

When the video is the agent's *own* browser or desktop session, **THE LOOP** closes the
circle: record the work, critique it against plain-language criteria, and show before and
after. That critique is *advisory* — a model describing pictures. To decide whether the
work actually succeeded, attach a **verification contract**: deterministic checks, frozen
before the run, that hold the verdict.

```bash
uvx --from "watch-skill[standard]" watch-skill setup
```

<p align="center">
  <img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/loop_before_after.gif" alt="A checkout flow fails with a NaN total, is fixed, and passes verification" width="720">
  <br>
  <sub>THE LOOP catching a <code>$NaN</code> total that an end-state screenshot misses, then showing the fix.</sub>
</p>

## What it does

| | |
|---|---|
| **Watch** | Scene-aware frames, on-screen text, and local-first transcription from 1,800+ sites, live HLS/DASH streams, local media, meetings, browsers, windows, and desktops. |
| **Watch live** | A session that reports what changed **while the source is still playing** — bounded queues, counted drops, cursor-addressed events, and a rolling buffer that pins the evidence around each one. [Guide](https://github.com/oxbshw/watch-skill/blob/main/docs/live.md) |
| **Remember** | A persistent, searchable index with timestamp citations, hybrid retrieval, cross-video synthesis, and reusable lessons. |
| **Verify** | A capture → critique → fix → re-capture loop for browser flows, interfaces, generated video, gameplay, and monitored streams — with deterministic contracts deciding pass or fail. |
| **Operate** | Drive a browser and prove the effect of each action — deterministic target resolution, per-step receipts, and verdicts that reject a page reporting success over a failed request. [Guide](https://github.com/oxbshw/watch-skill/blob/main/docs/browser-runtime.md) |

Available as Claude Code skills, 37 MCP tools, a CLI, a REST API, and native adapters for
LangChain/LangGraph, CrewAI, the OpenAI Agents SDK, LlamaIndex, and AutoGen.

Four things it will not do, because each one is a way of being confidently wrong:

- **Answer from a video that changed.** Identity follows the bytes, not the path. Overwrite
  `demo.mp4` and the next question returns `stale`, not yesterday's frames.
- **Upload a frame you did not agree to send.** A configured API key is not consent.
  `watch-skill plan` prints every network action before a run makes one.
- **Call an absent judgement a pass.** No frames, no OCR, an unreachable model, a timed-out
  check — all `inconclusive`. Only a required deterministic check produces a `pass`.
- **Claim a capability it has not checked.** `watch-skill capture-capabilities` says what this
  machine can actually record, and whether each answer was machine-tested or merely probed.

## Install

Two pieces, and you want both. The **engine** does the work; the **skills** teach your
agent when to reach for it.

```bash
# 1. the engine — installs, wires up every AI agent on the machine, backs up each config
uvx --from "watch-skill[standard]" watch-skill setup

# 2. the skills — into Claude Code, Codex, Cursor, Copilot, Gemini CLI, and 20+ more
npx skills add oxbshw/watch-skill -g
```

Watch Skill ships on PyPI, not npm. The second command runs
[Vercel's `skills` CLI](https://www.skills.sh), which reads the ten `SKILL.md`
files out of this repository and installs them into whichever agents you have —
there is no `watch-skill` npm package to install, and the engine is Python
either way.

Neither needs a clone, and the engine command works the same on macOS, Linux, and
Windows — [CI runs it on all three](https://github.com/oxbshw/watch-skill/actions/workflows/install.yml)
on every push.

Prefer a permanent install to `uvx` fetching on demand?

```bash
pipx install "watch-skill[standard]"     # or: pip install "watch-skill[standard]"
watch-skill setup
```

<details>
<summary>Other ways in — Claude Code plugin, Docker, from source</summary>

**Claude Code plugin** — skills, slash commands, and the MCP server in one:

```text
/plugin marketplace add oxbshw/watch-skill
/plugin install watch-skill@watch-skill
/watch-skill:setup-watch-skill
```

**Docker** — nothing on the host; the volume is where the index lives, so do not skip it:

```bash
docker run --rm -i -v watch-skill-data:/data ghcr.io/oxbshw/watch-skill serve
```

Built for `linux/amd64` and `linux/arm64`, with an SBOM and a signed build attestation.

**From source** (installs uv and Python if either is missing):

```bash
curl -fsSL https://raw.githubusercontent.com/oxbshw/watch-skill/main/scripts/install.sh | sh
```

```powershell
powershell -ExecutionPolicy Bypass -c "irm https://raw.githubusercontent.com/oxbshw/watch-skill/main/scripts/install.ps1 | iex"
```

**Wiring an agent by hand** — the block most MCP clients take:

```json
{ "mcpServers": { "watch-skill": {
    "command": "uvx",
    "args": ["--from", "watch-skill[standard]", "watch-skill", "serve"] } } }
```

Zed, Amp, and a few others name that key differently; each
[agent guide](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/README.md) shows the exact shape.

</details>

`standard` is frames, retrieval, and MCP — about 200 MB. `watch-skill[all]` adds OCR,
local Whisper, REST, and the browser THE LOOP drives. `watch-skill doctor` names anything
missing and prints the one command that installs it, so starting small is safe.

Coming from [claude-video](https://github.com/bradautomates/claude-video)? Your `/watch`
commands and flags work unchanged — see the [migration guide](https://github.com/oxbshw/watch-skill/blob/main/docs/migrate-from-claude-video.md).

## First run

```bash
watch-skill watch "https://youtu.be/..." "Summarize the important moments."
```

That prints a report and an id. Everything after it is a lookup against the index, not a
second download:

```bash
watch-skill ask <video_id> "when does the demo first fail?"
watch-skill search "pricing decision"        # across every video you've watched
watch-skill library ask "what did the team decide about auth?"
```

Useful flags on `watch`:

| Flag | Use it when |
|---|---|
| `--detail transcript` | You want the words, not the pictures — much faster |
| `--detail balanced` \| `token-burner` | More frames, more cost |
| `--start 4:10 --end 6:00` | Only a slice of a long video matters |
| `--word-timestamps` | You need the exact word, not the ten-second segment it sat in |
| `--no-cache` | Re-fetch a source that changed |

And the rest of the surface:

```bash
watch-skill serve                            # MCP over stdio — what agents connect to
watch-skill api                              # REST, port 8748
watch-skill doctor                           # check and repair the setup
watch-skill viewer <video_id> --out r.html   # one self-contained page to share
watch-skill loop viewer <loop_id>            # a run's iterations, compared
watch-skill bench providers                  # compare every provider you have a key for
```

Transcription, OCR, and search run locally and need no API key. Visual question
answering uses whichever provider you already pay for — Anthropic, OpenAI, Gemini,
OpenRouter, Groq, Together, Fireworks, DeepSeek, xAI, Mistral, MiniMax, Moonshot,
Z.ai, or Qwen — or nothing at all with a local Ollama model. Anything else that
speaks the OpenAI format (vLLM, LM Studio, llama.cpp, LiteLLM, Azure OpenAI, a
company gateway) works through the `custom` provider:

```bash
watch-skill setup-vision --provider groq            # or any of the above
watch-skill setup-vision --provider custom \
  --base-url http://127.0.0.1:8000/v1               # your own server
```

See [Getting started](https://github.com/oxbshw/watch-skill/blob/main/docs/getting-started.md) for manual installation and
[Configuration](https://github.com/oxbshw/watch-skill/blob/main/docs/configuration.md) for provider and privacy settings.

## Why use it

- **Evidence instead of frame dumps.** Scene detection and perceptual deduplication spend
  the frame budget on distinct moments. Answers include timestamps, confidence, and the
  evidence used to support them.
- **Persistent video memory.** Analyze once, ask again without downloading or transcribing
  the same video. Hybrid full-text and vector retrieval works within one video or across
  the entire library.
- **Local-first processing.** Original-language captions are preferred, local Whisper is
  the default fallback, and cloud speech-to-text is opt-in. An Ollama configuration keeps
  the complete pipeline on the machine.
- **Flow verification.** THE LOOP records an agent's browser, screen, or window and checks
  the result against plain-language criteria, producing a before/after comparison. The
  model's read of that recording is advisory; a [verification
  contract](https://github.com/oxbshw/watch-skill/blob/main/docs/verification.md) turns it into a decision, and its evidence bundle is
  hash-bound so an edited result stops verifying.
- **Corrections that persist.** `report_mistake` stores a local lesson, applies it to related
  questions, and turns it into a replayable evaluation.
- **Measured cost controls.** Text-first answers, semantic caching, configurable token
  budgets, and explicit `cheapest`, `quality_first`, and `offline_only` policies keep the
  trade-offs visible.
- **Multilingual retrieval.** Script-aware OCR routing, Arabic normalization, CJK substring
  matching, and multilingual embeddings support questions across languages.

Sixteen providers is a menu, not an answer, so there is a benchmark that
settles it on your own keys: `watch-skill bench providers` reads the same
committed frames with every provider you have configured and prints char-hit
rate, latency, and cost from each one's *reported* tokens — see
[method and results](https://github.com/oxbshw/watch-skill/blob/main/benchmarks/providers/README.md).

The repository includes reproducible [cost](https://github.com/oxbshw/watch-skill/blob/main/benchmarks/cost/RESULTS.md) and
[perception](https://github.com/oxbshw/watch-skill/blob/main/benchmarks/perception/RESULTS.md) benchmarks. Product claims in this README
link to the relevant implementation notes or testable example rather than relying on
unqualified marketing numbers.

## Works with your agent

The setup command detects supported clients and updates their configuration with a backup.
Manual guides are available for every entry below.

| | | | |
|:---:|:---:|:---:|:---:|
| [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/claude-code.webp" width="150" alt="Claude Code avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/claude-code.md)<br>[Claude Code](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/claude-code.md) | [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/claude-desktop.webp" width="150" alt="Claude Desktop avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/claude-desktop.md)<br>[Claude Desktop](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/claude-desktop.md) | [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/cursor.webp" width="150" alt="Cursor avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/cursor.md)<br>[Cursor](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/cursor.md) | [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/codex-cli.webp" width="150" alt="Codex CLI avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/codex-cli.md)<br>[Codex CLI](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/codex-cli.md) |
| [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/cline.webp" width="150" alt="Cline avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/cline.md)<br>[Cline](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/cline.md) | [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/windsurf.webp" width="150" alt="Windsurf avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/windsurf.md)<br>[Windsurf](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/windsurf.md) | [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/gemini-cli.webp" width="150" alt="Gemini CLI avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/gemini-cli.md)<br>[Gemini CLI](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/gemini-cli.md) | [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/vscode.webp" width="150" alt="VS Code avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/vscode.md)<br>[VS Code](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/vscode.md) |
| [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/github-copilot-cli.webp" width="150" alt="GitHub Copilot CLI avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/github-copilot-cli.md)<br>[GitHub Copilot CLI](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/github-copilot-cli.md) | [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/kimi-code.webp" width="150" alt="Kimi Code avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/kimi-code.md)<br>[Kimi Code](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/kimi-code.md) | [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/qwen-code.webp" width="150" alt="Qwen Code avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/qwen-code.md)<br>[Qwen Code](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/qwen-code.md) | [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/opencode.webp" width="150" alt="OpenCode avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/opencode.md)<br>[OpenCode](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/opencode.md) |
| [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/goose.webp" width="150" alt="Goose avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/goose.md)<br>[Goose](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/goose.md) | [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/openhands.webp" width="150" alt="OpenHands avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/openhands.md)<br>[OpenHands](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/openhands.md) | [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/kilocode.webp" width="150" alt="Kilo Code avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/kilocode.md)<br>[Kilo Code](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/kilocode.md) | [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/qodo.webp" width="150" alt="Qodo avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/qodo.md)<br>[Qodo](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/qodo.md) |
| [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/agent-zero.webp" width="150" alt="Agent Zero avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/agent-zero.md)<br>[Agent Zero](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/agent-zero.md) | [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/openclaw.webp" width="150" alt="OpenClaw avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/openclaw.md)<br>[OpenClaw](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/openclaw.md) | [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/pi.webp" width="150" alt="Pi avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/pi.md)<br>[Pi](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/pi.md) | [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/hermes.webp" width="150" alt="Hermes avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/hermes.md)<br>[Hermes](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/hermes.md) |
| [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/zed.webp" width="150" alt="Zed avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/zed.md)<br>[Zed](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/zed.md) | [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/roo-code.webp" width="150" alt="Roo Code avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/roo-code.md)<br>[Roo Code](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/roo-code.md) | [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/continue.webp" width="150" alt="Continue avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/continue.md)<br>[Continue](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/continue.md) | [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/jetbrains.webp" width="150" alt="JetBrains IDEs avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/jetbrains.md)<br>[JetBrains](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/jetbrains.md) |
| [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/amp.webp" width="150" alt="Amp avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/amp.md)<br>[Amp](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/amp.md) | [<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/aider.webp" width="150" alt="Aider avatar">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/aider.md)<br>[Aider](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/aider.md) | | |

[<img src="https://raw.githubusercontent.com/oxbshw/watch-skill/main/docs/assets/agents/frameworks.webp" width="360" alt="Framework agent avatars collaborating around a shared video engine">](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/frameworks.md)

Native tools are also available for [LangChain/LangGraph, CrewAI, OpenAI Agents SDK,
LlamaIndex, and AutoGen](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/frameworks.md); any other framework can use REST or
MCP.

### Why both skills and MCP

MCP gives an agent 37 tools. Skills give it the judgement about when to use them —
that a screen recording in the conversation is worth watching, that a follow-up
question should hit the index instead of re-processing, that a UI change deserves a
verification pass. An agent with only the tools waits to be told; an agent with the
skills reaches for them.

That is why `npx skills add oxbshw/watch-skill -g` is step two of the install and not
an optional extra. Pick individual ones with `--skill <name>`, or list them first:

```bash
npx skills add oxbshw/watch-skill --list
```

| Connection | How it reaches the agent |
|---|---|
| **Skills** | Every agent the [skills CLI](https://skills.sh) supports — Claude Code, Codex CLI, Cursor, GitHub Copilot, Gemini CLI, VS Code, and the rest — plus [OpenClaw](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/openclaw.md), [Pi](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/pi.md), and [Hermes-style agents](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/hermes.md) |
| **MCP** | [Claude Desktop](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/claude-desktop.md), [Cursor](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/cursor.md), [Codex CLI](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/codex-cli.md), [Cline](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/cline.md), [Windsurf](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/windsurf.md), [Gemini CLI](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/gemini-cli.md), [VS Code](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/vscode.md), [GitHub Copilot CLI](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/github-copilot-cli.md), [Zed](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/zed.md), [Roo Code](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/roo-code.md), [Continue](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/continue.md), [Kimi Code](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/kimi-code.md), [Qwen Code](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/qwen-code.md), [OpenCode](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/opencode.md), [Goose](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/goose.md), [OpenHands](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/openhands.md), [Kilo Code](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/kilocode.md), [Qodo](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/qodo.md), [Agent Zero](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/agent-zero.md) |
| **Native Python tools** | [LangChain/LangGraph, CrewAI, OpenAI Agents SDK, LlamaIndex, and AutoGen](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/frameworks.md) |
| **HTTP** | Vercel AI SDK, n8n, and any client that can call REST/OpenAPI |

The [full compatibility matrix](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/README.md) separates machine-tested,
machine-configured, and documentation-verified integrations. If your agent is missing,
the [adapter template](https://github.com/oxbshw/watch-skill/blob/main/templates/agent-adapter/README.md) provides a short contribution
path.

## Browser Runtime

Watch Skill has one browser subsystem with two modes. *Observer* mode watches
someone else work and verifies the result. *Operator* mode does the work and
holds itself to the same standard.

```python
from watch_skill.operate import (
    Action, ActionKind, BrowserRuntime, Expectation, SideEffect, Target,
)

runtime = BrowserRuntime(source)          # an already-running browser session
result = runtime.run_task("save the display name", [
    Action(kind=ActionKind.CLICK, intent="save",
           target=Target(role="button", name="Save"),
           side_effect=SideEffect.REVERSIBLE,
           expect=Expectation(text_present="Saved", network_ok=True)),
])

result.verified          # False — the page said Saved, PATCH /api/save returned 500
result.receipts[-1].reason
```

The rule the runtime is built around: **dispatching an action is not the same
as proving its effect.** Playwright returning from `click()` proves a click was
delivered and nothing more, so every action carries an expectation written down
beforehand, and the verdict is the comparison. An action with no expectation is
`UNVERIFIED`, never `SUCCEEDED`.

That is what catches the failure mode nothing else does — a page that renders
success over a request that failed. `network_ok` correlates the requests made
during the step, so "Saved" over a 500 is a failure with the request named in
the receipt.

Other properties worth knowing:

- **Targets resolve deterministically first** — accessible role and name, then
  label, test id, placeholder, selector, text. Vision is last because it is the
  most expensive signal and the least stable across a redeploy.
- **Ambiguity is refused, not guessed.** Two buttons named "Delete account" is
  not a case where the first one is probably right.
- **Retries respect side effects.** Clicking "Next" again is fine; clicking
  "Buy" again is not. Recovery never repeats an action that may have taken.
- **Every step produces a receipt** — how the target was found and with what
  confidence, what changed, which requests ran, the verdict, and any recovery.

Run the benchmark against the bundled local fixture site:

```bash
python -m watch_skill.operate.benchmark --out build/benchmark
```

It scores **false-success rate** — tasks where the runtime claimed the goal was
met and the server disagrees — because task success rate on its own counts a
confident wrong answer as a win. Ground truth comes from the fixture server's
own state, not from anything the browser reported.

On that nine-task fixture benchmark every ground-truth verdict was classified
correctly and no false-success verdict was produced. Nine tasks on one
synthetic site is a regression gate rather than a capability claim: it does not
cover real websites, authentication, or shadow DOM, and no other tool was
measured under the same method. [Full method and results](https://github.com/oxbshw/watch-skill/blob/main/docs/release-proof.md)
and the [design](https://github.com/oxbshw/watch-skill/blob/main/docs/browser-runtime.md).

## Common workflows

### Build a searchable video library

```bash
watch-skill batch ./recordings --limit 50
watch-skill library overview
watch-skill library ask "What did the team decide about authentication?"
```

`library ask` synthesizes evidence across videos and retains per-video timestamp
provenance. The [library example](https://github.com/oxbshw/watch-skill/blob/main/examples/12-library-memory/) demonstrates a question
whose answer is distributed across four clips.

### Verify an agent's browser work

```bash
watch-skill loop start "browser:http://127.0.0.1:3000" \
  "Checkout completes and the total is always a valid currency amount"
```

The loop captures the full interaction, critiques failures, and records the before/after
comparison once the agent applies a fix — the run shown at the top of this page.
[Example 14](https://github.com/oxbshw/watch-skill/blob/main/examples/14-browser-verification/) walks through that transient `$NaN` bug.

The critique is one model's reading of the recording. To make success a decision rather
than an opinion, attach deterministic checks:

```bash
watch-skill verify run checkout-contract.json --dir .
```

`pass` requires every **required** check to pass. A check that fails, times out, or never
runs makes the run `inconclusive` — never a pass. See
[Verification](https://github.com/oxbshw/watch-skill/blob/main/docs/verification.md).

### Export an offline report

```bash
watch-skill viewer <video_id> --out video-report.html
```

The generated page contains its frames, transcript, OCR, cached answers, and cited
evidence. It has no external runtime dependencies and can be opened without a server.

## Examples

The examples progress from a first watch to agent integration, cross-video memory, and
self-verification.

| Track | Examples |
|---|---|
| Learn the core | [01 Watch and ask](https://github.com/oxbshw/watch-skill/blob/main/examples/01-watch-and-ask), [02 Focused moment](https://github.com/oxbshw/watch-skill/blob/main/examples/02-focused-moment), [03 Cross-video search](https://github.com/oxbshw/watch-skill/blob/main/examples/03-cross-video-search) |
| Build with agents | [06 MCP and REST](https://github.com/oxbshw/watch-skill/blob/main/examples/06-agent-integration), [09 Framework adapters](https://github.com/oxbshw/watch-skill/blob/main/examples/09-framework-adapters), [15 Private offline workflow](https://github.com/oxbshw/watch-skill/blob/main/examples/15-private-offline-workflow) |
| Understand and organize | [05 Multilingual Arabic](https://github.com/oxbshw/watch-skill/blob/main/examples/05-multilingual-arabic), [10 Structured extraction](https://github.com/oxbshw/watch-skill/blob/main/examples/10-structured-extraction), [11 Batch mode](https://github.com/oxbshw/watch-skill/blob/main/examples/11-batch-mode), [12 Library memory](https://github.com/oxbshw/watch-skill/blob/main/examples/12-library-memory) |
| Verify and improve | [04 UI loop](https://github.com/oxbshw/watch-skill/blob/main/examples/04-ui-loop), [07 Lessons and stats](https://github.com/oxbshw/watch-skill/blob/main/examples/07-lessons-and-stats), [08 Loop types](https://github.com/oxbshw/watch-skill/blob/main/examples/08-loop-types), [13 Self-improvement](https://github.com/oxbshw/watch-skill/blob/main/examples/13-self-improvement), [14 Browser verification](https://github.com/oxbshw/watch-skill/blob/main/examples/14-browser-verification), [17 Freshness and offline](https://github.com/oxbshw/watch-skill/blob/main/examples/17-freshness-and-offline), [20 Observer loop](https://github.com/oxbshw/watch-skill/blob/main/examples/20-observer-loop) |
| Watch live | [18 Live watch](https://github.com/oxbshw/watch-skill/blob/main/examples/18-live-watch), [19 Live browser](https://github.com/oxbshw/watch-skill/blob/main/examples/19-live-browser) |
| Share results | [16 Export a self-contained viewer](https://github.com/oxbshw/watch-skill/blob/main/examples/16-shareable-viewer) |

See the [example catalog](https://github.com/oxbshw/watch-skill/blob/main/examples/README.md) for prerequisites, expected output, and a
recommended path through all 20 examples.

## Architecture

All interfaces call the same Python core. Skills and agent adapters decide *when* to use
Watch Skill; acquisition, perception, transcription, indexing, answering, and verification
remain in `src/watch_skill`.

```mermaid
flowchart LR
    A["Agents and frameworks"] --> S["Skills · MCP · CLI · REST"]
    S --> AC["Acquire"]
    AC --> P["Scenes · OCR · transcript"]
    P --> I[("Persistent index")]
    I --> Q["Answers · extraction · library"]
    I --> L["Lessons and evaluations"]
    V["Browser · screen · stream capture"] --> C["Loop critic"]
    C --> I
```

Read [Architecture](https://github.com/oxbshw/watch-skill/blob/main/docs/architecture.md) for the data model, provider boundaries, and
extension points.

## Documentation

| Guide | Use it for |
|---|---|
| [Documentation index](https://github.com/oxbshw/watch-skill/blob/main/docs/README.md) | Choose a guide by task or audience |
| [Getting started](https://github.com/oxbshw/watch-skill/blob/main/docs/getting-started.md) | Installation, first watch, and first agent connection |
| [Tool reference](https://github.com/oxbshw/watch-skill/blob/main/docs/tools/README.md) | All 37 MCP tools and their REST/CLI counterparts |
| [Configuration](https://github.com/oxbshw/watch-skill/blob/main/docs/configuration.md) | Storage, privacy, models, limits, and environment variables |
| [Agent matrix](https://github.com/oxbshw/watch-skill/blob/main/docs/agents/README.md) | Per-client setup and verification status |
| [Verification](https://github.com/oxbshw/watch-skill/blob/main/docs/verification.md) | Contracts, deterministic checks, assurance levels, attestations |
| [Use-case packs](https://github.com/oxbshw/watch-skill/blob/main/docs/packs/README.md) | Recipes for research, meetings, QA, content, and operations |
| [THE LOOP](https://github.com/oxbshw/watch-skill/blob/main/docs/guides/the-loop.md) | Capture, critique, iteration, and proof artifacts |
| [Cost policy](https://github.com/oxbshw/watch-skill/blob/main/docs/cost.md) | Routing, budgets, caching, and benchmark method |
| [Troubleshooting](https://github.com/oxbshw/watch-skill/blob/main/docs/troubleshooting.md) | Dependency repair and common runtime errors |
| [Comparison](https://github.com/oxbshw/watch-skill/blob/main/docs/comparison.md) | Honest trade-offs against the alternatives |
| [Engineering decisions](https://github.com/oxbshw/watch-skill/blob/main/docs/DECISIONS.md) | The reasoning behind non-obvious design choices |
| [Roadmap](https://github.com/oxbshw/watch-skill/blob/main/docs/ROADMAP.md) | Planned work and contribution opportunities |

## Development

```bash
git clone https://github.com/oxbshw/watch-skill
cd watch-skill
uv sync --extra all
uv run pytest
uv run ruff check .
```

See [CONTRIBUTING.md](https://github.com/oxbshw/watch-skill/blob/main/CONTRIBUTING.md) for test tiers, documentation standards, and the
agent-adapter checklist. Security and privacy reports are covered by
[SECURITY.md](https://github.com/oxbshw/watch-skill/blob/main/SECURITY.md).

## Listed on

Independent directories that index Watch Skill. They are maintained by their operators,
so the details there can lag a release.

- [Agent Skills Hub](https://agentskillshub.top/skill/oxbshw/watch-skill/)
- [Neuralbox](https://neuralbox.tech/oxbshw-watch-skill)

---

<div align="center">

Released under the [MIT License](https://github.com/oxbshw/watch-skill/blob/main/LICENSE) · Built by [oxbshw](https://github.com/oxbshw)

</div>
