Metadata-Version: 2.5
Name: quest1
Version: 0.2.0
Summary: Spoken dialogue frame extractor: find the exact video frame where a target spoken phrase occurs.
Project-URL: Homepage, https://github.com/sathyanarayanan/Quest1
Project-URL: Repository, https://github.com/sathyanarayanan/Quest1
Author-email: Sathya Narayanan <sathyanarayanan2548@gmail.com>
License: MIT
Keywords: asr,dialogue,frame-extraction,video,whisper
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.11
Requires-Dist: av>=12.0.0
Requires-Dist: faster-whisper>=1.0.3
Requires-Dist: ffmpeg-python>=0.2.0
Requires-Dist: numpy<3,>=1.26
Requires-Dist: pillow>=12.3.0
Requires-Dist: rapidfuzz>=3.10.1
Requires-Dist: yt-dlp>=2024.10.7
Provides-Extra: api
Requires-Dist: celery>=5.4; extra == 'api'
Requires-Dist: fastapi>=0.111; extra == 'api'
Requires-Dist: pydantic>=2.7; extra == 'api'
Requires-Dist: pymongo>=4.6; extra == 'api'
Requires-Dist: python-multipart>=0.0.9; extra == 'api'
Requires-Dist: redis>=5.0; extra == 'api'
Requires-Dist: uvicorn[standard]>=0.30; extra == 'api'
Provides-Extra: dev
Requires-Dist: httpx>=0.27; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff>=0.6; extra == 'dev'
Description-Content-Type: text/markdown

# Quest1

Spoken-dialogue frame extractor: find the exact video frame where a target
spoken phrase appears.

Given a video URL and a target phrase, this tool downloads the audio,
transcribes it with word-level timestamps, fuzzy-matches the phrase,
and saves the corresponding video frame as a PNG.

Available as a CLI (`python main.py`, `python -m q1`, `quest1`) and as an
async FastAPI service backed by Celery + Redis + MongoDB.

## Quick start

```bash
# Setup
uv venv --python 3.12 .venv
source .venv/bin/activate
uv pip install -e ".[dev]"             # CLI only
uv pip install -e ".[api]"             # CLI + FastAPI/Celery/Redis/Mongo

# Run on the spec example (ok.ru + 'My mind rebels at stagnation')
OMP_NUM_THREADS=14 python main.py

# Run on a different video / phrase
python main.py --url "https://example.com/video" --phrase "your phrase here"

# Re-run (uses cached WAV + transcript)
python main.py

# Force re-download
python main.py --force-redownload

# Force re-transcribe (reuse MP4 and WAV)
python main.py --force-retranscribe
```

## Configuration

Edit `config.toml` to set defaults:

```toml
[video]
url = "https://ok.ru/video/248244667877"

[target]
phrase = "My mind rebels at stagnation"
fuzzy_threshold = 78  # 0-100, lower = more lenient

[whisper]
model = "small"          # tiny | base | small | distil-large-v3
compute_type = "int8"
beam_size = 1
vad_filter = true
language = "en"

[paths]
cache_dir = "./cache"
output_dir = "./outputs"
frame_filename = "frame_output.png"
```

CLI args override config.toml (precedence: CLI > TOML > defaults).

## Output

Each run creates `outputs/<phrase-slug>/` containing:

- `frame_output.png` — the extracted video frame
- `result.json` — full match details (timestamp, frame number, text, video metadata, consumer stats)

Example output:

```
Timestamp : 00:05:25.390
Frame     : 7801
Text      : "My mind rebels at stagnation"
```

## Repo layout

```
.
├── README.md                ← you are here
├── pyproject.toml          ← [project.scripts] quest1 = q1.cli:main; [project.optional-dependencies] api = fastapi + celery + redis + pymongo
├── .gitignore
├── .python-version
├── config.toml              ← editable defaults
├── TODO.md                  ← problem statement (from PS)
├── main.py                  ← 2-line shim → q1.cli:main (backward compat)
├── src/
│   └── q1/                  ← the pip-installable package
│       ├── __init__.py
│       ├── config.py         ← Settings dataclass + TOML/CLI loader
│       ├── downloader.py     ← yt-dlp wrapper + cache
│       ├── audio.py          ← ffmpeg WAV extraction
│       ├── transcriber.py    ← faster-whisper with cache
│       ├── matcher.py        ← rapidfuzz sliding-window + gap refinement
│       ├── video_meta.py     ← ffprobe wrapper
│       ├── frame.py          ← PyAV frame extraction
│       ├── pipeline.py       ← run_pipeline() extracted as reusable fn (PipelineResult return)
│       └── cli.py            ← entry point used by [project.scripts] quest1
├── api/                      ← NEW: standalone FastAPI/Celery service
│   ├── __init__.py
│   ├── app.py                ← FastAPI app: POST /jobs, GET /jobs/<id>, /frame, /result.json, /healthz
│   ├── schemas.py            ← Pydantic request/response models
│   ├── celery_app.py         ← Celery instance (broker=Redis, backend=Redis)
│   ├── db.py                 ← MongoDB JobRepo (durable job metadata)
│   ├── worker.py             ← @celery_app.task process_job (runs run_pipeline, writes jobs/<id>/)
│   └── tests/
│       ├── conftest.py       ← shared fixtures (autouse _isolate_env + optional docker mongo/redis)
│       ├── test_app.py       ← FastAPI endpoint contract tests
│       └── test_worker.py    ← Celery task tests with FakePipelineResult
├── tests/
│   └── test_*.py            ← CLI module unit tests (unchanged)
├── docs/
│   ├── DESIGN.md            ← architectural guidance (external)
│   ├── AMBIGUITY.md         ← ambiguity-handling spec (external)
│   ├── research.md          ← my research deliverable
│   └── prompts.txt          ← the prompts i asked the AI (long form)
├── jobs/                     ← gitignored; per-job output dirs (created by worker at runtime)
└── .claude/
    └── commands/
        └── logprompt.md     ← /logprompt slash command definition
```

## Caching

Three layers of disk cache (in `./cache/`), shared by CLI and API workers:

```
cache/
  video/<id>.<ext>          ← yt-dlp output (skip if exists)
  audio/<id>.wav            ← ffmpeg WAV (skip if exists)
  transcript/<hash>.json    ← Whisper output (skip if exists)
```

The transcript cache is keyed on the first 8 MB of the WAV plus the model
parameters, so re-running with the same audio and same model is essentially
free.

## Tests

```bash
# CLI module unit tests
pytest -q tests/

# API endpoint + worker tests
pytest -q api/tests/
```

CLI: 27 unit tests covering all five pipeline modules. `test_frame.py`
uses a real ffmpeg-generated test video and skips automatically if
ffmpeg is not on PATH.

API: 13 tests (test_app.py + test_worker.py) using FastAPI TestClient
and a FakePipelineResult stand-in. Tests don't require Redis/Mongo
at runtime because JobRepo and celery_app.send_task are patched;
optional docker-based fixtures spin up real mongo/redis if available.

## Architecture (one paragraph)

`main.py` (CLI shim) -> `q1.cli.main()` -> `q1.pipeline.run_pipeline()` runs five
steps: `download_video` (yt-dlp, checks cache first) -> `extract_audio`
(ffmpeg subprocess to 16 kHz mono WAV) -> `transcribe` (faster-whisper with
`word_timestamps=True`, caches by audio hash + model params) ->
`find_phrase` (rapidfuzz sliding window, with gap refinement to skip
past Whisper's first-word timestamp drift) -> `probe` + `extract_frame`
(ffprobe for fps, PyAV seek and decode to PNG). The pipeline returns a
`PipelineResult` dataclass (never raises); the CLI prints the standard
output block and writes a sidecar JSON, the API worker writes the same
sidecar to `jobs/<job_id>/result.json` and updates the Mongo doc.

See `docs/DESIGN.md` for the full architecture and `docs/AMBIGUITY.md`
for how the tool handles uncertain matches.

## API service

The `api/` package exposes the pipeline as an async REST API.

### Run locally (4 terminals)

```bash
# 1. Start MongoDB and Redis (any way you like; the included conftest
#    uses docker run mongo:7 and redis:7 for tests)
mongod --dbpath /var/lib/mongodb
redis-server

# 2. Start the API server (FastAPI + uvicorn)
uv pip install -e ".[api]"
uvicorn api.app:app --host 0.0.0.0 --port 8000 --reload

# 3. Start the Celery worker (in another terminal)
celery -A api.celery_app worker --loglevel=info --concurrency=2

# 4. Submit + poll a job
JOB=$(curl -s -X POST http://localhost:8000/jobs \
   -H 'Content-Type: application/json' \
   -d '{"url":"https://...","dialogue":"...","options":{}}' | jq -r .job_id)

# poll
while true; do
  STATE=$(curl -s http://localhost:8000/jobs/$JOB | jq -r .state)
  echo "state=$STATE"
  case "$STATE" in success|no_match|failed) break;; esac
  sleep 5
done

# download artifacts
curl http://localhost:8000/jobs/$JOB/result.json | jq .
curl -o frame.png http://localhost:8000/jobs/$JOB/frame
```

### Endpoints

| Method | Path | Purpose |
|---|---|---|
| GET | `/healthz` | liveness check |
| POST | `/jobs` | enqueue a new job, returns 202 with `job_id` |
| GET | `/jobs/{job_id}` | full status (state, progress, result, error) |
| GET | `/jobs/{job_id}/frame` | the extracted PNG (404 until ready) |
| GET | `/jobs/{job_id}/result.json` | the sidecar JSON (404 until ready) |

### Environment variables (API + worker)

| Var | Default | Used for |
|---|---|---|
| `Q1_MONGO_URI` | `mongodb://localhost:27017` | Mongo client URI |
| `Q1_MONGO_DB` | `quest1` | Mongo database name |
| `Q1_BROKER_URL` | `redis://localhost:6379/0` | Celery broker (queue) |
| `Q1_RESULT_BACKEND` | `redis://localhost:6379/1` | Celery result cache |
| `Q1_CACHE_DIR` | `./cache` | shared with CLI |
| `Q1_JOBS_DIR` | `./jobs` | per-job output dirs |

Redis is the Celery broker AND Celery result backend (ephemeral).
MongoDB is the durable job metadata store. Both must be set so the API
and worker can find each other.

### Architecture flow

```
client --POST /jobs--> FastAPI
                       |  (1) JobRepo.create  -> MongoDB doc (state=queued)
                       |  (2) celery_app.send_task -> Redis queue
                       v
                 Celery worker (separate process)
                       |  (3) JobRepo.mark_started  -> MongoDB (state=started)
                       |  (4) run_pipeline(url, dialogue, ...)
                       |        uses shared cache/ dir (CLI/worker share)
                       |        writes outputs to jobs/<job_id>/
                       |  (5) JobRepo.mark_completed -> MongoDB (state=success|no_match|failed)
                       v
client --GET /jobs/<id>--> FastAPI reads MongoDB doc, returns state
client --GET /jobs/<id>/frame--> FastAPI streams jobs/<id>/frame_output.png
```

## Limitations

- No OCR fallback (per spec: "without using visual OCR")
- No cloud STT (local-first preference; documented for interview defense)
- No speaker diarization (out of scope for single-phrase search)
- No subtitle-track parsing (rarely present on test sources)
- No VFR per-frame PTS handling (rare; documented limitation)

## Sources cited

- yt-dlp: <https://github.com/yt-dlp/yt-dlp>
- faster-whisper: <https://github.com/SYSTRAN/faster-whisper>
- PyAV: <https://pyav.basswood.io/docs/stable/>
- RapidFuzz: <https://github.com/maxbachmann/RapidFuzz>
- ffmpeg/ffprobe: <https://ffmpeg.org/ffprobe.html>
- FastAPI: <https://fastapi.tiangolo.com/>
- Celery: <https://docs.celeryq.dev/>
