Metadata-Version: 2.5
Name: aind-video-utils
Version: 0.10.0
Summary: Tools to work with videos using ffmpeg
Project-URL: Repository, https://github.com/AllenNeuralDynamics/aind-video-utils/
Project-URL: Issues, https://github.com/AllenNeuralDynamics/aind-video-utils/issues
Project-URL: Changelog, https://github.com/AllenNeuralDynamics/aind-video-utils/blob/main/CHANGELOG.md
Author: Allen Institute for Neural Dynamics
Author-email: Galen Lynch <galen@galenlynch.com>
License-Expression: MIT
License-File: LICENSE
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.10
Requires-Dist: numpy
Provides-Extra: parquet
Requires-Dist: pyarrow>=14; extra == 'parquet'
Provides-Extra: plotting
Requires-Dist: matplotlib; extra == 'plotting'
Requires-Dist: opencv-python-headless>=4.13.0.90; extra == 'plotting'
Provides-Extra: transcode
Requires-Dist: pydantic-settings>=2.4; extra == 'transcode'
Requires-Dist: rich>=13.0; extra == 'transcode'
Description-Content-Type: text/markdown

# AIND Video Utils

![CI](https://github.com/AllenNeuralDynamics/aind-video-utils/actions/workflows/ci-call.yml/badge.svg)
[![License](https://img.shields.io/badge/license-MIT-brightgreen)](LICENSE)
[![ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![uv](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json)](https://github.com/astral-sh/uv)

Tools for working with video files using ffmpeg.

## Prerequisites

This package requires **ffmpeg** and **ffprobe** to be installed and available on your `PATH`.

- **Linux**: `sudo apt install ffmpeg` (Debian/Ubuntu) or `sudo dnf install ffmpeg` (Fedora)
- **macOS**: `brew install ffmpeg`
- **Windows**: Download from [ffmpeg.org](https://ffmpeg.org/download.html) and add to PATH

## Installation

```bash
# uv
uv add aind-video-utils                        # core only
uv add "aind-video-utils[transcode]"            # with transcode CLI
uv add "aind-video-utils[plotting]"             # with QC plotting

# pip
pip install aind-video-utils
pip install "aind-video-utils[transcode]"
pip install "aind-video-utils[plotting]"
pip install "aind-video-utils[parquet]"
```

| Extra | Adds |
|-------|------|
| `transcode` | pydantic-settings, rich — required for `aind-transcode` CLI |
| `plotting` | matplotlib, opencv — required for `aind-video-qc` CLI |
| `parquet` | pyarrow — required to write `preview_metadata.parquet` |

## Usage

```python
from aind_video_utils import probe, extract_luma_frame, extract_srgb_frame

# Probe video metadata
info = probe("video.mp4")
print(info["streams"][0]["pix_fmt"])  # e.g. "yuv420p", "gbrp"

# Extract luma (Y) plane at t=1.0s
luma, color_range, bit_depth = extract_luma_frame("video.mp4", 1.0)

# Extract an sRGB frame at t=1.0s
srgb = extract_srgb_frame("video.mp4", 1.0)
```

### Encoding Profiles

This package is the canonical Python source for the [AIND behavior video file standard](https://allenneuraldynamics.github.io/aind-file-standards/file_formats/behavior_videos/) encoding profiles. `SPEC_VERSION` names the revision of the standard they implement. Four profiles are
provided as frozen dataclass constants:

| Constant | Codec | Pixel Format | Container | Use Case |
|----------|-------|-------------|-----------|----------|
| `OFFLINE_8BIT` | libx264 | yuv420p | mp4 | Long-term storage (default) |
| `OFFLINE_10BIT` | libx264 | yuv420p10le | mp4 | Long-term storage, 10-bit |
| `ONLINE_8BIT` | h264_nvenc | yuv420p | mkv | Real-time acquisition |
| `ONLINE_10BIT` | hevc_nvenc | p010le | mkv | Real-time acquisition, 10-bit |

```python
from aind_video_utils import OFFLINE_8BIT, ONLINE_8BIT

# Inspect the exact ffmpeg args
OFFLINE_8BIT.ffmpeg_output_args()
ONLINE_8BIT.ffmpeg_input_args()

# Customize with replace()
fast = OFFLINE_8BIT.replace(codec_params=("-preset", "veryfast", "-crf", "18"))
```

For online acquisition pipelines that build their own ffmpeg command:

```python
from aind_video_utils import ONLINE_8BIT

profile = ONLINE_8BIT
cmd = [
    "ffmpeg",
    *profile.ffmpeg_input_args(),
    "-f", "rawvideo", "-pix_fmt", "bgr24",
    "-s", f"{w}x{h}", "-r", str(fps),
    "-i", "pipe:0",
    *profile.ffmpeg_output_args(),
    str(output_path),
]
```

### Transcoding Python API

```python
from fractions import Fraction

from aind_video_utils import transcode_video, OFFLINE_8BIT

# Simplest form — offline 8-bit with automatic colorspace fix
transcode_video(input_path, output_path)

# Explicit profile
transcode_video(input_path, output_path, profile=OFFLINE_8BIT)

# Custom profile with speed override
fast = OFFLINE_8BIT.replace(codec_params=("-preset", "veryfast", "-crf", "18"))
transcode_video(input_path, output_path, profile=fast)

# Skip automatic setparams probing
transcode_video(input_path, output_path, auto_fix_colorspace=False)

# Legacy h264-in-AVI sources: re-stamp timestamps so ffmpeg keeps every frame.
# The rate is measured from an MP4, MOV or AVI; pass frame_rate= for anything else.
transcode_video(input_path, output_path, normalize_cfr=True)
transcode_video(input_path, output_path, normalize_cfr=True, frame_rate=Fraction(500))

# Hold the decoder, filters and each encoder to 8 threads, e.g. in an 8-CPU job
transcode_video(input_path, output_path, threads=8)
```

`transcode_video()` raises `RuntimeError` when the output holds a different
number of frames than ffmpeg decoded from the source; pass
`fail_on_frame_drop=False` to accept such an output.

### Preview and Poster Derivatives

`transcode_video()` can also write a browser-playable preview and a JPEG
poster, as extra outputs of the same ffmpeg process, under the names the
standard gives them:

```python
from aind_video_utils import transcode_video

transcode_video(
    input_path,
    camera_dir / "video.mp4",                  # archival, frame-exact
    preview_fps=30.0,                          # preview.mp4: every N-th source frame
    metadata_csv=camera_dir / "metadata.csv",  # preview_metadata.parquet: ReferenceTime of each preview frame
    poster=True,                               # poster.jpg: sRGB still of the middle frame
)
```

The poster's middle frame comes from the frame count the container records.
Matroska records none, so for an `.mkv` it is estimated from the duration;
`count_frames=True` counts packets for the exact frame instead, at the cost of
reading the whole source once more, which for a URL is a second download. A
recording killed mid-write records no duration either, and its packets are
counted regardless. If a truncated file's duration places the poster past its
end, ffmpeg fails the run, though the archival video it wrote is complete.

The standard requires `preview_metadata.parquet` beside every preview. Its row
k is row kN of `metadata.csv`, so `metadata.csv` must have one row per frame of
the archival video; `transcode_video()` raises otherwise and writes no sidecar.
It needs the `parquet` extra. `write_preview_metadata()` writes one on its own.
The values are copied as recorded, flaws included. When `CameraFrameNumber`
skips or steps backwards, the times may need repair (see Acquisition Timing),
which needs the rows the preview drops, so a `PreviewTimingWarning` says to
repair from `metadata.csv` itself.

The derivatives take these names when the archival video is named `video.<ext>`,
as the standard names it. Otherwise they take its stem, so `BottomCamera.mp4`
gets `BottomCamera_preview.mp4`, `BottomCamera_preview_metadata.parquet` and
`BottomCamera_poster.jpg`, and two videos in one folder keep theirs apart.

N puts the preview between 25 and 35 fps, preferring a whole-number rate:

| Source | N | Preview |
|--------|---|---------|
| 1000 fps | 40 | 25 fps |
| 500 fps | 20 | 25 fps |
| 300 fps | 10 | 30 fps |
| 240 fps | 8 | 30 fps |
| 120 fps | 4 | 30 fps |
| 29.97 fps | 1 | 29.97 fps |

The poster branches off the source ahead of the BT.709 chain, because browsers
read JPEG as sRGB. The
[behavior video standard](https://allenneuraldynamics.github.io/aind-file-standards/file_formats/behavior_videos/#preview-videos)
gives the rationale for both.

To build the command yourself, attach derivatives to a conditioned profile.
Without `with_setparams`, the poster's `zscale` has no transfer function to
resolve, and its failure takes the archival encode down with it. The poster's
middle frame comes from the probe's frame count, or its duration when it has
none; `probe(input_path, count_packets=True)` gives an `.mkv` an exact count. Pass
`filename=None` to `with_preview` or `with_poster` to name a derivative from
the primary's stem and a suffix instead, as in `clip_preview.mp4`:

```python
from aind_video_utils import (
    OFFLINE_8BIT, probe, with_poster, with_preview, with_setparams,
)

probe_json = probe(input_path)
conditioned = with_setparams(OFFLINE_8BIT, probe_json)
profile = with_poster(with_preview(conditioned, probe_json), probe_json)

profile.output_paths(output_path)  # [video.mp4, preview.mp4, poster.jpg]
profile.ffmpeg_graph_args()        # ["-filter_complex", "[0:v]setparams=...,split=2[chain][d1];..."]
profile.ffmpeg_output_groups()     # one argument list per output, in the same order
```

`ffmpeg_output_args()` raises for a profile with derivatives, since one `-vf`
cannot branch. The `aind-transcode` CLI does not expose derivatives yet.

### Transcode CLI

With the `transcode` extra installed, the `aind-transcode` command is available:

```bash
aind-transcode videos/                                # defaults: offline-8bit, auto-fix
aind-transcode videos/ --profile offline-10bit        # explicit profile
aind-transcode videos/ --preset veryfast              # override speed
aind-transcode videos/ --crf 20 --preset veryfast     # override quality + speed
aind-transcode videos/ --no_auto_fix_colorspace true  # skip setparams probing
aind-transcode videos/ --normalize_cfr true           # legacy h264-in-AVI sources
aind-transcode videos/ --normalize_cfr true --frame_rate 500  # a container without a measurable rate
aind-transcode videos/ --jobs 4                       # parallel workers
```

Settings can also be stored in `aind-transcode.toml` in the working directory:

```toml
input = ["videos/"]
output_dir = "transcoded"
profile = "offline-8bit"
overwrite = false
```

### MP4 Timelines

An MP4 stitched from separately encoded chunks with `concat -c copy` can carry
notches: every frame after a seam is stamped a few frame periods late, so a
reader that seeks to frame `k` at `k / fps`, as NVIDIA DALI does, lands on the
wrong frame. `read_mp4_frame_index()` reads the sample tables from the `moov`
alone, locally or over HTTP, and `timeline_notches()` reports where the
timestamps step by anything other than one frame period. `restamp_uniform()`
removes the notches without re-encoding and checks the result before returning
it. `transcode_video()` warns with `NotchedTimelineWarning` when an MP4 or MOV
source is notched, since the output keeps the notches unless `normalize_cfr=True`.

```python
from pathlib import Path

from aind_video_utils import read_mp4_frame_index, restamp_uniform

notches = read_mp4_frame_index("bottom_camera.mp4").timeline_notches()
if not notches.is_uniform:
    restamp_uniform("bottom_camera.mp4", Path("/scratch/bottom_camera.mp4"))
```

The `aind-mp4-timeline` command needs no extra:

```bash
aind-mp4-timeline notches a.mp4 https://host/b.mp4 s3://aind-open-data/.../c.mp4 -o notches.csv
aind-mp4-timeline notches --list videos.txt -j 16      # one source per line
aind-mp4-timeline restamp bottom_camera.mp4 /scratch/bottom_camera.mp4
```

### Acquisition Timing

Each camera's per-frame CSV, the file standard's `metadata.csv` or the legacy
headerless `<camera>.csv`, reads into one table. `repair_frame_times()` corrects
what the recording got wrong and names each repair it applies, so the caller can
record it: a trigger time out of order with both its neighbours, frames paired
with triggers by arrival so that each dropped frame shifts every later time,
and a frame counter that steps backwards and so counts no exposures.

```python
from aind_video_utils import read_frame_timing, repair_frame_times

table = read_frame_timing(Path("bottom_camera.csv"))
times = repair_frame_times(table.reference_time, table.frame_number)
times.time, times.dropped_before, times.repairs  # e.g. ("zip_pairing_repaired",)
```

Whether frames were paired by arrival is detected from the times at the
counter's skips; pass `pairing="by_arrival"` or `"by_exposure"` to state it, as
you must when the skips fit neither.

### QC CLI

With the `plotting` extra installed, the `aind-video-qc` command is available:

```bash
# Compare linear-light input against BT.709-encoded output
aind-video-qc linear-to-bt709 input.mp4 output.mp4

# Compare ffmpeg luma extraction with OpenCV decode
aind-video-qc opencv input.mp4

# Options
aind-video-qc linear-to-bt709 input.mp4 output.mp4 --frame-time 1.5 --dpi 300 -o qc.png
```

### QC Python API

```python
from aind_video_utils.video_qc import compare_linear_to_bt709

fig = compare_linear_to_bt709("input.mp4", "output.mp4", frame_time=0)
fig.savefig("qc.png")
```

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md) for setup, checks and conventions.

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
