Metadata-Version: 2.4
Name: mono-aeb
Version: 0.2.0
Summary: Detector-agnostic monocular AEB/FCW decision chain: detection → tracking → ranging → TTC → risk decision (calibration-free scale TTC)
Author: mono-aeb contributors
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/H12Y10/mono-aeb
Project-URL: Repository, https://github.com/H12Y10/mono-aeb
Project-URL: Issues, https://github.com/H12Y10/mono-aeb/issues
Keywords: AEB,FCW,forward-collision-warning,monocular,time-to-collision,TTC,autonomous-driving
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: aeb/tracker/vendor/bytetrack/LICENSE
Requires-Dist: numpy
Requires-Dist: scipy
Requires-Dist: lap
Requires-Dist: opencv-python
Provides-Extra: yolo
Requires-Dist: ultralytics; extra == "yolo"
Provides-Extra: dfine
Requires-Dist: torch; extra == "dfine"
Requires-Dist: torchvision; extra == "dfine"
Requires-Dist: pillow; extra == "dfine"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Dynamic: license-file

# mono-aeb

[![CI](https://github.com/H12Y10/mono-aeb/actions/workflows/ci.yml/badge.svg)](https://github.com/H12Y10/mono-aeb/actions/workflows/ci.yml)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit)](https://github.com/pre-commit/pre-commit)

A detector-agnostic monocular **AEB (Autonomous Emergency Braking) / FCW (Forward Collision Warning)** decision chain: detection → tracking → in-path filtering → ranging → TTC → risk decision.

The two core strengths are **calibration-free scale TTC** — which relies only on the rate of change of a target's image size and does not depend on camera intrinsics or distortion calibration — and **speed-adaptive risk thresholds**, where the AEB trigger point shifts with ego speed. The detector is pluggable: the YOLOv8 backend (`yolo` extra) and the high-accuracy D-FINE backend (`dfine` extra) both implement the same `BaseDetector` interface, and the decision chain consumes only `[x1, y1, x2, y2, score, class]`.

> 中文说明见 [README.zh-CN.md](README.zh-CN.md)。

---

## Features

- **Detector-agnostic contract** — the decision chain is decoupled from any specific detector and works with the 7 AEB classes (person / rider / car / truck / bus / bike / motor).
- **Calibration-free scale TTC** — `τ = s / ṡ`, estimated via a Theil–Sen robust fit of `(t, 1/s)`; no camera calibration required.
- **Distance TTC + fusion** — a ground-plane ranging model recovers closing speed, fused with scale TTC by taking the minimum, with a `persist_k` frame hold to suppress jitter.
- **Speed-adaptive thresholds** — `τ_AEB(v) = t_react + v / (2·a_max)`, with fixed lead margins for FCW / ATTENTION on top; the whole decision gradient shifts with ego speed.
- **Friction-aware thresholds (opt-in)** — set `RiskThresholds.friction_mu` and the same formula uses a measured `a_eff(μ)` table (`aeb/friction.py`) instead of the dry-road `a_max`, so low-friction surfaces trigger earlier; the default (`None`) keeps the fixed-`a_max` behaviour bit-for-bit.
- **Per-video self-calibration** — jointly estimates the ranging scale `f·H` and the horizon `horizon_y` from GPS ego speed + calibration-free scale TTC, removing the per-video camera-parameter inconsistency in BDD100K.
- **Bundled ByteTrack** — the tracker is vendored into the package (`aeb/tracker/vendor/bytetrack/`) and distributed with the wheel; no separate installation needed.

## Pipeline

```
video frame → detection (YOLOv8 / D-FINE)
            → tracking (ByteTrack)
            → in-path filtering (fixed trapezoid ROI, ego-path corridor projection)
            → ranging (ground-plane model, D = f·H / (y_bottom − horizon))
            → TTC (scale τ and distance τ fused via min)
            → decision (state machine NORMAL → ATTENTION → FCW → AEB_WARNING)
```

The risk state machine only escalates while a target is inside the ego-path corridor (`in_path = True`); targets outside the path stay at `NORMAL`, avoiding false triggers from adjacent-lane vehicles.

---

## Installation

Requires Python ≥ 3.10.

From PyPI:

```bash
pip install mono-aeb
```

From source (for development):

```bash
pip install -e .
```

Runtime dependencies:

| Dependency | Purpose | Notes |
| --- | --- | --- |
| `numpy` / `scipy` | numeric computation / Theil–Sen fit | `scipy` is also a transitive dependency of ByteTrack |
| `opencv-python` | image I/O, ROI, visualization | |
| `lap` | ByteTrack matching (linear assignment) | see below |

**About `lap`**: `lap` is the linear-assignment solver used by ByteTrack matching. If no `lap` wheel exists for your platform/Python, install `lapx` instead — it is an alternative distribution of `lap` that ships a module named `lap`:

```bash
pip install lapx
```

Note: this project does **not** fall back in code. When `lap` is missing, installing `lapx` as above is enough (it provides the module name `lap`, so no code changes are needed).

ByteTrack itself is vendored as a **minimal subset** (`aeb/tracker/vendor/bytetrack/`, MIT License) and installed with the wheel, so both `pip install -e .` and a regular `pip install` work without fetching the upstream repository. To use an external/full ByteTrack instead:

| Environment variable | Meaning |
| --- | --- |
| `BYTETRACK_ROOT` | ByteTrack repository root (containing `yolox/tracker/`), overriding the bundled vendor default |

If `pip install -e .` aborts because no `lap` wheel is available for your platform, install `lapx` first, then install this package while skipping dependency resolution:

```bash
pip install lapx
pip install -e . --no-deps
```

**Optional high-accuracy backend D-FINE** additionally requires the PyTorch ecosystem:

```bash
pip install -e ".[dfine]"   # torch / torchvision / pillow
```

**About Windows**: `torch` requires the Microsoft Visual C++ runtime; on some machines (especially lab/clean-room environments) its absence shows up as `OSError: [WinError 126] ... c10.dll` failing to load. Installing the [VC++ Redistributable (x64)](https://aka.ms/vs/17/release/vc_redist.x64.exe) once resolves it. (The `--demo` zero-data demo does not import torch, so it does not need this.)

On Windows consoles whose code page is not UTF-8, non-ASCII text (for example in your own file paths, or in messages coming from third-party libraries) can still raise `UnicodeEncodeError`; run `set PYTHONUTF8=1` first. Linux/macOS and UTF-8 consoles are unaffected.

---

## Quick start

### Zero-data demo (no video / no weights)

```bash
python examples/quickstart.py --demo
```

This command drives a synthetic detector that generates a target closing at constant speed, and runs the full detection → tracking → ranging → TTC → decision chain, printing state changes frame by frame:

```
[quickstart] ego speed = 12 m/s, AEB threshold = 1.37s
[quickstart] target closing at a constant 10 m/s; printing the risk state per frame:
  frame   0: NORMAL     d= 45.0m  ttc=  infs
  frame  39: ATTENTION  d= 32.0m  ttc= 3.36s
  frame  69: FCW        d= 22.0m  ttc= 2.36s
  frame  99: AEB        d= 12.1m  ttc= 1.36s
[quickstart] done, 120 frames
```

Available options: `--out video.mp4` (write a visualization video), `--show` (live window), `--frames` / `--fps` / `--v-close`. `--demo` mode does **not** require `ultralytics` / `torch`; it depends only on numpy / scipy / opencv-python / lap.

Dependency self-check:

```bash
python examples/quickstart.py --check
```

### Real video

```bash
python examples/demo_aeb.py --source video.mp4 --coco --out out.mp4
```

- `--coco`: maps COCO classes to the 7 AEB classes when using COCO-pretrained weights (e.g. `yolov8n.pt`); omit it when using AEB-finetuned weights.
- `--ego-speed`: ego speed (m/s); when omitted, GPS speed is read from BDD100K `samples-1k/info/*.json` automatically, falling back to `12 m/s` when unavailable.
- `--detector dfine`: switch to the D-FINE backend (see below).

You may also pass an image directory via `--source ./frames_dir`.

---

## Detector backends

### YOLOv8 (optional backend, `yolo` extra)

```bash
pip install -e ".[yolo]"    # ultralytics (AGPL-3.0)
```

```python
from aeb.detectors import YOLOv8Detector, COCO_TO_AEB

det = YOLOv8Detector(
    "yolov8n.pt", conf=0.25, device="0", class_map=COCO_TO_AEB
)  # class_map=None for finetuned weights
```

### Optional high-accuracy backend: D-FINE

D-FINE depends on its upstream repository, located via environment variables to avoid hardcoding local paths:

| Environment variable | Meaning |
| --- | --- |
| `DFINE_ROOT` | D-FINE upstream repository root (containing `src/`, `configs/`) |
| `MONO_AEB_DFINE_WEIGHTS` | path to finetuned weights (`.pth`) |

```bash
export DFINE_ROOT=/path/to/D-FINE
export MONO_AEB_DFINE_WEIGHTS=/path/to/best_stg1.pth
python examples/demo_aeb.py --source video.mp4 --detector dfine
```

Or pass the arguments directly in code:

```python
from aeb.detectors import DFineDetector

det = DFineDetector(weights="/path/to/best_stg1.pth")
```

The `config` argument defaults to `<DFINE_ROOT>/configs/dfine/dfine_hgnetv2_m_aeb.yml`; pass `config=...` to use another file.

> **Note**: D-FINE's finetuned weights are trained on BDD100K and are subject to that dataset's license (see below); they are not distributed with this repository.

---

## Weights & data

> **Compliance note**: this repository does not contain any model weights or BDD100K videos/annotations.

- **COCO-pretrained YOLOv8 weights** (e.g. `yolov8n.pt`) are provided by ultralytics and obtained under ultralytics' license.
- **AEB-finetuned weights** (YOLOv8 or D-FINE) are finetuned on BDD100K, whose license is "academic/non-commercial use only, no redistribution of raw data". These weights are therefore not published here; they are distributed on a restricted basis (on request). With legitimate data and license access they can be reproduced by training the upstream detector on a 7-class AEB dataset, then pointing the backend at the resulting config and weights (`DFINE_ROOT` / `MONO_AEB_DFINE_WEIGHTS` for D-FINE). This repository does not ship a training configuration.

---

## Project structure

```
mono-aeb/
├── aeb/
│   ├── config.py           # camera / ego-path / risk thresholds / detection & tracking params
│   ├── friction.py         # measured μ → a_eff table for friction-aware thresholds
│   ├── types.py            # Detection / Track / RiskFeature / RiskLevel contracts
│   ├── pipeline.py         # AEBPipeline main chain
│   ├── calibration.py      # per-video self-calibration (joint f·H & horizon estimation)
│   ├── ego_speed.py        # GPS speed from BDD100K info/*.json
│   ├── detectors/          # BaseDetector + YOLOv8 + D-FINE
│   ├── tracker/            # ByteTrack adapter + cython_bbox numpy shim
│   │   └── vendor/bytetrack/   # vendored ByteTrack (MIT, minimal subset, shipped with wheel)
│   ├── distance/           # ground-plane ranging
│   ├── ttc/                # scale TTC / distance TTC / fusion / track history
│   ├── in_path/            # fixed trapezoid ROI (ego-path corridor)
│   └── decision/           # risk state machine
├── examples/
│   ├── quickstart.py       # one-shot demo (--demo / --check)
│   ├── demo_aeb.py         # offline real-video demo
│   └── calibrate_video.py  # per-video self-calibration CLI
├── tests/                  # zero-data test suite (no video / weights / network)
├── benchmarks/             # synthetic tracking benchmark, bbox_overlaps shim cost
├── .github/workflows/      # CI (tests / lint) and Release (tag → GitHub Release, PyPI)
├── Dockerfile              # python:3.12-slim + dev extra, runs the zero-data suite
├── CONTRIBUTING.md
├── ROADMAP.md
├── CHANGELOG.md
├── pyproject.toml
├── LICENSE
├── README.md               # this file (English, primary)
└── README.zh-CN.md         # Chinese counterpart
```

---

## Core principles

### Ranging (ground-plane model)

Assuming a flat ground plane with camera height `H` and focal length `f` (which enter ranging only through their product `f·H`, a scale degeneracy):

$$
D = \frac{f \cdot H}{y_{bottom} - y_{horizon}}
$$

`f·H` and `horizon_y` are calibrated per video (see below), not from global intrinsics.

### Calibration-free scale TTC

For a target's image size `s` (box height or width), its rate of change is related to closing speed, independent of camera calibration:

$$
\tau_{scale} = \frac{s}{\dot{s}}
$$

In implementation, a Theil–Sen robust linear fit is applied to `(t, 1/s)`, yielding slope `b`, so `τ_scale = −1 / (b · s_now)`. Theil–Sen takes the median of pairwise slopes and has a breakdown point of about `29%` (`1 − 1/√2`), robust to a significant fraction of outliers — better than ordinary least squares, whose breakdown point is `0`.

### Distance TTC and fusion

From the ranging sequence `(t, D)`, fit the closing speed `v_close = −slope`, then `τ_distance = D / v_close`. The two sources are fused by taking the more conservative one:

$$
\tau_{fused} = \min(\tau_{scale}, \tau_{distance})
$$

with a `persist_k` frame hold to suppress single-frame jitter.

### Risk state machine

```
NORMAL(0) → ATTENTION(1) → FCW(2) → AEB_WARNING(3)
```

The AEB trigger threshold adapts to ego speed (`t_react` reaction time, `a_max` emergency-braking max deceleration):

$$
\tau_{AEB}(v) = t_{react} + \frac{v}{2 \cdot a_{max}}
$$

FCW / ATTENTION add fixed lead margins on top (defaults `+1.0s` / `+2.0s`). Escalation is immediate; de-escalation has hysteresis to avoid oscillation around thresholds. Required deceleration (**reserved, not currently wired into the state machine**):

$$
a_{req} = \frac{v_{close}^2}{2 \cdot (D - d_{safe})}
$$

Defaults are in `aeb/config.py` (`t_react=0.5`, `a_max=6.86 m/s²`, `d_safe=5.0 m`, `history_len=10`, `persist_k=3`, `fps=30`).

With `RiskThresholds.friction_mu` set, `a_max` is replaced by a measured effective deceleration `a_eff(μ)` from `aeb/friction.py` — 10 measured points over `μ ∈ [0.4, 3.5]` (measurement protocol in that module's docstring). Values outside the range clamp to the nearest endpoint (no extrapolation) and the result is capped at `a_max`, so high-friction surfaces behave exactly like the fixed-`a_max` formula:

```python
from aeb.config import AEBConfig

cfg = AEBConfig()  # fixed dry-road a_max
cfg.risk.friction_mu = 0.6  # low-friction surface: a_eff(0.6) = 3.566 m/s², triggers earlier
```

---

## Per-video calibration

BDD100K is crowdsourced, so cameras differ per video and there are no unified intrinsics. This repository provides two calibrations:

1. **Ego speed + scale TTC** (primary entry point): for approximately stationary targets `D = v_ego · τ_scale`, then a Theil–Sen line fit over `(1/D, y_bottom)` jointly solves the slope `f·H` and the intercept `horizon_y` (`aeb/calibration.py: estimate_ground_plane`).
2. **Known target width (reserved)**: `CameraConfig.calibrate_from_known_width()` (currently no call site, kept as a reserved interface) recovers camera height from how pixel width changes with `(y_bottom − horizon)`.

CLI:

```bash
python examples/calibrate_video.py --stem 0571873b-faf718b2
# or
python examples/calibrate_video.py --video /path/to/samples-1k/videos/xxxx.mov --max-frames 200
```

`--stem` mode locates the video directory via the `BDD100K_VIDEO_DIR` environment variable (defaults to the current directory).

---

## Development & testing

Zero-data test suite: no video, detection weights, or network needed — only the core dependencies (numpy / scipy / opencv-python / lap).

```bash
pip install -e ".[dev]"     # or: pip install pytest pytest-cov (--cov needs pytest-cov)
pytest -q --cov=aeb --cov-report=term-missing
```

| Test | Verifies |
| --- | --- |
| `tests/test_ttc_synthetic.py` | synthetic closing scenarios: ranging recovery (median relative error < 10%), dual-source TTC, monotonic four-level state-machine escalation |
| `tests/test_calibration_horizon_synthetic.py` | joint calibration recovers `horizon_y` (truth 320 / 360 / 400, tolerance 8 px) |
| `tests/test_bytetrack_vendor.py` | vendored ByteTrack runs without Cython extensions (numpy shim, numpy≥1.24 alias restoration, real tracked output) |
| `tests/test_tracking_synth.py` | synthetic multi-target tracking regression: 3 targets keep stable ids (0 ID switches, recall ≥ 0.95) in a clean scene, and a target absent for 10 frames (< `track_buffer=30`) is re-associated to the same id |
| `tests/test_import_smoke.py` | import chain, ROI trapezoid derivation, 7-class per-class tracker |
| `tests/test_quickstart_demo.py` | end-to-end `examples/quickstart.py`, asserting the level progression promised in the README |
| `tests/test_friction_table.py` | friction-aware thresholds: the measured μ→`a_eff` table (exact nodes, interpolation, endpoint clamping without extrapolation, cap at `a_max`, NaN rejected) and `friction_mu=None` staying bit-identical to the fixed-`a_max` formula |

**Coverage**: the zero-data suite covers **71%** of `aeb/` overall (measured 2026-10-11 on `v0.2.0`); the decision chain itself is well covered — `pipeline.py` 98%, `in_path/fixed_roi.py` 96%, `tracker/bytetrack.py` 96%, `ttc/*` 84–92%, `calibration.py` 85%, `friction.py` 100% — while the detector implementation modules (`detectors/dfine_detector.py`, `detectors/yolov8_detector.py`) and `ego_speed.py` report 0% because they need the optional detector backends and real GPS data. CI prints the per-module table on every run (fully covered files are skipped).

**Lint / formatting**: [`ruff`](https://github.com/astral-sh/ruff), configured in `pyproject.toml` (the vendored ByteTrack subset is excluded; CJK punctuation in comments is allowed), plus a [`pre-commit`](https://pre-commit.com/) hook set in `.pre-commit-config.yaml`. Run `pip install pre-commit && pre-commit install` to enable it locally; a dedicated CI job runs `ruff check` and `ruff format --check`.

CI (`.github/workflows/ci.yml`) runs the above tests and the zero-data demo on Ubuntu / Windows × Python 3.10 / 3.12, plus a dedicated lint job; a further job runs `pip install -e ".[yolo,dev]"`, validating full dependency resolution including the optional detector backend (yolo extra).

### Container

A [`Dockerfile`](Dockerfile) builds the same environment in one step — `python:3.12-slim` plus the `dev` extra, and no compiler, because every core dependency has a manylinux wheel:

```bash
docker build -t mono-aeb .
docker run --rm mono-aeb                                          # pytest
docker run --rm mono-aeb python examples/quickstart.py --check    # zero-data demo
```

The image is built from this directory and is not published to a registry. The default command is the zero-data test suite, and the container needs no network access at test time.

---

## License

This project's code is released under **Apache-2.0**; see [LICENSE](LICENSE).

## Third-party components & acknowledgments

| Component | License | Use |
| --- | --- | --- |
| [ByteTrack](https://github.com/FoundationVision/ByteTrack) | MIT | multi-object tracking (minimal subset vendored at `aeb/tracker/vendor/bytetrack/`, license text shipped with the wheel) |
| [D-FINE](https://github.com/Peterande/D-FINE) | Apache-2.0 | optional high-accuracy detection backend |
| [ultralytics (YOLOv8)](https://github.com/ultralytics/ultralytics) | AGPL-3.0 | optional detection backend (yolo extra) |
| [BDD100K](https://bdd-data.berkeley.edu/) | academic/non-commercial, no redistribution | training data (weights restricted) |

> **License note**: `ultralytics` is AGPL-3.0, which imposes copyleft obligations on derivative works and network-service scenarios. If your distribution needs to avoid those obligations, use the Apache-2.0 D-FINE detector instead (`--detector dfine`). This repository's own code is Apache-2.0.
