Metadata-Version: 2.4
Name: opensportslib
Version: 0.3.1.dev14
Summary: OpenSportsLib is the professional library, designed for advanced video understanding in sports. It provides state-of-the-art tools for action recognition, spotting, retrieval, and captioning, making it ideal for researchers, analysts, and developers working with sports video data.
Author: Jeet Vora
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: LICENSE-COMMERCIAL
Requires-Dist: SoccerNet
Requires-Dist: av
Requires-Dist: decord; platform_system != "Darwin" and platform_machine != "arm64" and platform_machine != "aarch64"
Requires-Dist: evaluate
Requires-Dist: scikit-learn
Requires-Dist: torch
Requires-Dist: torchvision
Requires-Dist: transformers==4.57.3
Requires-Dist: tokenizers==0.22.1
Requires-Dist: accelerate
Requires-Dist: wandb
Requires-Dist: opencv-python
Requires-Dist: omegaconf
Requires-Dist: timm
Requires-Dist: seaborn
Requires-Dist: tabulate
Requires-Dist: pytorch-lightning
Requires-Dist: pandas
Requires-Dist: pyarrow
Requires-Dist: h5py==3.16.0
Requires-Dist: huggingface_hub
Requires-Dist: easydict
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Requires-Dist: pytest-cov; extra == "test"
Dynamic: license-file

# OpenSportsLib

OpenSportsLib inference servers support runtime model registration through
`RemoteModelRegistry`. Its unified `register_model()` method accepts Hugging
Face or server-local weights; the returned model ID is then used by the existing
remote task APIs. Unregistered model IDs are rejected.

See the [complete inference server guide](docs/server/inference-server.md) for
installation, registry administration, curl requests, single-video inference,
full-test-set and per-sample remote inference, job polling, and sessions.

For Docker administration, set `OSL_MODEL_ADMIN_TOKEN` in the private
`server/.env` file, recreate the containers after changes, and use the exact
same value as the Python client's `admin_token`. Never commit the real token.

## Configuration From Hugging Face

Prepare and inspect configuration before model weights are allocated:

```python
from opensportslib.apis import Config, ClassificationModel

config = Config.from_pretrained("OpenSportsLab/OSL-cls-action-mvitv2")
# Or: Config.from_file("opensportslib/configs/classification/video.yaml")

config.update(
    data={"data_root": "/datasets/fouls"},
    training={"epochs": 30, "batch_size": 8},
    inference={"batch_size": 4},
    overrides={"TRAIN.scheduler.step_size": 5},
)
print(config.options())
model = ClassificationModel(config=config)
```

`options()` reports editable parameters supported by the selected task and
backend. `get_config()` returns a detached canonical dictionary for discovering
advanced dotted paths. Dotted overrides must already exist, are validated as
one atomic update, and are intended for local execution. Initialization-sensitive
settings such as device and output directory must be changed before creating the
model. `model.update_config(...)` supports safe settings for the next operation.

When `weights` is a Hugging Face model ID, `config` may be omitted if the
repository contains a compatible OpenSportsLib `config.yaml`:

```python
from opensportslib.apis import ClassificationModel

model = ClassificationModel(weights="OpenSportsLab/OSL-cls-action-mvitv2")
```

This also applies to localization and VQA wrappers. Local checkpoints and
repositories containing only a Transformers `config.json` still require an
explicit OpenSportsLib config. Explicit configs retain existing merge behavior.
Provide your own input data when running inference; published dataset paths may
refer to the machine used for training.

Classification and localization accept a video directly, without a manifest:

```python
classification_predictions = classification_model.infer(video_path="/path/to/clip.mp4")
localization_predictions = localization_model.infer(video_path="/path/to/full-match.mp4")
```

Direct classification treats the file as one sample. Direct localization treats
it as one timeline and returns detected events. Both return the regular OSL JSON
prediction document; use `test_set=` instead when evaluating labeled data.

<img src="docs/assets/osl.jpg" height="400">

OpenSportsLib is a modular Python library for sports video understanding.

It provides a unified framework to **train, evaluate, and run inference** for key temporal understanding tasks in sports video, including:

- **Action classification**
- **Action localization / spotting**
- **Visual Question Answering (VQA)**
- **Action retrieval**
- **Action description / captioning**

OpenSportsLib is designed for **researchers, ML engineers, and sports analytics teams** who want reproducible and extensible workflows for sports video AI.

## Why OpenSportsLib?

- Unified workflow for training and inference
- Modular design for adding new tasks, datasets, and models
- Config driven experiments for reproducibility
- Optional SpoTTA test-time adaptation for E2ESpot inference
- Support for multiple modalities and sports workflows
- Research friendly while still usable in applied settings

## Quick links

- **Documentation:** https://opensportslab.github.io/opensportslib/
- **OSL JSON format:** https://opensportslab.github.io/opensportslib/data/osl-json-format/
- **PyPI:** https://pypi.org/project/opensportslib/
- **Issues:** https://github.com/OpenSportsLab/opensportslib/issues

---

## Installation

> Requires **Python 3.12+**.  
> Supports CUDA 12.6 / 12.8 / 13.0 (with CPU fallback).  
> PyTorch Geometric is supported up to PyTorch 2.10.*.

### Create conda env

```bash
conda create -n osl python=3.12 pip -y
conda activate osl
```

### Stable release

```bash
pip install opensportslib
```

### Pre release

```bash
pip install --pre opensportslib
```

### Source development version

```bash
pip install -e .
```

### Setup Environment (PyTorch, CUDA aware & Optional Dependencies)
```bash
# Install PyTorch (CPU/GPU auto-detected)
opensportslib setup

# Optional: install PyTorch Geometric support
opensportslib setup --pyg

# Optional: install for DALI support
opensportslib setup --dali

# Optional: install the X-VARS-compatible VQA dependency profile
opensportslib setup --vqa_xvars

# Optional: install the Qwen-compatible VQA dependency profile
opensportslib setup --vqa_qwen
```
---

**Note:**  
Run `opensportslib setup` to automatically configure dependencies.  
If issues occur, manually install compatible versions of `torch`, `torchvision`, and related libraries according to your CUDA version or system compatibility.

For VQA, use exactly one backend-specific dependency profile:

- `--vqa_xvars` installs the X-VARS-compatible Hugging Face stack from `XVARS_DEPENDENCY_PINS`
- `--vqa_qwen` installs the Qwen-compatible Hugging Face stack from `QWEN_DEPENDENCY_PINS`

The `vqa_qwen` config supports `Qwen/Qwen2.5-7B-Instruct` and `Qwen/Qwen3.5-9B-Base`.

---

## Data and pretrained models

OpenSportsLib uses external annotation files, datasets, and pretrained checkpoints.

Public assets are hosted under the **OpenSportsLab Hugging Face organization**:

**https://huggingface.co/OpenSportsLab**

Use it as the main entry point to find:
- datasets
- annotation files
- extracted features
- pretrained models and checkpoints

See the [Model Zoo](docs/model-zoo.md) for available pretrained models,
reported scores, datasets, and loading snippets.

---

## Dataset format

OpenSportsLib annotation files use the **OSL JSON v2.0** format. A dataset JSON
contains top-level metadata, a shared `labels` schema, and a `data` array where
each sample points to one or more inputs.

Minimal classification sample:

```json
{
  "labels": {
    "action": {
      "type": "single_label",
      "labels": ["pass", "shot"]
    }
  },
  "data": [
    {
      "id": "clip_0001",
      "inputs": [
        {
          "type": "video",
          "path": "clips/clip_0001.mp4",
          "fps": 25.0
        }
      ],
      "labels": {
        "action": {
          "label": "shot"
        }
      }
    }
  ]
}
```

Minimal localization sample:

```json
{
  "labels": {
    "action": {
      "type": "single_label",
      "labels": ["pass", "shot"]
    }
  },
  "data": [
    {
      "id": "game_0001",
      "inputs": [
        {
          "type": "video",
          "path": "games/game_0001.mp4",
          "fps": 25.0
        }
      ],
      "events": [
        {
          "head": "action",
          "label": "pass",
          "position_ms": 1240
        }
      ]
    }
  ]
}
```

Relative paths in `inputs[].path` are resolved from the split media root in the
YAML config, for example `DATA.common.splits.train.source_path`. Localization
records may also declare half-open physical-video ranges in
`metadata.intervals`; the OpenCV loader treats them as ordered logical videos
and evaluates only segments marked `verified`. See the full
[OSL JSON format guide](docs/data/osl-json-format.md) for field definitions,
multi-modal examples, prediction payloads, and conversion notes.

---

## Quickstart

### Import the library

```python
import opensportslib
print("OpenSportsLib imported successfully")
```

### Train a classification model

```python
from opensportslib.apis import ClassificationModel

my_model = ClassificationModel(
    config="/path/to/classification.yaml",
    weights=None,  # optional: path or Hugging Face model ID
)

my_model.train(
    train_set="/path/to/train_annotations.json",
    valid_set="/path/to/valid_annotations.json",
)
```

### Run inference

```python
from opensportslib.apis import ClassificationModel

my_model = ClassificationModel(
    config="/path/to/classification.yaml",
    weights=None,  # optional: path or Hugging Face model ID
)

predictions = my_model.infer(
    test_set="/path/to/test_annotations.json",
)

saved_predictions = my_model.save_predictions(
    output_path="/path/to/predictions.json",
    predictions=predictions,
)

metrics = my_model.evaluate(
    test_set="/path/to/test_annotations.json",
)

metrics_from_file = my_model.evaluate(
    test_set="/path/to/test_annotations.json",
    predictions=saved_predictions,
)

print(metrics)
```

### Localization example

```python
from opensportslib.apis import LocalizationModel

my_model = LocalizationModel(
    config="/path/to/localization_video_dali.yaml",
    weights=None,  # optional: path or Hugging Face model ID
)

predictions = my_model.infer(
    test_set="/path/to/test_annotations.json",
)

saved_predictions = my_model.save_predictions(
    output_path="/path/to/predictions.json",
    predictions=predictions,
)

metrics = my_model.evaluate(
    test_set="/path/to/test_annotations.json",
)

metrics_from_file = my_model.evaluate(
    test_set="/path/to/test_annotations.json",
    predictions=saved_predictions,
)
```

### VQA example

```python
from opensportslib.apis import VQAModel

my_model = VQAModel(
    config="opensportslib/configs/vqa/qwen.yaml",
    weights=None,  # optional: path or Hugging Face model ID
)

predictions = my_model.infer(
    test_set="/path/to/test_annotations.json",
)

# Headless single-video VQA uses the same prediction payload shape.
single_prediction = my_model.infer(
    video_path="/path/to/video.mp4",
    question="What card would you give? Why?",
)
```

Use `opensportslib/configs/vqa/xvars.yaml` with `opensportslib setup --vqa_xvars`
for the X-VARS backend. OpenSportsLib supports three VQA options:

- `opensportslib/configs/vqa/xvars.yaml`
  Original X-VARS / Video-ChatGPT path.
- CLIP features + Qwen
  Use `opensportslib/configs/vqa/qwen.yaml` for inference and
  `opensportslib/configs/vqa/qwen_lora.yaml` for LoRA training.
- `opensportslib/configs/vqa/qwen3_vl_native.yaml`
  Full end-to-end native QwenVL path. This is the single canonical QwenVL
  config; change `MODEL.components.llm_decoder.params.repo_id` to switch model
  IDs.

Use `opensportslib setup --vqa_qwen` for both the CLIP+Qwen and native QwenVL
paths. The CLIP+Qwen configs support `Qwen/Qwen2.5-7B-Instruct` and
`Qwen/Qwen3.5-9B-Base`. The native QwenVL config defaults to
`Qwen/Qwen3-VL-8B-Instruct` and supports:

- `Qwen/Qwen3-VL-8B-Instruct`
- `Qwen/Qwen2.5-VL-7B-Instruct`

For X-VARS, `feature_source: indexed_or_raw_clip` prefers indexed CLIP features
when available and falls back to extracting CLIP features from raw video during
`infer()`. Pre-extracted features remain the preferred path for parity, speed,
and reproducibility. See [docs/tools/vqa.md](docs/tools/vqa.md) for the full
VQA setup workflow.


---

## Hugging Face Dataset Transfer

OpenSportsLib provides APIs and scripts for downloading and uploading OSL datasets with Hugging Face.

### Python API

```python
from opensportslib.tools import (
    download_dataset_split_from_hf,
    download_dataset_sample_inputs_from_hf,
    upload_dataset_inputs_from_json_to_hf,
    upload_dataset_as_parquet_to_hf,
)
```

### Scripts

```bash
python tools/download/download_osl_hf.py --repo-id <org/repo> --revision main --split test --format parquet --output-dir downloaded_data --annotations-only
python tools/download/upload_osl_hf.py --repo-id <org/repo> --json-path <local_dataset.json> --split test --revision main
```

Downloads are placed under `<output-dir>/<revision>/<split>`.
Pass `annotations_only=True` to download or reconstruct only `<split>.json`.
The JSON records the resolved Hugging Face commit and can later be passed to
`download_dataset_sample_inputs_from_hf()` to fetch one sample or input. A full
Parquet/WebDataset download always completes the local split even when a
metadata-only `<split>.json` already exists.

Download APIs accept `byte_progress_cb(filename, downloaded_bytes,
total_bytes)`. When the repository file is Xet-backed, OpenSportsLib keeps the
accelerated Xet transfer and adapts Xet's byte updates to this callback. It
falls back to classic HTTP progress when Xet is unavailable, disabled, or not
used by the file.
When byte progress is enabled, Parquet downloads also emit `[current/total]`
file messages through `progress_cb` so clients can present file-count progress.
High-level split downloads also accept `file_plan_cb(filenames)`,
`file_completed_cb(filename, local_path)`, and
`json_ready_cb(split, json_path)`. These are transfer lifecycle notifications;
callers remain responsible for queue policy and presentation. For non-dry-run
JSON datasets, pinned source metadata is persisted before `json_ready_cb` runs.

JSON uploads support partially downloaded datasets: the JSON and all
referenced files available locally are committed, while missing referenced
files are skipped and reported. Remote files not included in that commit are
left untouched. Parquet/WebDataset uploads remain strict and require every
referenced file locally before conversion.

---

## What you can do with OpenSportsLib

### Action Classification
Classify clips or event centered samples into predefined categories.

### Action Localization / Spotting
Predict when key events happen in long untrimmed sports videos.

### Visual Question Answering (VQA)
Answer natural-language questions about sports video clips.

### Action Retrieval
Search and retrieve relevant clips or moments from a collection of sports videos.
This is part of the roadmap and OSL data model, not a first-class OpenSportsLib
training workflow yet.

### Action Description / Captioning
Generate text descriptions for sports events and temporal segments.
This is part of the roadmap and OSL data model, not a first-class OpenSportsLib
training workflow yet.

---

## Typical workflow

1. Prepare your dataset in the expected format
2. Select or create a YAML config
3. Initialize the task specific model
4. Train on your annotations
5. Run inference on new data
6. Extend the pipeline with your own datasets or models

---

## Examples and documentation

Use the README for the fast start, then go deeper through:

- Full documentation: https://opensportslab.github.io/opensportslib/
- OSL JSON format: [docs/data/osl-json-format.md](docs/data/osl-json-format.md)
- High-level API guide: [opensportslib/apis/README.md](opensportslib/apis/README.md)
- Configuration guide: https://opensportslab.github.io/opensportslib/config/configuration-guide/
- Example configs: [examples/configs/](examples/configs/)
- Quickstart scripts: [examples/quickstart/](examples/quickstart/)
- Contribution guide: [CONTRIBUTING.md](CONTRIBUTING.md)
- Developer guide: [DEVELOPERS.md](DEVELOPERS.md)

---

## Development setup

For contributors who want to work from source:

```bash
git clone https://github.com/OpenSportsLab/opensportslib.git
cd opensportslib
pip install -e .
```

### Conda option

If you prefer conda:

```bash
conda create -n osl python=3.12 pip
conda activate osl
pip install -e .
```

### Setup Environment (PyTorch, CUDA aware & Optional Dependencies)
```bash
# Install PyTorch (CPU/GPU auto-detected)
opensportslib setup

# Optional: install PyTorch Geometric support
opensportslib setup --pyg

# Optional: install for DALI support
opensportslib setup --dali

# Optional: install the X-VARS-compatible VQA dependency profile
opensportslib setup --vqa_xvars

# Optional: install the Qwen-compatible VQA dependency profile
opensportslib setup --vqa_qwen
```

### Git workflow

1. Make sure you are branching from `dev`
2. Create your feature or fix branch from `dev`
3. Open a pull request back into `dev`

---

## Contributing

We welcome contributions to OpenSportsLib.

Please check:

- [CONTRIBUTING.md](CONTRIBUTING.md)
- [DEVELOPERS.md](DEVELOPERS.md)

These documents describe:

- how to add models and datasets
- coding standards
- training pipeline structure
- how to run and test the framework

---

## License

OpenSportsLib is available under dual licensing.

### Open source license
[AGPL 3.0](LICENSE) for research, academic, and community use.

### Commercial license
For proprietary or commercial deployment, please refer to [LICENSE-COMMERCIAL](LICENSE-COMMERCIAL).

---

## Citation

If you use OpenSportsLib in your research, please cite the project.

```bibtex
@misc{opensportslib,
  title={OpenSportsLib},
  author={OpenSportsLab},
  year={2026},
  howpublished={\url{https://github.com/OpenSportsLab/opensportslib}}
}
```

---

## Acknowledgments

OpenSportsLib is developed within the broader OpenSportsLab effort for sports video understanding.

## Inference server

The optional FastAPI + Redis/RQ inference server lives in [`server/`](server/README.md),
beside the main library package. It supports classification, localization, VQA,
video/manifest uploads, and the library's remote inference client.

`pip install opensportslib` installs the library only. To run the server from
this repository, activate a fresh Python 3.12 or newer environment and install the server:

```bash
pip install -e ./server
bash server/scripts/setup_env.sh
bash server/scripts/start_all.sh
```

The server installs the OpenSportsLib release from PyPI pinned to the root project
version. That release must be published before installing or building the server.

See the [server guide](server/README.md) for uv setup, model configuration,
Redis and GPU deployment with Docker Compose. Server dependencies
and runtime data are managed separately from the library.
