Metadata-Version: 2.4
Name: visin-fusion
Version: 0.1.0
Summary: Camera and LiDAR fusion models and training tools
Author: Toomas Tahves
License-Expression: MIT
Project-URL: Homepage, https://github.com/visin-platform/visin-fusion
Project-URL: Documentation, https://visin-platform.github.io/visin-fusion/
Project-URL: Changelog, https://github.com/visin-platform/visin-fusion/blob/main/CHANGELOG.md
Project-URL: Issues, https://github.com/visin-platform/visin-fusion/issues
Keywords: semantic segmentation,sensor fusion,lidar,autonomous driving,pytorch
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Image Recognition
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.2
Requires-Dist: torchvision>=0.17
Requires-Dist: timm>=1.0
Requires-Dist: einops>=0.7
Provides-Extra: train
Requires-Dist: numpy; extra == "train"
Requires-Dist: scipy; extra == "train"
Requires-Dist: pydantic>=2; extra == "train"
Requires-Dist: tqdm; extra == "train"
Requires-Dist: tensorboard; extra == "train"
Requires-Dist: Pillow; extra == "train"
Requires-Dist: opencv-python; extra == "train"
Requires-Dist: pandas; extra == "train"
Requires-Dist: matplotlib; extra == "train"
Requires-Dist: psutil; extra == "train"
Requires-Dist: nvidia-ml-py; extra == "train"
Requires-Dist: GPUtil; extra == "train"
Requires-Dist: thop; extra == "train"
Requires-Dist: fvcore; extra == "train"
Provides-Extra: visin
Requires-Dist: visin>=0.2; extra == "visin"
Requires-Dist: python-dotenv; extra == "visin"
Requires-Dist: requests; extra == "visin"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Dynamic: license-file

# Visin Fusion

A Python library for semantic segmentation from camera images and projected LiDAR. Five research
model families share one call: `model(rgb, lidar)` returns class logits shaped
`[batch, classes, height, width]`.

## Install

Install from a source checkout:

```bash
git clone https://github.com/visin-platform/visin-fusion.git
cd visin-fusion
python -m pip install -e .
```

The base install provides the models. Add `.[train]` for the training and evaluation pipeline,
`.[visin]` for Visin reporting and datasets, or `.[dev]` for tests. See the
[installation and API guide](docs/library.md) for details.

## Use a model

```python
import torch
from visin_fusion.models import CLFTv2

model = CLFTv2(num_classes=4, mode="cross_fusion", pretrained=False).eval()
rgb = torch.randn(1, 3, 256, 256)
lidar = torch.randn(1, 3, 256, 256)  # camera-aligned LiDAR projection
with torch.inference_mode():
    logits = model(rgb, lidar)  # [1, 4, 256, 256]
```

| Model | Main difference |
| --- | --- |
| [CLFT](docs/architecture/CLFT.md) | Global ViT features fused during decoding |
| [CLFTv2](docs/architecture/CLFTv2.md) | Hierarchical Swin fusion pyramid |
| [MaskFormerFusion](docs/architecture/MaskFormer.md) | Class-labelled mask queries |
| [Mask2FormerFusion](docs/architecture/Mask2Former.md) | Multi-scale masked-attention queries |
| [DeepLabV3+](docs/architecture/DeepLabV3Plus.md) | Two convolutional branches with late fusion |

Each model page links its research papers. See [model comparison](docs/models.md) for the fusion points
and [Python API](docs/reference/python-api.md) for modes, training and checkpoints. The LiDAR input is
a three-channel projected image, not a raw point cloud.

## Optional pipeline

Train, test, visualize and benchmark CLFTv2 on the included sample dataset:

```bash
python -m pip install -e '.[train]'
visin-fusion run -c configs/quickstart.json
```

The [pipeline guide](docs/running.md) covers configs, Docker, SLURM and individual stages. Add
`.[train,visin]` and a Visin pipeline key to report runs; see the [Visin integration guide](integrations/README.md).

## Tests

```bash
python -m pip install -e '.[train,visin,dev]'
pytest tests/unit
python tools/coverage.py  # unit + end-to-end; requires >90% package coverage
```

See [tests/README.md](tests/README.md) for focused end-to-end commands. The full documentation is at
[visin-platform.github.io/visin-fusion](https://visin-platform.github.io/visin-fusion/).

## License

[MIT](LICENSE).
