Metadata-Version: 2.1
Name: aitviewer
Version: 1.4.0
Summary: Viewing and rendering of sequences of 3D data.
Home-page: https://github.com/eth-ait/aitviewer
Author: Manuel Kaufmann, Velko Vechev, Dario Mylonopoulos
License: UNKNOWN
Keywords: viewer,moderngl,machine learning,sequences,smpl,computer graphics,computer vision,3D,meshes,visualization
Platform: any
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE

# [![AITV](https://raw.githubusercontent.com/eth-ait/aitviewer/main/aitv_logo.svg)](https://github.com/eth-ait/aitviewer) AITViewer

A set of tools to visualize and interact with sequences of 3D data with cross-platform support on Windows, Linux, and Mac OS X.

![AITV Sample](https://raw.githubusercontent.com/eth-ait/aitviewer/main/aitv_sample.png)


## Features
* Easy to use Python interface.
* Load [SMPL[-H | -X]](https://smpl.is.tue.mpg.de/) / [MANO](https://mano.is.tue.mpg.de/) / [FLAME](https://flame.is.tue.mpg.de/) sequences and display them in an interactive viewer.
* Manually editable SMPL sequences.
* Render 3D data on top of images via weak-perspective or OpenCV camera models.
* Built-in extensible GUI (based on Dear ImGui).
* Export the scene to a video (mp4/gif) via the GUI or render videos/images in headless mode.
* Animatable camera paths.
* Prebuilt renderable primitives (cylinders, spheres, point clouds, etc).
* Support live data feeds and rendering (e.g., webcam).
* Modern OpenGL shader-based rendering pipeline for high performance (via ModernGL / ModernGL Window).

https://user-images.githubusercontent.com/5639197/188625409-1c86b12e-4e91-48ba-ab7e-b050508d586b.mp4

![AITV SMPL Editing](https://user-images.githubusercontent.com/5639197/188625764-351100e9-992e-430c-b170-69d4f142f5dd.gif)

## Installation
Basic Installation:
```commandline
pip install aitviewer
```

Or install locally (if you need to extend or modify code)
```commandline
git clone git@github.com:eth-ait/aitviewer.git
cd aitviewer
pip install -e .
```

Note that this does not install the GPU-version of PyTorch automatically. If your environment already contains it, you should be good to go, otherwise install it manually.

## Configuration
The viewer loads default configuration parameters from [`aitvconfig.yaml`](aitviewer/aitvconfig.yaml). There are three ways how to override these parameters:
  - Create a file named `aitvconfig.yaml` and have the environment variable `AITVRC` point to it. Alternatively, you can point `AITVRC` to the directory containing `aitvconfig.yaml`.
  - Create a file named `aitvconfig.yaml` in your current working directory, i.e. from where you launch your python program.
  - Pass a `config` parameter to the `Viewer` constructor.

Note that the configuration files are loaded in this order, i.e. the config file in your working directory overrides all previous parameters.

The configuration management is using [OmegaConf](https://omegaconf.readthedocs.io/en/2.1_branch/index.html). You will probably want to override the following parameters at your convenience:
- `datasets.amass`: where [AMASS](https://amass.is.tue.mpg.de/) is stored if you want to load AMASS sequences.
- `smplx_models`: where SMPLX models are stored, preprocessed as required by the [`smplx` package](https://github.com/vchoutas/smplx).
- `export_dir`: where videos and other outputs are stored by default.


## Quickstart
View an SMPL template

```py
from aitviewer.renderables.smpl import SMPLSequence
from aitviewer.viewer import Viewer

smpl_template = SMPLSequence.t_pose()

# Display in viewer.
v = Viewer()
v.scene.add(smpl_template)
v.run()
```

## Examples

Check out the [examples](examples/) for a few examples how to use the viewer:

 * [`camera_path.py`](examples/camera_path.py): Example how to use camera paths.

 * [`headless_rendering.py`](examples/headless_rendering.py): Example how to render a video in headless mode.

 * [`load_3DPW.py`](examples/load_3DPW.py): Loads an SMPL sequence from the 3DPW dataset and displays it in the viewer.

 * [`load_AMASS.py`](examples/load_AMASS.py): Loads an SMPL sequence from the AMASS dataset and displays it in the viewer.

 * [`load_DIP.py`](examples/load_DIP.py): Loads an SMPL and IMU sequence taken from the TotalCapture dataset as used by [DIP](https://github.com/eth-ait/dip18).

 * [`load_GLAMR.py`](examples/load_GLAMR.py): Loads a result obtained from [GLAMR](https://github.com/NVlabs/GLAMR) and displays it in the viewer both for 3D and 2D inspection.

 * [`load_obj.py`](examples/load_obj.py): Loads meshes from OBJ files.

 * [`load_ROMP.py`](examples/load_ROMP.py): Loads the result of [ROMP](https://github.com/Arthur151/ROMP) and overlays it on top of the input image using the OpenCV camera model.

 * [`load_template.py`](examples/load_template.py): Loads the template meshes of SMPL-H, MANO, and FLAME.

 * [`load_VIBE.py`](examples/load_VIBE.py): Loads the result of [VIBE](https://github.com/mkocabas/VIBE) and overlays it on top of the input image.

 * [`quickstart.py`](examples/quickstart.py): The above quickstart example.

 * [`render_primitives.py`](examples/render_primitives.py): Renders a bunch of spheres and lines.

 * [`stream.py`](examples/stream.py): Streams your webcam into the viewer.

 * [`vertex_clicking.py`](examples/vertex_clicking.py): An example how to subclass the basic Viewer class for custom interaction.

## Projects using the AITViewer
The following projects have used the AITViewer:
- Dong et al., [Shape-aware Multi-Person Pose Estimation from Multi-view Images](https://ait.ethz.ch/projects/2021/multi-human-pose/), ICCV 2021
- Kaufmann et al., [EM-POSE: 3D Human Pose Estimation from Sparse Electromagnetic Trackers](https://ait.ethz.ch/projects/2021/em-pose/), ICCV 2021
- Vechev et al., [Computational Design of Kinesthetic Garments](https://ait.ethz.ch/projects/2022/cdkg/), Eurographics 2021
- Guo et al., [Human Performance Capture from Monocular Video in the Wild](https://ait.ethz.ch/projects/2021/human-performance-capture/index.php), 3DV 2021
- Dong and Guo et al., [PINA: Learning a Personalized Implicit Neural Avatar from a Single RGB-D Video Sequence](https://zj-dong.github.io/pina/), CVPR 2022

## Citation
If you use this software, please cite it as below.
```commandline
@software{Kaufmann_Vechev_AITViewer_2022,
  author = {Kaufmann, Manuel and Vechev, Velko and Mylonopoulos, Dario},
  doi = {10.5281/zenodo.1234},
  month = {7},
  title = {{AITViewer}},
  url = {https://github.com/eth-ait/aitviewer},
  year = {2022}
}
```

## Contact & Contributions
This software was developed by [Manuel Kaufmann](mailto:manuel.kaufmann@inf.ethz.ch) and [Velko Vechev](mailto:velko.vechev@inf.ethz.ch) and Dario Mylonopoulos.
For questions please create an issue.
We welcome and encourage module and feature contributions from the community.


