Metadata-Version: 2.1
Name: aja-pose
Version: 0.0.1
Summary: Animal pose estimation using Vision Transformers and HRNet(VHR)
Home-page: https://github.com/Antony-gitau/2024_ICME_Challenge
Author: Austin Kaburia, Joan Kabura, Antony Gitau
Author-email: kaburiaaustin1@gmail.com, joankabura1@gmail.com, antonym.gitau9@gmail.com
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: absl-py
Requires-Dist: beautifulsoup4 ==4.12.3
Requires-Dist: cachetools ==5.3.3
Requires-Dist: certifi ==2024.2.2
Requires-Dist: charset-normalizer ==3.3.2
Requires-Dist: contourpy ==1.1.1
Requires-Dist: cycler ==0.12.1
Requires-Dist: Cython ==3.0.9
Requires-Dist: dataclasses ==0.6
Requires-Dist: easydict ==1.13
Requires-Dist: einops ==0.7.0
Requires-Dist: filelock ==3.13.1
Requires-Dist: fonttools ==4.49.0
Requires-Dist: fsspec
Requires-Dist: future ==1.0.0
Requires-Dist: grpcio ==1.62.0
Requires-Dist: idna ==3.6
Requires-Dist: imageio ==2.34.0
Requires-Dist: importlib-metadata ==7.0.2
Requires-Dist: importlib-resources ==6.1.3
Requires-Dist: Jinja2 ==3.1.3
Requires-Dist: json-tricks ==3.17.3
Requires-Dist: kiwisolver ==1.4.5
Requires-Dist: lazy-loader ==0.3
Requires-Dist: Markdown ==3.5.2
Requires-Dist: MarkupSafe ==2.1.5
Requires-Dist: matplotlib ==3.7.5
Requires-Dist: mpmath ==1.3.0
Requires-Dist: networkx ==3.1
Requires-Dist: numpy
Requires-Dist: oauthlib ==3.2.2
Requires-Dist: opencv-python ==4.9.0.80
Requires-Dist: packaging ==23.2
Requires-Dist: pandas
Requires-Dist: pillow ==10.2.0
Requires-Dist: protobuf ==3.20.3
Requires-Dist: pyasn1 ==0.5.1
Requires-Dist: pyasn1-modules ==0.3.0
Requires-Dist: pycocotools ==2.0.7
Requires-Dist: pyparsing ==3.1.2
Requires-Dist: PySocks ==1.7.1
Requires-Dist: python-dateutil ==2.9.0.post0
Requires-Dist: pytz ==2024.1
Requires-Dist: PyWavelets ==1.4.1
Requires-Dist: PyYAML ==6.0.1
Requires-Dist: requests ==2.31.0
Requires-Dist: requests-oauthlib ==1.3.1
Requires-Dist: rsa ==4.9
Requires-Dist: scikit-image ==0.21.0
Requires-Dist: scipy ==1.10.1
Requires-Dist: shapely ==2.0.3
Requires-Dist: six ==1.16.0
Requires-Dist: soupsieve ==2.5
Requires-Dist: sympy ==1.12
Requires-Dist: tensorboard ==2.15.0
Requires-Dist: tensorboard-data-server ==0.7.2
Requires-Dist: tensorboardX ==2.6.2.2
Requires-Dist: tifffile ==2023.7.10
Requires-Dist: timm
Requires-Dist: torch ==2.2.1+cu121
Requires-Dist: torchvision ==0.17.1+cu121
Requires-Dist: torchaudio ==2.2.1+cu121
Requires-Dist: tqdm ==4.66.2
Requires-Dist: typing-extensions ==4.10.0
Requires-Dist: tzdata ==2024.1
Requires-Dist: urllib3 ==2.2.1
Requires-Dist: werkzeug ==3.0.1
Requires-Dist: yacs ==0.1.8
Requires-Dist: zipp ==3.17.0

Welcome to our module!
# Aja-pose

AJA-pose helps you train, validate and test your animal pose estimation model.
Check out how we have done it in Google Colab.
We have evaluated our model (PCK@0.05) and the mean accuracy for the 23 keypoints for our model is 93.073%<br>
We recommend using at least Nvidia V100 GPU for faster inferencing but T4 GPUs will still work.  
<br><br>
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1N3v7Y-PN9uvw5V5PUbAYqh9vGkLfm_Km?usp=sharing)
## Getting Started

```bash
git clone https://github.com/Antony-gitau/AJA-pose.git
cd AJA-pose
pip install -e .
```
Getting our model and the dataset.
We have made our model public and can be downloaded here
```python
import urllib.request

# # Get the dataset
url = "https://storage.googleapis.com/figures-gp/animal-kingdom/dataset.zip"
destination = "dataset.zip"

urllib.request.urlretrieve(url, destination)

# Unzip the file
!unzip dataset.zip

# The model file
url = "https://storage.googleapis.com/figures-gp/animal-kingdom/all_animals_no_pretrain_106.pth"
destination = "all_animals_no_pretrain_60.pth"

urllib.request.urlretrieve(url, destination)
```
Test our model<br>
We require the path to the test images directory and the test.json file in MPII format
```python
from aja_pose import Model

# path to the images directory and annotation in mpii json format
images_directory = '' # Path to the images directory
mpii_json = '' # Path to the test.json file
model_file = 'all_animals_no_pretrain_60.pth' # Path to the model file 

# Initialize the class
model = Model()
# Test the model on Protocol 1
model.test(images_directory, protocol='P1', model=model_file)
# Test the model on Protocol 2
model.test(images_directory, protocol='P2', model=model_file)
# Test the model on birds class Protocol 3
model.test(images_directory, protocol='P3', model=model_file, animal_class='bird')
# Test the model on reptiles class Protocol 3
model.test(images_directory, protocol='P3', model=model_file, animal_class='reptile')
# Test the model on mammals class Protocol 3
model.test(images_directory, protocol='P3', model=model_file, animal_class='mammal')
# Test the model on fish class Protocol 3
model.test(images_directory, protocol='P3', model=model_file, animal_class='fish')
# Test the model on amphibian class Protocol 3
model.test(images_directory, protocol='P3', model=model_file, animal_class='amphibian')
```
You can also start to train your model or pretrain on top of ours
```python
# train a VHR model
train_json = '' # labels for the train set (train.json)
valid_json = '' # Labels for the validation set (test.json)
model_file = '' # A pytorch model file to pretrain on.
model.train(images_directory, train_json, valid_json, pretrained=model_file)

# Train a model on a particular class e.g (Ampibian)
model.train(images_directory, protocol='P3', animal_class='amphibian', model=model_file)
```

## Results
A sanity check on our model.

![image3](https://github.com/Antony-gitau/AJA-pose/assets/88529649/266e526c-48aa-4401-b411-5f161a734c83)
![image6](https://github.com/Antony-gitau/AJA-pose/assets/88529649/615f5498-1be9-4235-8df2-11e46bfb1384)
<br>
Ground Truth
![image](https://github.com/Antony-gitau/AJA-pose/assets/88529649/c7b8275d-04a5-420a-b8c5-1da70eaf6d9f)
<br>
Predictions
![image](https://github.com/Antony-gitau/AJA-pose/assets/88529649/efe360f8-3d5f-44b4-a396-a364096a2b4d)

## Performance
The performance of our model on the different animal classes is as shown below.

| Animal Class | Samples |  | Head | Shoulder | Elbow | Wrist | Hip | Knee | Ankle | Mouth | Tail | Mean |
|------|---------|--|------|----------|-------|-------|-----|------|-------|-------|------|------|
| Birds | 1705 |  | 95.756 | 93.637 | 89.774 | 88.179 | 98.975 | 97.582 | 94.326 | 98.447 | 95.112 | 95.164 |
| Reptiles | 1209 |  | 91.538 | 85.291 | 84.662 | 85.587 | 90.457 | 88.097 | 85.239 | 96.723 | 83.925 | 89.553 |
| Mammals | 1496 |  | 90.641 | 89.269 | 88.509 | 89.927 | 90.263 | 88.655 | 89.535 | 93.622 | 82.161 | 90.038 |
| Fish | 918  |  | 96.468 | 96.249 | 98.643 | 96.058 | 98.403 | 96.743 | 95.775 | 97.564 | 98.256 | 96.467 |
| Amphibian | 1279 |  | 98.128 | 94.342 | 97.948 | 98.508 | 95.491 | 94.957 | 94.319 | 98.702 | 99.568 | 95.493 |

The model performance on Protocol 1 and Protocol 2 is as shown below.

| Protocol | Samples |  | Head | Shoulder | Elbow | Wrist | Hip | Knee | Ankle | Mouth | Tail | Mean |
|----------|---------|--|------|----------|-------|-------|-----|------|-------|-------|------|------|
| P1       |    6620     |  | 94.230 | 91.054 | 90.806 | 90.920 | 94.414 | 93.233 | 92.094 | 96.867 | 92.346 | 93.073 |
| P2       | 2883    |  | 88.683 | 75.815 | 80.223 | 81.136 | 85.568 | 83.840 | 82.028 | 94.799 | 72.506 | 83.711 |


