Metadata-Version: 2.4
Name: ammico
Version: 0.3.0
Summary: AI Media and Misinformation Content Analysis Tool
Maintainer-email: Inga Ulusoy <ssc@iwr.uni-heidelberg.de>, Dmitrii Kapitan <ssc@iwr.uni-heidelberg.de>
License-Expression: MIT
Project-URL: homepage, https://github.com/ssciwr/AMMICO
Project-URL: documentation, https://ssciwr.github.io/AMMICO/build/html/index.html
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: colorgram.py
Requires-Dist: colour-science
Requires-Dist: dash
Requires-Dist: dash-bootstrap-components
Requires-Dist: einops
Requires-Dist: faiss-gpu-cu11<=1.13.0
Requires-Dist: google-cloud-vision
Requires-Dist: googletrans-py
Requires-Dist: grpcio
Requires-Dist: importlib_resources
Requires-Dist: matplotlib
Requires-Dist: numpy
Requires-Dist: opencv-python
Requires-Dist: pandas
Requires-Dist: pooch
Requires-Dist: Pillow
Requires-Dist: scipy
Requires-Dist: spacy
Requires-Dist: torch<2.9
Requires-Dist: sentence-transformers<5.2
Requires-Dist: timm
Requires-Dist: torchvision
Requires-Dist: transformers<4.58
Requires-Dist: webcolors
Requires-Dist: pip>=26.0.1
Provides-Extra: api
Requires-Dist: openai>=1.0; extra == "api"
Requires-Dist: python-dotenv; extra == "api"
Provides-Extra: dev
Requires-Dist: pre-commit; extra == "dev"
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: psutil; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Requires-Dist: mkdocs-material; extra == "dev"
Requires-Dist: mkdocstrings[python]; extra == "dev"
Provides-Extra: nb
Requires-Dist: datasets; extra == "nb"
Requires-Dist: huggingface-hub; extra == "nb"
Requires-Dist: ipython; extra == "nb"
Requires-Dist: jupyter; extra == "nb"
Requires-Dist: jupyter_dash; extra == "nb"
Dynamic: license-file

# AMMICO - AI-based Media and Misinformation Content Analysis Tool

![License: MIT](https://img.shields.io/github/license/ssciwr/AMMICO)
![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/ssciwr/AMMICO/ci.yml?branch=main)
![codecov](https://img.shields.io/codecov/c/github/ssciwr/AMMICO)
![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=ssciwr_ammico&metric=alert_status)
![Language](https://img.shields.io/github/languages/top/ssciwr/AMMICO)
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ssciwr/ammico/blob/main/docs/tutorials/ammico_demo_getting_started.ipynb)

This package extracts data from images such as social media posts that contain an image part and a text part. The analysis can generate a very large number of features, depending on the user input. See [our paper](https://dx.doi.org/10.31235/osf.io/v8txj) for a more in-depth description.

**_This project is currently under development!_**

Use pre-processed image files such as social media posts with comments and process to collect information:
1. Text extraction from the images
    1. Language detection
    1. Translation into English or other languages
1. Content extraction from the images
    1. Textual summary of the image content ("image caption") 
    1. Feature extraction from the images: User inputs query and images are matched to that query (both text and image query)
    1. Question answering about image content
1. Content extraction from videos
    1. Textual summary of the video content 
    1. Question answering about video content
1. Color analysis
    1. Analyse hue and percentage of color on image
1. Multimodal analysis
    1. Find best matches for image content or image similarity
1. Cropping images to remove comments from posts
 
## Installation

The `AMMICO` package can be installed using pip: 
```
pip install ammico 
```

Or install the development version from GitHub (currently recommended for the new features):

```bash
pip install git+https://github.com/ssciwr/AMMICO.git
```
This will install the package and its dependencies locally.

## Usage

Demonstration notebooks can be found in the `docs/tutorials` folder and also on google colab: 

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ssciwr/ammico/blob/main/docs/tutorials/ammico_demo_getting_started.ipynb)

## Features
### Text extraction
The text is extracted from the images using [google-cloud-vision](https://cloud.google.com/vision). For this, you need an API key. Set up your google account following the instructions on the google Vision AI website or as described [here](https://ssciwr.github.io/AMMICO/build/html/create_API_key_link.html).
You then need to export the location of the API key as an environment variable:
```
export GOOGLE_APPLICATION_CREDENTIALS="location of your .json"
```
The extracted text is then stored under the `text` key (column when exporting a csv).

[Googletrans](https://py-googletrans.readthedocs.io/en/latest/) is used to recognize the language automatically and translate into English. The text language and translated text is then stored under the `text_language` and `text_english` key (column when exporting a csv).

### Content extraction from images and videos

The image and video content ("caption") is extracted using [QWEN 2.5 Vision-Language model family](https://huggingface.co/collections/Qwen/qwen25-vl). Qwen2.5-VL is a multimodal large language model capable of understanding and generating content from both images and videos. With its help, `ammico` supports tasks such as image/video summarization and image/video visual question answering, where the model answers users' questions about the context of a media file.

The audio transcription, language detection and translation is carried out using the [WhisperX model family](https://github.com/m-bain/whisperX) for audio transcription as [developed by OpenAI](https://arxiv.org/abs/2303.00747).

### Color/hue detection

Color detection is carried out using [colorgram.py](https://github.com/obskyr/colorgram.py) and [colour](https://github.com/vaab/colour) for the distance metric. The colors can be classified into the main named colors/hues in the English language, that are red, green, blue, yellow, cyan, orange, purple, pink, brown, grey, white, black.

## Contributing

We welcome contributions to the ammico project! If you'd like to help improve the tool, add new features, or report or fix bugs, please follow [these guidelines](CONTRIBUTING.md).

## Reporting Issues

Please use the [issues tab](https://github.com/ssciwr/AMMICO/issues) to report bugs, request features, or start discussions.

## License

ammico is licensed under the [MIT license](LICENSE).

## Citing ammico

Ammico has been published in Comp. Comm. Res., please cite the paper as specified in the [Citations](CITATION.cff) file.
