Metadata-Version: 2.1
Name: africanwhisper
Version: 0.3.0
Summary: A package for fast fine-tuning and API endpoint deployment of Whisper model specifically developed to accelerate Automatic Speech Recognition(ASR) for African Languages.
Home-page: https://github.com/KevKibe/African-Whisper
Author: Kevin Kibe
Author-email: keviinkibe@gmail.com
License: MIT
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pip==24.0
Requires-Dist: transformers==4.39.2
Requires-Dist: datasets==2.17.0
Requires-Dist: librosa==0.10.1
Requires-Dist: evaluate==0.4.1
Requires-Dist: jiwer==3.0.3
Requires-Dist: bitsandbytes==0.42.0
Requires-Dist: accelerate==0.28.0
Requires-Dist: peft==0.9.0
Requires-Dist: numpy==1.26.4
Requires-Dist: wandb==0.16.5
Requires-Dist: holoviews==1.18.3
Requires-Dist: panel==1.3.8
Requires-Dist: gradio==4.24.0
Requires-Dist: pytube==15.0.0
Requires-Dist: tf-keras==2.16.0
Requires-Dist: tensorflow==2.16.1
Requires-Dist: keras==3.1.1
Requires-Dist: scipy==1.12.0
Requires-Dist: tensorflow-probability==0.24.0
Requires-Dist: yt-dlp==2023.11.14

<h1 align="center">African Whisper: ASR for African Languages</h1>

<p align="center">
  <a href="https://twitter.com/AfriWhisper">
    <img src="https://img.shields.io/twitter/follow/AfriWhisper?style=social" alt="Twitter">
  </a>
  <a href="https://github.com/KevKibe/African-Whisper/commits/">
    <img src="https://img.shields.io/github/last-commit/KevKibe/African-Whisper?" alt="Last commit">
  </a>
  <a href="https://github.com/KevKibe/African-Whisper/blob/main/LICENSE">
    <img src="https://img.shields.io/github/license/KevKibe/African-Whisper?" alt="License">
  </a>

  

</p>

<p align="center">
    <img src= "image.png" width="100">
</p>

# Description
African Whisper is an open-source project aimed at enhancing Automatic Speech Recognition (ASR): translation and transcription capabilities for African languages. 
This is done by developing a package to allow seamless fine-tuning and deployment of the Whisper ASR model developed by OpenAI to better recognize and transcribe African languages for all developers.

## Why Whisper?

Whisper is an open-source Automatic Speech Recognition (ASR) system developed by OpenAI.<br> 
Here’s why Whisper stands out:
<details>

  - **Extensive Training Data**: Trained on 680,000 hours of multilingual and multitask(translation and transcription) supervised data from the web.

  - **Sequence-based Understanding**: Unlike Word2Vec, which lacks sequential context, Whisper considers the full sequence of spoken words, ensuring accurate context and nuance recognition.

  - **Simplification for Developers**: Using Whisper, developers can deploy one model for transcribing a multitude of languages, including underrepresented ones, without sacrificing quality or context.

  For more details, you can refer to the [Whisper ASR model paper](https://cdn.openai.com/papers/whisper.pdf).

</details>



## Proof of Concept
<details>

  A successful proof of concept has been achieved by fine-tuning the Whisper-small model using a Google Colab Notebook and tested on an audiofile to test the performance. The results were promising, indicating the potential of this approach for ASR in African languages. You can explore the process and results in detail in the [repository](https://github.com/KevKibe/Finetuning-WhisperSmall-LoRA-Swahili)

</details>

## Objectives
<details>

To develop a quick-to-use fine-tuning and deployment pipeline utilizing audio datasets by the [Mozilla Foundation](https://commonvoice.mozilla.org/en), eventually having Automatic Speech Recognition (ASR) for African languages just as good as other non-African languages.

</details>

## Features

<details>
  
  1. Fine-tune a version of [whisper](https://huggingface.co/collections/openai/whisper-release-6501bba2cf999715fd953013) on any dataset from [Mozilla's](https://huggingface.co/mozilla-foundation) Common Voice datasets.

  2. View your training run metrics on [Wandb](https://wandb.ai/).

  3. Test your fine-tuned model using Gradio UI.

  4. Deploy a REST API endpoint fro transcription of Audio files.

  5. Containerize your REST API endpoint and push to DockerHub.
</details>

## Prerequisites

- Sign up to HuggingFace and get your token keys use this [guide](https://huggingface.co/docs/hub/en/security-tokens).

- Sign up for wandb and get your token keys use this [guide](https://app.wandb.ai/login?signup=true)


# A Guide to Usage on a Notebook

## Step 1: Installation

```python
!pip install africanwhisper
# If you're on Colab, restart the session due to issue with numpy installation on colab.
```

## Step 2: Set Parameters

```python
# Set the parameters (refer to the 'Usage on VM' section for more details)
huggingface_read_token = " "
huggingface_write_token = " "
dataset_name = "mozilla-foundation/common_voice_16_1" 
language_abbr= " " # Example 'af', 'sw'. see abbreviations here https://huggingface.co/datasets/mozilla-foundation/common_voice_16_1. 
                    # Note: choose a small dataset so as to not run out of memory,
model_id= "model-id" # Example openai/whisper-small, openai/whisper-medium
processing_task= "automatic-speech-recognition" 
wandb_api_key = " "
use_peft = True  # Note: PEFT only works on a notebook with GPU-support.

```

## Step 3: Prepare the Model
```python
from training.data_prep import DataPrep

# Initialize the DataPrep class and prepare the model
process = DataPrep(
    huggingface_read_token,
    dataset_name,
    language_abbr,
    model_id,
    processing_task,
    use_peft
)
tokenizer, feature_extractor, feature_processor, model = process.prepare_model()

```

## Step 4: Preprocess the Dataset
```python
# Load and preprocess the dataset
processed_dataset = process.load_dataset(
    feature_extractor=feature_extractor,
    tokenizer=tokenizer,
    processor=feature_processor
)
```

## Step 5: Train the Model

```python
from training.model_trainer import Trainer

# Initialize the Trainer class and train the model
trainer = Trainer(
    huggingface_write_token,
    model_id,
    processed_dataset,
    model,
    feature_processor,
    feature_extractor,
    tokenizer,
    language_abbr,
    wandb_api_key,
    use_peft
)
trainer.train(
    max_steps=100,
    learning_rate=1e-5,
    per_device_train_batch_size=96,
    per_device_eval_batch_size=64,
    optim="adamw_bnb_8bit"
)

# Optional parameters for training:
#     max_steps (int): The maximum number of training steps (default is 100).
#     learning_rate (float): The learning rate for training (default is 1e-5).
#     per_device_train_batch_size (int): The batch size per GPU for training (default is 96).
#     per_device_eval_batch_size (int): The batch size per GPU for evaluation (default is 64).
#     optim (str): The optimizer used for training (default is "adamw_bnb_8bit")

```

## Step 6: Generate a Demo using GradioUI
```python
from training.gradio_inference import WhisperDemo

# Generate a demo
model_name = "your-finetuned-model-name-on-huggingface-hub" # e.g., "KevinKibe/whisper-small-af"
demo = WhisperDemo(model_name, huggingface_read_token)
demo.generate_demo()
```

## Step 7: Test Model using Audio File

```python
from deployment.speech_inference import SpeechInference

model_name = "your-finetuned-model-name-on-huggingface-hub"  # e.g., "KevinKibe/whisper-small-af"
huggingface_read_token = " "
task = "desired-task"  # either 'translate' or 'transcribe'
audiofile_dir = "location-of-audio-file"  # filetype should be .mp3 or .wav

# Initialize the SpeechInference class and run inference
inference = SpeechInference(model_name, huggingface_read_token)
pipeline = inference.pipe_initialization()
transcription = inference.output(pipeline, audiofile_dir, task)

# Access different parts of the output
print(transcription.text)  # The entire text transcription.
print(transcription.chunks)  # List of individual text chunks with timestamps.
print(transcription.timestamps)  # List of timestamps for each chunk.
print(transcription.chunk_texts)  # List of texts for each chunk.

```

## Usage on a Virtual Machine

<details>

  - Clone the Repository: Clone or download the application code to your local machine.
  ```
  git clone https://github.com/KevKibe/African-Whisper.git
  ```

  - Create a virtual environment for the project and activate it.
  ```
  python3 -m venv env
  source venv/bin/activate
  ```

  - Install dependencies by running this command
  ```
  pip install -r requirements.txt
  ```
  - Navigate to:
  ```
  cd src
  ```

  - To start the training , use the following command:
  ```
  python -m training.main \
      --huggingface_read_token YOUR_HUGGING_FACE_READ_TOKEN_HERE \
      --huggingface_write_token YOUR_HUGGING_FACE_WRITE_TOKEN_HERE \
      --dataset_name DATASET_NAME \
      --language_abbr LANGUAGE_ABBREVIATION \
      --model_id MODEL_ID \
      --processing_task PROCESSING_TASK \
      --wandb_api_key YOUR_WANDB_API_KEY_HERE \
      --use_peft 
  ```
  Here's a short description of each argument used in the command:

  - **--huggingface_read_token**: Your Hugging Face authentication token for read access. It allows you to download datasets and models from Hugging Face.

  - **--huggingface_push_token**: Your Hugging Face authentication token for write access. It's used for uploading models to your Hugging Face account.

  - **--dataset_name**: The name of the dataset you wish to use for training. Example: 'mozilla-foundation/common_voice_16_1'. This should match the dataset's identifier on the Hugging Face Datasets Hub.

  - **--language_abbr**: The abbreviation of the language for the dataset you're using. Example: 'sw' for Swahili. This is used to specify the language variant of the dataset if it supports multiple languages.

  - **--model_id**: Identifier for the pre-trained model you wish to fine-tune. Example: 'openai/whisper-small'. This should match the model's identifier on the Hugging Face Model Hub.

  - **--processing_task**: Specifies the task for which the model is being trained. Example: 'transcribe'. This defines the objective of the model training, such as transcribing audio to text.

  - **--wandb_api_key**: Your Weights & Biases (W&B) API key. This is used for logging and tracking the training process if you're using W&B for experiment tracking.

  - **--use_peft**: Add this flag to fine-tune using PEFT method and omit it to do full fine-tuning. PEFT only works on a notbeook with GPU-support.

  ## Inference

  - To get inference from your fine-tuned model, follow these steps:

  - Ensure that ffmpeg is installed by running the following commands:
  ```
  # on Ubuntu or Debian
  sudo apt update && sudo apt install ffmpeg

  # on Arch Linux
  sudo pacman -S ffmpeg

  # on MacOS using Homebrew (https://brew.sh/)
  brew install ffmpeg

  # on Windows using Chocolatey (https://chocolatey.org/)
  choco install ffmpeg

  # on Windows using Scoop (https://scoop.sh/)
  scoop install ffmpeg
  ```

  - To get the Gradio inference URL:
  ```
  python -m training.gradio_demo \
      --model_name YOUR_FINETUNED-MODEL \
      --huggingface_read_token YOUR_HUGGING_FACE_READ_TOKEN_HERE \
  ```
  - **--model_name**: Name of the fine-tuned model to use in your huggingfacehub repo. This should match the model's identifier on the Hugging Face Model Hub.
  - **--huggingface_read_token**: Your Hugging Face authentication token for read access. It allows you to download datasets and models from Hugging Face.

</details>

## Deployment

- To deploy your fine-tuned model (assuming it's on Hugging Face Hub) as a REST API endpoint, follow this [instructions](https://github.com/KevKibe/African-Whisper/blob/master/DOCS/deployment.md).


## Contributing 
Contributions are welcome and encouraged.

Before contributing, please take a moment to review our [Contribution Guidelines](https://github.com/KevKibe/African-Whisper/blob/master/DOCS/CONTRIBUTING.md) for important information on how to contribute to this project.

If you're unsure about anything or need assistance, don't hesitate to reach out to us or open an issue to discuss your ideas.

We look forward to your contributions!


## License
This project is licensed under the MIT License - see the [LICENSE](https://github.com/KevKibe/African-Whisper/blob/main/LICENSE) file for details.

## Contact
For any enquiries, please reach out to me through keviinkibe@gmail.com
