Metadata-Version: 2.4
Name: supernova-v5
Version: 0.1.0
Summary: Supernova NepaliFast V5 tokenizer
License: MIT
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: huggingface_hub>=0.20
Requires-Dist: regex>=2024.0.0
Dynamic: license-file

# Supernova

Supernova NepaliFast V5 tokenizer.

## Installation

```bash
pip install supernova
```

## Usage

```python
from supernova import Tokenizer

tokenizer = Tokenizer("v5")

tokens = tokenizer.encode("नेपाल राम्रो छ।")
text = tokenizer.decode(tokens)

print(tokens)
print(text)
```
