Metadata-Version: 2.4
Name: tokenizebot
Version: 0.1.1
Summary: A simple rule-based tokenizer for English text
License-Expression: MIT
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# TokenizeBot

![PyPI version](https://img.shields.io/pypi/v/tokenizebot)
![License](https://img.shields.io/pypi/l/tokenizebot)

A lightweight, rule-based tokenizer for handling English clitics and punctuation.

## Installation
```bash
pip install tokenizebot
```

## Usage
```python
from tokenizebot import TokenizeBot
bot = TokenizeBot()

tokens = bot.tokenize("They've been busy.", lowercase=True)
print(tokens) 
```
