Metadata-Version: 2.4
Name: llm-usage-framework
Version: 0.2.0
Summary: Generic LLM usage and observability framework
Author: LLM Usage Framework Contributors
License-Expression: MIT
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Provides-Extra: ollama
Requires-Dist: httpx>=0.27; extra == "ollama"
Provides-Extra: tiktoken
Requires-Dist: tiktoken>=0.14; extra == "tiktoken"
Provides-Extra: openai
Requires-Dist: openai>=1.0; extra == "openai"
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.30; extra == "anthropic"
Provides-Extra: gemini
Requires-Dist: google-genai>=1.0; extra == "gemini"
Provides-Extra: bedrock
Requires-Dist: boto3>=1.34; extra == "bedrock"
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-mock>=3.0; extra == "dev"
Requires-Dist: build>=1.0; extra == "dev"

# LLM Usage Framework

A generic, provider-independent Python framework for tracking LLM usage and collecting standardized telemetry.

## Features

- Multiple LLM provider support
- Standardized `LLMTracker` interface
- Input, output, and total token tracking
- Cache usage tracking when reported by the provider
- Token source tracking
- Character count tracking
- Latency tracking
- Provider and model tracking
- Project, service, feature, and metadata support
- Request ID and session ID tracking
- Optional tokenizer fallback
- Optional `on_usage` callback
- `LLMConfig` support
- Provider-specific parameters through `provider_kwargs`
- Storage-agnostic
- No database dependency

## Supported Providers

| Provider | Status |
|---|---|
| Ollama | Supported |
| OpenAI | Supported |
| Anthropic | Supported |
| AWS Bedrock | Supported |
| Google Gemini | Planned |

---

# Installation

Install the core package:

```bash
pip install llm-usage-framework
```

## Usage

### Basic Example (Ollama)

Here is a simple example of how to use the `LLMTracker` with the Ollama provider:

```python
from llm_usage.tracker import LLMTracker

# Initialize the tracker
tracker = LLMTracker(
    provider="ollama",
    model="qwen3:8b",
    project="my_project",
    service="my_service",
    provider_kwargs={
        "base_url": "http://localhost:11434",
    },
)

# Generate a response
result = tracker.generate(
    prompt="Say hello in one sentence.",
)

# The result contains standardized usage metrics
print(result.text)
print(f"Input tokens: {result.usage.input_tokens}")
print(f"Output tokens: {result.usage.output_tokens}")
print(f"Total tokens: {result.usage.total_tokens}")
print(f"Latency: {result.latency_ms} ms")
```

### Usage with OpenAI

You can easily switch to a different provider by changing the initialization parameters:

```python
from llm_usage.tracker import LLMTracker
import os

tracker = LLMTracker(
    provider="openai",
    model="gpt-4",
    project="my_project",
    service="my_service",
    provider_kwargs={
        "api_key": os.getenv("OPENAI_API_KEY"),
    },
)

result = tracker.generate(
    prompt="Explain quantum computing in simple terms.",
)

print(result.text)
print(f"Usage: {result.usage.total_tokens} tokens")
```

### Usage with AWS Bedrock

You can also use AWS Bedrock by specifying the model and passing the `region_name` in the provider kwargs. Ensure you have your AWS credentials configured (e.g., via environment variables or the `~/.aws/credentials` file):

```python
from llm_usage.tracker import LLMTracker

tracker = LLMTracker(
    provider="bedrock",
    model="amazon.nova-micro-v1:0",
    project="my_project",
    service="my_service",
    provider_kwargs={
        "region_name": "us-east-1",
    },
)

result = tracker.generate(
    prompt="What is cloud computing?",
)

print(result.text)
print(f"Usage: {result.usage.total_tokens} tokens")
```

### Usage with LLMConfig

You can also encapsulate your configuration using the `LLMConfig` object. This is especially useful when you need to pass the configuration across different parts of your application:

```python
from llm_usage.tracker import LLMTracker
from llm_usage.config import LLMConfig

# Define the framework configuration
config = LLMConfig(
    provider="anthropic",
    model="claude-3-haiku-20240307",
    project="my_project",
    service="my_service",
    provider_kwargs={
        "api_key": "YOUR_ANTHROPIC_API_KEY",
    },
)

# Initialize the tracker with the config
tracker = LLMTracker(config=config)

result = tracker.generate(
    prompt="Write a short poem about the ocean.",
)

print(result.text)
print(f"Usage: {result.usage.total_tokens} tokens")
```
