Metadata-Version: 2.1
Name: autonomize-observer
Version: 0.0.10
Summary: Comprehensive LLM observability SDK with tracing, monitoring, and cost tracking
Home-page: https://github.com/autonomize-ai/autonomize-observer.git
License: Proprietary
Keywords: llm,observability,cost tracking,monitoring,kafka,mongodb
Author: Jagveer Singh
Author-email: jagveer@autonomize.ai
Requires-Python: >=3.12,<4.0
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Provides-Extra: anthropic
Provides-Extra: modelhub
Provides-Extra: openai
Requires-Dist: anthropic (>=0.55.0,<0.56.0) ; extra == "anthropic"
Requires-Dist: autonomize-core (>=0.1.7,<0.2.0)
Requires-Dist: azure-identity (>=1.21.0,<2.0.0)
Requires-Dist: azure-storage-blob (>=12.25.1,<13.0.0)
Requires-Dist: confluent-kafka (>=2.3.0,<3.0.0)
Requires-Dist: httpx (>=0.28.1,<0.29.0) ; extra == "modelhub"
Requires-Dist: jinja2 (>=3.1.6,<4.0.0)
Requires-Dist: langchain (==0.3.26)
Requires-Dist: openai (>=1.93.0,<2.0.0) ; extra == "openai"
Requires-Dist: pandas (>=2.2.2,<3.0.0)
Requires-Dist: tenacity (>=8.2.3,<10.0.0)
Requires-Dist: tiktoken (>=0.7.0)
Project-URL: Repository, https://github.com/autonomize-ai/autonomize-observer.git
Description-Content-Type: text/markdown

# Autonomize Observer SDK

[![PyPI version](https://badge.fury.io/py/autonomize-observer.svg)](https://badge.fury.io/py/autonomize-observer)
[![Python versions](https://img.shields.io/pypi/pyversions/autonomize-observer.svg)](https://pypi.org/project/autonomize-observer/)
[![License](https://img.shields.io/badge/License-Proprietary-red.svg)](LICENSE)

A production-ready SDK for comprehensive LLM observability with reliable monitoring that makes tracking effortless. Get complete visibility into your AI applications with automatic cost tracking, performance analytics, and reliable Kafka integration.

## 🎯 Core Monitor API

### 🚀 **Simple & Reliable LLM Monitoring**
**One-line setup** with reliable Kafka monitoring:

```python
from autonomize_observer import monitor
from openai import OpenAI

# Simple and reliable!
client = monitor(OpenAI(), project_name="my-app", user_id="user123")
response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello!"}]
)
```

### ✨ **Why Core Monitor API?**
The most reliable approach for production use:
- **🎯 One-line setup**: Simple `monitor(client, ...)` pattern
- **📡 Kafka Integration**: Direct message delivery to Kafka topics
- **🔧 Provider Support**: OpenAI and Anthropic with extensible design
- **💰 Cost tracking**: Automatic cost calculation for supported models
- **🔒 Production-ready**: Built-in error handling and monitoring

### 🏛️ **Architecture Components**
Built with modular design:
- **Provider System**: Extensible provider detection and management
- **Cost Strategies**: Provider-specific cost calculation implementations
- **Kafka Producer**: Reliable event streaming to Kafka
- **Monitoring Decorators**: Client method wrapping for observability

## 🚀 Quick Start

### Installation

```bash
# Basic installation
pip install autonomize-observer

# With provider support
pip install "autonomize-observer[openai]"        # OpenAI
pip install "autonomize-observer[anthropic]"     # Anthropic  
pip install "autonomize-observer[openai,anthropic]"  # Multiple providers
```

### Core Monitor API

```python
from autonomize_observer import monitor
from openai import OpenAI

def handle_query(user_question):
    """Handle customer query with reliable observability."""
    
    # One line setup with enhanced Kafka monitoring!
    client = monitor(
        OpenAI(), 
        project_name="customer-service",
        user_id="customer123",
        session_id="session456"
    )
    
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": "You are a helpful customer service agent."},
            {"role": "user", "content": user_question}
        ]
    )
    
    return response.choices[0].message.content

# Use normally - complete observability happens automatically
answer = handle_query("How do I reset my password?")
print(answer)
```

**What you get automatically:**
- ✅ **Cost tracking** with precise token-based pricing
- ✅ **Performance metrics** (latency, throughput, errors)
- ✅ **Complete traces** sent to Kafka/MongoDB
- ✅ **Provider detection** (OpenAI, Anthropic, etc.)
- ✅ **Model analytics** and usage patterns

### Multi-Step Workflows with Logging Helpers

```python
from autonomize_observer import monitor, agent, tool
from openai import OpenAI

# Note: @agent and @tool are basic logging decorators for workflow organization
# They do NOT provide monitoring - use monitor() for LLM observability

# Initialize monitored client - this provides the actual observability
client = monitor(OpenAI(), project_name="content-generation")

@agent("content-generator")  # Basic logging decorator
def generate_marketing_content(topic, target_audience):
    """Multi-step content generation with basic step logging."""
    
    @tool("research")  # Logs step execution
    def research_topic():
        # LLM calls through monitored client are tracked
        response = client.chat.completions.create(
            model="gpt-4o",
            messages=[{
                "role": "user", 
                "content": f"Research key insights about {topic} for {target_audience}"
            }]
        )
        return response.choices[0].message.content
    
    @tool("draft")  # Logs step execution
    def create_draft(research_data):
        response = client.chat.completions.create(
            model="gpt-4o",
            messages=[{
                "role": "user",
                "content": f"Create marketing content based on: {research_data}"
            }]
        )
        return response.choices[0].message.content
    
    # Execute workflow - @agent/@tool provide logging, monitor() provides observability
    research = research_topic()
    draft = create_draft(research)
    
    return {
        "research": research,
        "draft": draft
    }

# Run workflow - logging helpers track steps, monitor() tracks LLM usage
result = generate_marketing_content("AI automation", "tech startups")
```

### Async Support

```python
from autonomize_observer import monitor
from openai import AsyncOpenAI

async def async_ai_function():
    """Async function with automatic observability."""
    client = monitor(AsyncOpenAI(), project_name="async-app")
    
    response = await client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": "Hello async world!"}]
    )
    
    return response.choices[0].message.content

# Use with async/await
result = await async_ai_function()
```

## 🏗️ Architecture

```mermaid
graph TB
    subgraph "Autonomize Observer SDK v0.0.10"
        subgraph "API Layer"
            A[Core Monitor API<br/>monitor&#40;client, project_name&#41;]
            B[Agent/Tool Helpers<br/>@agent&#40;&#41; @tool&#40;&#41;<br/>Step logging only]
        end
        
        subgraph "Observability Engine"
            C[Provider Factory<br/>Provider Detection]
            D[Cost Strategies<br/>Cost Calculation]
            E[Monitoring Decorator<br/>Client Wrapping]
        end
        
        subgraph "Event Streaming"
            F[Kafka Producer<br/>Event Streaming]
            G[MongoDB<br/>Storage]
            H[Dashboard APIs<br/>Analytics Ready]
        end
        
        A -->|LLM Monitoring| C
        A --> D
        A --> E
        B -->|Step Logging| E
        
        C --> F
        D --> F
        E --> F
        
        F --> G
        F --> H
    end
    
    I[Your Application] --> A
    I --> B
    
    style A fill:#e1f5e1,stroke:#4caf50,stroke-width:2px
    style B fill:#fff3e0,stroke:#ff9800,stroke-width:2px
    style F fill:#e3f2fd,stroke:#2196f3,stroke-width:2px
```

## 🎯 Key Features

### **🤖 Automatic LLM Monitoring**
- **Zero Configuration**: No setup required for common use cases
- **Auto-Detection**: Automatically detects LLM providers and models
- **Auto-Monitoring**: Clients automatically monitored with `monitor()` function
- **Context-Aware**: Proper scoping and resource management

### **⚡ High Performance**
- **0.006ms Overhead**: 833x better than 5ms requirement
- **42M+ Operations/Second**: Proven scalability
- **Minimal Memory**: Optimized with weak references
- **Thread-Safe**: Full concurrent access support

### **🏢 Enterprise Ready**
- **Multi-Provider**: OpenAI, Anthropic, with extensible support for more
- **Kafka Streaming**: Real-time observability data pipeline
- **MongoDB Storage**: Scalable analytics database
- **Dashboard APIs**: Built-in endpoints for analytics platforms

### **🔄 Migration Friendly**
- **100% Backward Compatibility**: All existing code works unchanged
- **Gradual Adoption**: Easy integration into existing projects
- **Zero Risk**: No breaking changes for existing integrations
- **Simple Patterns**: Direct, straightforward API

## 📚 Configuration

### Environment Variables

```bash
# Kafka Configuration (Optional - defaults provided)
export AUTONOMIZE_KAFKA_BROKERS="your-kafka-brokers:9092"
export AUTONOMIZE_KAFKA_TOPIC="genesis-traces-streaming"
export AUTONOMIZE_KAFKA_USERNAME="your-username"
export AUTONOMIZE_KAFKA_PASSWORD="your-password"

# Security (Optional)
export AUTONOMIZE_KAFKA_SECURITY_PROTOCOL="SASL_SSL"
export AUTONOMIZE_KAFKA_SASL_MECHANISM="PLAIN"

# Provider API Keys
export OPENAI_API_KEY="your-openai-key"
export ANTHROPIC_API_KEY="your-anthropic-key"
```

### Programmatic Configuration

```python
from autonomize_observer import monitor
from openai import OpenAI

# Monitor with custom configuration
client = monitor(
    OpenAI(),
    project_name="my-app",
    user_id="user-123",
    session_id="session-456",
    # Optional: custom cost rates
    enable_cost_tracking=True
)
```

## 🧪 Testing & Validation

### Comprehensive Test Suite

```bash
# Run all tests
pytest tests/

# Test core monitoring
pytest tests/test_monitoring.py -v

# Performance validation
python performance_test_new_api.py

# Core API functionality
pytest tests/test_agent_tracer.py tests/core/ tests/monitoring/
```

### Performance Benchmarks

```python
# Performance test example
from autonomize_observer import monitor
from openai import OpenAI

def benchmark_function():
    client = monitor(OpenAI(), project_name="perf-test")
    # Your LLM calls here
    pass

# Overhead: 0.006ms (requirement: <5ms) ✅
# Throughput: 42M+ operations/second ✅
# Memory: Optimized with weak references ✅
```

## 📖 Examples & Documentation

### Jupyter Notebooks

Explore comprehensive examples in `examples/notebooks/`:

- **`00_modern_api_showcase.ipynb`** - Complete core API examples
- **`01_basic_monitoring.ipynb`** - Simple LLM call monitoring
- **`02_advanced_tracing.ipynb`** - Multi-step workflow monitoring
- **`03_cost_tracking.ipynb`** - Cost analytics and budgeting
- **`04_async_monitoring.ipynb`** - Async/await patterns

### Real-World Examples

```python
# Customer service chatbot
def handle_support_ticket(ticket_content, customer_id):
    client = monitor(OpenAI(), project_name="customer-service", user_id=customer_id)
    # Automatic observability for support interactions
    pass

# Content generation pipeline
@agent("content-pipeline")
def content_generation_pipeline(brief, target_audience):
    client = monitor(OpenAI(), project_name="marketing")
    # Step-by-step workflow tracking with basic logging
    pass

# Data analysis assistant
async def analyze_data_async(dataset_info):
    client = monitor(AsyncOpenAI(), project_name="data-analysis")
    # Async data analysis with observability
    pass
```

## 🚀 Advanced Usage

### Multiple Providers

```python
from autonomize_observer import monitor
from openai import OpenAI
from anthropic import Anthropic

# Auto-detection works with any provider
def multi_provider_workflow():
    openai_client = monitor(OpenAI(), project_name="multi-provider")
    anthropic_client = monitor(Anthropic(), project_name="multi-provider")
    
    # All automatically monitored with provider-specific optimizations
```

### Error Handling

```python
from autonomize_observer import monitor
from openai import OpenAI

def robust_ai_function():
    try:
        client = monitor(OpenAI(), project_name="robust-app")
        response = client.chat.completions.create(...)
        return response.choices[0].message.content
    except Exception as e:
        # Errors automatically captured in observability data
        print(f"AI call failed: {e}")
        return "I'm sorry, I couldn't process that request."
```

### Integration with Existing Systems

```python
# Works with your existing monitoring
import logging
from autonomize_observer import monitor
from openai import OpenAI

logger = logging.getLogger(__name__)

def integrated_function(user_input):
    logger.info("Processing user input")
    
    client = monitor(OpenAI(), project_name="existing-system")
    response = client.chat.completions.create(...)
    
    logger.info("AI processing complete")
    return response.choices[0].message.content
```

## 🔧 Migration Guide

### From Manual Setup

```python
# OLD: Manual initialization (deprecated)
from autonomize_observer import monitor, initialize
initialize()
client = monitor(OpenAI(), provider="openai")

# NEW: Direct monitoring (recommended)
from autonomize_observer import monitor
client = monitor(OpenAI(), project_name="my-app", user_id="user123")
```

### From Other Observability Tools

```python
# Replace complex setups with simple monitoring
from autonomize_observer import monitor
from openai import OpenAI

def previously_complex_setup():
    # No more manual instrumentation needed
    client = monitor(OpenAI(), project_name="migrated-app")
    return client.chat.completions.create(...)
```

## 📈 Analytics & Dashboards

### Data Access

The SDK automatically creates rich observability data:

```javascript
// MongoDB Collections Created
{
  "traces": {           // Complete execution traces
    "trace_id": "...",
    "project_name": "my-app", 
    "total_cost": 0.045,
    "duration_ms": 1250,
    "steps": [...]
  },
  "model_usage": {      // Individual LLM calls
    "model": "gpt-4o",
    "provider": "openai",
    "input_tokens": 150,
    "output_tokens": 75,
    "cost": 0.025
  },
  "realtime_metrics": { // Aggregated analytics
    "entity_type": "model",
    "entity_name": "gpt-4o", 
    "total_calls": 1247,
    "total_cost": 45.67
  }
}
```

### Dashboard APIs

```python
# Query your observability data
GET /api/v1/observability/dashboard/summary
GET /api/v1/observability/dashboard/costs
GET /api/v1/observability/dashboard/models
```

## 🔬 Technical Details

### Design Patterns Used

- **Factory Pattern**: Automatic provider detection and instantiation
- **Strategy Pattern**: Provider-specific cost calculation algorithms  
- **Decorator Pattern**: Non-invasive client monitoring
- **Observer Pattern**: Event-driven observability system

### Thread Safety

```python
# Safe for concurrent use
import threading
from autonomize_observer import monitor
from openai import OpenAI

def thread_safe_function():
    client = monitor(OpenAI(), project_name="concurrent-app")
    return client.chat.completions.create(...)

# Multiple threads can safely use monitoring
threads = [threading.Thread(target=thread_safe_function) for _ in range(10)]
```

### Performance Characteristics

- **Latency Overhead**: 0.006ms average
- **Memory Overhead**: <1MB baseline + weak references
- **CPU Overhead**: <0.1% in production workloads
- **Network Overhead**: Async Kafka publishing (non-blocking)

## Available Functions

### Core Monitoring
- ✅ `monitor(client, ...)` - Primary monitoring function for LLM observability
- ✅ `initialize()` - Initialize the monitoring system
- ✅ `CostTracker` - Cost calculation and tracking

### Logging Helpers (Not Monitoring)
- ✅ `@agent(name)` - Basic logging decorator for workflow steps
- ✅ `@tool(name)` - Basic logging decorator for tool functions
- Note: These decorators only provide logging, not LLM monitoring

### Advanced Features
- ✅ `AgentTracer` - Advanced tracing capabilities
- ✅ `setup_logger()` - Logging configuration
- ✅ `KafkaLLMMonitor` - Direct Kafka integration for LLM events

## 🤝 Contributing

We welcome contributions! Here's how to get started:

```bash
# Setup development environment
git clone https://github.com/autonomize-ai/autonomize-observer.git
cd autonomize-observer
pip install -e ".[dev]"

# Run tests
pytest tests/ -v

# Check performance
python performance_test_new_api.py
```

### Development Priorities

- **New Provider Support**: Add more LLM providers
- **Enhanced Analytics**: Advanced cost optimization features
- **Performance Optimizations**: Further reduce overhead
- **Documentation**: More examples and use cases

## 📜 License

Proprietary - Autonomize AI

## 🆘 Support

- **Documentation**: Complete examples in `examples/notebooks/`
- **Issues**: [GitHub Issues](https://github.com/autonomize-ai/autonomize-observer/issues)
- **Migration Help**: Contact [jagveer@autonomize.ai](mailto:jagveer@autonomize.ai)

---

**Autonomize Observer SDK v0.0.10** - Production-ready LLM observability with reliable monitoring that makes tracking effortless.

### 🎉 Ready to Start?

```python
pip install autonomize-observer

from autonomize_observer import monitor
from openai import OpenAI

# Simple, reliable monitoring
client = monitor(OpenAI(), project_name="your-app", user_id="user123")
response = client.chat.completions.create(...)
```

**That's it!** Complete observability with one function call. 🚀
