Metadata-Version: 2.4
Name: atla-insights
Version: 0.0.20
Summary: Atla is a platform for monitoring and improving AI agents.
Project-URL: Homepage, https://atla-ai.com
Author-email: Atla Team <support@atla-ai.com>
License: Apache-2.0
License-File: LICENSE
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Requires-Python: >=3.10
Requires-Dist: httpx>=0.24.0
Requires-Dist: openai>=1.77.0
Requires-Dist: openinference-instrumentation>=0.1.32
Requires-Dist: openinference-semantic-conventions>=0.1.17
Requires-Dist: opentelemetry-api>=1.32.1
Requires-Dist: opentelemetry-exporter-otlp-proto-grpc>=1.32.1
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.32.1
Requires-Dist: opentelemetry-instrumentation>=0.54b0
Requires-Dist: pydantic>=2.11.4
Requires-Dist: python-dateutil>=2.8.0
Requires-Dist: rich>=13.9.4
Requires-Dist: wrapt>=1.17.2
Provides-Extra: agno
Requires-Dist: agno>=1.5.0; extra == 'agno'
Requires-Dist: openinference-instrumentation-agno>=0.1.4; extra == 'agno'
Provides-Extra: all
Requires-Dist: agno>=1.5.0; extra == 'all'
Requires-Dist: anthropic[bedrock]>=0.52.1; extra == 'all'
Requires-Dist: baml-py>=0.201.0; extra == 'all'
Requires-Dist: boto3>=1.38.17; extra == 'all'
Requires-Dist: claude-code-sdk>=0.0.19; extra == 'all'
Requires-Dist: crewai>=0.130.0; extra == 'all'
Requires-Dist: google-genai>=1.19.0; extra == 'all'
Requires-Dist: google-generativeai>=0.7.0; extra == 'all'
Requires-Dist: langchain-openai>=0.3.4; extra == 'all'
Requires-Dist: langchain>=0.3.18; extra == 'all'
Requires-Dist: langgraph>=0.3.20; extra == 'all'
Requires-Dist: litellm<1.74.0,>=1.72.0; extra == 'all'
Requires-Dist: mcp>=1.9.0; extra == 'all'
Requires-Dist: openai-agents>=0.0.7; extra == 'all'
Requires-Dist: openinference-instrumentation-agno>=0.1.4; extra == 'all'
Requires-Dist: openinference-instrumentation-anthropic>=0.1.18; extra == 'all'
Requires-Dist: openinference-instrumentation-bedrock>=0.1.26; extra == 'all'
Requires-Dist: openinference-instrumentation-crewai>=0.1.10; extra == 'all'
Requires-Dist: openinference-instrumentation-google-genai==0.1.2; extra == 'all'
Requires-Dist: openinference-instrumentation-langchain==0.1.43; extra == 'all'
Requires-Dist: openinference-instrumentation-mcp>=1.3.0; extra == 'all'
Requires-Dist: openinference-instrumentation-openai-agents>=0.1.12; extra == 'all'
Requires-Dist: openinference-instrumentation-openai>=0.1.30; extra == 'all'
Requires-Dist: openinference-instrumentation-smolagents>=0.1.14; extra == 'all'
Requires-Dist: smolagents<1.21.0,>=1.19.0; extra == 'all'
Provides-Extra: anthropic
Requires-Dist: anthropic[bedrock]>=0.52.1; extra == 'anthropic'
Requires-Dist: openinference-instrumentation-anthropic>=0.1.18; extra == 'anthropic'
Provides-Extra: baml
Requires-Dist: baml-py>=0.201.0; extra == 'baml'
Provides-Extra: bedrock
Requires-Dist: boto3>=1.38.17; extra == 'bedrock'
Requires-Dist: openinference-instrumentation-bedrock>=0.1.26; extra == 'bedrock'
Provides-Extra: ci
Requires-Dist: mypy>=1.15.0; extra == 'ci'
Requires-Dist: pytest-asyncio>=0.26.0; extra == 'ci'
Requires-Dist: pytest-httpserver>=1.1.3; extra == 'ci'
Requires-Dist: pytest>=8.3.4; extra == 'ci'
Requires-Dist: ruff>=0.11.7; extra == 'ci'
Provides-Extra: claude-code-sdk
Requires-Dist: claude-code-sdk>=0.0.19; extra == 'claude-code-sdk'
Provides-Extra: crewai
Requires-Dist: crewai>=0.130.0; extra == 'crewai'
Requires-Dist: litellm<1.74.0,>=1.72.0; extra == 'crewai'
Requires-Dist: openinference-instrumentation-crewai>=0.1.10; extra == 'crewai'
Provides-Extra: google-genai
Requires-Dist: google-genai>=1.19.0; extra == 'google-genai'
Requires-Dist: openinference-instrumentation-google-genai==0.1.2; extra == 'google-genai'
Provides-Extra: google-generativeai
Requires-Dist: google-generativeai>=0.7.0; extra == 'google-generativeai'
Requires-Dist: openinference-instrumentation-google-genai==0.1.2; extra == 'google-generativeai'
Provides-Extra: langchain
Requires-Dist: langchain-openai>=0.3.4; extra == 'langchain'
Requires-Dist: langchain>=0.3.18; extra == 'langchain'
Requires-Dist: langgraph>=0.3.20; extra == 'langchain'
Requires-Dist: openinference-instrumentation-langchain==0.1.43; extra == 'langchain'
Provides-Extra: litellm
Requires-Dist: litellm<1.74.0,>=1.72.0; extra == 'litellm'
Provides-Extra: mcp
Requires-Dist: mcp>=1.9.0; extra == 'mcp'
Requires-Dist: openinference-instrumentation-mcp>=1.3.0; extra == 'mcp'
Provides-Extra: openai
Requires-Dist: openinference-instrumentation-openai>=0.1.30; extra == 'openai'
Provides-Extra: openai-agents
Requires-Dist: openai-agents>=0.0.7; extra == 'openai-agents'
Requires-Dist: openinference-instrumentation-openai-agents>=0.1.12; extra == 'openai-agents'
Requires-Dist: openinference-instrumentation-openai>=0.1.30; extra == 'openai-agents'
Provides-Extra: smolagents
Requires-Dist: openinference-instrumentation-smolagents>=0.1.14; extra == 'smolagents'
Requires-Dist: smolagents<1.21.0,>=1.19.0; extra == 'smolagents'
Description-Content-Type: text/markdown

# Atla Insights

Atla Insights is a platform for monitoring and improving AI agents.

<p align="center">
  <a href="https://badge.fury.io/py/atla-insights"><img src="https://badge.fury.io/py/atla-insights.svg" alt="PyPI version"></a>
  <a href="https://github.com/atla-ai/atla-insights-sdk/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-Apache%202.0-blue?style=flat-square" alt="license" /></a>
  <a href="https://app.atla-ai.com"><img src="https://img.shields.io/badge/Atla_Insights_platform-white?logo=data:image/svg%2bxml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBzdGFuZGFsb25lPSJubyI/Pgo8IURPQ1RZUEUgc3ZnIFBVQkxJQyAiLS8vVzNDLy9EVEQgU1ZHIDIwMDEwOTA0Ly9FTiIKICJodHRwOi8vd3d3LnczLm9yZy9UUi8yMDAxL1JFQy1TVkctMjAwMTA5MDQvRFREL3N2ZzEwLmR0ZCI+CjxzdmcgdmVyc2lvbj0iMS4wIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciCiB3aWR0aD0iMjAwLjAwMDAwMHB0IiBoZWlnaHQ9IjIwMC4wMDAwMDBwdCIgdmlld0JveD0iMCAwIDIwMC4wMDAwMDAgMjAwLjAwMDAwMCIKIHByZXNlcnZlQXNwZWN0UmF0aW89InhNaWRZTWlkIG1lZXQiPgoKPGcgdHJhbnNmb3JtPSJ0cmFuc2xhdGUoMC4wMDAwMDAsMjAwLjAwMDAwMCkgc2NhbGUoMC4xMDAwMDAsLTAuMTAwMDAwKSIKZmlsbD0iIzAwMDAwMCIgc3Ryb2tlPSJub25lIj4KPHBhdGggZD0iTTEyODUgMTQ1MCBjLTM2IC0zNyAtNDcgLTU0IC00MCAtNjMgNDAgLTU0IDc4IC0xMTkgOTYgLTE2NyAxNyAtNDYKMjAgLTcyIDE3IC0xNTUgLTMgLTg3IC04IC0xMDkgLTM1IC0xNjggLTQwIC04NCAtMTI1IC0xNzEgLTIwNSAtMjA5IC0xNTIgLTc0Ci0zMjcgLTU2IC00NjIgNDcgbC00OSAzNyAtNDggLTQ4IGMtMjcgLTI3IC00OSAtNTMgLTQ5IC01OSAwIC0xNyA5MCAtODIgMTYyCi0xMTUgMjM1IC0xMTAgNTA0IC01OSA2ODMgMTMxIDE1MCAxNjAgMjAyIDM4NyAxMzQgNTkzIC0yNCA3MyAtMTE1IDIxOSAtMTM5CjIyNCAtOSAxIC0zOCAtMjAgLTY1IC00OHoiLz4KPHBhdGggZD0iTTgxNSAxNDE5IGMtMjYwIC04NCAtMzMwIC00MTQgLTEyOCAtNTk5IDc2IC02OSAxNDQgLTkzIDI1MyAtODggMTQyCjcgMjQxIDcxIDMwMiAxOTYgMzAgNjEgMzMgNzUgMzMgMTU3IC0xIDc3IC01IDk4IC0yOCAxNDUgLTQ2IDk1IC0xMjEgMTYxCi0yMTggMTkxIC03NiAyNCAtMTM3IDIzIC0yMTQgLTJ6Ii8+CjwvZz4KPC9zdmc+Cg==" alt="Atla Insights platform"></a>
  <a href="https://arxiv.org/abs/2501.17195"><img src="https://img.shields.io/badge/ArXiv-Selene_Mini-darkred?logo=arxiv" alt="ArXiv Selene Mini"></a>
  <a href="https://discord.com/invite/qFCMgkGwUK"><img src="https://img.shields.io/badge/Discord-Join_Chat-7289DA.svg?logo=discord" alt="Discord"></a>
  <a href="https://x.com/Atla_AI"><img src="https://img.shields.io/twitter/follow/Atla_AI?style=social" alt="Twitter Follow"></a>
</p>

## Getting started

To get started with Atla Insights, you can either follow the instructions below, or let an agent instrument your code for you.

-   **If you are using Claude Code**: copy the contents of `Claude.md`.
-   **If you are not using Claude Code**: copy the contents of `onboarding.txt` and paste them into your AI agent of choice.

## Installation

```bash
pip install atla-insights
```

To install package-specific dependencies:

```bash
pip install "atla-insights[litellm]"
```

## Usage

### Configuration

Before using Atla Insights, you need to configure it with your authentication token:

```python
from atla_insights import configure

# Run this command at the start of your application.
configure(token="<MY_ATLA_INSIGHTS_TOKEN>")
```

You can retrieve your authentication token from the [Atla Insights platform](https://app.atla-ai.com).

### Environment Configuration

Separate traces between development and production environments:

```python
# Development environment
configure(token="<TOKEN>", environment="dev")

# Production environment (default)
configure(token="<TOKEN>", environment="prod")

# Via environment variable
export ATLA_INSIGHTS_ENVIRONMENT=dev
configure(token="<TOKEN>")  # Uses "dev" from env var
```

### Instrumentation

In order for spans/traces to become available in your Atla Insights dashboard, you will
need to add some form of instrumentation.

As a starting point, you will want to instrument your GenAI library of choice.

See the section below to find out which frameworks & providers we currently support.

All instrumentation methods share a common interface, which allows you to do the following:

-   **Session-wide (un)instrumentation**:
    You can manually enable/disable instrumentation throughout your application.

```python
from atla_insights import configure, instrument_my_framework, uninstrument_my_framework

configure(...)
instrument_my_framework()

# All framework code from this point onwards will be instrumented

uninstrument_my_framework()

# All framework code from this point onwards will **no longer** be instrumented
```

-   **Instrumented contexts**:
    All instrumentation methods also behave as context managers that automatically handle (un)instrumentation.

```python
from atla_insights import configure, instrument_my_framework

configure()

with instrument_my_framework():
    # All framework code inside the context will be instrumented

# All framework code outside the context **not** be instrumented
```

### Instrumentation Support

#### Providers

We currently support the following LLM providers:

| Provider         | Instrumentation Function  | Notes                                                  |
| ---------------- | ------------------------- | ------------------------------------------------------ |
| **Anthropic**    | `instrument_anthropic`    | Also supports `AnthropicBedrock` client from Anthropic |
| **Google GenAI** | `instrument_google_genai` | E.g., Gemini                                           |
| **LiteLLM**      | `instrument_litellm`      | Supports all available models in the LiteLLM framework |
| **OpenAI**       | `instrument_openai`       | Includes Azure OpenAI                                  |
| **Bedrock**      | `instrument_bedrock`      |                                                        |

⚠️ Note that, by default, instrumented LLM calls will be treated independently from one
another. In order to logically group LLM calls into a trace, you will need to group them
as follows:

```python
from atla_insights import configure, instrument, instrument_litellm
from litellm import completion

configure(...)
instrument_litellm()

# The LiteLLM calls below will belong to **separate traces**
result_1 = completion(...)
result_2 = completion(...)

@instrument("My agent doing its thing")
def run_my_agent() -> None:
    # The LiteLLM calls within this function will belong to the **same trace**
    result_1 = completion(...)
    result_2 = completion(...)
    ...
```

#### Frameworks

We currently support the following frameworks:

| Framework           | Instrumentation Function     | Notes                                                                                                       |
| ------------------- | ---------------------------- | ----------------------------------------------------------------------------------------------------------- |
| **Agno**            | `instrument_agno`            | Supported with `openai`, `google-genai`, `litellm` and/or `anthropic` models\*                              |
| **BAML**            | `instrument_baml`            | Supported with `openai`, `anthropic` or `bedrock` models\*                                                  |
| **Claude Code SDK** | `instrument_claude_code_sdk` |                                                                                                             |
| **CrewAI**          | `instrument_crewai`          |                                                                                                             |
| **LangChain**       | `instrument_langchain`       | This includes e.g., LangGraph as well                                                                       |
| **MCP**             | `instrument_mcp`             | Only includes context propagation. You will need to instrument the model calling the MCP server separately. |
| **OpenAI Agents**   | `instrument_openai_agents`   | Supported with `openai`, `google-genai`, `litellm` and/or `anthropic` models\*                              |
| **Smolagents**      | `instrument_smolagents`      | Supported with `openai`, `google-genai`, `litellm` and/or `anthropic` models\*                              |

⚠️ \*Note that some frameworks do not provide their own LLM interface. In these cases, you will
need to instrument both the framework _and_ the underlying LLM provider(s) as follows:

```python
from atla_insights import configure, instrument, instrument_agno

configure(...)

# If you are using a single LLM provider (e.g., via `OpenAIChat`).
instrument_agno("openai")

# If you are using multiple LLM providers (e.g., `OpenAIChat` and `Claude`).
instrument_agno(["anthropic", "openai"])
```

#### Manual instrumentation

It is also possible to manually record LLM generations using the lower-level span SDK.

```python
from atla_insights.span import start_as_current_span

with start_as_current_span("my-llm-generation") as span:
    # Run my LLM generation via an unsupported framework.
    input_messages = [{"role": "user", "content": "What is the capital of France?"}]
    tools = [
        {
            "type": "function",
            "function": {
                "name": "get_capital",
                "parameters": {"type": "object", "properties": {"country": {"type": "string"}}},
            },
        }
    ]
    result = my_client.chat.completions.create(messages=input_messages, tools=tools)

    # Manually record LLM generation.
    span.record_generation(
        input_messages=input_messages,
        output_messages=[choice.message for choice in result.choices],
        tools=tools,
    )

```

Note that the expected data format are OpenAI Chat Completions compatible messages / tools.

### Adding metadata

You can attach metadata to a run that provides additional information about the specs of
that specific workflow. This can include various system settings, prompt versions, etc.

```python
from atla_insights import configure

# We can define some system settings, prompt versions, etc. we'd like to keep track of.
metadata = {
    "environment": "dev",
    "prompt-version": "v1.4",
    "model": "gpt-4o-2024-08-06",
    "run-id": "my-test",
}

# Any subsequent generated traces will inherit the metadata specified here.
configure(
    token="<MY_ATLA_INSIGHTS_TOKEN>",
    metadata=metadata,
)
```

### Tool invocations

If you want to ensure your function-based tool calls are logged correctly, you can wrap
them using the `@tool` decorator as follows:

```python
from atla_insights import tool

@tool
def my_tool(my_arg: str) -> str:
    return "some-output"
```

⚠️ Note that if you are using an instrumented framework, you do **not** need to manually
decorate your tools in this way.

### Sampling

By default, Atla Insights will instrument & log all traces. In high-throughput scenarios,
you may not want to log every trace you produce. In these cases, you can specify a
sampler at configuration time.

-   **Using a built-in sampling method**:

If you want a basic, reliable sampler, you can use one of our pre-built sampling methods.

```python
from atla_insights import configure
from atla_insights.sampling import TraceRatioSamplingOptions

# We want to log 10% of traces
sampling_options = TraceRatioSamplingOptions(rate=0.10)

configure(
    token="<MY_ATLA_INSIGHTS_TOKEN>",
    sampling=sampling_options,
)
```

-   **Using a custom sampling method**:

If you want to implement your own custom sampling method, you can pass in your own
[OpenTelemery Sampler](https://opentelemetry-python.readthedocs.io/en/latest/sdk/trace.sampling.html).

```python
from atla_insights import configure
from opentelemetry.sdk.trace.sampling import Sampler

class MySampler(Sampler):
    ...

my_sampler = MySampler()

configure(
    token="<MY_ATLA_INSIGHTS_TOKEN>",
    sampling=my_sampler,
)
```

⚠️ Note that the Atla Insights platform is **not** intended to work well with partial
traces. Therefore, we highly recommend using either `ParentBased` or `StaticSampler`
samplers. This ensures either all traces are treated the same way or all spans in the
same trace are treated the same way.

```python
from atla_insights import configure
from opentelemetry.sdk.trace.sampling import ParentBased, Sampler

class MySampler(Sampler):
    ...

my_sampler = ParentBased(root=MySampler())

configure(
    token="<MY_ATLA_INSIGHTS_TOKEN>",
    sampling=my_sampler,
)
```

### Adding custom metrics

You can add custom evaluation metrics to your trace.

```python
from atla_insights import instrument, set_custom_metrics

@instrument()
def my_function():
    # Some GenAI logic here
    eval_result = False
    set_custom_metrics({"my_metric": {"data_type": "boolean", "value": eval_result}})
```

The permitted `data_type` fields are:

-   `likert_1_to_5`: a numeric 1-5 scale. In this case, `value` is expected to be an `int` between 1 and 5.
-   `boolean`: a boolean scale. In this case, `value` is expected to be a `bool`.

The primary intended use case is logging custom code evals that benefit from being in the
active runtime environment. You can, however, log any arbitrary metric - including custom
LLMJ eval results.

### Marking trace success / failure

The logical notion of _success_ or _failure_ plays a prominent role in the observability
of (agentic) GenAI applications.

Therefore, the `atla_insights` package offers the functionality to mark a trace as a
success or a failure like follows:

```python
from atla_insights import (
    configure,
    instrument,
    instrument_openai,
    mark_failure,
    mark_success,
)
from openai import OpenAI

configure(...)
instrument_openai()

client = OpenAI()

@instrument("My agent doing its thing")
def run_my_agent() -> None:
    result = client.chat.completions.create(
        model=...,
        messages=[
            {
                "role": "user",
                "content": "What is 1 + 2? Reply with only the answer, nothing else.",
            }
        ]
    )
    response = result.choices[0].message.content

    # Note that you could have any arbitrary success condition, including LLMJ-based evaluations
    if response == "3":
        mark_success()
    else:
        mark_failure()
```

⚠️ Note that you should use this marking functionality within an instrumented function.

### Compatibility with existing observability

As `atla_insights` provides its own instrumentation, we should note potential interactions
with our instrumentation / observability providers.

`atla_insights` instrumentation is generally compatible with most popular observability
platforms.

E.g., the following code snippet will make tracing available in both Atla and Langfuse.

```python
from atla_insights import configure, instrument_openai
from langfuse.openai import OpenAI

configure(...)

instrument_openai()

client = OpenAI()
client.chat.completions.create(...)
```

#### OpenTelemetry compatibility

The Atla Insights SDK is built on the OpenTelemetry standard and fully compatible with
other OpenTelemetry services.

If you have an existing OpenTelemetry setup (e.g., by setting the relevant otel
environment variables), Atla Insights will be _additive_ to this setup. I.e., it will add
additional logging on top of what is already getting logged.

If you do not have an existing OpenTelemetry setup, Atla Insights will initialize a new
(global) tracer provider.

Next to the above, you also have the ability to add any arbitrary additional span
processors by following the example below:

```python
from atla_insights import configure
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace.export import SimpleSpanProcessor

# This is the otel traces endpoint for my provider of choice.
my_otel_endpoint = "https://my-otel-provider/v1/traces"

my_span_exporter = OTLPSpanExporter(endpoint=my_otel_endpoint)
my_span_processor = SimpleSpanProcessor(my_span_exporter)

configure(
    token="<MY_ATLA_INSIGHTS_TOKEN>",
    # This will ensure traces get sent to my otel provider of choice
    additional_span_processors=[my_span_processor],
)
```

### More examples

More specific examples can be found in the `examples/` folder.

## Data API

The Atla Insights SDK includes a data API client for programmatically accessing your traces and analytics data.

### Usage

```python
from atla_insights.client import Client

# Initialize the client with your API key
client = Client(api_key="your_api_key_here")

# List traces with optional filters
traces = client.list_traces(
    page_size=50,
    metadata_filter=[{"key": "environment", "value": "prod"}]
)

# Get detailed trace information
trace_details = client.get_trace("trace_id_123")

# Bulk retrieve multiple traces
traces = client.get_traces(["trace_1", "trace_2", "trace_3"])
```

### Available Methods

- `list_traces()` - Retrieve paginated list of traces with optional filtering
- `get_trace(trace_id)` - Get detailed information for a specific trace
- `get_traces(trace_ids)` - Bulk retrieve multiple traces by ID

The client returns structured data objects with full type hints for easy integration into your workflows.

See the `examples/` directory for additional usage examples and integration patterns.
