Metadata-Version: 2.3
Name: agent-substrate
Version: 0.1.0
Summary: Production-ready async Python framework for building autonomous AI agents with tool calling, memory, and durable execution.
Keywords: ai,agents,llm,multi-agent,autonomous,fastapi,async
Author: Ravikumar Chavva
Author-email: Ravikumar Chavva <chavvaravikumarreddy2004@gmail.com>
License: MIT
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Framework :: AsyncIO
Classifier: Typing :: Typed
Requires-Dist: httpx>=0.27.0
Requires-Dist: mcp>=1.0.0
Requires-Dist: openai>=2.15.0
Requires-Dist: opentelemetry-api>=1.20.0
Requires-Dist: opentelemetry-sdk>=1.20.0
Requires-Dist: pydantic>=2.12.5
Requires-Dist: pydantic-settings>=2.12.0
Requires-Dist: python-json-logger>=2.0.7
Requires-Dist: tiktoken>=0.12.0
Requires-Dist: fastapi>=0.128.0
Requires-Dist: redis[hiredis]>=5.0.0
Requires-Dist: sqlalchemy>=2.0.46
Requires-Dist: asyncpg>=0.30.0
Requires-Dist: psycopg[binary]>=3.2.0
Requires-Dist: uvicorn[standard]>=0.34.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: python-multipart>=0.0.9
Requires-Dist: rich>=14.3.2
Requires-Dist: pyjwt>=2.12.0
Requires-Dist: cryptography>=46.0.6
Requires-Dist: apscheduler>=4.0.0a5
Requires-Dist: anthropic>=0.95.0
Requires-Dist: google-genai>=1.73.0
Requires-Dist: msgspec>=0.21.1
Requires-Dist: pillow>=10.4.0
Requires-Dist: opentelemetry-exporter-otlp
Requires-Dist: opentelemetry-instrumentation-fastapi<0.60b1
Requires-Dist: httpx2>=2.7.0
Requires-Dist: confusable-homoglyphs>=3.3.1
Requires-Dist: onnxruntime>=1.20.0
Requires-Dist: tokenizers>=0.20.0
Requires-Dist: huggingface-hub>=0.25.0
Requires-Dist: k8s-agent-sandbox>=0.4.6 ; extra == 'code'
Requires-Dist: doc-firewall>=0.5.1 ; extra == 'extraction'
Requires-Dist: paddleocr[doc-parser]>=3.7.0 ; extra == 'extraction'
Requires-Dist: paddlepaddle==3.3.0 ; extra == 'extraction'
Requires-Dist: pypdf>=6.13.0 ; extra == 'rag'
Requires-Dist: pinecone>=9.0.0 ; extra == 'rag-pinecone'
Requires-Dist: aiobotocore>=3.7.0 ; extra == 's3'
Requires-Dist: botocore>=1.43.0 ; extra == 's3'
Requires-Dist: pandas>=2.2.0 ; extra == 'sandbox'
Requires-Dist: matplotlib>=3.9.0 ; extra == 'sandbox'
Requires-Dist: openpyxl>=3.1.0 ; extra == 'sandbox'
Requires-Dist: python-docx>=1.1.0 ; extra == 'sandbox'
Requires-Dist: python-pptx>=1.0.0 ; extra == 'sandbox'
Requires-Dist: reportlab>=4.2.0 ; extra == 'sandbox'
Requires-Dist: polars>=1.0.0 ; extra == 'sandbox'
Requires-Dist: seaborn>=0.13.0 ; extra == 'sandbox'
Requires-Dist: plotly>=5.24.0 ; extra == 'sandbox'
Requires-Dist: scikit-learn>=1.5.0 ; extra == 'sandbox'
Requires-Dist: pdfplumber>=0.11.0 ; extra == 'sandbox'
Requires-Dist: agent-substrate[web,code,rag,s3,sandbox] ; extra == 'server'
Requires-Dist: ddgs>=9.14.4 ; extra == 'web'
Requires-Dist: crawl4ai>=0.8.7 ; extra == 'web'
Requires-Dist: playwright>=1.60.0 ; extra == 'web'
Requires-Dist: exa-py>=2.14.0 ; extra == 'web'
Requires-Dist: tavily-python>=0.7.26 ; extra == 'web'
Requires-Python: >=3.11
Project-URL: Homepage, https://docs.agent-substrate.com
Project-URL: Documentation, https://docs.agent-substrate.com
Project-URL: Repository, https://github.com/Ravikumarchavva/agent-substrate
Project-URL: Issues, https://github.com/Ravikumarchavva/agent-substrate/issues
Provides-Extra: code
Provides-Extra: extraction
Provides-Extra: rag
Provides-Extra: rag-pinecone
Provides-Extra: s3
Provides-Extra: sandbox
Provides-Extra: server
Provides-Extra: web
Description-Content-Type: text/markdown

<center><h1>Agent Substrate</h1></center>

**A production-ready, protocol-oriented Python framework for building robust, observable, and composable autonomous AI agents and multi-agent workflows.**

[![Python 3.13+](https://img.shields.io/badge/python-3.13+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Docs](https://img.shields.io/badge/docs-docs.agent-substrate.com-teal)](https://docs.agent-substrate.com)

---

## 🚀 Features

*   **🤖 ReAct Agent Loop**: Production-grade Reasoning + Action loop with HITL gates, supervision budgets, and priority preemption.
*   **🔧 Safe Tool Execution**: JSON-schema-validated tools, risk-tiered approval gating, sandboxed code-mode chaining, and MCP integration.
*   **💾 Pluggable Memory**: `CachedHistoryProvider` (fast Redis cache, self-heals from durable storage on a cold session) is the production default; in-memory, Redis-only, and Postgres providers are also available standalone. Sliding-window, token-budget, and summarization compaction strategies included.
*   **🎯 Multi-Provider LLM**: OpenAI, Anthropic, Gemini, Groq, Ollama — auto-detected from model name prefix via `LLMFactory`.
*   **📊 Guardrails & Middleware**: Async tripwire pipeline evaluating inputs, outputs, and tool calls with mutation policies.
*   **🕷️ Composable Flows**: `SequentialFlow`, `ParallelFlow`, and `ConditionalFlow` nest recursively in `fabric/`.
*   **📡 Durable Execution**: Postgres-backed event log + inbox + scheduler for at-most-once delivery and crash recovery.
*   **📊 Observability**: OpenTelemetry traces, structured logging, lifecycle hooks, and a Grafana dashboard out of the box.

---

## 📋 Table of Contents

*   [Quick Start](#-quick-start)
*   [Core Architecture](#-core-architecture)
*   [Key Patterns](#-key-patterns)
*   [Multi-Agent Workflows](#-multi-agent-workflows)
*   [Installation & Setup](#-installation--setup)
*   [Testing](#-testing)
*   [Documentation](#-documentation)

---

## ⚡ Quick Start

### Installation

```bash
# Clone the repository
git clone https://github.com/Ravikumarchavva/agent-substrate
cd agent-substrate

# Sync dependencies
uv sync

# Start infrastructure (Postgres, Redis, MinIO, observability)
make infra-up
```

### Your First Agent

`Runtime.run(agent, prompt)` is the one-shot entry point: it registers the
agent, submits the prompt, and returns a `RunOutcome` once the agent's final
answer is available.

```python
import asyncio
from substrate.agents import ReActAgent, Runtime
from substrate.integrations.llm import LLMFactory

async def main():
    llm = LLMFactory("gpt-4o", api_key="sk-...").build()

    agent = ReActAgent(
        "assistant",
        model=llm,
        system_instructions="You are a helpful assistant.",
    )

    async with Runtime() as runtime:
        result = await runtime.run(agent, "Write a Python function to compute Fibonacci numbers.")
        print(result.output)

if __name__ == "__main__":
    asyncio.run(main())
```

### Agent with Tools

```python
import asyncio
from substrate.agents import ReActAgent, Runtime
from substrate.capabilities.tools.compute.calculator import CalculatorTool
from substrate.integrations.llm import LLMFactory

async def main():
    llm = LLMFactory("gpt-4o", api_key="sk-...").build()

    agent = ReActAgent(
        "math_expert",
        model=llm,
        tools=[CalculatorTool()],
        system_instructions="Always use the calculator tool to solve math problems.",
    )

    async with Runtime() as runtime:
        result = await runtime.run(agent, "Calculate 1234 * 5678.")
        print(result.output)

if __name__ == "__main__":
    asyncio.run(main())
```

`CalculatorTool` evaluates arithmetic via a whitelisted AST walk — no
`eval()` — so LLM-controlled input can never reach arbitrary code. Writing
your own tool that evaluates expressions? Reuse `substrate.capabilities.
tools.compute.calculator.safe_eval` rather than calling `eval()` yourself.

---

## 🏛️ Core Architecture

Agent Substrate is partitioned into **four strict dependency layers**. Imports flow strictly downward — lower layers never depend on higher ones:

```
fabric (L3)        ← Flows (Sequential/Parallel/Conditional), Evals, durable execution
  capabilities (L2)  ← Tools, Skills, Knowledge/RAG, Memory, Vector/Graph stores, Triggers
    agents (L1)      ← ReActAgent, OrchestratorAgent, Runtime, Middleware, Guardrails
      kernel (L0)    ← FROZEN. Protocols, ContentBlock types, AgentId, Tool contracts
```

**Orthogonal layers** (implement kernel Protocols, cross-cut all layers):

| Layer | Responsibility |
|---|---|
| `integrations/` | Third-party adapters: LLM providers, MCP, event bus, connectors |
| `infrastructure/` | Engine backends: Postgres, Redis, MinIO, durable runtime |
| `serving/` | Deployment shells: monolith FastAPI app + 12 microservices |

Import-linter enforces the layer contract on every CI run (`uv run lint-imports`).

**[→ Capability Map](docs/capability-map.md)** — the platform organized by concern: context (the RAM tier), memory (short-term + long-term with pluggable backends), storage, guardrails, governance, evals, observability, and tools. Every item names a real, shipped class.

**[→ Kernel Board](docs/kernel-board.html)** — the contract-level view: every kernel protocol drawn as a socket, traced to the real implementations that plug into it.

**[→ Agent Builder](docs/agent-builder.html)** — pick a memory backend, tools, guardrails, and budgets; get real, accurate `ReActAgent` construction code back, generated from the actual constructor signatures.

---

## 🔑 Key Patterns

### Adding a Tool

Drop a file at `src/substrate/capabilities/tools/<name>/tool.py` — `CatalogScanner` discovers it automatically, no registration needed:

```python
from substrate.kernel.tools import ToolExecutionResult
from substrate.kernel.core.content import TextBlock

class MyTool:
    name = "my_tool"
    description = "What it does"
    input_schema = {"type": "object", "properties": {...}, "required": [...]}

    async def execute(self, *, ctx=None, **kwargs) -> ToolExecutionResult:
        return ToolExecutionResult(content=[TextBlock(text="result")])
```

### LLM Client

```python
from substrate.integrations.llm import LLMFactory

# Provider auto-detected from model name prefix
client = LLMFactory("gpt-4o", api_key).build()
client = LLMFactory("claude-opus-4-8", api_key).build()
client = LLMFactory("groq/llama-3.3-70b-versatile", api_key).build()
client = LLMFactory("ollama/llama3.2", "ollama").build()   # local, no key
```

### MCP Tools

```python
from substrate.integrations.tools.mcp import MCPClient, MCPTool

client = MCPClient(url="http://localhost:9000/sse")
tools = await MCPTool.from_mcp_client(client)   # list[MCPTool]
```

### Knowledge / RAG

```python
from substrate.capabilities.vector import PgVectorStore
from substrate.capabilities.knowledge import RAGPipeline

pipeline = RAGPipeline(embedding_client=embed_client, vector_store=PgVectorStore(...))
await pipeline.ingest("Long document …", collection="kb")
results = await pipeline.query("What is X?", collection="kb")
```

---

## 🕸️ Multi-Agent Workflows

### OrchestratorAgent — Hub & Spoke

`OrchestratorAgent` delegates to sub-agents via an LLM-driven tool call, and
also works with `Runtime.run()` — its final synthesized answer streams
through the same mechanism as `ReActAgent`'s.

```python
from substrate.agents import OrchestratorAgent, SubAgentConfig, ReActAgent

researcher = ReActAgent("researcher", model=llm, system_instructions="Research the web.")
writer = ReActAgent("writer", model=llm, system_instructions="Write content.")

orchestrator = OrchestratorAgent(
    "coordinator",
    model=llm,
    sub_agents=[
        SubAgentConfig(agent=researcher, description="Web research"),
        SubAgentConfig(agent=writer, description="Content writing"),
    ],
)

async with Runtime() as runtime:
    result = await runtime.run(orchestrator, "Research and draft a blog post about Rust vs Go.")
    print(result.output)
```

### Flows — Coordination Primitives

`SequentialFlow`, `ParallelFlow`, and `ConditionalFlow` (in `fabric/flows/`)
are `Agent`-shaped coordinators: instead of streaming text via an LLM call,
they reply to their caller via `ctx.reply()` — the same mechanism any agent
uses to answer an `ctx.ask()`. Use **`Runtime.ask()`**, not `Runtime.run()`,
to invoke one directly and read its result — register each step with the
`Runtime` first:

```python
from substrate.agents.runtime import Runtime
from substrate.fabric.flows import SequentialFlow
from substrate.kernel.core.identity import AgentId

class FetchStep:
    id = AgentId(type="step", key="fetch")
    async def run(self, ctx, inbox):
        for msg in inbox:
            await ctx.reply(msg, {"text": "Fetched 3 records."})

class AnalyzeStep:
    id = AgentId(type="step", key="analyze")
    async def run(self, ctx, inbox):
        for msg in inbox:
            await ctx.reply(msg, {"text": "Analysis: all records valid."})

async def main():
    fetch, analyze = FetchStep(), AnalyzeStep()
    pipeline = SequentialFlow(steps=[fetch, analyze], name="demo_pipeline")

    async with Runtime() as runtime:
        await runtime.register(fetch)
        await runtime.register(analyze)
        result = await runtime.ask(pipeline, "Process the latest dataset.")
        print(result.output)
        # Process the latest dataset.
        #
        # Fetched 3 records.
        #
        # Analysis: all records valid.
```

`SequentialFlow`'s reply is the **full accumulated trace** (input + every
step's output, joined by blank lines) — not just the last step's output.
`ParallelFlow` (`branches=[...]`, `merge="concat"|"vote"|callable`) and
`ConditionalFlow` (`predicate`, `if_true`, `if_false`) follow the same
`Runtime.ask()` pattern.

---

## 🔧 Installation & Setup

### Environment Variables (`.env`)

```bash
# LLM providers (set at least one)
OPENAI_API_KEY=sk-proj-...
ANTHROPIC_API_KEY=sk-ant-...

# Database
DATABASE_URL=postgresql+asyncpg://postgres:postgres@localhost:5432/agentdb

# Redis
REDIS_URL=redis://localhost:6379/0

# Auth (required)
JWT_SECRET=<32+ char random string>

# Observability
OTLP_ENDPOINT=http://localhost:4318
```

The monolith server (`uv run start` / `substrate start`) listens on port
**8000** by default.

---

## 🧪 Testing

```bash
# Run full test suite
uv run pytest

# Single file
uv run pytest tests/test_foo.py

# Architecture + import-linter checks
uv run lint-imports

# Full CI preflight (lint → typecheck → test → security)
make ci
```

---

## 📖 Documentation

Full architecture reference, layer guides, and API docs at **[agent-substrate.pages.dev](https://docs.agent-substrate.com)**.

---

**Built with ❤️ for the AI agent engineering community**
