Metadata-Version: 2.4
Name: rail-engine-ingest
Version: 0.5.1
Summary: Python SDK for Railtown AI Railengine - Ingestion
Author-email: Railtown AI <support@railtown.ai>
License: MIT
Project-URL: Homepage, https://railengine.ai
Keywords: railtown,rail-engine,ingestion,sdk
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: certifi>=2026.10.19
Requires-Dist: httpx>=0.24.0
Requires-Dist: pydantic>=1.10.0
Dynamic: license-file

# Railengine Ingestion SDK

Python SDK for ingesting data into Railtown AI Railengine - handles embeddings, storage, indexing, and webhook publishing.

## Overview

The `rail-engine-ingest` package provides a Pythonic interface for publishing data to Railengine. It supports async/await patterns, Pydantic model validation, and familiar configuration patterns.

## Installation

```bash
uv pip install rail-engine-ingest
```

## Quick Start

```python

# Copy your ENGINE_TOKEN from https://cndr.railtown.ai Project Settings

import asyncio
import os
from railtown.engine.ingest import RailengineIngest
import base64
import json

engine_token = os.getenv("ENGINE_TOKEN")

async def main():
    # Initialize client
    async with RailengineIngest(engine_token=engine_token) as client:
        # Ingest data
        data = {
            "name": "Chicken Shawarma",
            "meal": "lunch",
            "calories": 95
        }
        response = await client.upsert(data)
        print(f"Status: {response.status_code}")

asyncio.run(main())
```

## Configuration

### Environment Variable

- `ENGINE_TOKEN` - Base64-encoded JSON string containing ingestion credentials

### Constructor Parameters

- `engine_token` (optional) - ENGINE_TOKEN string (if not provided, reads from env)
- `model` (optional) - Pydantic model type for validating ingested data
- `timeout` (optional) - Request timeout in seconds or `httpx.Timeout` (default `30`)
- `verify` (optional) - TLS verification. Defaults to `True`. Pass a CA bundle path or `ssl.SSLContext` for custom trust. `False` disables verification and must be set explicitly.
- `http_client` (optional) - Caller-owned `httpx.AsyncClient` for proxies, custom transports, or tests. The SDK does not close this client.
- `logging_policy` (optional) - `LoggingPolicy` for SDK logs. Default INFO logs include engine id, status, duration, and payload size. The request body is not logged unless `log_content=True`.

Earlier SDK versions disabled TLS verification. 0.4.0 verifies certificates by default. Use `verify=` with your CA bundle if the ingestion host uses a private certificate.

SDK loggers are named (`railtown.engine.*`). Default INFO logging does not write the upsert body. Pass `logging_policy=LoggingPolicy(log_content=True)` only when you explicitly want redacted, truncated payload logs.

## Features

- **Single `upsert()` method** - Unified interface for ingesting data
- **Multiple data formats** - Accepts Pydantic models, dictionaries, or JSON strings
- **Model validation** - Optional Pydantic model validation
- **Direct JSON payload** - Sends data directly as JSON to the ingestion endpoint
- **Webhook handler** - `get_webhook_handler()` returns a payload parser when the client is constructed with `model=`
- **Async/await support** - Built for modern async Python applications

## Error Handling

The SDK provides custom exception classes:

- `RailtownError` - Base exception
- `RailtownBadRequestError` - 400 Bad Request
- `RailtownUnauthorizedError` - 401 Unauthorized
- `RailtownNotFoundError` - 404 Not Found
- `RailtownConflictError` - 409 Conflict
- `RailtownServerError` - 5xx Server errors

Exceptions are raised on errors. In 0.5.0, ingestion shares the retrieval error classification: HTTP 403 and 429 use `RailtownForbiddenError` and `RailtownRateLimitError`, transport failures use `RailtownTransportError`, and timeouts use its subclass `RailtownTimeoutError`. HTTP 404 and 409 use `RailtownNotFoundError` and `RailtownConflictError`. These types are exported from `railtown.engine.ingest`; all inherit from `RailtownError`.

Request errors expose `status_code` when available, `operation`, `engine_id`, `request_id` when available, and `retryable`. Retryable is true for timeouts, connection failures, and HTTP 408, 429, 502, 503, and 504. It does not mean replaying an ingestion request is safe, and no automatic retries are performed.

**Migration to 0.5.0:** SDK-generated errors leave `response_text` as `None`; existing exception constructors still accept explicitly supplied values. SDK-generated messages exclude request/response payloads, but the preserved original cause can retain sensitive data. Full traceback logging is not guaranteed to be sanitized. Ingestion always raises on request failures, independent of the retrieval compatibility mode. User-model validation behavior is unchanged.

## Requirements

- uv
- Python 3.11+
- httpx >= 0.24.0
- pydantic >= 1.10.0

## Related package

For retrieving and searching data from Railengine, install and use [`rail-engine`](https://pypi.org/project/rail-engine/).

## License

MIT
