Metadata-Version: 2.4
Name: rail-engine
Version: 0.5.0
Summary: Python SDK for Railtown AI Railengine - Retrieval
Author-email: Railtown AI <support@railtown.ai>
License: MIT
Project-URL: Homepage, https://railengine.ai
Keywords: railtown,rail-engine,retrieval,sdk
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: certifi>=2026.10.19
Requires-Dist: httpx>=0.24.0
Requires-Dist: pydantic>=1.10.0
Dynamic: license-file

# Railengine Retrieval SDK

Python SDK for retrieving and searching data from Railtown AI Railengine - handles embeddings, storage documents, and indexed content.

## Overview

The `rail-engine` package provides a Pythonic interface for retrieving and searching data from Railengine. It supports async/await patterns, client-side filtering, explicit storage list paging via `StorageListPage`, and Pydantic model deserialization.

## Installation

```bash
uv pip install rail-engine
```

## Quick Start

```python
import asyncio
from railtown.engine import Railengine

async def main():
    # Initialize client (reads from ENGINE_PAT and ENGINE_ID env vars)
    async with Railengine() as client:
        page = await client.list_storage_documents(
            page_number=1,
            page_size=25,
        )
        print(f"total_count={page.total_count} total_pages={page.total_pages}")
        for doc in page.items:
            print(doc)

asyncio.run(main())
```

## Configuration

### Environment Variables

- `ENGINE_PAT` (required) - Personal Access Token
- `ENGINE_ID` (required) - Engine ID (can also be passed to constructor)
- `RAILTOWN_API_URL` (optional) - Base API URL (defaults to `https://cndr.railtown.ai/api`)

### Constructor Parameters

- `pat` (optional) - PAT token (if not provided, reads from `ENGINE_PAT` env)
- `engine_id` (optional) - Engine ID (if not provided, reads from `ENGINE_ID` env, required if not in env)
- `api_url` (optional) - Base API URL (if not provided, reads from `RAILTOWN_API_URL` env or defaults to production)
- `model` (optional) - Pydantic model type for deserializing retrieved data
- `timeout` (optional) - Request timeout in seconds or `httpx.Timeout` (default `30`). Applied to the SDK-created client and to each request, including vector search.
- `verify` (optional) - TLS verification. Defaults to `True`. Pass a CA bundle path or `ssl.SSLContext` for custom trust. `False` disables verification and must be set explicitly.
- `http_client` (optional) - Caller-owned `httpx.AsyncClient` for proxies, custom transports, or tests. The SDK does not close this client.
- `retrieval_error_mode` (optional) - `"raise"` (default) or `"fallback"`; see the error contract below.
- `logging_policy` (optional) - `LoggingPolicy` for SDK logs. Default logs operation metadata only. Search text, JSONPath, and response bodies are not logged unless `log_content=True`.

Earlier SDK versions disabled TLS verification. 0.4.0 verifies certificates by default. Use `verify=` with your CA bundle if the API host uses a private certificate.

## Features

- **Multiple retrieval methods**:
  - `search_vector_store()` - Semantic search in vector stores
  - `search_vector_store_filtered()` - Semantic search restricted to the documents an index query matches (`search_query` takes the same `IndexingSearchQuery`/dict as `search_index`; requires Indexing and Embeddings enabled on the engine)
  - `get_storage_document_by_event_id()` - Get document by EventId
  - `get_storage_document_by_customer_key()` - Same paging as `list_storage_documents` (`page_number`, `page_size`) but scoped to a CustomerKey; returns `StorageListPage` with `items`, `total_pages`, and `total_count`
  - `query_storage_by_jsonpath()` - Query using JSONPath
  - `list_storage_documents()` - Use `page_number` and `page_size` to paginate; returns `StorageListPage` with `items`, `total_pages`, and `total_count`
  - `search_index()` - Index search (default `raw=True`: each hit is the API dict; use `raw=False` to parse each hit's `body` with `model` or the client default); returns `IndexingSearchResult` with `items` and optional `total_count`
  - `delete_event()` - Delete an event from storage, embeddings, and indexing (raises on API errors)
- **Client-side filtering on storage reads** - Optional `filter_fn` on storage/list/query helpers (not on `search_vector_store` or `search_vector_store_filtered`; filter vector hits in your loop)
- **Storage list pagination** - One HTTP GET per page; use `page_number` / `page_size` on the request and `total_pages` / `total_count` on `StorageListPage` when looping
- **Model deserialization** - Optional Pydantic model support with per-call override
- **Actionable error handling** - Retrieval failures raise by default; explicit fallback mode supports legacy integrations
- **Async/await support** - Built for modern async Python applications

## Error Handling

The SDK provides custom exception classes:

- `RailtownError` - Base exception
- `RailtownBadRequestError` - 400 Bad Request
- `RailtownUnauthorizedError` - 401 Unauthorized
- `RailtownNotFoundError` - 404 Not Found
- `RailtownConflictError` - 409 Conflict
- `RailtownServerError` - 5xx Server errors

### Error contract and migration to 0.5.0

Version 0.5.0 changes retrieval failures to **raise by default**. Successful empty queries keep their existing result types and pagination metadata. To retain the previous retrieval fallback behavior, explicitly construct `Railengine(retrieval_error_mode="fallback")`. Fallback logs the failure and returns an empty list, empty result/page, or `None` for an event lookup. It does not suppress ingestion or deletion failures, invalid arguments, cancellation, or application callback errors. User-model validation and per-record deserialization fallback are unchanged.

A 404 raises `RailtownNotFoundError` by default, including event lookups. The current API contract does not establish a reliable discriminator between a missing document and another missing resource; the SDK does not infer which one is absent or perform another request to guess.

Catch the common base class for all request/response errors:

```python
from railtown.engine import Railengine, RailtownError

async def search():
    async with Railengine() as client:
        try:
            return await client.search_index(query={"search": "example"})
        except RailtownError as error:
            print(error.operation, error.status_code, error.request_id)
            raise
```

HTTP 400, 401, 403, 404, 409, and 429 map to `RailtownBadRequestError`, `RailtownUnauthorizedError`, `RailtownForbiddenError`, `RailtownNotFoundError`, `RailtownConflictError`, and `RailtownRateLimitError`. Other 4xx responses use `RailtownError`; 5xx responses use `RailtownServerError`. Transport failures use `RailtownTransportError`, timeouts its subclass `RailtownTimeoutError`, and invalid response JSON/envelopes use `RailtownResponseError`. For retrieval (`rail-engine`), import these exceptions from `railtown.engine`. For ingestion (`rail-engine-ingest`), import them from `railtown.engine.ingest`. The `railtown.engine.ingest` namespace requires `rail-engine-ingest`; it is not included in the retrieval wheel.

Request/response errors expose `status_code` when available, `operation`, the effective `engine_id`, `request_id` when available, and `retryable`. Retryable is true for timeouts, connection failures, and HTTP 408, 429, 502, 503, and 504. It describes a potentially transient failure, not whether replaying a write is safe. The SDK does not retry automatically.

**Diagnostic compatibility change:** SDK-generated errors now leave `response_text` as `None`. The constructor still accepts an explicitly supplied value. Messages exclude query strings and payloads, but the preserved original exception cause can retain sensitive data. Full traceback logging is not guaranteed to be sanitized. Explicit content logging remains governed by `LoggingPolicy` and does not populate error attributes.

## Requirements

- uv
- Python 3.11+
- httpx >= 0.24.0
- pydantic >= 1.10.0

## Related package

For ingesting data into Railengine, install and use [`rail-engine-ingest`](https://pypi.org/project/rail-engine-ingest/).

## License

MIT
