Metadata-Version: 2.4
Name: cluebase-backend-sdk
Version: 0.0.11
Requires-Dist: opentelemetry-api>=1.41.1,<2.0
Requires-Dist: opentelemetry-sdk>=1.41.1,<2.0
Requires-Dist: rfc8785>=0.1.4,<0.2
Requires-Dist: opentelemetry-instrumentation-asgi>=0.63b1,<1.0 ; extra == 'instrumentation-asgi'
Requires-Dist: opentelemetry-instrumentation-celery>=0.63b1,<1.0 ; extra == 'instrumentation-celery'
Requires-Dist: opentelemetry-instrumentation-django>=0.63b1,<1.0 ; extra == 'instrumentation-django'
Requires-Dist: fastapi>=0.100,<1.0 ; extra == 'instrumentation-fastapi'
Requires-Dist: opentelemetry-instrumentation-fastapi>=0.63b1,<1.0 ; extra == 'instrumentation-fastapi'
Requires-Dist: opentelemetry-instrumentation-httpx>=0.63b1,<1.0 ; extra == 'instrumentation-httpx'
Requires-Dist: opentelemetry-instrumentation-requests>=0.63b1,<1.0 ; extra == 'instrumentation-requests'
Requires-Dist: opentelemetry-instrumentation-sqlalchemy>=0.63b1,<1.0 ; extra == 'instrumentation-sqlalchemy'
Requires-Dist: mutmut>=3.2,<4.0 ; extra == 'mutation'
Requires-Dist: pytest>=8.0,<9.0 ; extra == 'test'
Requires-Dist: pytest-cov>=5.0,<7.0 ; extra == 'test'
Requires-Dist: pyright>=1.1.411,<1.2 ; extra == 'test'
Requires-Dist: build>=1.2,<2.0 ; extra == 'test'
Requires-Dist: packaging>=25,<26 ; extra == 'test'
Requires-Dist: twine>=6.1,<7.0 ; extra == 'test'
Requires-Dist: opentelemetry-instrumentation-requests>=0.63b1,<1.0 ; extra == 'test'
Requires-Dist: opentelemetry-instrumentation-httpx>=0.63b1,<1.0 ; extra == 'test'
Requires-Dist: opentelemetry-instrumentation-fastapi>=0.63b1,<1.0 ; extra == 'test'
Requires-Dist: opentelemetry-instrumentation-asgi>=0.63b1,<1.0 ; extra == 'test'
Requires-Dist: opentelemetry-instrumentation-sqlalchemy>=0.63b1,<1.0 ; extra == 'test'
Requires-Dist: opentelemetry-instrumentation-django>=0.63b1,<1.0 ; extra == 'test'
Requires-Dist: opentelemetry-instrumentation-celery>=0.63b1,<1.0 ; extra == 'test'
Requires-Dist: requests>=2.31 ; extra == 'test'
Requires-Dist: httpx>=0.26 ; extra == 'test'
Requires-Dist: sqlalchemy>=2.0 ; extra == 'test'
Requires-Dist: django>=5.0,<6.0 ; extra == 'test'
Requires-Dist: fastapi>=0.100,<1.0 ; extra == 'test'
Requires-Dist: celery>=5.0,<6.0 ; extra == 'test'
Requires-Dist: langchain-core>=0.3,<1.0 ; extra == 'test'
Requires-Dist: llama-index-core>=0.12,<1.0 ; extra == 'test'
Requires-Dist: crewai>=1.15.1,<1.16 ; extra == 'test'
Requires-Dist: openai==2.45.0 ; extra == 'test'
Requires-Dist: anthropic==0.116.0 ; extra == 'test'
Requires-Dist: opentelemetry-instrumentation-openai==0.62.1 ; extra == 'test'
Requires-Dist: opentelemetry-instrumentation-anthropic==0.62.1 ; extra == 'test'
Requires-Dist: aiosqlite==0.21.0 ; extra == 'test'
Requires-Dist: setuptools>=68,<83 ; extra == 'test'
Provides-Extra: instrumentation-asgi
Provides-Extra: instrumentation-celery
Provides-Extra: instrumentation-django
Provides-Extra: instrumentation-fastapi
Provides-Extra: instrumentation-httpx
Provides-Extra: instrumentation-requests
Provides-Extra: instrumentation-sqlalchemy
Provides-Extra: mutation
Provides-Extra: test
Summary: Cluebase Backend SDK (Python) — OTel-based observation-source event capture for Django, FastAPI, Celery, SQLAlchemy, LangChain, LlamaIndex, CrewAI, MCP, and AI providers.
Requires-Python: >=3.10
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM

# cluebase-backend-sdk

Shared Python backend core for Cluebase SDK wrappers (Django, FastAPI,
Celery). Events flow through an OTel `TracerProvider` + `ConsentSpanProcessor` +
`ObservationSourceEventExporter` and the shared delivery queue to
`/api/v1/ingest/backend`.

## Install

The base install carries only the OTel API and SDK. The packages that observe
a framework, an outbound HTTP client, an ORM or a task queue are extras, so
install the ones the application actually uses:

```
pip install 'cluebase-backend-sdk[instrumentation-fastapi,instrumentation-httpx,instrumentation-sqlalchemy]'
```

| Extra | Observes |
| --- | --- |
| `instrumentation-fastapi` | FastAPI requests (pulls `fastapi` as well) |
| `instrumentation-asgi` | any other ASGI application's requests |
| `instrumentation-django` | Django requests |
| `instrumentation-requests` | outbound calls made with `requests` |
| `instrumentation-httpx` | outbound calls made with `httpx` |
| `instrumentation-sqlalchemy` | SQLAlchemy statements |
| `instrumentation-celery` | Celery tasks |

Without the matching extra the setup helper reports `False` and the affected
signal is not collected. The helpers never raise and never stop the
application, so the return value is the only place the missing package shows
up.

## Minimal integration

The SDK exposes a single ``cluebase`` namespace with twelve methods (``init`` /
``identify`` / ``group`` / ``link`` / ``reset`` / ``track`` /
``set_operation_discriminator`` / ``mark_background_task`` / ``set_consent`` /
``flush`` / ``close`` / ``get_diagnostics``). The cross-SDK surface is
documented in ``docs/system/contracts/sdk/public-api.md``.

``cluebase.init`` starts the SDK and the delivery pipeline, but on its own it
does **not** register any framework instrumentation. A FastAPI or Django
application should use the framework setup helper below instead; it performs
``init`` for you. Use bare ``cluebase.init`` only when there is no supported
framework to hook into, such as a plain script or a custom worker.

```python
import asyncio
from cluebase_backend_sdk import cluebase

cluebase.init({
    "project_key": "...",
    "api_key": "...",
    "service_key": "backend-api",
})

cluebase.identify("user_42", {"name": "Alice", "email": "alice@example.com"})
cluebase.group("organization", "org_7", {"name": "Acme Inc"})
cluebase.track("order_placed", {"product_name": "shirt", "amount": 100})
cluebase.set_consent("granted")

# Shutdown idiom — both are coroutines for cross-SDK parity.
asyncio.run(cluebase.flush())
asyncio.run(cluebase.close())
```

Do not pass ``environment`` or ``service_name`` in the MVP setup path.
Cluebase derives dev/prod from the project key prefix, and ``service_key`` is the
single literal customer-facing backend service label.

``endpoint`` is optional and defaults to ``https://api.cluebase.io``. Set
``CLUEBASE_API_BASE_URL`` only when the SDK must send to another Cluebase API
base URL, such as a local environment.

``cluebase.track`` custom event names must match
``^[A-Za-z0-9_.:-]{1,128}$``. Invalid names are ignored and do not emit
a source-signal custom track intent. ``cluebase.set_consent`` only records local
SDK state in the MVP backend SDK; it does not emit consent transition events.

Framework wrappers and runtime helpers are imported from their owning modules.
The package root is intentionally small: customer-facing lifecycle methods plus
stable setup helpers. Direct event builders are not public extension
points.

## Framework setup

These helpers are what actually register instrumentation. Each one calls
``init`` internally, configures the OTel exporter, installs the request
middleware, and enables the optional instrumentation it finds, so do not also
call ``cluebase.init``. Both return ``bool`` and never raise: a ``False``
result means setup did not complete and nothing is being collected.

```python
import os

from fastapi import FastAPI
from cluebase_backend_sdk._integrations.fastapi import cluebase_init_fastapi

app = FastAPI()
cluebase_init_fastapi(
    app,
    project_key=os.environ["CLUEBASE_PROJECT_KEY"],
    api_key=os.environ["CLUEBASE_API_KEY"],
    service_key="backend-api",
)
```

Django takes the settings module mapping instead of an app object, and appends
its middleware to ``MIDDLEWARE``:

```python
import os

from cluebase_backend_sdk._integrations.django import cluebase_init_django

cluebase_init_django(
    globals(),
    project_key=os.environ["CLUEBASE_PROJECT_KEY"],
    api_key=os.environ["CLUEBASE_API_KEY"],
    service_key="backend-api",
)
```

Both accept the same optional keyword arguments:

| Argument                          | Default            | Effect                                                                 |
| --------------------------------- | ------------------ | ---------------------------------------------------------------------- |
| ``service_key``                   | ``python-service`` | Customer-facing backend service label.                                 |
| ``instrument_outbound_requests``  | ``True``           | Capture outbound HTTP through ``requests`` and ``httpx``.              |
| ``instrument_celery_if_available``| ``True``           | Capture Celery task execution when Celery is importable.               |
| ``instrument_orm_if_available``   | ``True``           | Capture write-side ORM mutations through SQLAlchemy or the Django ORM. |
| ``state_fields``                  | ``()``             | Field names to observe as named state transitions.                     |

The optional instrumentation is skipped silently when the underlying library
is not installed, so a ``True`` return does not by itself prove that Celery or
ORM capture is active. Use ``cluebase.get_diagnostics()`` to confirm what the
process actually started.

## Recording internal processing boundaries

``cluebase.init`` and the framework helpers capture requests, outbound calls,
jobs, and write-side ORM mutations. Use these boundaries to mark the use-case
and state-machine steps inside a request, which cannot be inferred from a route
name. Each works as both a context manager and a decorator, including on
``async def``:

```python
from cluebase_backend_sdk import CluebaseCommand, CluebaseStateTransition

with CluebaseCommand("orders.approve", "Approve order"):
    ...

@CluebaseCommand("orders.approve")
async def approve_order(order_id: str) -> None:
    ...

with CluebaseStateTransition(
    "orders.approval", from_state="pending", to_state="approved"
):
    ...
```

Keys use ``dot.case``. ``from_state`` and ``to_state`` are required raw
evidence, not display labels. Reserve these for durable, business-meaningful
transitions; see
[`backend_sdk.md`](../../../docs/engineering/rules/backend_sdk.md) for what
must not be sent as a state transition.

## Architecture boundary

The Python SDK contains a shared instrumentation core and framework/provider
adapters. Adapters register native hooks and extract framework/provider facts;
the shared SDK core owns span lifecycle, request-scoped correlation context,
privacy handling, common attributes, failure status, and delivery batching. The
exporter serializes raw source signals, and the Cluebase backend ingest core
performs canonical classification, normalization, privacy projection, and
raw-ingest shaping.

```text
Python framework / library hook
  -> native facts passed to shared SDK instrumentation core
  -> OTel source signal
  -> ObservationSourceEventExporter
  -> /api/v1/ingest/backend
  -> apps/api/src/modules/ingest canonical core
```

Rules:

- Keep adapters limited to native hook registration, native fact extraction,
  and calls into the shared SDK core.
- Keep run/step tracking, common span lifecycle, context projection, privacy
  policy, failure classification, and request delivery timing outside
  framework/provider adapters.
- Keep canonical event classification in the backend ingest core so every SDK
  language uses the same event rules.
- Preserve ``interaction_id``, ``request_span_id``, ``request_id``, and
  ``trace_id`` whenever they are available through the shared SDK core.
- Run ``pnpm sdk:architecture:check`` when changing SDK source. CI runs this
  gate across every SDK registered in the descriptor.

The stable boundary contract is
[`architecture-gates.md`](../../../docs/system/contracts/sdk/architecture-gates.md).

## Privacy and PII handling

### 1. Hard-deny: PII / secrets are stripped before transport

The SDK strips a built-in set of property keys **before** the event
leaves the customer process. Caller-supplied `denied_keys` are added
on top of `DEFAULT_DENIED_KEYS` — they cannot remove a default-denied
key.

Hard-deny categories (case- and separator-insensitive — `userEmail`,
`user-email`, `USER_EMAIL`, `email_address` all match):

- Auth credentials: `authorization`, `cookie`, `set-cookie`,
  `password`, `passwd`, `secret`, `token`, `access_token`,
  `refresh_token`, `session`, `session_token`, `api_key`, `apikey`,
  `private_key`
- PII categories: `email`, `phone`, `credit_card`, `ssn`

This list is a strict superset of the server-side ingest hard-deny
(`@cluebase/shared` `INGEST_HARD_DENY_KEYS`), enforced by the
`tests/test_privacy_parity.py` cross-SDK parity test.

### 2. HTTP request values

This SDK does not read HTTP request or response bodies. The value surface it can
observe is the request query, and its values are masked by default. A value is
sent in plaintext only when the customer names that field on the Cluebase screen; the
decision is delivered to the SDK and re-checked by ingest. Hard-denied keys such
as `email` can never be released, whatever the screen says. See
`docs/system/architecture/sdk/backend-sdk.md` for the boundary and for what would be required to
bring bodies into scope.

### 3. Analysis projection and later registration

Properties that pass the hard-deny gate are projected by default unless a
cardinality, free-text, or complex-value guard keeps them out of the analysis
index. Guarded non-private values remain available in
`analysisProperties.raw_only` so they can be registered and promoted later.
The map stores each original value as JSON so its type can be restored.
Hard-denied values are dropped and never stored in `raw_only`. After a privacy
review, use the project service allowlist to promote a guarded non-private
property into the analysis projection.

### 4. Identity traits stay privacy-safe

`cluebase.identify(user_id, traits={...})` accepts profile traits such as
`name`, `email`, and `avatarUrl`. Other customer-defined values remain generic
subject traits instead of becoming fixed Cluebase profile fields. Use a stable opaque
`user_id`; the SDK converts `email` into privacy-safe contact-derived
fields before canonical identity events leave the process.

## Build / test

The wheel bundles the compiled Rust core (`crates/cluebase-sdk-core-py`) as
`cluebase_backend_sdk._native` through maturin, so building from source needs a
Rust toolchain. Published wheels already contain the extension; installing one
needs nothing beyond `pip`.

```sh
rustup toolchain install stable
python3 -m venv .venv
source .venv/bin/activate
pip install -e '.[test]'
python -m pytest
```

