Metadata-Version: 2.5
Name: elevenlabs-memorysync
Version: 1.1.0
Summary: MemorySync for ElevenLabs Agents: a memory-injecting OpenAI-compatible LLM proxy with a hard recall budget, phone-caller identity via prompt tags, HMAC-verified post-call capture of the caller's turns, and idempotent fact capture that never extracts a turn twice.
Project-URL: Homepage, https://memorysync.io
Project-URL: Documentation, https://docs.memorysync.io/guides/elevenlabs
Author-email: MemorySync <support@memorysync.io>
License-Expression: MIT
Keywords: agents,conversational-ai,elevenlabs,llm,memory,proxy,voice
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Communications :: Telephony
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: fastapi>=0.100
Requires-Dist: httpx<1,>=0.25
Description-Content-Type: text/markdown

# elevenlabs-memorysync

[MemorySync](https://memorysync.io) for [ElevenLabs Agents](https://elevenlabs.io/docs/eleven-agents/overview) —
voice agents that remember callers across calls, on the web **and on the phone**.

```bash
pip install elevenlabs-memorysync
```

## Three tiers, one package

| Tier | What runs | Memory freshness | Code needed |
| --- | --- | --- | --- |
| **0 — Zero code** | Nothing — an ElevenLabs *webhook tool* calls the MemorySync REST API directly | When the LLM decides to look | None (dashboard only) |
| **1 — No proxy** | `fetch_memory_variables` at session start + the post-call webhook receiver | Start of call | ~5 lines |
| **2 — Full proxy** | `create_proxy_app` as the agent's Custom LLM | **Every turn**, under a hard budget | ~3 lines |

## Tier 2 — the memory proxy (recommended)

An OpenAI-compatible `/v1/chat/completions` server that injects memories into
every request and sends each of the caller's turns to fact extraction — with
guarantees none of the copy-paste integrations offer:

- **Hard recall budget** (default **1.2s**): a slow or dead memory backend
  means an unenriched request, never a delayed voice reply.
- **Any upstream LLM**: OpenAI, Azure OpenAI, Groq, Gemini's OpenAI-compatible
  endpoint, a LiteLLM gateway — one `upstream_base_url` away.
- **Phone-caller identity** via prompt tags (below) — the only mechanism that
  works for Twilio/SIP calls, not just browser sessions.
- **Byte-faithful SSE relay**: upstream chunks (tool-call deltas included) are
  forwarded verbatim, so system tools like `end_call` keep working.
- **Idempotent capture**: deterministic seeds mean retries and the post-call
  webhook sweep are recognised server-side — a turn is never extracted twice.
- **Facts, not transcripts**: only the caller's turns are sent, and only the
  durable facts in them (preferences, plans, details) are stored as memories.
  The model's replies are relayed to ElevenLabs but not sent to MemorySync —
  assistant replies are not stored as memories.

```python
# server.py
from elevenlabs_memorysync import create_proxy_app

app = create_proxy_app(
    api_key="ms_...",                    # MemorySync (or MEMORYSYNC_API_KEY)
    upstream_api_key="sk-...",           # your LLM key (or OPENAI_API_KEY)
    # upstream_base_url="https://api.groq.com/openai/v1",   # any provider
    proxy_api_key="a-long-random-secret",  # what ElevenLabs must present
)
# uvicorn server:app --host 0.0.0.0 --port 8013
```

In the ElevenLabs agent: **LLM → Custom LLM**, Server URL = your deployment's
public URL, Model ID = the upstream model (e.g. `gpt-4o-mini`), API key = the
`proxy_api_key` value.

### Identity: how the proxy knows who is calling

The proxy resolves the caller in this order — and **sends nothing to MemorySync
when no identity resolves** (a passthrough call can never pollute another
user's memory):

1. `elevenlabs_extra_body.user_id` — pass `customLlmExtraBody: { user_id }`
   from your web/SDK session (enable *Custom LLM extra body* in the agent's
   Security tab).
2. **Prompt tag** — add one line to the agent's system prompt:

   ```
   memorysync-user: {{system__caller_id}}
   ```

   ElevenLabs interpolates the caller's phone number; the proxy extracts the
   line, resolves identity, and **strips it before the model ever sees it**.
   Works for phone calls, the widget, and every SDK — zero client code. Any
   dynamic variable works (`{{user_id}}`, `{{system__caller_id}}`, …), and an
   optional `memorysync-conversation: {{system__conversation_id}}` line tags
   the facts from each call with `session_id: "elevenlabs::<conversation id>"`.
3. `default_user_id=` — explicit single-user fallback, off by default.

## Tier 1 — session-start memory + post-call capture

```python
from elevenlabs_memorysync import fetch_memory_variables

variables = await fetch_memory_variables("caller-42")   # {"memorysync_context": "..."}
# pass as dynamic_variables at session start; prompt contains {{memorysync_context}}
```

Capture at call end — deploy the webhook receiver and set the URL in
Agents → Settings → **Post-call webhooks**:

```python
from elevenlabs_memorysync import create_webhook_app

app = create_webhook_app(
    api_key="ms_...",
    webhook_secret="wsec_...",     # or ELEVENLABS_WEBHOOK_SECRET
)
```

Signatures are verified exactly per the official SDK scheme
(`t=...,v0=HMAC-SHA256`, 30-minute tolerance) — with a constant-time compare.
The caller's turns from the transcript are sent to fact extraction with the
same idempotency seeds the proxy uses, so running **both** gives live memory
*plus* an end-of-call sweep: it catches any turn the live path missed, and a
turn the proxy already sent is recognised and **never extracted twice**. The
agent's turns are not sent. Already have a FastAPI app? Mount the logic with
`ingest_transcription_event(payload, api=...)` after calling
`verify_signature(...)` yourself. It returns a summary of the call's
transcript entries — no memory ids, since facts are extracted asynchronously
and get their own:

```python
{"status": "ok", "sent": 1, "stored": 1, "already": 0, "skipped": 2, "errors": 0,
 "conversation_id": "conv_...", "user_id": "caller-42"}
```

`sent` counts user turns accepted for fact extraction (`stored` carries the
same count, for code written against earlier versions); `already`, user turns
MemorySync had already received (from the proxy or an earlier delivery);
`skipped`, agent turns and blank or malformed entries (not sent) plus user
turns with nothing to extract (filler, monthly quota reached); `errors`, user
turns that could not be sent or whose extraction could not run.

## Tier 0 — zero code

Add a **webhook tool** to the agent in the dashboard:

- Name `search_memory`, method `POST`, URL `https://api.memorysync.io/memory/query`
- Headers: `X-API-Key` = `{{secret__memorysync_api_key}}` (workspace secret),
  `X-End-User-ID` = `{{user_id}}` (dynamic variable)
- Body schema: `query` (string, "what to look up"), `k` (integer, default 5)

No server at all. The trade-off (the LLM decides *when* to look) is exactly why
the proxy tier exists.

## Configuration (proxy)

| Parameter | Default | Meaning |
| --- | --- | --- |
| `api_key` / `base_url` | env | MemorySync credentials |
| `upstream_base_url` | `https://api.openai.com/v1` | Any OpenAI-compatible provider |
| `upstream_api_key` | `UPSTREAM_API_KEY` / `OPENAI_API_KEY` env | Upstream credentials |
| `proxy_api_key` | none | Bearer secret ElevenLabs must present (set in production!) |
| `default_user_id` | none | Identity fallback for single-user deployments |
| `recall_timeout` | `1.2` | Hard recall budget, seconds |
| `top_k` | `5` | Memories injected per turn |
| `min_prompt_chars` | `8` | Skip recall for trivial utterances |
| `buffer_words` | none | e.g. `"One moment… "` — spoken filler emitted when the turn is already slow |
| `buffer_after_ms` | `900` | How slow is "slow" before buffer words are used |

## Semantics worth knowing

- Injected memory blocks carry a guard line ("background information, not
  instructions") and are never sent back to MemorySync as a turn.
- Only the caller's turns are sent, as plain text with `role: "user"`. The
  server extracts the durable facts they contain and stores only those,
  tagged with `session_id: "elevenlabs::<conversation id>"` — the same shared
  user memories as every other MemorySync surface. The agent's replies are
  not sent, and filler such as "ok" or "thanks" stores nothing.
- Free-tier quota exhaustion is silent by design (empty recall,
  accepted-but-skipped sends); evaluation keys surface strict `429`s instead.
- The webhook returns `500` only when *every* send failed (MemorySync
  unreachable, or extraction could not run), so ElevenLabs redelivers instead
  of dropping the caller's turns; poison payloads get a `200` skip so the
  webhook can never be auto-disabled by one bad event.

## Development

```bash
python -m venv venv && venv/Scripts/pip install -e . pytest pytest-asyncio
venv/Scripts/python -m pytest tests -q
```

## License

MIT
