Metadata-Version: 2.3
Name: prme
Version: 0.17.0a2
Summary: Portable Relational Memory Engine - local-first memory substrate for LLM-powered systems
Keywords: memory,llm,ai,knowledge-graph,vector-search,event-sourcing,duckdb,agents,rag,long-term-memory,mcp
Author: Dwamian McLeish
Author-email: Dwamian McLeish <dwamian@journey.ly>
License: Apache-2.0
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Database
Requires-Dist: duckdb>=1.4.0
Requires-Dist: usearch>=2.26.2
Requires-Dist: tantivy>=0.22.0
Requires-Dist: pydantic>=2.12
Requires-Dist: pydantic-settings>=2.9
Requires-Dist: fastembed>=0.7.4
Requires-Dist: structlog
Requires-Dist: pytz>=2025.2
Requires-Dist: instructor>=1.14
Requires-Dist: openai>=2.20
Requires-Dist: anthropic>=0.80
Requires-Dist: ollama>=0.6
Requires-Dist: dateparser>=1.2
Requires-Dist: cryptography==50.0.2
Requires-Dist: filelock>=3.16
Requires-Dist: fsspec==2026.2.0
Requires-Dist: httpx>=0.27
Requires-Dist: tiktoken>=0.12
Requires-Dist: numpy>=1.26
Requires-Dist: pillow>=10.1
Requires-Dist: fastapi>=0.115 ; extra == 'api'
Requires-Dist: uvicorn[standard]>=0.30 ; extra == 'api'
Requires-Dist: tiktoken>=0.12 ; extra == 'evaluation'
Requires-Dist: langchain-core>=0.3 ; extra == 'langchain'
Requires-Dist: llama-index-core>=0.12 ; extra == 'llamaindex'
Requires-Dist: keyring==25.7.0 ; extra == 'local-privacy'
Requires-Dist: mcp>=1.26.0,<2 ; extra == 'mcp'
Requires-Dist: asyncpg>=0.29.0 ; extra == 'postgres'
Requires-Dist: pgvector>=0.3.0 ; extra == 'postgres'
Requires-Dist: presidio-analyzer==2.2.364 ; python_full_version < '3.15' and extra == 'privacy'
Requires-Dist: spacy==3.8.11 ; python_full_version < '3.15' and extra == 'privacy'
Requires-Dist: sentence-transformers>=4.0 ; extra == 'reranker'
Requires-Dist: sentence-transformers>=4.0 ; extra == 'verification'
Requires-Python: >=3.11
Project-URL: Changelog, https://github.com/dwamianm/prism/blob/main/CHANGELOG.md
Project-URL: Documentation, https://github.com/dwamianm/prism/tree/main/docs
Project-URL: Homepage, https://github.com/dwamianm/prism
Project-URL: Issues, https://github.com/dwamianm/prism/issues
Project-URL: Repository, https://github.com/dwamianm/prism
Provides-Extra: api
Provides-Extra: evaluation
Provides-Extra: langchain
Provides-Extra: llamaindex
Provides-Extra: local-privacy
Provides-Extra: mcp
Provides-Extra: postgres
Provides-Extra: privacy
Provides-Extra: reranker
Provides-Extra: verification
Description-Content-Type: text/markdown

# PRME — Portable Relational Memory Engine

[![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/)
[![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)
[![CI](https://github.com/dwamianm/prism/actions/workflows/ci.yml/badge.svg)](https://github.com/dwamianm/prism/actions/workflows/ci.yml)

**Local-first, embeddable memory for LLM-powered applications.**

PRME gives agents and chatbots persistent memory through an append-only event
log, typed relationships, and hybrid retrieval. It preserves source evidence,
tracks changing claims, and packs relevant memories into a configurable token
budget. The default backend runs locally in a portable directory; PostgreSQL is
available for server deployments.

**Current release: v0.16.1.** Python 3.11+, synchronous and async clients, HTTP,
MCP, and LangChain/LlamaIndex integrations.

## Architecture

Four storage layers sit behind a unified memory API:

| Layer | Local implementation | Purpose |
|---|---|---|
| Event store | DuckDB | Immutable source events and durable operation records |
| Graph store | DuckDB tables and recursive CTEs | Typed memories, relationships, temporal validity, confidence, and provenance |
| Vector index | USearch | Semantic similarity search with versioned embeddings |
| Lexical index | Tantivy | BM25 full-text search |

The optional PostgreSQL backend uses PostgreSQL tables, pgvector, and full-text
search behind the same API. Both backends default to exact vector search;
approximate search is configurable.

```text
Source text → event log → direct storage or LLM extraction → graph + indexes

Query → intent, entity, and time analysis
      → graph, vector, lexical, and recent-memory candidates
      → deterministic scoring and filtering
      → token-bounded context with provenance
```

- **Ingestion:** `store()` writes a memory directly without an LLM.
  `ingest()` extracts entities, facts, and relationships through a configured
  provider. Deferred ingestion records durable work that can resume after restart.
- **Memory state:** typed nodes carry owner, scope, lifecycle, confidence,
  validity windows, and source references. Explicit corrections and contradiction
  operations preserve earlier claims; automatic store-time supersedence is opt-in.
- **Retrieval:** combines multiple search paths and packs selected evidence into
  context. Structured assertion-state and aggregation APIs support exact stored
  claim queries and decimal totals beyond semantic top-k retrieval.
- **Default retrieval behavior:** candidates are ranked by reciprocal rank
  fusion of their semantic and lexical ranks, with a recency boost on
  current-state questions and an event-time tie-break. Packing uses balanced
  ordering and renders each record as one plain line (the `reader` context
  format), while the full records stay in the bundle's sections and the
  retrieval receipt. These defaults changed after v0.12.0;
  [Default retrieval settings](docs/PACKING.md#default-retrieval-settings)
  lists the earlier values and the environment variables that restore them.
- **Organizer:** maintains promotion, decay, summaries, deduplication, archival,
  and index compaction. It runs opportunistically or through `prme organize`;
  recurring schedules use an external timer. Shared stores should pass `user_id`.
- **Workspaces:** `MemoryWorkspace` manages named local packs or PostgreSQL
  schemas with identity checks and scoped leases.

A local memory pack is a directory by default, or one file when you opt in:

```text
my_memories/            # directory pack (the default)
├── memory.duckdb
├── vectors.usearch
├── lexical_index/
└── manifest.json       # only when encryption is on

memories.prme           # single-file pack (opt-in)
```

Giving `MemoryClient` a path that ends in `.prme`, or setting `PRME_PACK_PATH`,
selects the single-file layout: one DuckDB file holds the event log, the graph,
the vector snapshot and a pack manifest, and the lexical index is rebuilt in
memory when the pack opens. `prme migrate` copies a directory pack into a new
`.prme` file, and [single-file packs](docs/INTEGRATION.md#single-file-packs)
covers encryption, compaction and how retrieval compares across the layouts.

Search indexes can be regenerated from the durable graph with `prme rebuild`. A store can move to another embedding
model without losing its memories: `enable_multi_model_embeddings` (off by default) keeps vectors by model, and
`prme reembed` builds the new model's vectors in the background before you switch (see
[switching embedding models](docs/EMBEDDING-MODELS.md)).
Event time, ingestion time, and validity time remain distinct. The `knowledge_at`
filter applies an ingestion cutoff to current graph/index state; exact historical
state replay is not currently available.

## Current benchmarks

| Benchmark | Correct answers | Answer accuracy |
|---|---:|---:|
| LongMemEval-S | 453 / 500 | **90.6%** |
| LoCoMo, non-adversarial questions | 1,250 / 1,540 | **81.2%** |

### LongMemEval-S by question type

| Question type | Correct answers | Answer accuracy |
|---|---:|---:|
| Single-session user | 69 / 70 | 98.6% |
| Single-session assistant | 54 / 56 | 96.4% |
| Knowledge update | 71 / 78 | 91.0% |
| Temporal reasoning | 120 / 133 | 90.2% |
| Single-session preference | 26 / 30 | 86.7% |
| Multi-session | 113 / 133 | 85.0% |

### LoCoMo by question type

| Question type | Correct answers | Answer accuracy |
|---|---:|---:|
| Single-hop | 756 / 841 | 89.9% |
| Temporal | 256 / 321 | 79.8% |
| Multi-hop | 182 / 282 | 64.5% |
| Open-domain | 56 / 96 | 58.3% |

Measured on September 25, 2026 with the v0.13.0 retrieval defaults and a
3,996-token memory-context budget per question, over memory packs built with
`store()`, which never runs extraction. The defaults adopted on October 6
change what `ingest()` stores and how repeated text is packed, so they were
measured on a separate track over packs built through `ingest()`. See
[BENCHMARKS.md](BENCHMARKS.md) for both tracks, the method, earlier results
and reproducibility artifacts.

## Installation

```bash
pip install prme
```

Optional integrations:

```bash
pip install "prme[postgres]"   # PostgreSQL backend
pip install "prme[api]"        # HTTP API
pip install "prme[mcp]"        # MCP server
pip install "prme[langchain]"  # LangChain retriever and chat history
pip install "prme[llamaindex]" # LlamaIndex retriever and chat store
```

## Quickstart

```python
from prme import MemoryClient

with MemoryClient("./my_memories") as memory:
    memory.store("Alice prefers dark mode in her editors.", user_id="alice")
    memory.store("The team decided to use PostgreSQL.", user_id="alice")

    response = memory.retrieve("What are Alice's preferences?", user_id="alice")
    for result in response.results:
        print(f"[{result.composite_score:.3f}] {result.node.content}")
```

`MemoryClient` is synchronous; `MemoryEngine` provides the async API. Direct
storage and retrieval use local embeddings by default and require no LLM API key.
Use `ingest()` with an OpenAI, Anthropic, or Ollama extraction provider when you
want structured memories extracted from conversation text.

Configure PRME through `PRMEConfig`, `PRME_` environment variables, or a `.env`
file. Constructor values override the environment, which overrides `.env`.
The [configuration reference](documentation/configuration.md) lists the
settings and their defaults. To return to the v0.12.0 retrieval defaults, set
`PRME_SCORING__FUSION=weighted`, `PRME_PACKING__CONTEXT_FORMAT=auditable` and
`PRME_PACKING__MULTIPATH_ORDERING=balanced` (the last is the default again and
only makes it explicit; see
[Default retrieval settings](docs/PACKING.md#default-retrieval-settings)).
See the [integration reference](docs/INTEGRATION.md) and
[examples](examples/) for configuration and application patterns.

## Interfaces and tools

Use the CLI to initialize, inspect, search, and maintain a local pack:

```bash
prme init ./my_memories
prme doctor ./my_memories
prme search ./my_memories/memory.duckdb "Alice's preferences" --user-id alice
prme organize ./my_memories/memory.duckdb --user-id alice
prme compact ./my_memories/memory.duckdb
prme migrate ./my_memories ./memories.prme
```

`prme compact` rewrites a closed pack file to give back the space DuckDB never
returns, and `prme migrate` copies a directory pack into a new single-file pack,
leaving the directory in place. Every command that takes `memory.duckdb` also
takes a `.prme` file, and `prme init ./memories.prme` creates one. See the
[CLI reference](documentation/cli.md).

For a local assistant, install the MCP extra and run with a fixed owner:

```bash
PRME_MCP_USER_ID=alice prme-mcp --db-path ./my_memories
```

`--db-path` also accepts a single-file pack, such as `./memories.prme`.

The HTTP extra provides a FastAPI service via `python -m prme.api`, which binds to
`127.0.0.1` and refuses a network bind without API credentials (see
[Running the Server](documentation/http-api.md#running-the-server)).
Shared HTTP deployments bind owners to credentials through `PRME_API_USER_KEYS`;
MCP Streamable HTTP uses its separate `PRME_MCP_USER_KEYS` setting.
See the [HTTP guide](docs/HTTP-API.md) and
[framework integration guide](docs/FRAMEWORK-INTEGRATIONS.md) for application setup.

To browse stored entries and explore their graph connections from a source
checkout, run `uv run --extra api python -m web.server` and open
**http://127.0.0.1:8080**. The standalone [memory explorer](web/README.md) lives in
`web/` and includes filters, source evidence, and an interactive relationship map.

## Milestones

| Milestone | Delivered |
|---|---|
| Core memory foundation, through v0.9 | Event sourcing, typed graph, hybrid retrieval, organizer jobs, encryption, CLI, evaluation harness, and index rebuilds |
| v0.10 | Deterministic vector search, batched ingestion indexing, and context-budget enforcement |
| v0.11 | Tenant-scoped maintenance, ownership and retrieval-filter fixes, and broader backend CI coverage |
| v0.12 | Durable ingestion and recovery, atomic lifecycle and correction records, scoped workspaces, structured assertion and quantity operations, retrieval receipts, and evaluated ranking profiles |
| v0.13 | Rank fusion with current-state recency, the one-line reader context format and balanced ordering as retrieval defaults; conversation participants with speaker names; refusal of unauthenticated network binds for the HTTP API; an offline evidence gate and paired answer runs for measuring retrieval changes |
| v0.14, current | A claim's own sentences as its text, a named speaker's I, me and my bound to the speaker, and each source text packed once, as `ingest()` and reader context defaults; repeated extracted claims merged into one current record; a browser memory explorer; benchmark packs built through `ingest()`, with the evidence gate and a DeepSeek answer track over them |

Experimental reranking (including the cross-encoder rank order), temporal-relation
guidance, model-assisted verification, temporal-first query intent, event-time
recency for the weighted formula, session context packing, windowed extraction,
self-contained fact text and speaker grounding remain opt-in.
Detailed changes are in the [changelog](CHANGELOG.md).

Current work follows [epic #77](https://github.com/dwamianm/prism/issues/77),
which acts on the 2026-09-23 benchmark gap audit: a reader-facing context,
better ranking, the context budget, and dated facts extracted at write time for
multi-hop and list questions. See the [roadmap](ROADMAP.md) for what has been
delivered and what is still open.

Images and video as memory sources are
[RFC-0018](docs/RFC-0018-Visual-Memory.md). Its phase 1 is implemented and
off by default: with `vision.enabled` set, `MemoryClient.observe()` and
`prme observe` turn a photo or a saved video into dated observation events
described by a local vision model, with a thumbnail of each keyframe kept in
the pack for provenance. The benchmark with real recordings is still open.

## Documentation

- [Developer guides](documentation/README.md) and [configuration reference](documentation/configuration.md)
- [Architecture and RFC index](docs/INDEX.md)
- [Integration reference](docs/INTEGRATION.md) and [framework adapters](docs/FRAMEWORK-INTEGRATIONS.md)
- [Context packing and default retrieval settings](docs/PACKING.md) and [retrieval feedback and learning](docs/LEARNING.md)
- [Experimental retrieval policies](docs/EXPERIMENTAL-RETRIEVAL-POLICIES.md)
- [Memory corrections](docs/MEMORY-CORRECTIONS.md)
- [Entity profiles](docs/ENTITY-PROFILES.md) and [entity identity](docs/ENTITY-IDENTITY.md)
- [Workspaces](docs/WORKSPACES.md) and [local resource control](docs/LOCAL-RESOURCES.md)
- [Custom embeddings](docs/CUSTOM-EMBEDDINGS.md) and [typed value bindings](docs/VALUE-BINDINGS.md)
- [HTTP API](docs/HTTP-API.md)

## Development

```bash
git clone https://github.com/dwamianm/prism.git
cd prism
uv sync --dev
uv run pytest tests/ -q
```

PostgreSQL tests require a live database configured through
`PRME_TEST_DATABASE_URL`. See [CONTRIBUTING.md](CONTRIBUTING.md) for development
setup, checks, and contribution guidelines.

## License

Apache License 2.0 — see [LICENSE](LICENSE).
