Metadata-Version: 2.4
Name: engraph
Version: 0.1.1
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python
Classifier: Programming Language :: Rust
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Dist: pytest>=8.0 ; extra == 'dev'
Requires-Dist: httpx>=0.27 ; extra == 'dev'
Requires-Dist: fastembed>=0.3 ; extra == 'local'
Requires-Dist: mcp>=1.0 ; extra == 'mcp'
Requires-Dist: openai>=1.30 ; extra == 'openai'
Requires-Dist: fastapi>=0.110 ; extra == 'server'
Requires-Dist: uvicorn>=0.29 ; extra == 'server'
Provides-Extra: dev
Provides-Extra: local
Provides-Extra: mcp
Provides-Extra: openai
Provides-Extra: server
License-File: LICENSE
Summary: Temporal knowledge-graph memory for AI agents, engineered for the lowest retrieval latency
Keywords: knowledge-graph,memory,agents,llm,temporal,rag
Author: Engraph contributors
License: Apache-2.0
Requires-Python: >=3.9
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Homepage, https://github.com/engraph/engraph
Project-URL: Repository, https://github.com/engraph/engraph

# Engraph

**Temporal knowledge-graph memory for AI agents — engineered for the lowest retrieval latency.**

Engraph is an embedded, Rust-core reimagining of the [Graphiti](https://github.com/getzep/graphiti) model: raw **episodes** go in, a **bi-temporal knowledge graph** of entities and facts comes out, and hybrid search answers queries in **microseconds, not hundreds of milliseconds** — with no database server, and a fully-local mode that needs no API keys at all.

```
pip install engraph        # Python (prebuilt wheels)
cargo add engraph-core           # Rust
```

## Why Engraph?

Graphiti popularised the right data model — bi-temporal facts, episodes, entity resolution — but every query pays network round-trips to Neo4j/FalkorDB. Engraph keeps the model and changes the physics:

| | **Engraph** | **Graphiti** |
|---|---|---|
| Storage | **Embedded** (redb, pure-Rust ACID KV) — single file, no server | Neo4j / FalkorDB server required |
| Vector / lexical / graph indexes | **In-RAM, in-process** (HNSW + BM25 + adjacency) | Remote per query |
| Retrieval latency (10k episodes) | **p50 ≈ 17 µs, p95 ≈ 1.2 ms** (see below) | typically tens–hundreds of ms (network + query engine) |
| Zero-dependency mode | **Yes** — hashing embedder + heuristic extractor, runs offline | LLM + embedding API required |
| Fully-local neural mode | fastembed (ONNX) + Ollama | via OpenAI-compatible endpoints |
| Latency toolkit | result cache, adaptive planner, **latency budgets**, stage traces, p50/p95/p99 API | — |
| Similarity floor (`min_vector_sim`) | **Yes** | — |
| Bi-temporal model, groups/tenancy, communities, MCP server | Yes | Yes |

> Latency claims are for Engraph's embedded engine measured on commodity hardware (see [Benchmarks](#benchmarks)); Graphiti figures depend on deployment and network. Different architectures — Engraph optimises for single-node agent memory; a remote graph DB makes sense when many processes share one huge graph.

## Benchmarks

Measured with the built-in benchmark (`cargo run --release --example bench -- 10000 2000`), 10,000 episodes → 14,159 edges / 416 entities, 2,000 random queries, 2 vCPU sandbox, hashing embeddings, Rust 1.97:

| search mode | p50 | p95 | p99 |
|---|---|---|---|
| **hybrid (vector + BM25 + RRF)** | **17 µs** | 1.2 ms | 1.5 ms |
| lexical only (BM25) | 17 µs | 0.4 ms | 0.7 ms |
| vector only (HNSW) | 15 µs | 0.8 ms | 1.0 ms |
| full (graph expansion + MMR, k=20) | 5.9 ms | 6.5 ms | 7.0 ms |
| **cache hit** | **18 µs** | — | — |

End-to-end from Python (`benchmarks/bench_retrieval.py`, 2k episodes, includes hashing query embedding + FFI round-trip): **hybrid p50 ≈ 70 µs, p95 ≈ 0.7 ms**.

Ingestion: ~62 episodes/s (durable, one ACID commit per episode, zero-dep extraction). Cold open: 284 ms (index snapshots). Numbers on your hardware will differ — run the benchmark yourself, and see `benchmarks/` for the CI regression harness.

## Quickstart

### Python — zero dependencies

No API keys, no model downloads: the built-in hashing embedder and heuristic extractor run inside the Rust engine.

```python
from engraph import Engraph

g = Engraph("memory.db")                      # embedded, single file
g.add_episode("intro", "Alice works at Acme Corp. Acme Corp is a robotics company.")
g.add_episode("update", "Alice left Acme and joined Globex.")

res = g.search("Where does Alice work?")
for hit in res["edges"]:
    print(hit["edge"]["fact"], round(hit["score"], 4))

# What did we believe last week? (bi-temporal, transaction-time queries)
old = g.search("Where does Alice work?", as_of=last_week_ms)

# Explain exactly where the microseconds go
res = g.search("Alice", explain=True)
print(res["trace"])     # vector_us, bm25_us, graph_us, fusion_us, fetch_us
```

### Python — fully local, neural quality

```python
from engraph import Engraph, FastEmbedder, OllamaExtractor

g = Engraph(
    "memory.db",
    embedder=FastEmbedder(),        # ONNX embeddings locally (pip install engraph[local])
    extractor=OllamaExtractor(),    # LLM extraction via local Ollama
)
```

Or with OpenAI (`pip install engraph[openai]`, `OPENAI_API_KEY` set): `OpenAIEmbedder()` + `OpenAIExtractor()`.

### Rust

```rust
use engraph_core::{Engine, EngineConfig, Episode, EpisodeType, SearchConfig, HashingEmbedder};
use engraph_core::types::{now_millis, new_uuid};

let eng = Engine::open(None, EngineConfig::default())?;   // in-memory
eng.ingest_simple(Episode {
    uuid: new_uuid(), tenant_id: "local".into(), group_id: "demo".into(), name: "ep1".into(),
    content: "Alice works at Acme Corp.".into(),
    source: EpisodeType::Text, source_description: "docs".into(),
    reference_time: now_millis(), created_at: now_millis(),
})?;
let hits = eng.search("Alice work", None, &SearchConfig::default())?;
```

## Feature tour

- **Bi-temporal model** — every fact carries transaction time (`created_at`/`expired_at`) and event time (`valid_at`/`invalid_at`). Ask "what is true now?" *and* "what was true at T?" (`as_of`, `facts_as_of`).
- **Hybrid retrieval** — vector + BM25 + optional graph expansion, fused with RRF; rerankers: plain RRF, center-node distance (Graphiti-style), MMR diversity.
- **Latency toolkit** — result caching, adaptive query planner, per-query **latency budgets**, stage-level traces (`explain=True`), rolling p50/p95/p99 via `stats()`.
- **Similarity floor** — `min_vector_sim` drops weak ANN matches for out-of-domain queries.
- **Entity resolution** — exact + embedding-similarity merge with configurable threshold; custom labels and attributes.
- **Communities** — one-call clustering of the live graph with LLM-updatable names/summaries.
- **Tenancy** — `group_id` partitions everywhere; `delete_group()` for erasure.
- **MCP server + REST API** — bundled, see below.
- **Pluggable everything** — embedders/extractors are traits (Rust) and protocols (Python).

## MCP server (Claude, Cursor, …)

```bash
pip install engraph[mcp]
engraph-mcp --db memory.db          # or: python -m engraph.mcp_server --db memory.db
```

```json
{ "mcpServers": { "engraph": {
    "command": "engraph-mcp", "args": ["--db", "~/.engraph/memory.db"]
} } }
```

Tools: `add_memory`, `search_memory`, `search_facts`, `get_entity`, `recent_facts`.

## REST server

Production deployments use the native HTTP + gRPC server:

```bash
cargo run -p engraphd -- --insecure-local --db memory.db
# authenticated deployments: --auth-config grants.json
```

Native HTTP endpoints are `POST /v1/episodes`, `POST /v1/search`, and
`GET /v1/groups/{tenant}/{group}/submissions/{episode_uuid}`. The submission
endpoint reports `wal_only`, `queued`, `recovering`, `committed`, or `failed`.
Equivalent protobuf RPCs are exposed by `EngraphService`.

The Python/FastAPI server remains a compatibility shim:

```bash
pip install engraph[server]
uvicorn engraph.server:app --app-dir python --port 8484
# or ENGRAPH_DB=memory.db python -m engraph.server
```

`POST /episodes` · `POST /search` · `GET /nodes/{uuid}` · `GET /edges/{uuid}` · `GET /nodes/{uuid}/edges` · `GET /facts/{group}` · `GET /stats` · `DELETE /groups/{group}`

## Production deployment

- **Container**: `docker build -t engraphd .` (multi-stage, non-root, healthcheck);
  `docker compose up` brings up engraphd + Prometheus + Grafana with the
  provisioning in [`deploy/`](deploy/).
- **Backups / tiering**: `engraph export` bundles, or `engraph tier-out --s3`
  against any S3-compatible store (AWS/MinIO/R2; build feature `s3`).
- **Encryption at rest**: `ENGRAPH_MASTER_KEY` (EnvKms) for single-operator
  setups, or `engraphd --vault-kms` for HashiCorp Vault Transit with per-tenant
  derived keys (build feature `vault-kms`).
- **API keys**: `engraph hash-key` generates a key + argon2 hash for
  `grants.json` ([example](deploy/grants.example.json)).
- **Runbooks**: crash recovery, disk-full, rotation, erasure, restore, upgrade,
  failover — [`docs/RUNBOOKS.md`](docs/RUNBOOKS.md).
- **Validation**: [`scripts/load_test.py`](scripts/load_test.py),
  [`scripts/drill_kill_recovery.py`](scripts/drill_kill_recovery.py),
  [`scripts/adversarial_tenant.py`](scripts/adversarial_tenant.py),
  [`benchmarks/eval_memory.py`](benchmarks/eval_memory.py).
- Full path to production: [`docs/PROD.md`](docs/PROD.md).

## Database migration

v1 uses versioned protobuf records. Existing v0.1 databases must be converted offline:

```bash
cargo run -p engraph-cli -- migrate memory.db --output memory.v1.db
# or use --in-place; the original is retained as memory.db.v0.bak
```

## Architecture

```mermaid
flowchart LR
    subgraph Python
        A[add_episode / search] --> P{providers}
        P -->|LLM: OpenAI / Ollama / heuristic| E[extraction JSON]
        P -->|embed: fastembed / OpenAI / hashing| V[vectors]
    end
    subgraph Rust core
        E --> I[ingest pipeline<br/>resolve · dedupe · invalidate]
        V --> I
        I --> S[(redb store<br/>ACID, single file)]
        I --> X[in-RAM indexes<br/>HNSW · BM25 · adjacency]
        Q[query] --> H[hybrid search<br/>vector ∥ BM25 ∥ graph]
        X --> H
        H --> F[RRF · distance · MMR] --> R[top-N hydration]
        S --> R
    end
```

- Search never touches the network or the store's read path except to hydrate the final top-N; candidate generation, filtering (group + bi-temporal) and fusion run entirely on RAM indexes.
- Index snapshots make cold starts ~300 ms at 10k episodes; records are always the source of truth.
- The LLM is only ever on the *ingestion* path — retrieval is model-free (unless your query embedder is a model, which can also be the zero-cost hashing one).

Phase 1 correctness and concurrency acceptance results, including reproducible
Loom and performance-gate commands, are recorded in
[`docs/PHASE1_VALIDATION.md`](docs/PHASE1_VALIDATION.md).

## Repo layout

```
crates/engraph-core    # the embedded engine (pure Rust)
crates/engraph-proto   # durable records and gRPC contracts
crates/engraph-py      # PyO3 bindings (engraph._core)
crates/engraph-cli     # migration and administration CLI
crates/engraphd        # native authenticated HTTP/gRPC server
python/engraph         # Python facade, providers, REST + MCP servers
benchmarks/           # latency harness + CI baseline
tests/                # Rust integration + Python tests
```

## Roadmap

- [x] Batch ingest API (one group transaction + one view publication)
- [ ] mmap index snapshots (instant open at 1M+ edges)
- [ ] Learned cross-encoder reranker (Python callback)
- [ ] TypeScript bindings (napi-rs)
- [ ] Label-propagation communities with LLM summarisation in one call
- [ ] GraphQL subscriptions / change feed


## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md). The latency regression CI runs the benchmark on every PR — treat it as a budget, not a suggestion.

## License

[Apache-2.0](LICENSE) © Engraph contributors. Not affiliated with Zep Software, Inc.; "Graphiti" refers to their open-source project used here purely as a design reference.

