Metadata-Version: 2.5
Name: mark-sdk
Version: 0.2.0a8
Summary: Local-first cognitive memory runtime for agent applications.
Project-URL: Homepage, https://github.com/emsoftanalytics/MARK-SDK
Project-URL: Repository, https://github.com/emsoftanalytics/MARK-SDK.git
Project-URL: Issues, https://github.com/emsoftanalytics/MARK-SDK/issues
Project-URL: Changelog, https://github.com/emsoftanalytics/MARK-SDK/blob/main/CHANGELOG.md
Author: Somtochukwu Owen Ikeanyi
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: agents,ai,memory,runtime,sdk
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Python: >=3.10
Requires-Dist: pydantic>=2.0
Provides-Extra: adapters
Requires-Dist: langchain-core>=0.3; extra == 'adapters'
Requires-Dist: langchain>=1.0; extra == 'adapters'
Requires-Dist: mcp>=1.0; extra == 'adapters'
Provides-Extra: all
Requires-Dist: beautifulsoup4>=4.12; extra == 'all'
Requires-Dist: cryptography>=42.0; extra == 'all'
Requires-Dist: duckduckgo-search>=6.0; extra == 'all'
Requires-Dist: langchain-core>=0.3; extra == 'all'
Requires-Dist: langchain-mcp-adapters>=0.1; extra == 'all'
Requires-Dist: langchain-ollama>=0.2; extra == 'all'
Requires-Dist: langchain>=1.0; extra == 'all'
Requires-Dist: langgraph>=0.2; extra == 'all'
Requires-Dist: mcp>=1.0; extra == 'all'
Requires-Dist: numpy>=1.26; extra == 'all'
Requires-Dist: pyarrow>=14.0; extra == 'all'
Requires-Dist: python-dotenv>=1.0; extra == 'all'
Requires-Dist: requests>=2.31; extra == 'all'
Requires-Dist: sentence-transformers>=2.7; extra == 'all'
Requires-Dist: wikipedia-api>=0.6; extra == 'all'
Provides-Extra: crypto
Requires-Dist: cryptography>=42.0; extra == 'crypto'
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: tomli>=2.0; (python_version < '3.11') and extra == 'dev'
Provides-Extra: fast
Requires-Dist: numpy>=1.26; extra == 'fast'
Provides-Extra: fastembed
Requires-Dist: fastembed>=0.5; extra == 'fastembed'
Provides-Extra: langchain
Requires-Dist: langchain-core>=0.3; extra == 'langchain'
Requires-Dist: langchain>=1.0; extra == 'langchain'
Provides-Extra: mcp
Requires-Dist: mcp>=1.0; extra == 'mcp'
Provides-Extra: middleware
Provides-Extra: middleware-compression
Provides-Extra: middleware-gap-healing
Provides-Extra: middleware-governance
Provides-Extra: middleware-lifecycle
Provides-Extra: middleware-media-continuity
Provides-Extra: middleware-observability
Provides-Extra: middleware-observe
Provides-Extra: middleware-query-expansion
Provides-Extra: middleware-recall
Provides-Extra: middleware-sandbox
Provides-Extra: middleware-sync
Provides-Extra: middleware-trust-bus
Provides-Extra: model2vec
Requires-Dist: model2vec>=0.4; extra == 'model2vec'
Provides-Extra: ollama
Requires-Dist: requests>=2.31; extra == 'ollama'
Provides-Extra: rl
Requires-Dist: numpy>=1.26; extra == 'rl'
Requires-Dist: pyarrow>=14.0; extra == 'rl'
Provides-Extra: sentence-transformers
Requires-Dist: sentence-transformers>=2.7; extra == 'sentence-transformers'
Provides-Extra: tutorial
Requires-Dist: langchain-core>=0.3; extra == 'tutorial'
Requires-Dist: langchain-mcp-adapters>=0.1; extra == 'tutorial'
Requires-Dist: langchain-ollama>=0.2; extra == 'tutorial'
Requires-Dist: langchain>=1.0; extra == 'tutorial'
Requires-Dist: langgraph>=0.2; extra == 'tutorial'
Requires-Dist: mcp>=1.0; extra == 'tutorial'
Requires-Dist: nest-asyncio>=1.6.0; extra == 'tutorial'
Requires-Dist: python-dotenv>=1.0; extra == 'tutorial'
Requires-Dist: requests>=2.31; extra == 'tutorial'
Provides-Extra: web
Requires-Dist: beautifulsoup4>=4.12; extra == 'web'
Requires-Dist: duckduckgo-search>=6.0; extra == 'web'
Requires-Dist: wikipedia-api>=0.6; extra == 'web'
Description-Content-Type: text/markdown

# MARK (Memory-Augmented Agent Recall Kit) Python SDK

`mark-sdk` is an agent memory runtime. It installs as `mark-sdk` from PyPI and
imports as `mark` in Python.

MARK helps agents remember what they create, decide, observe, and learn across
long-running workflows. It is built for more than chat history: stories,
characters, images, videos, software projects, plans, design decisions, tool
results, and multi-agent workflow state all need continuity.

The SDK combines memory storage, retrieval, context injection, observability,
graph-scoped memory blocks, and tamper-evident provenance into a small
developer API. Everything runs locally by default: no account, network, or API
key is required.

```python
from mark import Mark

with Mark.local(project_path=".") as mark:
    mark.memory.block("project").write(
        "The API framework is FastAPI.",
        importance=0.9,
    )

    context = mark.memory.retrieve("Which API framework does this project use?")
    print(context.as_text())
```

## Why MARK

Most memory integrations start and end with `store()` and `retrieve()`. MARK is
designed as an agent memory runtime: it participates in the agent loop, injects
useful context before work, observes outcomes after work, and keeps memory
inspectable as connected evidence rather than flat chunks.

Use MARK when an agent needs to:

- remember creative continuity: characters, objects, shots, styles, plot
  points, and generated artifacts;
- remember project decisions: architecture, conventions, constraints,
  implementation plans, and test results;
- coordinate across agents: one agent plans, another implements, another tests,
  while all share durable local context;
- preserve trust and provenance: group related memory into blocks, seal them,
  verify them later, and quarantine bad memory without breaking the rest of the
  agent;
- adopt memory incrementally: add middleware or tools to an existing agent
  rather than rebuilding the application around a database.

MARK is not a managed memory database. The open SDK is the Apache-2.0-licensed
runtime and integration layer.

## Install

MARK supports Python 3.10 and newer.

```bash
pip install mark-sdk
```

Optional framework adapters are bundled under `mark.adapters` and installed
through extras:

```bash
pip install "mark-sdk[langchain]"   # LangChain tools + agent middleware
pip install "mark-sdk[mcp]"        # MCP server over MARK memory
pip install "mark-sdk[adapters]"   # both of the above
pip install "mark-sdk[middleware]" # all local middleware batteries
pip install "mark-sdk[fast]"       # numpy-vectorised search (recommended)
```

The core package depends only on `pydantic`. Without numpy, vector search runs
in pure Python; with `mark-sdk[fast]` it uses one vectorised scan (about 1 ms at
10k memories, 7 ms at 100k).

Specific middleware install targets are also available, for example
`mark-sdk[middleware-governance]`, `mark-sdk[middleware-trust-bus]`,
`mark-sdk[middleware-sandbox]`, and `mark-sdk[middleware-media-continuity]`.
Most middleware batteries are pure Python today, so the extras mainly provide a
stable install contract as dependency-bearing middleware grows.

Verify the installed package:

```bash
python -c "from mark import Mark; print(Mark.local('.').memory.list_blocks())"
```

## Quickstart

### 1. Store and recall project memory

```python
from mark import Mark

with Mark.local(".") as mark:
    mark.memory.block("architecture").write(
        "Authentication uses FastAPI dependencies and JWT.",
        importance=0.8,
    )

    result = mark.memory.retrieve("How is authentication implemented?")
    print(result.as_text())
```

### 2. Run the same agent with memory

```python
import asyncio

from mark import Mark

def coding_agent(prompt: str) -> str:
    if "FastAPI" in prompt:
        return "Use FastAPI dependency injection for this endpoint."
    return "I need more project context."

with Mark.local(".") as mark:
    mark.memory.block("project").write("This backend uses FastAPI.")
    agent = mark.wrap_agent(coding_agent, blocks=["project"])

    result = asyncio.run(agent.run("Add a health-check endpoint."))
    print(result.output)
```

### 3. Track continuity across creative sessions

```python
from mark import Mark

with Mark.local(".") as mark:
    memory = mark.runtime.memory("video-agent")

    memory.observe(
        "Elena enters the North Warehouse wearing the red scarf.",
        session_id="season-01/episode-01/scene-04",
        memory_type="scene",
    )

    scene_context = memory.retrieve_sync(
        "What must stay consistent for Elena?",
        session_prefix="season-01/",
    )
    print(scene_context.as_context())
```

### 4. Group creations and decisions into provenance blocks

Memory blocks bundle related fragments, nodes, and edges into one unit per
topic, session, or world-bible scope. Blocks link to each other forward and
backward, can be sealed into a tamper-evident hash chain, and can be
quarantined — isolating bad memory without affecting the rest of the agent.

```python
from mark import Mark

with Mark.local(".") as mark:
    memory = mark.runtime.memory("video-agent")
    graph  = memory.blocks()
    chain  = memory.chain()

    shot = graph.create_block("ep01-scene04", session_id="season-01/episode-01")
    graph.add_fragment(shot.id, memory.store_sync("Elena hides the key in the rafters."))

    # Retrieval scoped to one block
    result = memory.retrieve_sync("Where is the key?", block_id=shot.id)

    # Seal the block into the agent's provenance chain
    sealed = chain.seal(shot.id)
    print(sealed.content_hash)

    # Verify integrity later — pinpoints any corrupted member
    print(chain.verify(shot.id).valid)
    print(chain.verify_chain().valid)

    # Isolate a bad block without touching anything else
    chain.quarantine(shot.id)
```

### 5. Add memory to a LangChain agent as middleware

`MarkAgentMiddleware` turns MARK into a transparent context-window manager for
any LangChain v1 agent: it watches the LangChain message history, retrieves
only relevant memory before each model call, injects a compact memory block,
and archives useful AI/tool evidence in the background. The LLM remains the
reasoning machine; MARK supplies durable working memory.

```python
from langchain.agents import create_agent

from mark import Mark
from mark.adapters.backend import LocalMarkBackend
from mark.adapters.langchain.middleware import MarkAgentMiddleware

with Mark.local(".") as mark:
    middleware = MarkAgentMiddleware(
        backend=LocalMarkBackend(mark),
        max_context_chars=1400,     # context budget injected per model call
        write_outcomes=True,        # archive AI reasoning steps
        observe_tool_results=True,  # archive tool outputs as they happen
    )

    agent = create_agent(model, tools, middleware=[middleware])
    result = agent.invoke({"messages": [("user", "Fix the failing build.")]})
    # or: await agent.ainvoke(...) / agent.astream(...)
```

The middleware is safe to share across concurrent runs: per-run memory state is
isolated, every model call in a run receives memory, and memory failures are
logged and degrade gracefully instead of breaking the agent. Gap healing
(`auto_heal=True`) is opt-in because it can reach the network.

MARK adapters default to the `__mark__` namespace used by `mark.memory`, so
memory written with `mark.memory.block(...).write(...)` is recalled
automatically. Pass `agent_id=` / `default_agent_id=` only for a separate
namespace, and write into it with `mark.runtime.memory("<id>")`.

Prefer explicit control? `create_mark_tools` exposes memory as ordinary
LangChain tools the model calls itself. The tools support both sync `invoke()`
and async `ainvoke()` paths, including notebook environments with an already
running event loop:

```python
from mark.adapters.langchain import create_mark_tools

tools = create_mark_tools(LocalMarkBackend(mark))
agent = create_agent(model, [*tools, *my_other_tools])
```

Use one surface by default: middleware for automatic context loading, or tools
when the model should deliberately write canonical facts. Combining both is an
advanced mode for targeted recall/write operations; keep injected context small
so MARK does not duplicate information already loaded by the middleware.

And `mark.adapters.mcp` serves the same memory to any MCP-compatible client:

```python
from mark.adapters.backend import LocalMarkBackend
from mark.adapters.mcp import create_mark_mcp_server

server = create_mark_mcp_server(
    LocalMarkBackend(mark),
    allowed_agent_ids={"__mark__"},  # optional: restrict client-selected namespaces
)
server.run()  # stdio MCP server
```

The adapter supports both `mcp` 2.x (`MCPServer`) and 1.x (`FastMCP`).

The full middleware walkthrough — the same agent run with and without MARK,
side by side — lives in the tutorial notebook under
[examples/](examples/).

### 6. Keep canonical creative facts in a world bible

```python
from mark import Mark

with Mark.local(".") as mark:
    mark.world_bible.remember(
        "Elena is left-handed and always wears the red scarf in episode 01.",
        tags=["character:elena", "wardrobe"],
    )

    facts = mark.world_bible.check("Elena wardrobe and physical traits")
    for fact in facts:
        print(fact.content)
```

### 7. Use any local embedding model

The zero-dependency default (`HashEmbeddingProvider`) needs no model download.
For semantic recall, plug in any local model server or in-process model:

```python
from mark import Mark
from mark.embeddings import (
    OllamaEmbeddingProvider,            # Ollama: batch /api/embed
    OpenAICompatibleEmbeddingProvider,  # llama.cpp server, LM Studio, vLLM, LocalAI
    FastEmbedProvider,                  # ONNX, no torch:   mark-sdk[fastembed]
    Model2VecProvider,                  # static, numpy:    mark-sdk[model2vec]
)

embedder = OpenAICompatibleEmbeddingProvider(
    base_url="http://localhost:1234/v1",   # LM Studio
    model="nomic-embed-text-v1.5",
)
with Mark.local(".", embedder=embedder) as mark:
    ...
```

HTTP providers batch requests, use connect/read timeouts, and retry transient
failures with jittered backoff. MARK records each embedder's fingerprint, so
switching models re-embeds existing memory instead of mixing vector spaces.

### 8. Track values that change over time

Facts record the current value of something and keep its full history.
Later values supersede earlier ones by when they became true, not by when
they were written:

```python
with Mark.local(".") as mark:
    memory = mark.runtime.memory("ops")
    memory.facts.set("atlas-core", "port", "6430", valid_from="2026-03-01")
    memory.facts.set("atlas-core", "port", "4444", valid_from="2026-03-30")

    memory.facts.get("atlas-core", "port").value              # "4444"
    [f.value for f in memory.facts.history("atlas-core", "port")]   # ["6430", "4444"]

    # Retrieval understands "now" and "originally":
    result = memory.retrieve_sync("Which port does atlas-core use now?")
    result.facts            # current facts about atlas-core
    result.missing_anchors  # names in the query that memory has never seen
```

Free-text memories can carry an event time too
(`store_sync(text, occurred_at="2026-03-30")`, or a leading `[2026-03-30]`
stamp), and "current"/"original" questions order matching records by it.

### 9. Let MARK anticipate what the agent needs

`ContextWatch` follows a conversation and keeps relevant memory ready before
the agent asks: facts about the subjects being discussed (and subjects they
link to), corrections when the agent states an outdated value, and notices for
subjects memory has never seen.

```python
from mark import ContextWatch

with Mark.local(".") as mark, ContextWatch(mark.runtime.memory("ops")) as watch:
    watch.observe("Please look into ticket HAL-4433.", role="user")
    system_prompt = base_prompt + watch.briefing().text   # before each model call
    watch.observe(model_reply, role="assistant")
```

`MarkAgentMiddleware` does this automatically for LangChain agents
(`anticipate=True` by default).

### 10. Use memory from the command line

```bash
mark ingest ops.log --session day-1
mark fact set atlas-core port 4444 --valid-from 2026-03-30
mark recall "Which port does atlas-core use now?"
printf 'Who owns atlas-core?\nWhat is HAL-4433 about?\n' | mark recall-many -
mark serve &          # keep the store warm: each call then takes about 0.1 s
```

The store comes from `--db` or `$MARK_DB`; the server binds to loopback only
and requires a token kept next to the database.

## Comparative Samples

MARK examples are written as A/B comparisons: the same task runs once with
ordinary short-term context and once with MARK-backed recall.

| Scenario | Without MARK | With MARK | What to inspect |
| --- | --- | --- | --- |
| Coding agent | The agent answers from the current prompt only. Project conventions must be repeated. | MARK recalls stored conventions before the model reasons. | `examples/02_agent_with_memory.py` and `examples/getting_started_with_mark.ipynb` |
| LangChain agent | Message history is whatever LangChain keeps in the active loop. | `MarkAgentMiddleware` retrieves compact memory before model calls and observes useful tool/model evidence. | README section 5 and the tutorial notebook |
| Creative continuity | A generator without MARK only sees its prompt and short-term carryover. | A MARK-aware agent recalls canonical identity/world facts and writes continuity observations between steps. | `MarkAgent`, sessions, and `world_bible` |

Generated media demo outputs and heavyweight provider clients are not included
in `mark-sdk`.

## Features

- **Agent-loop integration:** wrap simple callables, attach LangChain
  middleware, expose MARK as tools, or serve memory over MCP.
- **Pluggable middleware:** compose framework-neutral middleware for recall
  defaults, observation metadata, compression, query expansion, gap healing,
  lifecycle maintenance, observability, and sync envelope preparation.
- **Structured memory:** fragments, sessions, graph nodes, graph edges,
  world-bible facts, and graph-scoped blocks.
- **Hybrid retrieval:** SQLite FTS5 (BM25) lexical search and vector search
  fused with Reciprocal Rank Fusion, so exact identifiers, names, and
  paraphrases all hit. Agent- and session-scoped filters are applied before
  scoring, graph expansion is damped per hop, and gap reports are calibrated on
  real query evidence. No LLM calls on the write or read hot path.
- **Creative continuity:** session prefixes, tags, scopes, world-bible facts,
  and blocks make it natural to track characters, objects, locations, shots,
  styles, and generated artifacts.
- **Workflow memory:** store plans, conventions, decisions, tool outputs, test
  results, and implementation state for coding or autonomous agents.
- **Provenance and integrity:** SHA-256 block sealing, whole-chain verification,
  pinpointed corruption reports, and quarantine isolation.
- **Memory lifecycle:** working memory with TTL expiry, consolidation,
  deduplication, pruning, reinforcement, and local governance gates.
- **Multi-agent sharing:** trust-aware local bus for publisher trust,
  subscriptions, snapshots, and trust-filtered retrieval.
- **Local observability:** JSONL traces, replay support, session activity logs,
  and middleware observation of reasoning/tool outcomes.

Everything in this package runs locally under the Apache-2.0 license.

## Performance

Measured with `benchmarks/bench_runtime.py` (offline, default configuration,
SQLite on disk, numpy installed):

| Memories | Store | Retrieve p50 | Retrieve p95 | hit@1 / hit@5 |
| --- | --- | --- | --- | --- |
| 2,000 | 0.6 ms | 12.5 ms | 23 ms | 14/14 / 14/14 |
| 10,000 | 0.7 ms | 32 ms | 81 ms | 14/14 / 14/14 |

Reproduce with `uv run --extra fast python benchmarks/bench_runtime.py`.

## MARK Cloud — coming soon

Watch out for MARK Cloud updates.

## Examples

See [examples/](examples/) for runnable developer usage tests and walkthroughs:

```bash
python examples/run_live_examples.py
```

The examples cover recalling memory, an LLM answering with and without MARK,
sessions, facts that change over time, anticipatory context with
`ContextWatch`, and redacted sync. Examples that need a model call any local
OpenAI-compatible server (Ollama, LM Studio, llama.cpp) via `MARK_LLM_URL`.

The notebook in [examples/](examples/) provides a longer step-by-step
walkthrough for memory, retrieval, sessions, middleware, tools, MCP exposure,
memory inspection, and provenance sealing.

Planned proof-oriented examples:

- `examples/multi-agent-coding/`: demonstrate planner, implementer, and tester
  agents sharing decisions and workflow state through MARK.

## Development

```bash
python examples/run_live_examples.py
uv run --extra dev pytest
uv build
```

## Middleware

Core middleware lives under `mark.middlewares` and wraps existing public runtime
operations. It is framework-neutral, so LangChain, MCP, and future adapters can
share the same behavior instead of reimplementing recall, observation,
compression, tracing, and sync logic.

```python
from mark import Mark, ObserveMiddleware, ObservabilityMiddleware, RecallMiddleware

with Mark.local(
    ".",
    middleware=[
        RecallMiddleware(compress=True, expand=True),
        ObserveMiddleware(source="agent", tags=["workflow"]),
        ObservabilityMiddleware(),
    ],
) as mark:
    memory = mark.runtime.memory("coder")
    memory.observe("Use FastAPI dependency injection for routes.")
    result = memory.retrieve_sync("How should routes be implemented?")
```

Middleware is local and Apache-2.0-safe. Optional integrations can be attached
through explicit hooks or caller-supplied clients.

Middleware batteries are opt-in. Without a middleware instance in the runtime,
MARK stores and retrieves through the core memory primitives only; governance,
trust-bus publication, sandbox execution, media continuity defaults, sync
envelopes, compression, expansion, and automatic observation structuring are
not activated.

Available local middleware includes recall, observe/writeback, compression,
query expansion, governance, lifecycle, observability, sync, trust bus,
sandbox, gap healing, and media continuity wrappers. Each middleware lives in
its own package under `mark.middlewares`.

## Contributing & support

- **Found a bug? Have an idea?** [Open an issue](https://github.com/emsoftanalytics/MARK-SDK/issues/new/choose) —
  the templates take two minutes, and "this surprised me" reports are welcome too.
- **Want to contribute?** Start with [CONTRIBUTING.md](CONTRIBUTING.md) and the
  [`good first issue`](https://github.com/emsoftanalytics/MARK-SDK/labels/good%20first%20issue) label.
  Contributions should be made from a separate branch and submitted through a
  Pull Request for review before they land on `main`. Draft PRs and questions
  are encouraged.
- **Want to know where MARK is going?** See [ROADMAP.md](ROADMAP.md) for the
  current SDK roadmap and contributor-friendly work areas.
- **Using MARK in a project?** Tell us in
  [Discussions](https://github.com/emsoftanalytics/MARK-SDK/discussions) — real
  workloads drive the roadmap.
- **Security issues:** see [SECURITY.md](SECURITY.md) — please report privately.

## License

Apache-2.0 — see [LICENSE](LICENSE).
