Metadata-Version: 2.4
Name: mirrorneuron-python-sdk
Version: 1.2.31
Summary: MirrorNeuron Python SDK
License-Expression: MIT
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: grpcio>=1.82.1
Requires-Dist: protobuf>=7.35.1
Provides-Extra: dev
Requires-Dist: build>=1.0; extra == "dev"
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: ruff<0.16,>=0.8; extra == "dev"
Requires-Dist: twine>=5.0; extra == "dev"
Dynamic: license-file

# MirrorNeuron Python SDK

`mn-python-sdk` provides the Python gRPC client and workflow-bundle helpers used
by the CLI, API, and Python-defined workflows.

## Blueprint payload contract

`mn_sdk.payload_contract` validates blueprint-owned skill and agent source
packages or wheels under `payloads/skills` and `payloads/agents`. It also
validates GGUF, Safetensors, and DDUF declarations below `payloads/models`,
including auxiliary files and SHA-256 digests. Payload packages override exact
matching GAR dependencies.

`iter_bundle_assets` and `stage_payload_assets` stream large files into
content-addressed storage; model files are never eagerly buffered into the
submission payload. `package_payload_models` prepares physical model payloads
with Docker Model Runner. Submission preparation renders a bundled agent index
and injects payload Python packages into HostLocal environments. When a
HostLocal upload source imports `mn_sdk`, submission preparation stages the
SDK package beside that source so the isolated worker receives the same client
contract used to prepare its job. Compiled bundles marked
`python_source_mode=false` keep their generated dependency-light source
fallback and are not expanded with the full SDK at submission time.

## Built-in LLM and runtime model access

`mn_sdk.llm` provides `LLMClient.from_env()`, text and strict JSON completions,
retries, deterministic fallbacks, and token usage accounting. The lower-level
`mn_sdk.model_access.runtime` wrapper is shared by LLM, RAG embedding, and OCR
calls.

Prepared blueprint submissions stage run-store writes in shared storage and
register a run-scoped copy-back to the configured local `MN_RUNS_ROOT`.
Terminal copy-back observes the authoritative source tree until it is stable
and recopies late-arriving files, preventing a completed local run directory
from retaining only a partial artifact set.

Managed Docker Model Runner models are selected and prepared on first use.
Logical `default` prefers Nemotron 3 on a healthy compatible cluster node and
otherwise uses catalog fallback Gemma 4. Non-LLM consumers pass a complete
model specification to the generic wrapper; for example, the RAG and OCR
skills own their concrete model, backend, context, and hardware requirements.
Those models are not SDK catalog defaults or blueprint declarations. Successful
bindings are cached per worker process; the native runtime service supplies
install single-flight and LiteLLM routing. External OpenAI-compatible, LiteLLM,
and Ollama endpoints bypass installation.

Blueprint actor configuration preserves `docker_model_runner` as the provider
for managed models. LiteLLM is the node-local gateway transport after model
preparation, not a replacement provider contract that bypasses preparation.

The node-local LiteLLM gateway applies one shared FIFO admission queue across
parallel workers and model routes. It defaults to one active request and 64
waiters, so excess calls wait instead of starting enough local decoders to
starve Core. Chat/completion calls and embeddings use separate bounded FIFO
lanes so long local decodes cannot starve short RAG requests. Configure them
with `MN_LITELLM_MAX_CONCURRENT_REQUESTS`,
`MN_LITELLM_MAX_CONCURRENT_EMBEDDINGS`, `MN_LITELLM_MAX_QUEUED_REQUESTS`,
`MN_LITELLM_QUEUE_TIMEOUT_SECONDS`, and `MN_LITELLM_MAX_SLOT_SECONDS`. Queue transitions are emitted as structured
`runtime_llm_request_*` log events and responses report queue wait time in the
`x-mn-llm-queue-wait-ms` header.

Runtime model preparation uses the dedicated
`MN_RUNTIME_MODEL_PREPARE_TIMEOUT_SECONDS` deadline (20 minutes by default), so
first-use model downloads are not cut off by the general 10-second gRPC client
deadline.

DockerWorker model selection prefers the worker's actual execution node before
considering installation reuse or capacity on another node. This keeps a
node-local workflow on its selected machine and returns the reachable
`mn-litellm-proxy` address. If that node cannot satisfy the model requirements,
selection may use another compatible cluster node and its advertised gateway
address. Core supplies the actual worker placement as `MN_EXECUTION_NODE`; that
value takes precedence over the submission-time `MN_SELECTED_RUNTIME_NODE`.
Submission preparation also injects the Docker-reachable
`MN_RUNTIME_MODEL_CONTROL_TARGET` into managed DockerWorker nodes so their
first-use preparation RPC reaches Core instead of container loopback.

## Declarative Step Handlers

`mn.workflow.source/v2` manifests declare direct DAG dependencies with `needs`
and select Python behavior modules with `run.handler`:

```json
{
  "id": "research",
  "needs": ["intake"],
  "run": {
    "handler": "my_blueprint.steps.research",
    "with": {"operation": "company_identity"}
  }
}
```

The module defines `run()`; the manifest does not need a `:run` suffix. During
expansion, the standard `blueprint` profile configures each handler-backed
worker to run `python3 -m mn_sdk.step_runtime`. The SDK entrypoint invokes only
the scheduler-selected handler and passes a `StepContext` containing the step
id, run id, attempt metadata, incoming message, and embedded config.

A logical step can instead reference a Python `StepSpec`. Registry entries own
immutable agent handlers and parameters, while the step module owns its input
contract, output contract, and internal collaboration graph:

```json
{
  "agents": {
    "registry": {
      "extractor": {"handler": "my_blueprint.agents.extractor"},
      "normalizer": {"handler": "my_blueprint.agents.normalizer"}
    }
  },
  "workflow": {
    "steps": [{
      "id": "prepare",
      "needs": [],
      "run": {"definition": "steps.prepare"}
    }]
  }
}
```

```python
from mn_sdk.step_graph import (
    InputSpec,
    OutputSpec,
    StepSpec,
    agent,
    flow_output,
    run_input,
    sequence,
)

STEP = StepSpec(
    input=InputSpec(fields={"document_folder": run_input("document_folder")}),
    flow=sequence(
        agent("extractor", as_="extract"),
        agent("normalizer", as_="normalize"),
    ),
    output=OutputSpec(fields={"company_evidence": flow_output()}),
)
```

The compiler expands each logical step into a start boundary, its internal
agent graph, and an end boundary. It supports `sequence`, all-required
`parallel`, `choice` with a
default, `fallback`, and `bounded_loop`. Workflow edges connect only a previous
step's end boundary to the next step's start boundary. Agent handlers use
`receive_input(context)` and `send_output(...)`; Redis routing, retries,
fan-out, and fan-in remain outside agent code.

Blueprint runtime contexts keep persisted metadata across retries. If a saved
run or output path is not mounted in the current runner, the SDK retains the
currently resolved path instead; this keeps durable state portable across
DockerWorker, OpenShell, and host execution boundaries.

## Quick Start

Install locally and run tests:

```bash
python3.11 -m venv .venv
. .venv/bin/activate
.venv/bin/python -m pip install -e ".[dev]"
.venv/bin/python -m pytest -q
.venv/bin/python -m ruff check .
```

Minimal client example:

```python
from mn_sdk import Client

client = Client(target="localhost:55051")
print(client.list_jobs(limit=5))
```

### Stable jobs and multiple runs

Use the v2 methods when work must retain configuration or data across
executions:

```python
import json
from mn_sdk import Client

client = Client(target="localhost:55051")
job = json.loads(client.create_stable_job(manifest_json, payloads))
first = json.loads(client.start_run(job["job_id"], inputs={"source": "manual"}))
second = json.loads(client.start_run(job["job_id"], inputs={"source": "scheduled"}))

assert first["run_id"] != second["run_id"]
```

`job_id` owns the durable definition and `$MN_HOME/job-data/<job-id>`.
`run_id` owns one execution and all control/observability calls. Retries keep
their run ID and use `attempt_id`. `RuntimeService` exposes create, get, list,
update, archive, reset-data, delete, start/list-run, run-control, run-delete,
and stable-job schedule adapters for `mirrorneuron.job.v2.JobService`.
`update_stable_job(..., manifest_json=..., payloads=...)` atomically replaces
an inactive job's executable bundle. The graph and blueprint identities must
match; job data, schedules, and prior run records are preserved.
Transport-size exhaustion is reported separately from runtime resource
pressure, so an oversized server response is not presented as a busy cluster.

Running jobs can expose narrowly scoped commands through
`contracts.live_inputs`. Callers submit only the public input ID and payload;
the manifest supplies the entrypoint and message type:

```python
accepted = json.loads(
    client.send_run_input(
        first["run_id"],
        "steer_monitoring",
        {"instruction": "Watch the loading dock", "analyze_now": True},
        idempotency_key="operator-command-001",
    )
)
```

The manifest compiler validates each live-input ID, object schema, declared
entrypoint, and permitted message route. Shared blueprint-support dashboards
derive their controls from those declarations and can optionally publish a
credential-free HLS preview through Core's supervised media-relay node.

Runtime payloads must read `MN_RUN_ID` for execution identity. The SDK no
longer treats `MN_JOB_ID` as a fallback run ID. `MN_JOB_ID`, `MN_RUN_ID`,
`MN_ATTEMPT_ID`, and `MN_JOB_DATA_DIR` therefore have distinct meanings.

## Details

- [MirrorNeuron Component Guide](../mn-docs/component-guide.md#python-sdk)
- [Python SDK](../mn-docs/SDK.md)
- [Job Bundle Format](../mn-docs/bundle.md)
- [Environment Variables](../mn-docs/env_variables.md)

`prepare_job_submission(..., env=...)` accepts an explicit environment mapping
for local DockerWorker preparation. This lets adapters enable diagnostic Docker
build output or provide runtime-local settings without changing process-global
environment variables.

Prepared submissions preserve stable-job, run, and attempt identity as
separate fields. Run stores and staged artifacts are keyed by `run_id`; durable
job data is mounted by Core and is never synthesized from a caller-provided
host path.
The caller generates `job_id` before preparation. Shared storage and
DockerWorker services use a job-scoped definition-revision ID, never a
`run_id`, so ordinary starts and scheduled dispatches reuse the stored
definition resources.

When that mapping contains `MN_SELECTED_RUNTIME_NODE`, DockerWorker preparation
treats it as the workflow's hard placement target. This is required for
source/v2 manifests because DockerWorker nodes are generated after the CLI has
made the hardware-fitness decision. The SDK resolves the selected node from the
injected/current cluster reports and uses its native SDK client; it does not
fall back to a local Docker build when the selected node is remote.
The shared workflow resolver also applies a hard `node.name` constraint to
every executor and to every source, sink, join, router, aggregator, or other
control node generated during lowering. Nodes with a divergent or read-only
coordination store are not placement candidates.

When the mapping also enables `MN_DEBUG` or `MN_BLUEPRINT_DEBUG`, that intent is
carried through remote native-SDK preparation. The returned DockerWorker service
record includes the build action, image, command, context digest, and complete
captured output so the calling CLI can show remote Docker build diagnostics.

Small DockerWorker build contexts are sent directly to the selected node's
native SDK so a newly joined cluster does not block behind an unrelated
Syncthing backlog. Only the build-context subtree is included in that native
request. The default bound is 3 MiB, below the standard gRPC message ceiling.
Larger contexts continue to use digest-verified shared-storage staging. Set
`MN_DOCKER_WORKER_INLINE_PAYLOAD_MAX_BYTES` or
`MN_DOCKER_WORKER_INLINE_CONTEXT_MAX_BYTES` to tune the bounds; set either to
`0` to force shared-storage staging.

The boundary is service-free in unit tests: inject `cluster_client`,
`native_client_factory`, and `command_runner` into
`prepare_docker_worker_compose_services`. The focused regression is:

```bash
../mn-system-tests/.venv/bin/python -m pytest -q \
  tests/test_native_resources.py \
  -k "selected_runtime_node or cuda_docker_worker"
```

## Source Manifests

Blueprints may use `apiVersion: mn.workflow.source/v1` for a compact,
CSS-like `manifest.json` that declares intent and overrides while SDK profiles
provide common defaults. Generate the executable runtime manifest with:

```bash
mn-manifest-converter expand manifest.json --output build/manifest.executable.json
mn-manifest-converter check manifest.json --against build/manifest.executable.json
```

Pre-submission validation accepts `input_validation` either as a source-level
section or under the source manifest's `manifest` section. Both forms are
validated before source expansion and runtime preparation. In local-development
mode, command validators can import only the local skill sources declared by
that manifest.

The CLI/API expand source manifests automatically before validation and
submission. Expansion returns the schema-valid catalog form; submission
preparation then lowers it to Core's compatibility topology, including
`graph_id`. Existing `mn.workflow/v1` executable manifests continue to work.

For source/v2 blueprints, `config.manifest_defaults` can expose authoritative
manifest descriptors through resolved runtime configuration without copying
them into `config/default.json`. A dotted string keeps the same path; a mapping
projects it to another config path:

```json
{
  "config": {
    "manifest_defaults": [
      "llm",
      {"from": "requirements", "to": "resources"}
    ]
  }
}
```

Manifest values are merged first, followed by the default config file and the
invocation overlay. Both manifest compilation and `load_runtime_config()` use
this order.

### DAG dependencies and trigger rules

`workflow.requires` and `workflow.provides` compile into runtime DAG edges.
Declare an explicit `workflow.edges` list when an edge needs a custom event or
otherwise cannot be inferred from a provided capability. A step can declare a
runtime trigger at `trigger_rule` (or `control.trigger_rule`): `all_success`,
`all_done`, `one_success`, `one_done`, `one_failed`,
`none_failed_min_one_success`, or `quorum_success` with a positive `quorum`.
The generated manifest places these under `flow.steps` and `flow.graph.edges`,
which are consumed by the Core workflow ledger.

## Configuration

Configuration is loaded by `mn_sdk.config` in this order:

```text
real environment variables
> .env.${MN_ENV}
> .env
> built-in safe defaults
```

`MN_ENV` defaults to `dev` when unset. `MN_ENV=development` loads `.env.dev`;
`MN_ENV=test` loads `.env.test`; `MN_ENV=prod` or `MN_ENV=production` loads
`.env.prod` when present. Production does not require any `.env` file.

Development example:

```bash
export MN_ENV=dev
cp .env.example .env.dev
mn-cli ...
```

Test example:

```bash
export MN_ENV=test
mn-cli ...
```

Production example:

```bash
export MN_ENV=production
export MN_HOME=/var/lib/mirrorneuron
export MN_LOG_LEVEL=info
export MN_API_HOST=0.0.0.0
export MN_API_PORT=8080
mn-api ...
```

### Model catalog overrides

The SDK uses the packaged `mn_sdk/model_catalog.json` as its baseline catalog.
If present, `$MN_HOME/models/catalog.json` is loaded next; `$MN_HOME` defaults
to `~/.mn`. Entries are deep-merged by model ID, so an external entry can
override selected fields while unmentioned built-in models remain available.

Set `MN_MODEL_CATALOG_PATH` to load a final, highest-priority catalog file.
The file may be a model list, an object with a `models` list, or an object keyed
by model ID. Paths support `~`, `$MN_HOME`, and normal environment-variable
expansion.

For example, this changes the bundled Gemma model endpoint and adds a new
catalog entry without copying the entire packaged catalog:

```bash
mkdir -p "$MN_HOME/models"
cat > "$MN_HOME/models/catalog.json" <<'JSON'
{
  "models": [
    {
      "id": "gemma4:e2b",
      "model": "local/gemma4:E2B",
      "requirements": {"min_vram_gb": 4}
    },
    {
      "id": "my-local-model",
      "model": "local/my-model",
      "aliases": ["my-model"]
    }
  ]
}
JSON
```

Catalog precedence is:

1. Packaged `mn_sdk/model_catalog.json`.
2. `$MN_HOME/models/catalog.json`, when present.
3. `MN_MODEL_CATALOG_PATH`, when configured.

Matching entries are merged by `id`. Nested objects are merged recursively;
scalar values and lists from the higher-priority catalog replace lower-priority
values. A malformed existing catalog raises a validation error rather than
being silently ignored.

Do not commit real `.env` files. Use `.env.example` for placeholders only, and
put secrets in real environment variables or token files.

## Notes

- A running MirrorNeuron core is required for live client calls.
- Constructor arguments take precedence over environment variables.
- Generated protocol modules are included with the package.

### Durable group operations

`Client.start_operation(kind, options)` starts a server-defined durable bulk
operation and returns its JSON snapshot. Use `get_operation(operation_id)` to
read the latest state and `stream_operation_events(operation_id,
after_sequence=..., follow=True)` to replay/continue progress after a detach.
The operation kinds are `cancel_all_jobs`, `clear_jobs`, `reconcile_node`, and
`drain_node`; target selection and concurrency remain Core-owned.

`cancellation_pending` is an accepted cancellation result: the Core has fenced
the old owner and queued cleanup for its rejoin. It is not a per-item failure.
