Metadata-Version: 2.4
Name: shugonet
Version: 0.5.3
Summary: Networking integration for connecting multiple Shugocore agents together for collaboration between Shugocore agents in simulation and physical spaces, designed to be used over 5G, 4G, EDGE, LoRa, Wifi-Halow, Wifi, and Bluetooth networks.
Author: SamurAI-Official
License: MIT
Keywords: agent,networking,mesh,memory,fallback,shugocore
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: ws
Requires-Dist: websocket-client>=1.6; extra == "ws"
Provides-Extra: relay
Requires-Dist: requests>=2.32.0; extra == "relay"
Provides-Extra: serial
Requires-Dist: pyserial>=3.5; extra == "serial"
Provides-Extra: ble
Requires-Dist: bleak>=0.20; extra == "ble"
Provides-Extra: telemetry
Requires-Dist: opentelemetry-api>=1.20; extra == "telemetry"
Provides-Extra: postgres
Requires-Dist: psycopg2-binary>=2.9; extra == "postgres"
Dynamic: license-file

# Shogunet

> Networking integration for connecting multiple Shugocore agents together for
> collaboration between Shugocore agents in simulation and physical spaces,
> designed to be used over 5G, 4G, EDGE, LoRa, Wifi-Halow, Wifi, and Bluetooth
> networks.

[![PyPI](https://img.shields.io/pypi/v/shugonet)](https://pypi.org/project/shugonet/)
![Release](https://img.shields.io/badge/release-v0.5.3-blue)
![Python](https://img.shields.io/badge/python-3.9%E2%80%933.12-blue)
![Platform](https://img.shields.io/badge/platform-Linux%20%7C%20macOS%20%7C%20Android%20%28Termux%2FChaquopy%29-lightgrey)
![License](https://img.shields.io/badge/license-MIT-green)

## Installation

```bash
pip install shugonet
```

The core is dependency-free (stdlib-only). Optional extras:

```bash
pip install shugonet[postgres]   # PostgreSQL mesh backend (PgSemanticMemory parity)
pip install shugonet[ws]         # rosbridge-style WebSocket transport
pip install shugonet[relay]      # HTTPS relay transport (5G/4G/EDGE)
pip install shugonet[serial]     # LoRa SX126x/SX127x serial modules
pip install shugonet[ble]        # BLE transport
pip install shugonet[telemetry]  # OpenTelemetry integration
```

The compiled fleet dashboard ships inside the wheel — no JavaScript toolchain
is needed at install time.

Shogunet is the networking layer for [ShugoCore](https://github.com/SamurAI-Official/ShugoCore).
It lets a fleet of ShugoCore agents — running in Gazebo simulation, on servers,
on robots, or on Android handsets — discover each other, exchange tasks and
events, and consolidate a *codependent* memory mesh, over whatever networks the
mission has available, with deterministic fallback between them.

## Design principles

**One protocol, every network.** A single versioned envelope is spoken over all
transports, from 5G to LoRa. Two codecs exist for it: a JSON codec for
broadband links and a compact binary codec (16-byte header + TLV payload) for
constrained links such as LoRa (0.3–27 kbit/s, ~220-byte frames) and BLE.

**Memory invariants survive the network.** ShugoCore's memory tiers map to
network tiers: Tier 0/1 never leave the agent (N0). Tier 2 semantic facts sync
between paired agents (N1). Tier 3 is read-only everywhere: only *promotion
proposals* travel, and application stays operator-attributed (N2).

**Codependent memory.** Facts propagate with provenance
`(origin_agent_id, fact_id)`; salience merges additively; pruning propagates as
tombstones; and reinforcement is a *feedback loop* — when a peer's fact proves
useful, a tiny `reinforce` message flows back so useful memories survive decay
fleet-wide. Over LoRa, the loop runs on ~50-byte digest frames; bulk fact
content moves later over any broadband link (digest anti-entropy).

**Pairing is consent.** Only operator-allowlisted agent IDs pair, grants carry
TTLs, topics are namespaced (`/shugunet/{agent_id}/{topic}`), and every
ingress/egress field is sanitized and size-capped. Every network event lands in
a hash-chained audit log.

**Deterministic fallback.** Transports form an ordered, health-ranked chain
with per-transport circuit breakers. If the LAN drops, traffic falls to the
relay; if everything drops, a bounded store-and-forward WAL replays on
reconnect. Total transport exhaustion latches the ShugoCore governor through
its fallback triggers instead of failing silently.

## Network tiers

| Class | Networks | Protocol response |
|---|---|---|
| Broadband IP | 5G, 4G, WiFi | JSON codec, mesh query fan-out, bulk sync; LAN = direct TCP, cellular = relay hub |
| Midband IP | EDGE, WiFi-Halow | Compact codec, batching, latency-tolerant profiles |
| Constrained | LoRa | 16-byte header frames ≤ 220 B, P0/P1 priority only, digest anti-entropy, duty-cycle honored |
| Short-range | Bluetooth (RFCOMM/BLE) | Stream framing or segmented compact codec; pairing-based discovery |

Message priority classes: **P0** control (announce/heartbeat/ack/tombstone) →
all links · **P1** memory deltas (facts/reinforce/digests) → constrained-capable
· **P2** tasks and queries → IP links · **P3** bulk snapshots and audit
shipping → broadband only, deferred under low power / thermal pressure.

## Module map

| Module | Responsibility |
|---|---|
| `protocol.py` | Envelope schema, 16-byte binary header, JSON + compact TLV codecs, segmentation, validation |
| `transports.py` | `BaseTransport`, `LinkProfile` registry (7 networks + loopback), loopback bus |
| `link_simulator.py` | In-process impairment: bandwidth, latency, jitter, loss, MTU, duty cycle |
| `discovery.py` | Per-medium peer discovery (multicast beacons, hub rendezvous, pairing, duty-cycled LoRa) |
| `agent_registry.py` | Pairing = consent, allowlists, TTL grants, topic ACLs, sim/phys manifests |
| `tcp_transport.py` | Length-prefixed TCP transport with announce handshake (WiFi / WiFi-Halow profiles) |
| `relay_transport.py` / `relay_server.py` | Cellular path: HTTPS relay hub + long-poll client (5G/4G/EDGE, cross-NAT) |
| `lora_transport.py` | Serial SX126x/SX127x point-to-point transport (optional `pyserial`) |
| `bluetooth_transport.py` | RFCOMM (AF_BLUETOOTH) and BLE (optional `bleak`) transports |
| `transport_fallback.py` | Link-aware chain: eligibility, EWMA health ranking, breakers, QoS |
| `store_forward.py` | Bounded JSONL WAL outbox/inbox with replay + dedup |
| `memory_sync.py` / `mesh_query.py` | Codependent memory mesh: fact replication, conflict resolution, fan-out queries |
| `audit.py`, `fallbacks.py`, `policy.py`, `telemetry.py` | ShugoCore-aligned safety surface |
| `shugocore_bridge.py` | Loads a real ShugoCore checkout's primitives (`sanitize_text` / `redact` / `validate_url` / `AuditChain` / Tier-2 `SemanticMemory`), with stdlib-first fallbacks |
| `shugocore_adapter.py` | ShugoCore-side handler: registers the `network_*` action types on a `DecisionEngine` and merges the network fallback severities |

## Hosting a fleet

Shogunet runs as a single server process that owns the fleet's trust plane so
every ShugoCore agent only needs a ~20-line client runtime to join.

### Start a host

```python
from host import ShugonetHost

host = ShugonetHost(agent_id="fleet-1", tcp_port=9000, relay_port=9001)
host.start()

# Grant consent to a joining agent (pairing = consent)
host.pair("agent-001", manifest={"realm": "phys", "role": "perception"})

print(host.status())   # roster, alive count, breaker health, mesh counts
```

Or from the CLI:

```bash
python3 host.py --tcp-port 9000 --relay-port 9001
```

### Join from a ShugoCore process

```python
from shugonet_runtime import ShugonetAgentRuntime

runtime = ShugonetAgentRuntime(
    agent_id="agent-001",
    host_tcp_host="127.0.0.1",
    host_tcp_port=9000,
    host_relay_url="http://127.0.0.1:9001",
    on_message=lambda sender, msg: print(f"from {sender}: {msg}"))
runtime.connect_to_host()

# Send a task to another agent
runtime.send("agent-002", "/shugunet/agent-001/task",
             {"action": "scan", "zone": "north"})

# Search the fleet's memory
results = runtime.query("obstacle in zone north")

# Sync memory with the fleet
runtime.sync()

# Leave the fleet
runtime.stop()
```

### Driving Shogunet from a ShugoCore `DecisionEngine`

`shugocore_adapter.py` (vendored into a ShugoCore checkout as
`shugonet_bridge.py`) exposes the Shogunet network stack as an ordinary
execution-layer handler, so network actions clear the same consent/approval
gate as the robotics and mobile handlers:

```python
from shugonet_bridge import (
    attach_network_fallbacks,
    register_network_handlers,
)

register_network_handlers(engine.execution_layer, runtime)
attach_network_fallbacks(engine.fallback_controller)
```

Ten action types are registered — `network_send`, `network_query`,
`network_sync`, `network_spatial_observe`, `network_nrr_render`,
`network_nrr_scene` (side-effecting, consent-gated) and
`network_list_agents`, `network_status`, `network_spatial_query`,
`network_fleet_map` (read-only). The spatial/NRR actions require a runtime
that implements `publish_observation` / `query_nearby` / `get_fleet_map` /
`render_frame` / `query_scene`; against an older runtime they return
`{"status": "refused"}` rather than failing.

`register_network_handlers` patches **three** policy sets —
`NETWORK_ACTION_TYPES`, `NETWORK_READ_ACTION_TYPES` and
`KNOWN_ACTION_TYPES` — because `ExecutionLayer.register_handler` validates
against the first two. It mutates them in place (consumers hold references to
those objects) and registers each type independently, so one type the engine
refuses never costs you the rest. It returns the types that registered; only
a *total* failure raises, so `attach_network_fallbacks` still runs.

### Using ShugoCore's primitives (optional)

Point `SHUGOCORE_PATH` at a ShugoCore checkout to have Shogunet prefer
ShugoCore's hardened `sanitize_text` / `redact` / `validate_url` and
hash-chained `AuditChain` over its own equivalents:

```python
import shugocore_bridge

shugocore_bridge.configure()          # reads $SHUGOCORE_PATH
shugocore_bridge.shugocore_loaded()   # True only if the load is verified
```

Shogunet and ShugoCore both ship `security.py`, `audit.py` and `policy.py`,
so the bridge loads ShugoCore's modules by explicit file path under private
names and temporarily evicts the same-named entries from `sys.modules` (it
restores them afterwards). `shugocore_loaded()` verifies each module's
`__file__` really sits inside the checkout rather than trusting the import.
Without a checkout every primitive degrades to the local stdlib-first
equivalent and the mesh is unchanged.

### Cross-platform CLI client

### Spatial awareness (3D multi-agent tracking)

Shogunet includes a built-in spatial awareness system for tracking entities
across a multi-agent fleet in 3D space:

- **Spatial observations**: agents publish observations of entities (robots,
  obstacles, people) with 3D coordinates, confidence, and labels.
- **Octree spatial index**: fast sphere and AABB queries over the indexed
  volume. Thread-safe for concurrent insert/query.
- **Multi-agent fusion**: when multiple agents observe the same entity,
  confidence-weighted centroid fusion merges their views into a single
  consolidated position.
- **Position-enriched heartbeats**: agents can set their own position;
  it rides piggyback in heartbeat messages so the host tracks the whole
  fleet's positions without extra messages.
- **Coordinate frames**: transforms between local and world frames are
  shared via ``coordinate_frame`` messages.

CLI spatial commands:

```bash
# Observe an entity
shugonet-client observe robot-1 10.0 20.0 0.0 --label robot --confidence 0.95

# View the fleet's spatial map
shugonet-client map

# Locate a specific agent
shugonet-client locate agent-a

# Find what's near a point
shugonet-client nearby 10.0 20.0 0.0 5.0
```
Shogunet ships a standard CLI client that works on macOS, Linux, and Windows:

```bash
# Install (ships with the wheel)
pip install shugonet

# Connect and stay connected (foreground daemon)
shugonet-client run

# One-shot status
shugonet-client status

# One-shot send to a peer
shugonet-client send agent-002 /shugunet/agent-001/task '{"action":"scan"}'

# One-shot memory query
shugonet-client query "obstacle in zone north"

# One-shot digest sync
shugonet-client sync agent-002
```

The client is configured via environment variables:

| Variable | Default | Description |
|---|---|---|
| `SHUGONET_AGENT_ID` | hostname | Agent identifier |
| `SHUGONET_HOST` | 127.0.0.1 | ShugonetHost address |
| `SHUGONET_TCP_PORT` | 9000 | TCP port |
| `SHUGONET_RELAY_URL` | *(none)* | Relay hub URL |
| `SHUGONET_REALM` | phys | sim or phys realm |
| `SHUGONET_LOG_LEVEL` | info | Log verbosity |

Or use it programmatically:

```python
from shugonet_client import ShugonetClient

client = ShugonetClient(agent_id="robot-1", host="10.0.0.5", tcp_port=9000)
client.connect()
client.send("robot-2", "/shugunet/robot-1/status", {"battery": 87})
print(client.status())
client.disconnect()
```

### Routing model

Agents connect **to** the host (hub-and-spoke). The host forwards addressed
mail to its recipient and fans broadcasts out as addressed copies to every
other paired agent. Host-addressed mail — heartbeats, mesh queries, memory
broadcasts — is processed locally by the host's own chain handlers.

**Pairing is enforced twice** — at TCP admission (handshake hook) and again on
every host-processed or forwarded envelope (the relay path has no registry gate
of its own, so the host re-checks the sender there).

### Failure semantics

- **Agent crash/disconnect** → host governor latches `pause` (ShugoCore
  deterministic latch contract).
- **Message durability** → per-agent `OutboxStore` + `at_least_once` QoS + hub
  mailbox TTL bounds.
- **Cross-talk** → chain and mesh recipient guards (hardened in the
  concurrency suite) hold at host scale.

### Fleet dashboard

Every `ShugunetHost` can expose an operator console — a compiled single-page
app (TypeScript + SolidJS, built with Vite) served directly by the host over a
loopback HTTP port. No JavaScript toolchain is needed at install time: the
built assets ship inside the wheel (`shugonet_web/static`).

```python
host = ShugonetHost(agent_id="fleet-1", tcp_port=9000, relay_port=9001,
                    dashboard_port=9002)
host.start()
# open http://127.0.0.1:9002 in a browser
```

The console streams a live event feed (Server-Sent Events), shows the roster,
transport-chain health, memory-mesh counters, and audit-chain integrity, and
exposes pair / unpair / resume / broadcast controls. State-changing POSTs are
audited and can be gated by a token (`dashboard_token=...`).

The SPA source lives in `dashboard/`; rebuild it with:

```bash
cd dashboard && npm ci && npm run build
```

## Module map (continued)

| Module | Responsibility |
|---|---|
| `host.py` | `ShugunetHost`: admit paired agents, route traffic, seed the mesh |
| `shugonet_runtime.py` | `ShugonetAgentRuntime`: client half a ShugoCore process instantiates |
| `shugonet_client.py`  | `ShugonetClient`: cross-platform CLI and programmatic client |
| `spatial.py`          | `SpatialIndex`: octree-based 3D spatial index, fusion engine |
| `spatial_sync.py`     | `SpatialMemoryNode`: cross-agent spatial observation sharing |
| `dashboard.py` | `DashboardServer`: stdlib HTTP operator plane (REST + SSE + SPA) |
| `pg_store.py` | `PgFactStore`: optional PostgreSQL mesh backend (ShugoCore `PgSemanticMemory` parity) |

## Testing

```bash
python3 -m unittest discover -s tests -v
```

The suite is dependency-free and runs everywhere: transports are exercised
through the loopback bus and the in-process link simulator, which models every
network's bandwidth, latency, loss, MTU and duty-cycle constraints. The
integration suite spins up a real `ShugonetHost` with multiple threaded
`ShugonetAgentRuntime` clients, validates cross-talk isolation, memory
convergence, peer-lost latching, and unpaired-agent refusal.

Most of the suite is hermetic. Three modules are not, and are skipped
automatically when no ShugoCore checkout is present (`$SHUGOCORE_PATH`, or a
sibling `../Shugocore`):

| Test | What it covers |
|---|---|
| `tests/test_shugocore_integration.py` | Drives a **real** `ExecutionLayer` / `FallbackController` in a subprocess, with `sys.modules` deliberately shadowed, to prove registration and primitive delegation work end to end |
| `tests/test_bridge_sync.py` | Keeps `shugocore_adapter.py` and the vendored `shugonet_bridge.py` from drifting apart |
| `tests/test_memory_compat.py` | Fact-schema and embedding parity against a real `SemanticMemory` / `vector_db` |

The ShugoCore integration test runs out-of-process on purpose: module shadowing
and `sys.path` order are process-global, so an in-process test could not
reproduce the failure mode it guards against.

## Changelog

### 0.5.3

ShugoCore 1.30 compatibility. The `network_*` action registration was broken
against a real `DecisionEngine` and the failures were all silent.

- **Fixed: `register_network_handlers` registered nothing.** It patched only
  `policy.KNOWN_ACTION_TYPES`, but `ExecutionLayer.register_handler` validates
  against `NETWORK_ACTION_TYPES` / `NETWORK_READ_ACTION_TYPES` and raises
  `ValueError` for anything else. The adapter registers 10 types (5 more than
  ShugoCore's `policy` knows), so the call aborted on the fifth and left the
  engine with **zero** network handlers — and, because the documented
  `attach_network_fallbacks` call came after it, with no fallback severities
  either. Both sets are now patched, **in place**: `execution_layer` holds a
  reference to those objects and re-reads them per call, so a rebind would
  have left it validating a stale set forever.
- **Fixed: registration is no longer all-or-nothing.** Each type is registered
  independently; a refused type is logged and skipped. The function returns
  the types that registered, and raises only on *total* failure.
- **Fixed: the bridge never actually used ShugoCore's primitives.** Both repos
  ship `security.py` / `audit.py` / `policy.py`, and a module already in
  `sys.modules` wins over `sys.path` order — so `import security` returned
  Shogunet's own module while `shugocore_loaded()` reported `True`. Modules
  are now loaded by explicit file path under private names, with same-named
  `sys.modules` entries temporarily evicted (and restored) so ShugoCore's
  internal imports bind ShugoCore's copies. `shugocore_loaded()` now verifies
  each module's `__file__` is inside the checkout.
- **Fixed: `validate_url` silently fell back to the weak local check.**
  ShugoCore ≥1.30 exposes `validate_url(url, hosts, schemes) -> (ok, reason)`;
  the bridge still called the removed `allow_all=` keyword, and the resulting
  `TypeError` was swallowed — leaving the host allowlist unenforced and
  embedded URL credentials accepted. Both signatures are handled, the result is
  always `bool`, and a validator error now **fails closed**. The local fallback
  also rejects embedded credentials.
- **Fixed: `register_network_handlers` could patch the wrong `policy` module**
  when both trees are on `sys.path`; ShugoCore's is now resolved by file path.
- **Fixed: `nrr_adapter` was missing from the wheel.** `shugonet_runtime`
  imports it unconditionally, so a 0.5.3 build without it would have shipped a
  client runtime that raised `ModuleNotFoundError` on install. Added to
  `py-modules`.
- **Tests**: new `tests/test_shugocore_integration.py` (10 tests) drives a real
  `ExecutionLayer`/`FallbackController` in a subprocess with `sys.modules`
  deliberately shadowed — the first test in the suite able to reproduce these
  failures, since duck-typed doubles cannot raise `ValueError` or model
  import shadowing. Plus 6 hermetic tests for the registration semantics.
  Suite now 348 tests.
- **Known gap**: ShugoCore's `ShugonetAgentRuntime` does not yet implement
  `publish_observation` / `query_nearby` / `get_fleet_map` / `render_frame` /
  `query_scene`, so the five spatial/NRR actions register but return
  `{"status": "refused"}` at dispatch.
- **Known gap**: `host.py` is not in `py-modules`, so `pip install shugonet`
  does not ship the fleet host even though the *Start a host* section above
  shows `from host import ShugonetHost`. This predates 0.5.3 and is left as-is
  (the host is normally run from a checkout via `python3 host.py`); the client
  runtime, mesh and `shugonet-client` are unaffected. Tracked for a follow-up.
- Version bumped 0.5.2 → 0.5.3 (patch only — no wire-format change, so the
  0.5.x dialect peers keep interoperating).

### 0.5.0

- **ShugoCore 1.20.0 compatibility**: renamed ``agent_runtime.py`` to
  ``shugonet_runtime.py`` to avoid naming collision with ShugoCore's own
  ``agent_runtime`` module. All imports updated accordingly.
- **Cross-platform CLI client**: ``shugonet-client`` command (installed by
  ``pip``) with ``run``, ``status``, ``send``, ``query``, ``sync``,
  ``observe``, ``map``, ``locate``, and ``nearby`` subcommands.
- **Standard programmatic client**: ``shugonet_client.ShugonetClient`` wraps
  the full runtime in a simple ``connect()`` / ``send()`` / ``disconnect()``
  interface.
- **3D spatial awareness**: ``spatial.py`` provides ``SpatialIndex`` (octree
  with sphere/AABB queries and confidence-weighted fusion);
  ``spatial_sync.py`` provides ``SpatialMemoryNode`` for cross-agent
  observation sharing. Agent positions ride piggyback on heartbeats.
- **New protocol messages**: ``spatial_observation``, ``spatial_query``,
  ``spatial_response``, ``spatial_merge``, ``coordinate_frame``.
- Version bumped 0.4.0 → 0.5.0.

### 0.5.2

- **Hardening**: fixed truncated ``effective_confidence`` decay math, removed
  orphaned duplicate code paths, deduplicated ``status()`` in the runtime.
- **Observation dedup**: ``SpatialIndex.insert`` now skips identical payloads
  (content-based key) so re-delivery via broadcast + merge cannot inflate the
  index; surfaced as ``duplicates_skipped`` in ``stats()``.
- **Client API**: spatial CLI commands now use public ``ShugonetClient``
  methods instead of runtime internals.
- **42 new tests**: ``tests/test_spatial.py`` (27) and
  ``tests/test_spatial_sync.py`` (15) covering the octree index, fusion,
  cross-agent sync, dedup and the new protocol types. Suite now 301 tests.
- Version bumped 0.5.1 → 0.5.2.

### 0.5.1

- **3D spatial awareness**: ``spatial.py`` (octree SpatialIndex with
  sphere/AABB queries and confidence-weighted fusion), ``spatial_sync.py``
  (SpatialMemoryNode for cross-agent observation sharing). Agent positions
  ride piggyback on heartbeats.
- **New protocol messages**: ``spatial_observation`` (14), ``spatial_query``
  (15), ``spatial_response`` (16), ``spatial_merge`` (17),
  ``coordinate_frame`` (18).
- **Extended CLI**: ``shugonet-client observe``, ``map``, ``locate``,
  ``nearby`` subcommands for spatial awareness.
- **Host/runtime integration**: ShugonetHost maintains fleet-wide spatial
  index; ShugonetAgentRuntime enriches heartbeats with position data.
- Version bumped 0.5.0 → 0.5.1.

### 0.4.0

- **Fleet dashboard**: stdlib HTTP operator plane (`dashboard.py`) with a
  compiled TypeScript + SolidJS SPA (`dashboard/` → `shugonet_web/static`).
  Serves JSON REST, a live SSE event stream, and the console; ships inside the
  wheel so no JS toolchain is needed at install time.
- **ShugoCore memory compatibility**: fact schema aligned with
  `SemanticMemory` / `PgSemanticMemory` columns (`kind`, `metadata`,
  `created_at`); `pg_store.PgFactStore` mirrors mesh facts into PostgreSQL
  (optional `postgres` extra).
- **Version handshake**: `shugonet_version` + `protocol_version` exchanged at
  TCP admission; mismatches emit a `version_mismatch` audit event and refuse
  the join.
- **Bridge sync test**: `tests/test_bridge_sync.py` keeps Shogunet's
  `shugocore_adapter` and ShugoCore's vendored `shugonet_bridge` from drifting.
- Version bumped 0.1.0 → 0.4.0.
