Metadata-Version: 2.4
Name: rustygate
Version: 0.1.19
Classifier: Programming Language :: Rust
Classifier: License :: Other/Proprietary License
Requires-Dist: fastcore>=2.2.11
Requires-Dist: pytest ; extra == 'dev'
Requires-Dist: pytest-timeout ; extra == 'dev'
Requires-Dist: httpx ; extra == 'dev'
Requires-Dist: websockets ; extra == 'dev'
Requires-Dist: jupyter-client ; extra == 'dev'
Requires-Dist: jupygate ; extra == 'dev'
Requires-Dist: jupyasyncclient>=0.2.16 ; extra == 'dev'
Requires-Dist: ipymini ; extra == 'dev'
Requires-Dist: miniapl>=0.1.4 ; extra == 'dev'
Provides-Extra: dev
Summary: Jupyter kernel gateway with terminals, files, and cells APIs, as a single binary
Author: Jeremy Howard
Maintainer-email: "fast.ai" <infos@fast.ai>
License: Proprietary
Requires-Python: >=3.10
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Homepage, https://github.com/AnswerDotAI/rustygate
Project-URL: Issues, https://github.com/AnswerDotAI/rustygate/issues
Project-URL: Repository, https://github.com/AnswerDotAI/rustygate

# rustygate

A Jupyter kernel gateway as one binary: kernels, terminals, files, search, and per-cell notebook operations, over the same REST and websocket API shapes `jupyter_server` uses.

## Running

    rustygate [--host H] [--port N] [--token T | --token-file PATH] [--trust-loopback] [--root DIR] [--tls] [--autorun DIR]

The defaults are 127.0.0.1, port 8787, no auth, and the working directory as the files root. With a token set, requests carry `Authorization: Bearer T`; Jupyter's `Authorization: token T` form and `?token=T` are also accepted. `--token-file` reads the token without putting it in the process arguments. `--trust-loopback` lets clients whose actual TCP peer is loopback omit the token. Do not use it behind an untrusted same-host reverse proxy, since the proxy is the TCP peer. Forwarded headers never affect authentication. `--tls` serves https and wss from a self-signed certificate generated at startup; clients opt out of verification (jupyasyncclient's `verify=False`).

The Python helper `rustygate.tools.start_gateway()` starts a background gateway with inherited stdout and stderr. Gateway and kernel startup errors appear in the calling process's console.

### Working directories

Notebook-bound kernels default to the notebook's parent directory under the served root. Folder-bound terminals default to their folder. Explicit `cwd` overrides either binding; relative values use the served root. Restart recomputes a kernel's default cwd from its current binding. See [DEV.md](DEV.md#kernel-lifecycle) for unbound processes and path rules.

`GET /api/kernels/{kid}/realpath?path=...` returns an on-disk path as a JSON string. Empty and relative paths use the bound notebook's parent, regardless of the kernel's cwd. A leading slash selects the gateway root. The target need not exist. Notebook-relative paths require a notebook binding; root-qualified paths need only an existing kernel. Existing symlinks and `..` resolve within the root; escapes and dangling symlinks are rejected. Notebook bindings separately require one canonical spelling, without symlink aliases.

### Notebook state and connections

Clients sharing a notebook share its kernel and authoritative notebook state. Accepted edits broadcast to every subscriber, including their author. Saving is debounced; contents reads, copies, search and command execution first save pending changes in their file or directory scope, then operate on disk normally.

`POST /api/kernels` with a `path` returns its existing non-dead kernel unchanged (200), ignoring startup options, or starts a new kernel (201). A concurrent creation still starting returns 409. Without a path, each request starts a new kernel.

A kernel id identifies the logical kernel across rename, restart and failed-restart retry. `GET/POST /api/kernels/{kid}/cells` operates on its current notebook binding. `PATCH /api/kernels/{kid}` with `{"path":"new.ipynb"}` renames that notebook. Deletion and new creation produce a new id. Path-addressed file and cell operations remain available for unopened notebooks.

Cell reads support `section=ID` for a heading and its descendants, `ancestors=ID` for enclosing headings, and `before=ID` or `after=ID` for every cell on one side of a cell. All use ordinary cell IDs and ignore collapse flags. `fields=id` returns IDs in cell dictionaries; `fields=id,source` returns IDs and sources. Omit `fields` for full cells without attachments, or use `fields=*` for all cell fields including attachments. `fields=*,meta` also returns notebook metadata. See [DEV.md](DEV.md#cells-api) for heading detection and selector composition.

Notebook settings use the same ordered ops: `{"ops":[{"op":"meta","metadata":{"solveit":{"mode":"learn"}}}]}` merge-patches notebook metadata without transferring or replacing cells and outputs. Accepted metadata broadcasts carry the resulting metadata, so subscribers need no follow-up read.

File and cell edits use last-accepted-edit-wins semantics, without hashes or stale-write checks. File writes and copies can require a new destination with `overwrite=false`; renames never overwrite. The watcher reloads valid external changes when the file's mtime changes.

Kernel traffic uses bounded queues with backpressure before notebook updates. A slow frontend's full queue closes its whole connection rather than silently dropping individual messages. Disconnected messages are not replayed: reconnecting clients load current notebook state. Kernel execution and persistence continue without connected clients.

### Startup notebooks

`--autorun AUTORUN` runs the direct `*.ipynb` children of `AUTORUN` under the files root once per gateway startup. Each notebook gets its own path-bound kernel, with its parent directory as the working directory. Nonempty code cells run in notebook order; outputs persist normally. Other cell types and CRAFT inheritance are not processed. Shared initialization belongs in explicit imports in the notebook.

Notebooks run independently in the background, without delaying HTTP readiness. A cell error stops the remaining cells in that notebook; errors are logged and its kernel remains available for inspection. Other notebooks and the gateway keep running. A missing directory is empty; unreadable directories or malformed notebooks produce an error. Attaching a client or restarting an individual kernel does not rerun AUTORUN. Omit the flag to disable it. `rustygate service install` accepts the same flag.

## Running as a service

`rustygate service` installs and controls a user service using launchd on macOS and systemd on Linux:

    rustygate service install --root ~/git --workdir ~/git/project
    rustygate service status
    rustygate service restart
    rustygate service stop
    rustygate service start
    rustygate service uninstall

`install` starts the service immediately; running it again replaces the configuration. Add `--host 0.0.0.0` for direct network access, `--token-file ~/.config/rustygate/token` for authentication, `--trust-loopback` when same-host clients should not need that token, and `--tls` when wanted. A missing token file is generated with user-only permissions. `--env-file` sources environment variables inherited by kernels. The service PATH is the rustygate binary's own directory (the venv bin, under a pip install) plus the standard system directories, so kernels find `python` despite the minimal launchd and systemd defaults. The default loopback service can instead be reached remotely through an SSH tunnel.

## Built-in Luau kernel

Start with `help()` in a Luau cell (or `create(kernel="luau")` then
`exec(code="help()")` through MCP). It
returns a bundled quick-start guide and API index; `help("ex.edit_file")` gives
function details, using the same documentation as Jupyter inspection. Inside a
longer cell use `print(help())` to display it. No external docs/install needed.

Create with `POST /api/kernels` and `{"name":"luau"}` (optionally `cwd` and
`path`). The gateway starts its own executable as
`rustygate native-kernel -f CONNECTION_FILE`; no Python, Lua installation, or
external search command is needed. `name="ipymini"` selects Python; omitting
both `name` and `argv` also defaults to ipymini in the REST API. A named
implementation cannot be combined with custom `argv`; unknown names are errors.

Native APIs include `rg.search`, `rg.find`, `fs.read_text`, `ex.edit_text`,
file/cell viewing and editing, `os.execute`, `io.popen`, `print`, and `display`. For example, in successive cells:

```lua
hits = rg.search("TODO", {root=".", exts={"rs"}, max_results=100})
```

```lua
#hits.matches
```

```lua
original = fs.read_text("notes.txt")
edited = ex.edit_text(original, "%s/TODO/DONE/g")
return edited.text
```

`ex.edit_text` edits private in-memory text. To edit files or notebook cells:

```lua
print(ex.view_file("notes.txt")) -- use addresses from this view
-- replace ADDRESS with a verified address from the view
-- ex.edit_file("notes.txt", {{ADDRESS, "s", "TODO", "DONE"}})
-- ex.edit_cell("analysis.ipynb", "cell-id-prefix", {{ADDRESS, "c", "new source"}})
```

File commands use arrays of strings (nested arrays for global subcommands),
like Python command tuples. `path:address` and `path.ipynb:cellid:address`
prefixes allow cross-file/cell moves and copies. `ex.edit_cell` also accepts
a compact script. Options are `{inplace=false}` for preview and `{sw=4}` for
shift width; writes are enabled by default. `ex.edit_file` returns
`{files, changed, diff}`; `ex.edit_cell` returns one result. Every command is
validated/applied before any writes; this is **not** an atomic multi-file
transaction. Notebook metadata, outputs, source form and trailing newlines
are preserved.

```lua
local success, kind, code = os.execute("printf hello | tr a-z A-Z")
local p = assert(io.popen("git status --short"))
local output = p:read("*a")
local success, kind, code = p:close()
return output
```

Both launch `/bin/sh -c` in the kernel cwd/environment; Bash-specific syntax
requires explicitly calling `bash -c`. `os.execute` publishes stdout/stderr
and returns Lua 5.4-style success/nil, exit/signal, and code. No-command
`os.execute()` checks shell availability. Commands are noninteractive: stdin
is closed unless using `io.popen(command, "w")`. Programs you invoke must
exist, but Python or a separate Lua install is not needed.

Pipes support `read`, `lines`, `write`, `flush`, `close`, and `seek` (which
returns a pipe-not-seekable error). Reads accept byte counts or `l`, `L`, `a`,
`n` (also `*l`, etc.), with multiple formats and binary-safe strings. Writes
are unbuffered and return the handle; `close` returns the command status.
This is not the full Lua `io` library: no `io.open`, default streams, or
`setvbuf`. `io.type` and `io.close(pipe)` are available. stderr and write-mode
stdout are forwarded during pipe operations. A pipe can persist across cells;
close it explicitly. GC, interruption and kernel shutdown terminate outstanding
processes; an interrupt closes all kernel-owned pipes.

Search records carry path,
line number, lnhash and match spans. Search/discovery return all results by
default; an optional `max_results` truncates them and sets `complete=false` when
results are omitted. There are no imposed byte or result-count caps. This is
trusted local scripting, not an OS sandbox or memory-bounded execution.

`fs.read_text` uses a standard UTF-8 file read with normal filesystem/encoding
errors and no custom size limit. Interrupts are checked before and after the
read; they do not abort an OS call in progress.

Globals/functions persist; `local` variables remain cell-local. A cell may be a
whole expression or an ordinary chunk with an explicit `return`. Stream output,
MIME display, errors, user expressions, basic host-API completion/help, and the
tail retrieval of session input history are supported. `display` takes a MIME bundle,
for example `display({['text/markdown']='**hello**'})`.

SIGINT and control-channel interruption share an execution-specific token with
the VM and native operations. Text edits run normally, with interrupt checks
before and after the call; an interrupt waits for the edit to finish and discards
its result (file writes that completed are not rolled back). Shell process groups
are killed and their direct children reaped on interruption. Search workers are cancelled and joined before execution completes. Earlier
statements are not rolled back. Individual regex/compiler/filesystem calls and
allocation/free remain cooperative boundaries, not hard real-time preemption.
Restart/delete still terminate the child independently.

This initial implementation has no IPython magics, debugger, typechecker/LSP,
subshells, or alternate interpreter backend. MCP uses `create(kernel="luau")`
followed by language-independent `exec` calls. Standalone executable
release artifacts and Linux validation are follow-up work. The design and
experiment record are in `meta/luau.md`.

## MCP

Choose a kernel implementation, then execute code in it:

```python
create(kernel="miniapl")
exec(code="avg←+/÷≢ ⋄ avg 2 4 9")
```

Available implementations are `ipymini` (Python/IPython), `luau` (bundled Luau),
and `miniapl` (APL). Miniapl must be installed on the gateway's PATH; it runs as
`miniapl --kernel -f CONNECTION_FILE`. New kernels require an explicit implementation; `exec` never creates or
switches kernels. `create(dlgname="work", kernel="luau")` binds a kernel to a
dialog. Omit `kernel` only to reuse an existing binding; an explicit
implementation must match it. Omit `dlgname` to create an unbound kernel.

Creation and selection return kernel-specific guidance, including Luau's
`help()` entry point. Creation, selection, and listing report both the
implementation and the language from `kernel_info_reply`. Python
startup/inspectors run only for Python kernels, including on restart. Clikernel
forwards the tools and arguments directly, including creation on named gateways.

`POST /mcp` is a stateful Streamable HTTP MCP server for Claude Code and other MCP clients. Its tools are `list_kernels`, `create`, `use_kernel`, `delete_kernel`, `exec`, `restart`, and `interrupt`. Code runs exactly as written; Python/IPython magics such as `%%bash` work in Python kernels. The MCP session remembers its current kernel, so no id is needed between calls. Call `create` or `use_kernel` before executing. Ending the session (`DELETE /mcp`) stops every kernel it created with autoclose, enabled by default; `autoclose=false` keeps a new kernel. Reusing or selecting an existing kernel never claims ownership. An execution may name another existing binding with `dlgname` for that call without changing selection. Clients may send creation defaults (cwd, env, Python startup code, quiet) in initialize params; see [DEV.md](DEV.md#mcp-api).

`POST /mcp/{path}` is the stateless alternative for one dialog. The URL-decoded relative path fixes the binding. It exposes `create(kernel?)`, `exec(code)`, and `interrupt()`. Creation requires an implementation when unbound; otherwise it reuses the kernel, checking any explicitly requested implementation. Execution and interrupt require an existing binding. With no owning session, these kernels persist until deleted or the gateway stops. Executions use priority 1, ahead of queued normal-priority notebook runs without preempting an active cell.

Kernels actually spawned through MCP receive `RUSTYGATE_LAUNCHER=mcp`, available during their startup profiles and preserved across restart. Selecting or get-or-creating a kernel that was already running does not alter its environment. See [DEV.md](DEV.md#mcp-api) for the tool and execution contract.

