Metadata-Version: 2.4
Name: nativegate
Version: 0.1.7
Summary: Make legacy C++/C/Fortran AI-ready and microservice-ready: expose native code as an MCP server, REST API and container
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: click>=8.1
Requires-Dist: pyyaml>=6.0
Requires-Dist: rich>=13.7
Provides-Extra: clang
Requires-Dist: libclang>=16; extra == "clang"
Provides-Extra: fparser
Requires-Dist: fparser<0.3,>=0.2; extra == "fparser"
Provides-Extra: docs
Requires-Dist: mkdocs>=1.6; extra == "docs"
Requires-Dist: mkdocs-material>=9.5; extra == "docs"
Provides-Extra: build
Requires-Dist: scikit-build-core>=0.10; extra == "build"
Requires-Dist: pybind11>=3.0; extra == "build"
Requires-Dist: numpy>=1.26; extra == "build"
Requires-Dist: fastapi>=0.110; extra == "build"
Requires-Dist: uvicorn>=0.29; extra == "build"
Requires-Dist: meson>=1.1; extra == "build"
Requires-Dist: ninja>=1.11; extra == "build"
Provides-Extra: test
Requires-Dist: pytest>=8.0; extra == "test"
Requires-Dist: httpx>=0.27; extra == "test"
Requires-Dist: setuptools>=68; extra == "test"
Requires-Dist: build>=1.0; extra == "test"
Requires-Dist: wheel>=0.42; extra == "test"
Requires-Dist: fastmcp>=3.0; extra == "test"
Requires-Dist: anyio>=4.0; extra == "test"

# nativegate — make legacy C++ and Fortran AI-ready and microservice-ready

[![CI](https://github.com/ravikings/nativegate/actions/workflows/ci.yml/badge.svg)](https://github.com/ravikings/nativegate/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/nativegate)](https://pypi.org/project/nativegate/)
[![Python](https://img.shields.io/pypi/pyversions/nativegate)](https://pypi.org/project/nativegate/)
[![MCP server (Glama)](https://glama.ai/mcp/servers/ravikings/nativegate/badges/score.svg)](https://glama.ai/mcp/servers/ravikings/nativegate)

Expose existing C++ and Fortran code to Python as deployable microservices —
without hand-writing bindings, and without touching the numerics.

Point it at a header or a Fortran source and it generates the pybind11/f2py
bindings, the CMake build, an installable Python package, a FastAPI service,
smoke tests, a numerical regression baseline, and a Dockerfile.

```bash
pip install "nativegate[clang,build]"   # from PyPI; needs cmake/ninja + a C/C++/Fortran toolchain for the build step

ngate quickstart libraries/geometry/geometry.hpp --name demo --build
pip install services/demo/dist/*.whl
```

```python
from demo import Geometry

Geometry().hypotenuse(3.0, 4.0)   # 5.0
```

...plus a FastAPI service over the same binding:

```bash
ngate serve demo
curl -X POST "http://localhost:8000/hypotenuse?a=3&b=4"
```

Built for re-hosting the kind of code that runs an engineering business:
1990s C++ over F77, fixed-form decks with COMMON blocks and INCLUDE files,
PVT correlations nobody wants to rewrite and nobody can afford to get wrong.
The worked example of that in this repo is `services/petro_api`, generated
from the Fortran decks in `libraries/petro/`.

## Install

```bash
cd tools/nativegate
./scripts/bootstrap.sh            # creates .venv, installs everything
```

Or piecemeal, into an environment of your own:

```bash
pip install -e ".[clang,build,test,docs]"
```

| Extra | What it adds |
|---|---|
| `clang` | libclang — the C++ AST parser. Without it, C++ falls back to a much weaker regex reader |
| `build` | scikit-build-core, pybind11, numpy, fastapi, uvicorn — needed to build generated services |
| `test` | pytest, httpx |
| `docs` | mkdocs + material |

Python 3.10+. `ngate build` also needs a system toolchain that pip cannot
install: `cmake`, `ninja`, a C++ compiler, and `gfortran` for Fortran services.

### Global install (`ngate` on your PATH everywhere)

To get the `ngate` command in every shell/project without activating a
venv, install this checkout editable with [pipx](https://pipx.pypa.io):

```bash
python3 -m pip install --user pipx
python3 -m pipx ensurepath        # adds ~/.local/bin to PATH, restart your shell after
pipx install -e tools/nativegate   # run from the repo root
```

`-e` keeps it editable, so changes under `nativegate/` take effect immediately
— no reinstall needed. Verify with `ngate --version` from any directory.

If `pyproject.toml`'s `dependencies` list changes (e.g. a new package is
added), `pipx install -e` won't pick it up on its own — the venv pipx made
already exists. Re-run the install to rebuild it:

```bash
pipx reinstall nativegate
```

## Using it in another project

nativegate is an ordinary pip package with a console entry point. Nothing in it
refers to this repo: `services/` and `libraries/` resolve against your
**current working directory**, so "use it elsewhere" means install it into
that project's environment and run it from that project's root.

```bash
cd /path/to/other-project
python3 -m venv .venv
.venv/bin/pip install "nativegate[clang,build,test] @ git+https://github.com/ravikings/nativegate.git@<sha>#subdirectory=tools/nativegate"

.venv/bin/ngate quickstart src/mylib.hpp --build
```

`quickstart` scaffolds `services/mylib` itself — no separate `init` needed
unless you also want the empty `libraries/`/`infrastructure/` directories.
`--build` compiles the wheel in the same run. Progress prints as a live
checklist (scaffold → copy source → generate bindings → build):

```
✔ Scaffold service
✔ Copy native source
✔ Generate bindings & package
✔ Build wheel

Done — services/mylib is ready.
```

Pin `@<sha>` (or a tag). This is a code generator: an unpinned upgrade can
change the bindings under a service that was working, and you want that to be
a deliberate commit rather than a surprise on someone's laptop.

Installing from a local checkout works the same way, and is what you want
while changing the tool itself:

```bash
pip install -e "/path/to/haliburtion/tools/nativegate[clang,build,test]"
```

Three things worth knowing before rolling it out to a team:

- **Extras are not optional in practice.** Without `clang` the C++ front end
  silently falls back to the regex reader and your macro-defined and
  `#include`d symbols quietly vanish — set `parser: clang` in
  `nativegate.yaml` so a mis-provisioned machine fails loudly instead. Without
  `build`, `ngate build` cannot run; without `test`, `ngate test`
  cannot.
- **Run from the project root.** `libraries:` and `include_paths:` in
  `nativegate.yaml` resolve from there too.
- **Generated services do not depend on nativegate.** `services/<name>/` is a
  self-contained wheel plus a FastAPI app — hand it to a team that has never
  installed the generator and it still builds.

## Use it as an agent skill

The tool ships a Claude Code skill at `tools/nativegate/skill/SKILL.md`, so a
coding agent reaches for `ngate` instead of hand-writing a binding. This
repo already exposes it — `.claude/skills/nativegate` is a symlink to that
directory, so editing `SKILL.md` updates the installed skill with no copy step.

To install it in another project:

```bash
mkdir -p .claude/skills
cp -r /path/to/nativegate/tools/nativegate/skill .claude/skills/nativegate
# or, to track the tool: ln -s /path/to/nativegate/tools/nativegate/skill .claude/skills/nativegate
```

The skill covers what the agent has no way to infer from the CLI's `--help`:
that `suggest` exists so it should not read headers itself to pick a starting
file, that a `regex reader` line means the parse is missing symbols and must
not be trusted, that skipped-binding reasons should be relayed rather than
summarized, and that generated files are rewritten every run so hand-edits are
lost. It also carries the production-readiness gap list, so the agent raises
the process-wide call lock and the single-tenant boundary unprompted rather
than handing over a service as if it were ready for anything.

### Why an agent should use it: the token argument

For a human the saving is hours. For an agent it is context, and context is
the binding constraint on whether the task finishes at all.

Hand-writing the binding for one 20-method C++ class means *emitting* the
pybind11 translation unit, `CMakeLists.txt`, `pyproject.toml`, the package
`__init__`, a FastAPI app and a test module — on the order of 500 lines, call
it 7–10k output tokens if every line lands correctly on the first attempt.
They do not. Each compile or link error costs another round of reading the
error, re-reading the file, and re-emitting a corrected version; three or four
rounds is normal for template-adjacent code, and `include/`-vs-`src/` layout
and undefined-symbol-at-import failures are exactly the kind of thing that
burns several. A realistic figure is **20–40k tokens per file**, most of it
output, and all of it resident in context afterwards — crowding out the actual
engineering question the agent was asked.

The same work through nativegate is one command and a four-line checklist:
roughly **300–500 tokens**, and the generated files never enter context
because the agent has no reason to read them. Those are estimates from the
size of the generated artifacts, not a measured benchmark — but the ratio is
not a close call, and it is structural rather than a matter of prompting the
agent better.

Three effects beyond the raw count:

- **The repair loop disappears, not just shrinks.** The generator emits code
  that already compiles for the constructs it accepts, and refuses the ones it
  cannot bind *with a reason*. The agent never enters the read-error →
  re-emit → rebuild cycle that dominates hand-written binding cost, and never
  spends a round discovering that a header does not compile.
- **`golden verify` is a verification signal the agent cannot fake.** Left to
  itself, an agent asked whether a re-hosted library still works writes tests
  from its own reading of the source — which validates its interpretation, not
  the numbers. `golden record` / `golden verify` gives a real pass/fail on the
  answers for a couple of hundred tokens.
- **Correctness stops depending on how much context is left.** A hand-written
  binding degrades as the window fills — later methods get less attention than
  earlier ones. Generated output is identical at the start of a session and at
  80% context.

## Which file do I point it at?

For a codebase you did not write, `suggest` parses every header it can find
and ranks them by how cleanly they would bind — preferring self-contained
files, because a good first service is one that drags nothing else in:

```
$ ngate suggest libraries/petro/cpp

      file                        binds                     skipped  notes
  ~   include/Units.hpp           11 fn, 1 class, 9 method        1  Units.cpp
  ~   include/WellModel.hpp       1 class, 21 method                 WellModel.cpp; needs FortranBridge.hpp
  ~   include/FluidModel.hpp      1 class, 18 method, 1 struct       FluidModel.cpp; needs FortranBridge.hpp
  ~   include/Simulator.hpp       1 class, 24 method             10  Simulator.cpp; needs DeckReader.hpp, FluidModel.hpp +2
  ✘   include/FortranBridge.hpp   2 fn, 3 struct                 26  no .cpp found

Start with libraries/petro/cpp/include/Units.hpp:

  ngate quickstart libraries/petro/cpp/include/Units.hpp --name units --build
```

`✔` binds everything it declares and needs no other header; `~` binds more
than it skips; `✘` binds nothing usable. Dependencies outrank skip count in
the ranking — one skipped array method costs you that method, while one
`#include` of another subsystem costs you that subsystem's whole build.

If the header line says *regex reader* rather than *clang AST*, install the
parser first (`pip install "nativegate[clang]"`) and re-run: the fallback
reader silently misses free functions and anything behind a macro, so the
ranking will be scored on an incomplete picture.

## The loop

Run everything from the repo root — commands resolve `services/<name>/`
relative to your working directory.

```bash
ngate init                            # optional: empty libraries/, infrastructure/ dirs
ngate suggest src/                    # which file should I start with?
ngate quickstart src/pvt.hpp --build  # scaffold + expose + generate + build, one shot
ngate inspect src/pvt.hpp             # what the parser sees, before generating
ngate generate pvt                    # re-run codegen after editing nativegate.yaml
ngate build pvt                       # pip wheel . -> scikit-build-core -> CMake
ngate test pvt                        # the generated pytest suite
ngate golden record pvt               # pin the numbers (commit golden.json)
ngate golden verify pvt               # prove a rebuild returns the same answers
ngate lock pvt                        # pin deps by version + SHA-256
ngate docker pvt --build              # multi-stage image, non-root, healthcheck
ngate k8s pvt                         # -> infrastructure/kubernetes/pvt.yaml
ngate gateway platform-api --service pvt
```

Every generated service and gateway also serves an [MCP endpoint](docs/mcp.md)
at `/mcp`, so the same native routines are callable as tools by Claude or any
other MCP client. It is the same routes — FastMCP derives one tool per route
from the router's OpenAPI schema — so the tool surface cannot drift from the
HTTP surface, and the COMMON-block lock still applies.

## What it parses

**C++** — via Clang (libclang), the same front end that compiles the code.
The preprocessor runs, so `#include`d types, `#define`d names, `#ifdef`
branches and macro-mangled `extern "C"` bridge declarations resolve. Handles
classes, structs, overloads, public inheritance, constructors, typedefs and
`using` aliases, enums, namespaces, `std::string`, static methods, abstract
classes, `= default` / `= delete`. Also: `std::vector<T>` of scalars,
`const char*` inputs, operators bound as Python special methods (`__add__`,
`__eq__`, `__getitem__`, …), and a raw `T*` paired with a length argument
bound as a numpy buffer — including on class methods.

Refused *with a reason*, never silently: templates (a typedef does not help),
a non-const `std::vector<T>&` (pybind11 would discard what you write into it),
a non-const `char*` (an output buffer with no length convention), a raw
pointer with no length argument to pair it with, pointer returns of class
types (ownership), and types only forward-declared. A header that does not
compile is refused outright rather than half-bound — clang recovers from an
unknown type by pretending it was `int`, and binding that produces a service
whose signatures disagree with the C++ it calls.

`extern "C"` functions whose bare `T*` arguments are single scalars passed by
reference (the Fortran-linkage convention) can be opted in per function with
`clang.scalar_ref_functions`. It is opt-in and never inferred, because an
array whose extent lives in a COMMON block or PARAMETER looks identical in a C
prototype. It applies to free functions under the clang backend only.

**Fortran** — you name the routines you want, and a large legacy deck costs
only what you expose. Fixed-form F77 and free-form F90+, INCLUDE expansion,
kind-parameter resolution (`real(dp)` → `real(8)`), inferred argument intent,
module-nested routines, array arguments, and `.F90`/`.F` sources run through
`gfortran -cpp` first.

Two newer Fortran features are narrower than they sound, so they are worth
stating precisely. **Derived types** are bound by generating a flattening
shim, but only for a *subroutine*, in *free-form* source, parsed by the
*fparser2* backend, where the routine is inside a module, the type is defined
in the same file, and every component is a scalar `real`/`integer`/`logical`.
**CHARACTER outputs** are bound only when the declaration fixes the length
(`CHARACTER*80`); an assumed-length output is demoted to an input and
reported, because f2py builds it and then silently returns an empty string.
That screening currently runs on the fixed-form path only.

Everything the parser recognises but cannot bind is reported at `generate`
time with the reason — the failure mode that costs the most time is a header
with thirty methods producing a module with twenty-six and nothing saying
which four are missing.

## Verification

For re-hosted engineering code, "it builds and imports" is not the acceptance
criterion. Three gates sit beside each other, each answering a different
question that the others structurally cannot:

| | asks | compares against | catches |
|---|---|---|---|
| `golden.json` | did it change? | its own past | compiler/flag drift, a regenerate picking a different overload, a refactor of the native source |
| `oracle` | is it faithful? | the legacy binary | transposed arrays, wrong intent, narrowing, units, argument-order swaps, a wrong first recording |
| `invariants.json` | is it possible? | mathematics | wrong away from the sample point, NaN at domain edges, hidden process-global state |

Run every layer at once, in the order CI runs them — oracle first (a
faithful binding is a precondition for the other two meaning anything), then
golden, then invariants — with each layer reported by name so one layer's
failure never masks another's:

```bash
ngate verify pvt
# oracle: passed (9 covered, 0 skipped)
# golden: passed
# invariants: passed
```

### Layer 1 — golden.json: did the answer change?

```bash
ngate golden record pvt  # services/pvt/golden.json — commit this
ngate golden verify pvt  # 4 entry point(s) unchanged (0 not covered).
```

`golden.json` records every bound entry point, the inputs it was called with,
and the answer. Edit the inputs to values your engineers recognise (a real
pressure, a real API gravity); they survive future re-records. The generated
`tests/test_golden.py` replays the same calls inside the service's own suite,
so CI catches drift with no extra wiring. Changing a conversion constant from
`14.5037738` to `14.5038` fails the check — a change no compiler, type
checker or unit test would flag.

Its limit is structural: golden only asserts about the one input tuple it
recorded, and only proves the binding is *unchanged*, never that the first
recording was *right*.

### Layer 2 — oracle: is the binding faithful to the legacy code?

```bash
ngate oracle check pvt
# 9 covered, 0 skipped
```

`oracle check` generates a driver **in the original language** from
`golden.json`'s recorded entries (never from a fresh sample plan), compiles
only the driver translation unit, links it against the extension's own built
objects (never a second compilation of the library sources), runs both
paths in the same build, and compares every observable value **bitwise** —
16 hex digits of IEEE-754, no tolerance, because the same machine code
produced both sides. It needs a compiler; it needs no committed file, and
regenerates the driver on every run so a stale one can never pass. A failing
comparison is classified (argument order, transposition, `float32`
narrowing, a missing in-place output, ...) so the fix is obvious from the
message rather than from a difference alone.

### Layer 3 — invariants.json: is it possible?

```bash
ngate invariants verify pvt
# 6 function(s) checked, 1 uncovered (services/pvt/invariants.json):
#   solution_gor: pass (6 propert(y/ies), 33 point(s))
#   [uncovered] tubing_bhp: no range declared for parameter(s) rate
```

Declare what a function's answer must satisfy in `nativegate.yaml`:

```yaml
state:
  setup: [pvt_set_fluid]     # replayed before every property evaluation
  mutating: [pvt_set_fluid]  # this routine's job IS to change state
  error_flag: last_error

invariants:
  solution_gor:
    - bounds: {min: 0.0}
    - monotone: {in: pressure, direction: nondecreasing}

ranges:
  pressure: [14.7, 10000.0]  # swept, 33 points, inclusive
```

`invariants verify` checks two kinds of property over a fixed, declared
lattice (a per-parameter sweep, never random, never a cartesian explosion):
**structural** properties (`finite`, `total`, `no_error_flag`, `idempotent`,
`order_independent` — derived from the IR plus the `state:` declaration, no
domain knowledge needed) and **declared** properties (`bounds`, `monotone`,
`sum_to_one` — a closed, non-`eval`'d vocabulary authored by a human). A
failing declared property reports the first failing lattice point and
bisects to the tightest bracket it can prove. A swept parameter with no
declared range is not skipped silently — it is recorded under `uncovered`,
and **a run whose `checked` block comes out empty is a hard failure**, never
a quiet pass: an invariants file that checks nothing must not look like one
that does.

## Layout

```
nativegate/
  cli.py            the CLI (click)
  ir.py             language-neutral IR — the seam every layer meets at
  config.py         nativegate.yaml
  golden.py         numerical regression harness
  discovery.py      language/dialect detection
  preprocess.py     INCLUDE expansion and gfortran -cpp for .F90/.F
  suggest.py        ranks candidate sources
  locking.py        dependency pinning for requirements.lock
  parsers/
    cpp.py              front door: picks a C++ backend, reports which ran
    cpp_ast.py          Clang AST parser (primary)
    cpp_regex.py        token/brace reader (fallback, no libclang needed)
    fortran.py          front door: picks a Fortran backend
    fortran_fparser.py  fparser2 parse tree (primary)
    fortran_regex.py    targeted per-routine reader (fallback)
    fixed_form.py       F77 fixed-form helpers (IMPLICIT, CHARACTER lengths)
  generators/       pybind11, f2py, CMake, Python package, FastAPI router,
                    middleware, error handling, tests, golden test,
                    Dockerfile, gateway, MCP server, Kubernetes, pyproject
```

Parsers normalise source into `ir.ModuleIR`; generators consume it without
knowing which language it came from. Both C++ backends emit the identical IR,
which is why swapping the regex reader for a real compiler front end changed
nothing below `parsers/`. The Fortran side has since gone the same way: the
fparser2 backend replaced the regex reader as the default, and the regex
reader is still there for machines without the package.

## Developing

```bash
.venv/bin/python -m pytest tests -q                           # 694 tests
NATIVEGATE_CPP_PARSER=regex .venv/bin/python -m pytest -q      # C++ fallback backend
NATIVEGATE_FORTRAN_PARSER=regex .venv/bin/python -m pytest -q  # Fortran fallback backend
.venv/bin/python -m mkdocs serve                              # docs at :8000
```

The suite runs green under both C++ backends — that is the contract that lets
either one be selected. Beyond unit tests, both language paths are verified by
really building them: generated CMake fed to actual `cmake`/`clang++`/
`gfortran`, the resulting extension imported, the FastAPI service exercised
over HTTP. Two real bugs (f2py module nesting, the missing-`.cpp` link
failure) were found that way and not by inspection.

## Docs

`docs/`, or `mkdocs serve`:

- [Getting started](docs/getting-started.md) — C++ end to end
- [Exposing C++](docs/cpp-guide.md) / [Exposing Fortran](docs/fortran-guide.md)
- [Numerical regression](docs/golden-values.md)
- [Troubleshooting](docs/troubleshooting.md)
- [nativegate.yaml reference](docs/configuration.md) · [CLI reference](docs/cli-reference.md)
- [Architecture](docs/architecture.md) — the IR, the parser seam, and what is
  generated versus what is yours
- [Deployment topologies](docs/deployment-topologies.md) — one image per
  service, or one composed gateway
- [MCP endpoint](docs/mcp.md) — calling the native routines as tools from an
  LLM client

## Before you ship

**Honest positioning: this is usable today for internal, single-tenant
deployments, where a team is putting an HTTP or Python interface on legacy
code it already trusts. It is not ready for internet-facing or multi-tenant
use.**

The compile-and-serve path is real and verified. Generated services do now
carry API-key auth (`api.auth: api_key`), rate limiting, a request size cap,
request IDs and access logging, an exception handler, `/healthz` and `/readyz`
with SIGTERM draining, and Kubernetes manifests via `ngate k8s`.

What remains, and why it is not just a to-do item:

- **Fortran endpoints are serialised process-wide.** Each holds one lock
  across the native call, because COMMON blocks are process-global storage and
  two concurrent requests that configure different states would otherwise read
  back each other's numbers silently. Throughput scales with processes, not
  threads — do not plan capacity as if these endpoints were concurrent. C++
  services are not locked (a fresh instance per request covers the usual
  case), which also means a C++ library holding file-scope static state has no
  protection here and the parser cannot detect that it needs any.
- **Stateful libraries get a session-shaped API** — "configure, then read".
  That is fine when one tenant owns the process and wrong when it does not.
- **`MAX_ARRAY_ITEMS` is a memory guard, not a bounds check.** Emitted for
  services that take array arguments. The IR records that a parameter is an
  array, not the extent the routine actually declares, so one configurable cap
  (`NATIVEGATE_MAX_ARRAY_ITEMS`, default 65536) stands in for all of them. A
  real extent check waits on the fparser2 front end.
- **A segfault still takes the worker down.** No sandbox, no per-call timeout,
  no per-call isolation.
- No CI/CD templates; `infrastructure/docker/` is an empty skeleton.

Read [Is this production-ready?](docs/production-readiness.md) and
[DEFECTS.md](DEFECTS.md) before a paying workload touches this. Those are
honest gap lists, not a sales pitch.

### Refusals are part of the contract

When the parser recognises a symbol but will not bind it, that is a decision,
not a gap it forgot to fill: a non-const `char*` output buffer has no length,
a pointer return has no ownership, a derived-type Fortran result has no
mapping. Each refusal is printed at `generate` time with its reason, and the
generated service publishes them at `GET /_unexposed` so they stay visible to
whoever is calling the API. An empty mapping there means this build refused
nothing; a 404 means the service predates the route and cannot tell you.
