Metadata-Version: 2.4
Name: gpuhive
Version: 0.1.7
Summary: RL-based runtime abstraction for transparent multi-GPU CUDA execution
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: numpy>=1.23
Requires-Dist: gymnasium>=0.29
Requires-Dist: jinja2>=3.1
Requires-Dist: requests>=2.28
Provides-Extra: gpu
Requires-Dist: nvidia-ml-py>=12.0; extra == "gpu"
Provides-Extra: plots
Requires-Dist: matplotlib>=3.6; extra == "plots"

# RL-Split / GPUHive

**An intelligent reinforcement-learning based runtime abstraction layer for
transparent multi-GPU CUDA execution.**

Final year project, Department of Computer Science & Engineering, University of
Moratuwa. Supervisor: Dr. Adeesha Wijayasiri.
Team GPUHive — Kajaluxan S. (220297K), Parishith R. (220444K),
Sukithan T. (220625R), Vinushaanth S. (220668B).

---

## The problem, in one paragraph

A machine with several GPUs does not present itself as one larger GPU. CUDA and
NCCL provide every *mechanism* needed to spread one task across several devices,
and no *policy* for deciding how. So programmers pick devices, split ratios,
chunk sizes and communication paths by hand — and the right choices depend on the
kernel, the input size, which devices are currently busy, and which pairs happen
to be well connected. Equal splitting, the practical default, is wrong the moment
the devices differ. This project puts a learned policy where that hand-tuning
goes.

## What it does

```
                 ┌──────────────────────────────────────────────┐
   application → │  DECISION PLANE      RL agent, live telemetry │
                 └──────────────────┬───────────────────────────┘
                        GATE 1 ─── Security & Feasibility Validator
                 ┌──────────────────┴───────────────────────────┐
                 │  decision-parameter contract  {gpus, shares,  │
                 │   chunk, block, mode, comm_path, config_hash} │
                 └──────────────────┬───────────────────────────┘
                 ┌──────────────────┴───────────────────────────┐
                 │  CODE-GEN PLANE     LLM + hash-sealed cache   │
                 └──────────────────┬───────────────────────────┘
                        GATE 2 ─── Code Security & Integrity Gate
                 ┌──────────────────┴───────────────────────────┐
   result     ←  │  EXECUTION PLANE    CUDA / NCCL, hidden       │
                 └──────────────────────────────────────────────┘
```

The agent never issues a CUDA call and never writes source. It emits a small
declarative object; everything downstream consumes only that object. That
separation is the project's architectural claim, and it is what lets every
baseline policy in the evaluation drive the *same* execution path — so a measured
difference is attributable to the decision and nothing else.

## New here? Read [GUIDE.md](GUIDE.md)

[`GUIDE.md`](GUIDE.md) is the complete walkthrough written for someone picking
this up cold: what the project is, how a task flows through it, what every file
does, every command explained, and how to run it on the university server. Start
there.

## Try it now — no GPU required

```bash
git clone <this repo> && cd gpuhive
make install          # or: . scripts/dev_env.sh   (no install, works in place)

gpuhive doctor                                    # what this machine can do
gpuhive explain --kind matmul --dim M=8192 --dim N=8192 --dim K=8192
gpuhive run --kind reduce --dim N=67108864 --verify
gpuhive security-audit
gpuhive evaluate --repeats 10 --no-optimum
```

Every command works on a laptop with no GPU, against a simulated machine, and
**says so** in its output. Nothing reports a modelled number as a measurement.

## Repository layout

Twelve independently installable packages sharing the `gpuhive` namespace, plus
the CUDA layer and the workload suite. Each package has its own README explaining
what it is for and which part of the proposal it implements.

| Package                                             | Proposal             | What it is                                                                          |
| --------------------------------------------------- | -------------------- | ----------------------------------------------------------------------------------- |
| [`gpuhive-core`](packages/gpuhive-core/)           | Sec. 3.6.1, Fig. 7   | The decision-parameter contract, task descriptors, telemetry schema, state encoding |
| [`gpuhive-telemetry`](packages/gpuhive-telemetry/) | Sec. 3.3 (3), M3     | NVML monitoring, measured interconnect matrix, machine characterisation             |
| [`gpuhive-costmodel`](packages/gpuhive-costmodel/) | Gap G5, M4           | Eq. (1) made executable, calibration with reported error, exhaustive optimum        |
| [`gpuhive-security`](packages/gpuhive-security/)   | Sec. 2.8, Fig. 4     | Gate 1 (validity), Gate 2a (code scan), Gate 2b (integrity), the injection suite    |
| [`gpuhive-exec`](packages/gpuhive-exec/)           | Sec. 3.3 (8)         | The hidden execution plane: CUDA, CPU multi-device, and cost-model backends         |
| [`gpuhive-codegen`](packages/gpuhive-codegen/)     | Sec. 3.3 (6,7)       | LLM providers, deterministic templates, compiler, hash-sealed cache                 |
| [`gpuhive-env`](packages/gpuhive-env/)             | Sec. 3.5, M4         | Gymnasium environment, Eq. (2) reward, training curriculum                          |
| [`gpuhive-agent`](packages/gpuhive-agent/)         | Sec. 3.5.5, M5–M7   | Masked PPO with a Dirichlet allocation head, TD3, bandit, three-stage pipeline      |
| [`gpuhive-baselines`](packages/gpuhive-baselines/) | Sec. 3.9.3           | All seven comparison policies, driving the same execution path                      |
| [`gpuhive-runtime`](packages/gpuhive-runtime/)     | Sec. 3.6.2, Fig. 8   | The`Agent` entry path that walks a task down the spine                            |
| [`gpuhive-eval`](packages/gpuhive-eval/)           | Sec. 3.9, M10        | Shared measurement protocol, Table 4, ablations, transfer, targets                  |
| [`gpuhive-cli`](packages/gpuhive-cli/)             | —                   | One command per milestone                                                           |
| [`cpp/`](cpp/)                                     | Sec. 3.3 (8), Fig. 8 | `libgpuhive_exec.so` and the `gpuopt::Agent` C++ header                         |
| [`workloads/`](workloads/)                         | Sec. 3.9.1           | The five divisible workload classes, with references and CUDA kernels               |

## Using it from an application

Python, and the C++ header in [`cpp/include/gpu_optimizer/agent.h`](cpp/include/gpu_optimizer/agent.h)
mirrors it field for field:

```python
from gpuhive.runtime import Agent
from gpuhive.core import TaskSpec

agent = Agent("artifacts/policies/rl-split-hetero4/policy.npz")

task = TaskSpec(kind="matmul", dims={"M": 100_000, "N": 100_000, "K": 100_000},
                dtype="fp16", divisible_dim=0, merge="concat")

params = agent.optimize(task)     # decision plane  -> a declarative object
result = agent.execute(task, params)   # execution plane -> the merged result
```

`optimize` and `execute` are separate on purpose. A configuration can be
inspected, logged, diffed between policy versions, or replaced by a hand-written
one — and the execution layer will run any of them.

## Running it on the departmental server

The project targets a server with **one GPU today and a second one later**, and
nothing about that transition requires redoing work.

```bash
# 0. One command that does all of the below, and checks it worked:
bash scripts/server_setup.sh

# ...or do it by hand:

# 1. Build the CUDA execution layer (once).
make -C cpp                       # or: make -C cpp NCCL=1
export GPUHIVE_EXEC_LIB=$PWD/cpp/build/libgpuhive_exec.so

# 2. Measure the machine. This is milestone M3 and it is real data.
gpuhive characterise --machine nvml --backend cuda \
    --out artifacts/characterisation.json

# 3. Train. Stages 1 and 2 need no GPU at all, so they can run anywhere;
#    only stage 3 touches the hardware.
gpuhive train --machine uni-server-1gpu --pretrain-steps 500000 \
    --finetune-steps 20000 --finetune-backend cuda

# 4. Evaluate against every baseline, on hardware.
gpuhive evaluate --machine nvml --backend cuda --policy artifacts/policies/…/policy.npz
```

**With one GPU** the multi-device actions are masked out, so the framework
exercises decision, validation, code generation and execution end to end, and the
comparison it can make is single-device against single-device. That is a real but
limited result, and the framework labels it as such.

**With two GPUs** the interesting comparisons open up. The two cards will be
identical, so heterogeneity has to be *induced* — by clock locking or calibrated
background load, per Sec. 3.9.1 — and the framework keeps genuine and induced
settings separate throughout:

```bash
gpuhive evaluate --machine uni-server-2gpu-induced   # clock-locked, reported as induced
gpuhive evaluate --machine hetero-4                  # simulated genuine heterogeneity
```

Note honestly what induced heterogeneity cannot show: clock locking scales
arithmetic and memory throughput together, so it does **not** change which kernel
each device is relatively good at. The kernel-dependent device ordering that
motivates the whole project (Sec. 1.2) is only observable on genuinely different
cards.

## What is honest about the numbers

This matters more than any individual result, so it is stated plainly:

- **Simulated is labelled simulated.** Every `ExecutionMetrics` carries a
  `backend` field; every results table prints whether it was MEASURED on hardware
  or PREDICTED by the cost model; the Markdown report puts a warning block at the
  top of a predicted run.
- **Measured interconnect is distinguished from estimated.** `TopologyProbe`
  carries `measured=True/False` and the two are never pooled.
- **The cost model reports its own error.** `gpuhive calibrate` prints median
  APE, p90 APE and rank correlation, and warns when the error is above 30%.
- **The targets can fail.** `gpuhive evaluate` checks Sec. 3.8's falsifiable
  targets and prints PASS or FAIL with the measured value either way.
- **The security gates are measured against benign controls too.** A gate that
  rejects everything scores 100% coverage and is useless, so the audit reports
  false positives and cases caught by the *wrong* check.

## Development

```bash
make test          # the full suite
make lint          # syntax and import checks across every package
make audit         # security validation coverage
make demo          # train briefly, then evaluate, end to end
make cpp           # build the CUDA execution layer
make cpp-stub      # build the CPU ABI stub (no CUDA needed)
```

The test suite runs in seconds and needs no GPU. It includes gradient checks
against finite differences for every autodiff operation and every action
distribution, because hand-written backward passes are exactly the kind of code
that is wrong in a way ordinary tests do not notice.

## Documentation

- [`docs/architecture.md`](docs/architecture.md) — how the planes fit together and why
- [`docs/proposal-map.md`](docs/proposal-map.md) — every section, objective and gap of the proposal, mapped to code
- [`docs/deployment.md`](docs/deployment.md) — the one-GPU to two-GPU path, in detail
- [`docs/experiments.md`](docs/experiments.md) — how to reproduce every result in the evaluation plan
- [`docs/findings.md`](docs/findings.md) — what building it revealed that the proposal did not anticipate; several of these are results

## Status

Implemented end to end and running: the contract, both security gates, the cost
model and its calibration, all three execution backends, the CUDA/NCCL layer, the
code-generation plane with four LLM providers and a deterministic fallback, the
Gymnasium environment, masked PPO with the Dirichlet allocation head, all seven
baselines, the runtime, and the evaluation harness with ablations and transfer
studies.

Not yet done, and requiring hardware: measured characterisation, cost-model
calibration against real timings, hardware fine-tuning, and every number in the
evaluation reported as measured rather than modelled.
