Metadata-Version: 2.5
Name: morbidlabs
Version: 0.1.0
Summary: A local-LLM benchmark bench that records what actually happened, including what it could not measure
Project-URL: Homepage, https://github.com/vansh-comprint/local-llm
Project-URL: Source, https://github.com/vansh-comprint/local-llm
Author-email: Vansh <vansh@ctsipl.co.in>
Keywords: benchmark,gpu,llama-cpp,llm,ollama,telemetry,vllm
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: System :: Benchmark
Requires-Python: >=3.11
Requires-Dist: duckdb>=1.3.0
Requires-Dist: fastapi>=0.135.0
Requires-Dist: httpx>=0.27
Requires-Dist: psutil>=5.9
Requires-Dist: qrcode>=7.4
Requires-Dist: uvicorn
Provides-Extra: dev
Requires-Dist: httpx; extra == 'dev'
Requires-Dist: httpx2; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Description-Content-Type: text/markdown

# Morbid Labs

A local-LLM benchmark bench that records what actually happened on this machine —
**including what it could not measure**.

```bash
npx morbidlabs up
```

It configures itself, discovers what is already running, greys out what this box
cannot do *with a reason*, and launches a benchmark from your browser while the
numbers move. Every run is stored, stamped and comparable across runs and machines.

## It exists to not lie

Most benchmark tools report a number whatever happens. This one reports a number, or
it reports `—` and why.

- **A missing measurement is `null` with a reason, never `0`.** A gauge reading 0 W
  is a lie; a gauge reading `—` beside `no_unprivileged_sensor` is a fact.
- **UNKNOWN, blocked and not-attempted are three different things.** Collapsing them
  turns "never ran" into "can't run".
- **Two series measured at different scopes are never drawn on one axis.** GPU-rail
  watts beside CPU-package watts passes every null check and is still meaningless,
  so `/compare` refuses the overlay instead of drawing it.
- **Every run is commit-stamped**, with a dirty flag — so you can tell "this exact
  tree" from "this commit, plus uncommitted edits of unknown content".

## The six tests

| | | |
|---|---|---|
| **A** | Concurrency ramp | prefill t/s against *concurrency* — the curve nothing else plots |
| **B** | Context-depth decay | what depth costs, measured against what you asked for |
| **C** | KV budget | bytes per token, and where the box actually stops |
| **D** | Sustained hour | drift, thermals and tokens-per-watt with an honest `power_scope` |
| **E** | Memory spill | a model larger than RAM, running — slow is the content |
| **F** | Determinism gate | run this first: a throughput number off an unstable box is noise |

Run **F first**. **C and E last** — both deliberately walk into the freeze zone.

## The dashboard is a recording instrument

It is built to be screen-recorded while a benchmark runs, so it refuses to perturb
the thing it is measuring: paint is capped at 30 fps through one rAF loop, SSE events
never touch the DOM directly, and there is no `backdrop-filter`, no animated
background and no webfont fetch. *A dashboard that perturbs the benchmark invalidates
the benchmark.*

## Engines

Benchmarks whatever is already listening — Ollama, LM Studio, llama-server, vLLM —
and **never restarts a server it did not start**. With `--manage-engine` it launches
one itself, which is what lets Test C sweep context size and Test F sweep `-np`.

## Install

```bash
npx morbidlabs up          # node launcher; installs uv if needed
uvx --from morbidlabs morbidlabs up
pip install morbidlabs && morbidlabs up
```

Requires Python 3.11+. `up` prints a Local URL, a Network URL and a QR code, each
gated by a per-launch token.

## Status

Pre-1.0 and honest about it. Six harnesses, a dashboard, a compare view and an
engine-lifecycle manager are built and tested; publishing, cross-machine verification
and the Spark-side measurements are in progress. Numbers this suite has actually
taken are recorded in `docs/superpowers/STATE.md`, with the ones that are **not**
citable marked as such and why.
