Metadata-Version: 2.4
Name: infrune
Version: 0.1.1
Summary: Measured inference performance intelligence
Project-URL: Homepage, https://pypi.org/project/infrune/
Author: Infrune
License-Expression: LicenseRef-Proprietary
License-File: LICENSE
Keywords: benchmark,inference,llm,optimization,performance,vllm
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Requires-Dist: aiohttp>=3.9
Requires-Dist: click>=8.0
Description-Content-Type: text/markdown

# Infrune

Measured inference performance intelligence.

Get optimized vLLM server configurations backed by benchmark evidence, then verify the improvement on your own hardware.

## Supported (v0.1)

- **Engine**: vLLM 0.25.1
- **Model**: Meta-Llama-3-8B-Instruct
- **GPU**: NVIDIA A100 80GB
- **Config**: Infrune Config 108

Other engine/model/GPU combinations are not yet validated.

## Evidence

Infrune Config 108 achieved +24.1% mean throughput versus factory defaults in our historical A100 validation (9/9 workloads won). That experiment used the documented benchmark settings, including sampling parameters. Your measured result may differ. Run `infrune recommend` for full details including methodology notes.

## Install

```bash
pip install infrune
```

## Quick Start

```bash
# Get a measured config recommendation
infrune recommend --engine vllm --model llama-3-8b --gpu a100

# Benchmark your current setup
infrune benchmark --save current

# Restart your server with the recommended config, then:
infrune benchmark --save infrune

# Compare results
infrune compare current infrune
```

## Commands

| Command | Description |
|---------|-------------|
| `infrune recommend` | Get optimized server config for your setup |
| `infrune benchmark` | Benchmark a running endpoint |
| `infrune compare` | Compare two benchmark results |
| `infrune list` | List saved benchmarks |
| `infrune supported` | Show supported engine/model/GPU combinations |

## How It Works

Infrune ships a curated database of server configurations derived from large-scale benchmark sweeps (LLMSYS-HPOBench). These configurations optimize infrastructure parameters only — your sampling behavior (temperature, top_k, etc.) is never changed.

## License

Proprietary. See [LICENSE](LICENSE) for terms.
