Metadata-Version: 2.5
Name: lutflow
Version: 0.3.1
Summary: Local cost intelligence for LLM inference: measure vLLM economics and enforce in-process budgets.
Project-URL: Homepage, https://lutflow.dev
Project-URL: Documentation, https://pypi.org/project/lutflow/
Author-email: "Lutflow Inc." <oscar@lutflow.com>
License: Business Source License 1.1
        
        Parameters
        
        Licensor:             Lutflow Inc.
        Licensed Work:        Lutflow SDK
                              The Licensed Work is (c) 2026 Lutflow Inc.
        Additional Use Grant: You may make use of the Licensed Work, provided that
                              you do not use the Licensed Work for a GPU Cost Management
                              Service.
        
                              A "GPU Cost Management Service" is a commercial offering
                              that allows third parties to access the functionality of
                              the Licensed Work by performing any of the following:
                              (a) offering budget enforcement, cost tracking, or kill-path
                                  functionality for GPU or AI inference workloads as a
                                  managed service;
                              (b) offering the Licensed Work to third parties as a
                                  commercial cloud service, hosted service, or
                                  software-as-a-service.
        
                              For clarity, the following uses are permitted:
                              - Internal use within your organization
                              - Development, testing, and evaluation
                              - Non-commercial and educational use
                              - Integration into your own products that do not compete
                                with Lutflow's commercial offerings
        
        Change Date:          March 29, 2030
        Change License:       Apache License, Version 2.0
        
        For information about alternative licensing arrangements for the Licensed Work,
        please contact: licensing@lutflow.com
        
        -----------------------------------------------------------------------------
        
        Business Source License 1.1
        
        License text copyright (c) 2017 MariaDB Corporation Ab, All Rights Reserved.
        "Business Source License" is a trademark of MariaDB Corporation Ab.
        
        Terms
        
        The Licensor hereby grants you the right to copy, modify, create derivative
        works, redistribute, and make non-production use of the Licensed Work. The
        Licensor may make an Additional Use Grant, above, permitting limited
        production use.
        
        Effective on the Change Date, or the fourth anniversary of the first publicly
        available distribution of a specific version of the Licensed Work under this
        License, whichever comes first, the Licensor hereby grants you rights under
        the terms of the Change License, and the rights granted in the paragraph
        above terminate.
        
        If your use of the Licensed Work does not comply with the requirements
        currently in effect as described in this License, you must purchase a
        commercial license from the Licensor, its affiliated entities, or authorized
        resellers, or you must refrain from using the Licensed Work.
        
        All copies of the original and modified Licensed Work, and derivative works
        of the Licensed Work, are subject to this License. This License applies
        separately for each version of the Licensed Work and the Change Date may vary
        for each version of the Licensed Work released by Licensor.
        
        You must conspicuously display this License on each original or modified copy
        of the Licensed Work. If you receive the Licensed Work in original or
        modified form from a third party, the terms and conditions set forth in this
        License apply to your use of that work.
        
        Any use of the Licensed Work in violation of this License will automatically
        terminate your rights under this License for the current and all other
        versions of the Licensed Work.
        
        This License does not grant you any right in any trademark or logo of
        Licensor or its affiliates (provided that you may use a trademark or logo of
        Licensor as expressly required by this License).
        
        TO THE EXTENT PERMITTED BY APPLICABLE LAW, THE LICENSED WORK IS PROVIDED ON
        AN "AS IS" BASIS. LICENSOR HEREBY DISCLAIMS ALL WARRANTIES AND CONDITIONS,
        EXPRESS OR IMPLIED, INCLUDING (WITHOUT LIMITATION) WARRANTIES OF
        MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, AND
        TITLE.
        
        MariaDB hereby grants you permission to use this License's text to license
        your works, and to refer to it using the trademark "Business Source License",
        as long as you comply with the Covenants of Licensor below.
        
        Covenants of Licensor
        
        In consideration of the right to use this License's text and the "Business
        Source License" name and trademark, Licensor covenants to MariaDB, and to all
        other recipients of the licensed work to be provided by Licensor:
        
        1. To specify as the Change License the GPL Version 2.0 or any later version,
           or a license that is compatible with GPL Version 2.0 or a later version,
           where "compatible" means that software provided under the Change License can
           be included in a program with software provided under GPL Version 2.0 or a
           later version. Licensor may specify additional Change Licenses without
           limitation.
        
        2. To either: (a) specify an additional grant of rights to use that does not
           impose any additional restriction on the right granted in this License, as
           the Additional Use Grant; or (b) insert the text "None".
        
        3. To specify a Change Date.
        
        4. Not to modify this License in any other way.
License-File: LICENSE
Keywords: ai,anthropic,budget-tracking,cost-management,energy-efficiency,finops,gpu,llm,openai
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: 3.15
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.9
Requires-Dist: click<9.0.0,>=8.1.0
Requires-Dist: httpx<1.0.0,>=0.25.0
Requires-Dist: rich<16.0.0,>=13.0.0
Requires-Dist: tiktoken<1.0.0,>=0.5.0
Provides-Extra: all
Requires-Dist: anthropic<2.0.0,>=1.0.0; extra == 'all'
Requires-Dist: google-generativeai<1.0.0,>=0.8.0; extra == 'all'
Requires-Dist: openai<4.0.0,>=1.0.0; extra == 'all'
Provides-Extra: anthropic
Requires-Dist: anthropic<2.0.0,>=1.0.0; extra == 'anthropic'
Provides-Extra: dev
Requires-Dist: mypy>=1.0.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.21.0; extra == 'dev'
Requires-Dist: pytest-cov>=4.0.0; extra == 'dev'
Requires-Dist: pytest>=7.0.0; extra == 'dev'
Requires-Dist: ruff>=0.1.0; extra == 'dev'
Provides-Extra: gemini
Requires-Dist: google-generativeai<1.0.0,>=0.8.0; extra == 'gemini'
Provides-Extra: openai
Requires-Dist: openai<4.0.0,>=1.0.0; extra == 'openai'
Description-Content-Type: text/markdown

# Lutflow — AI Cost Intelligence for LLM inference

Lutflow measures the allocated cost of a vLLM deployment you already run,
including idle GPU time. It reports cost per useful token, throughput and the
source or limitation of each number. The SDK also tracks and enforces provider
spend **inside your Python process** (OpenAI · Anthropic · Gemini).

Live vLLM measurement requires Lutflow `0.2.1` or newer. Check the installed
version before using `measure --endpoint`: older releases support only the
offline `--from-telemetry` path. For a published release, install from PyPI as
shown below. For an unreleased first-operator candidate, use the one audited wheel
and checksum in the supplied `TIER0_FIRST_OPERATOR.md` guide.

Tier 0 is local: no Lutflow account, hosted backend, Kubernetes agent or GPU
sensor is needed. To measure a live deployment, you need vLLM metrics reachable
from the machine running this command. A sourced USD hourly rate is needed only
for monetary fields.

### First vLLM measurement (version 0.3.0+)

Use Python 3.9 or newer (CI tests 3.9–3.15). Before installing, check that your vLLM server exposes
token counters (replace the URL if it is not on this machine):

```bash
curl -fsS http://localhost:8000/metrics | grep -E 'vllm:(prompt|generation)_tokens_total'
```

The quickest way to use the `lutflow` command is to run it with
[uv](https://docs.astral.sh/uv/) or [pipx](https://pipx.pypa.io/): each keeps
lutflow in its own isolated environment, so nothing you already have installed
changes.

```bash
# uv: run it without installing anything permanently
uvx --from 'lutflow>=0.3.0,<0.4' lutflow --version

# uv: install the command for good
uv tool install 'lutflow>=0.3.0,<0.4'

# pipx: install the command for good
pipx install 'lutflow>=0.3.0,<0.4'
```

The version range makes the install fail instead of quietly picking an older
release that lacks the 0.3.0 window checks. With `uvx`, write every command as
`uvx --from 'lutflow>=0.3.0,<0.4' lutflow …` (for example `… lutflow measure …`);
after `uv tool install` or `pipx install`, the command is just `lutflow`.

To use lutflow from your own Python code, install it in a virtual environment:

```bash
python3 --version
python3 -m venv .venv
source .venv/bin/activate
python -m pip install 'lutflow>=0.3.0,<0.4'
lutflow --version
```

In a uv project, `uv add 'lutflow>=0.3.0,<0.4'` does the same.

Then measure:

```bash
lutflow measure --endpoint http://localhost:8000 \
  --duration 300 --interval 10 \
  --json measurement.json
```

With no real traffic, the cost per useful token is undefined; use `lutflow
probe` only if you are allowed to generate requests on that server. It shows a
load plan and asks for `yes` before it starts. The report and JSON stay on your
machine. Without a sourced USD hourly rate, monetary fields are unavailable.

---

## Capabilities in version 0.2.1+

| Capability | Status | Notes |
|-----------|--------|-------|
| **In-process budget enforcement** | ✅ Ships, real | Tracks token/GPU-time spend in your process and, on breach, `raise`s, warns, runs a callback, or `SIGKILL`s the current process. |
| **Provider wrappers** | ✅ Ships, real | OpenAI, Anthropic, Google Gemini — wrap the client, spend is metered per call. |
| **Pricing tables** | ✅ Ships, real | Token and GPU-hour pricing with per-model overrides. |
| **`lutflow measure --from-telemetry`** | ✅ Ships, real | Small offline utility: cost & energy metrics from a telemetry export. Read-only, no network. |
| **`lutflow measure --endpoint`** | Included | Reads an existing vLLM server's Prometheus metrics and emits `tier0.measure.v1`; no account, cluster or GPU sensor. |
| **`lutflow probe`** | Included | Generates bounded, explicitly confirmed synthetic load when the server has no traffic. |

> Budget enforcement acts only on the **current Python process** (raise / warn
> / callback / `SIGKILL` of `os.getpid()`) — it is not cluster-level workload
> termination.

*The connected Lutflow platform — fleet-wide cost intelligence — is in private
development.*

---

## In-process budget enforcement

Wrap your provider client; Lutflow meters every call and enforces the limit
locally. This runs entirely in your process — no account, no network.

```python
from lutflow import Client, BudgetStrategy
import openai

client = Client(
    tenant_id="acme",
    budget_usd=10.00,
    on_budget_exceeded=BudgetStrategy.RAISE_ERROR,
)
wrapped = client.wrap(openai.OpenAI())
response = wrapped.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
)

print(f"Spent:     ${client.accumulated_cost_usd:.4f}")
print(f"Remaining: ${client.remaining_budget_usd:.4f}")
```

When the budget is exceeded, the configured strategy fires **in this process**:

| Strategy | Behavior (in-process) |
|----------|-----------------------|
| `RAISE_ERROR` | Raises `BudgetExceededError` (default) |
| `WARN_ONLY` | Logs a warning, continues |
| `CALLBACK` | Calls a function you provide |
| `SELF_KILL` | Sends `SIGKILL` to the current process (`os.getpid()`) |

A context-manager form is also available:

```python
from lutflow import budget_session

with budget_session(budget_usd=0.50, tenant_id="acme") as session:
    wrapped = session.wrap(openai.OpenAI())
    ...
```

### GPU-time pricing (self-hosted models)

```python
from lutflow import Client, PricingMode

client = Client(
    tenant_id="acme",
    budget_usd=5.00,
    pricing_mode=PricingMode.GPU_TIME,
    gpu_type="nvidia-l4",
)
client.start_gpu_timer()
# ... run inference ...
cost = client.stop_gpu_timer()
```

### `lutflow measure --endpoint` — measure a vLLM server you already run

> Available from version 0.2.1. Check `lutflow --version` after installation;
> 0.2.0 supports only the offline `--from-telemetry` path.

Point it at a vLLM server that is already serving. It reads that server's own
Prometheus metrics across a time window and reports what your tokens cost. No
Kubernetes, no Lutflow account, no GPU sensor, no agent to install.

```bash
lutflow measure --endpoint "$METRICS_URL" \
  --duration 300 \
  --gpu-hourly-rate "$GPU_RATE_USD_PER_HOUR" --rate-basis "$RATE_BASIS" \
  --gpus-in-basis "$GPUS_IN_BASIS" --gpu-count "$GPU_COUNT" \
  --json measurement.json
```

Set these variables from your own sourced USD/hour quote and deployment record;
record the quote's source and effective date. `RATE_BASIS` is `instance` or
`per_gpu`; `GPUS_IN_BASIS` is 1 for a per-GPU quote. Without a verifiable USD
rate, omit all four pricing options and measure without monetary fields. No
currency conversion is performed. `--rate-basis` has no default. Some cloud
offers quote an entire multi-GPU **instance**. Reading that price as a per-GPU
rate overstates cost by the number of GPUs included, so Lutflow asks which
basis the quote uses.

Add your API price to get a comparison and a break-even point:

```bash
lutflow measure --endpoint "$METRICS_URL" \
  --gpu-hourly-rate "$GPU_RATE_USD_PER_HOUR" --rate-basis "$RATE_BASIS" \
  --gpus-in-basis "$GPUS_IN_BASIS" --gpu-count "$GPU_COUNT" \
  --api-provider "$API_PROVIDER" --api-model "$API_MODEL" \
  --api-input-rate "$API_INPUT_USD_PER_MILLION" \
  --api-output-rate "$API_OUTPUT_USD_PER_MILLION" \
  --api-price-effective-at "$API_PRICE_EFFECTIVE_AT" \
  --api-price-source "$API_PRICE_SOURCE"
```

Every field is required for a comparison, including where the price came from and
when it was in force. There is no built-in price table: prices move, and a
substituted rate produces a comparison against a bill you never received.

The output has two halves, in this order:

- **Allocated economics** — the whole window, idle seconds included. This is what
  the deployment costs. A reserved GPU bills while it waits.
- **Active efficiency** — only the seconds that produced useful tokens. This is how
  well the deployment runs when it is running.

They coincide at 100% duty cycle and diverge everywhere else. The gap between them
is what you spend on idle capacity.

Useful tokens are `prompt + generation`. Anything a GPU sensor would have provided
reports `unavailable`, never `0` — this slice reads the inference engine only, so
power, energy, VRAM and GPU utilisation are all `unavailable`. vLLM does not publish
GPU utilisation, and KV-cache occupancy is a different quantity that is not
substituted for it.

The CLI fetches metrics from your vLLM endpoint. The report and JSON export stay
on your machine; the JSON contains no endpoint, hostname, secret or prompt text,
and nothing is sent to Lutflow. `probe` sends synthetic requests to the endpoint
only after you confirm its load plan.

It does not guess a rate, substitute a similar model's price, assume you bought
one GPU, or add two models' tokens together. Missing prices stay unavailable;
incomplete cost inputs and ambiguous model selection produce actionable errors.

Currently supported: **vLLM**. TGI and Ollama are not implemented, and asking for
them says so rather than silently using vLLM's metric names.

#### Limits

- **A window comes from one engine process, with continuous counters.** From 0.3.0, a
  token counter or latency histogram that goes backwards, a token counter that disappears
  from some scrapes, or a change in the engine's process identity ends the measurement with
  exit 1 and no report. The tokens around a restart were never observed, so no total is
  published.
- **Restart detection depends on the engine publishing its identity.** If the engine
  publishes neither `process_start_time_seconds` nor the `_created` series of its token
  counters, a restart whose counters keep rising can go unnoticed. The report then states
  `engine_process_identity_unpublished_restart_with_rising_counters_not_detectable` in
  `warnings[]` and is not rejected. The absence of a reset error does not prove that the
  window came from a single continuous process.
- **TPOT is `unavailable` on recent vLLM.** 0.3.0 reads `vllm:time_per_output_token_seconds`,
  which recent vLLM (v0.30.0, for example) no longer publishes. `tpot_ms` is then reported
  `unavailable`, never estimated; TTFT, end-to-end latency and tokens are unaffected. See
  [Known limitations](CHANGELOG.md#known-limitations).
- **Wrapper costs use a dated price table.** Model names outside it are mapped to older
  entries without a warning, so the rate applied may not match the model you call, and
  a cost can be off by an order of magnitude. Pass `price_per_1k_usd` to `Client` when
  you rely on them. `lutflow measure` is not affected.
  See [Known limitations](CHANGELOG.md#known-limitations).
- **The p95 resolves only to a histogram bucket.** It is interpolated inside the engine's
  bucket that contains it: with vLLM's default end-to-end buckets, a window whose requests
  all take 2.5–5 s reports 4875 ms whatever the real tail. Read it as "within that bucket".
  See [Known limitations](CHANGELOG.md#known-limitations).

### `lutflow probe` — measure a vLLM server with no traffic yet

> Available from version 0.2.1, with explicit confirmation before load starts.

`--endpoint` measures traffic your server is already handling. `probe` is for the
opposite case: a server that is up but idle, with nothing yet to observe. It
generates its own load against the server, then measures that load with the exact
same path `--endpoint` uses.

```bash
lutflow probe --endpoint "$METRICS_URL" --model "$MODEL_ID" \
  --concurrency 8 --duration 90 \
  --json measurement.json
```

Because it generates traffic on a server you may not own alone, four guards apply
and none of them can be turned off:

- **Opt-in.** `probe` is a separate command; `measure --endpoint` never generates
  load on its own.
- **Declared, not disguised.** Every `probe` document is marked
  `measurement_mode: "probe"` and `load_origin: "induced"`, and the terminal output
  leads with a banner saying the traffic was generated, not observed.
- **Synthetic prompts, hard caps.** The load uses a fixed, packaged prompt pool
  (never your data) under concurrency, duration and total-request ceilings that are
  compiled into the package. No flag, environment variable, or config file can
  raise them.
- **Confirmation, every time.** `probe` prints exactly what it is about to do and
  requires you to type `yes`. There is no `--force` or `--yes` flag.

If `--max-requests` finishes the load before the observation window closes, the
rest of the window is idle time the probe itself created, and the document's
`warnings[]` (and the terminal) say so rather than let a collapsed duty cycle look
like something the endpoint did.

### `lutflow measure --from-telemetry` — offline pass over an existing export

The original offline path, unchanged: point it at a GPU telemetry export for a
per-concurrency breakdown of cost and energy per token. Read-only, no network.

```bash
lutflow measure --from-telemetry telemetry.jsonl --price-per-hr "$GPU_RATE_USD_PER_HOUR"
```

Its `$/1M useful` column is an **active** figure — cost over each concurrency
bucket's productive intervals. A concurrency bucket is a subset of a window by
construction, so it carries no idle share of its own; use `--endpoint` when you need
the allocated cost of a real window.

---

## Installation

```bash
pip install 'lutflow>=0.3.0,<0.4'            # Core + CLI
pip install 'lutflow[openai]>=0.3.0,<0.4'    # + OpenAI wrapper
pip install 'lutflow[anthropic]>=0.3.0,<0.4' # + Anthropic wrapper
pip install 'lutflow[gemini]>=0.3.0,<0.4'    # + Google Gemini wrapper
pip install 'lutflow[all]>=0.3.0,<0.4'       # All providers
```

Provider wrappers support **OpenAI**, **Anthropic** and **Google Gemini**.
Live endpoint measurement supports **vLLM only**. In-process GPU-time pricing
can use an operator-supplied rate for other self-hosted inference processes;
it does not scrape their endpoint metrics.

---

## Links

- **Website**: [lutflow.dev](https://lutflow.dev)
- **PyPI**: [pypi.org/project/lutflow](https://pypi.org/project/lutflow/)
- **Support**: <oscarmatiasg@lutflow.com>

## License

Source-available under the Business Source License 1.1 (BSL 1.1). Internal use,
development, testing, evaluation, and non-commercial use are expressly permitted.
Converts to Apache-2.0 on 2030-03-29 (the Change Date).

See [LICENSE](LICENSE) for the full terms, including the commercial-use
exclusion, which is not summarized here. For commercial licensing:
licensing@lutflow.com
