Metadata-Version: 2.5
Name: lutflow
Version: 0.3.3
Summary: Local cost intelligence for LLM inference: measure vLLM economics and enforce in-process budgets.
Project-URL: Homepage, https://lutflow.dev
Project-URL: Documentation, https://pypi.org/project/lutflow/
Author-email: Lutflow EAS <oscar@lutflow.com>
License: Business Source License 1.1
        
        Parameters
        
        Licensor:             Lutflow EAS
        Licensed Work:        Lutflow SDK
                              The Licensed Work is (c) 2026 Lutflow EAS
        Additional Use Grant: You may make use of the Licensed Work, provided that
                              you do not use the Licensed Work for a GPU Cost Management
                              Service.
        
                              A "GPU Cost Management Service" is a commercial offering
                              that allows third parties to access the functionality of
                              the Licensed Work by performing any of the following:
                              (a) offering budget enforcement, cost tracking, or kill-path
                                  functionality for GPU or AI inference workloads as a
                                  managed service;
                              (b) offering the Licensed Work to third parties as a
                                  commercial cloud service, hosted service, or
                                  software-as-a-service.
        
                              For clarity, the following uses are permitted:
                              - Internal use within your organization
                              - Development, testing, and evaluation
                              - Non-commercial and educational use
                              - Integration into your own products that do not compete
                                with Lutflow's commercial offerings
        
        Change Date:          March 29, 2030
        Change License:       Apache License, Version 2.0
        
        For information about alternative licensing arrangements for the Licensed Work,
        please contact: licensing@lutflow.com
        
        -----------------------------------------------------------------------------
        
        Business Source License 1.1
        
        License text copyright (c) 2017 MariaDB Corporation Ab, All Rights Reserved.
        "Business Source License" is a trademark of MariaDB Corporation Ab.
        
        Terms
        
        The Licensor hereby grants you the right to copy, modify, create derivative
        works, redistribute, and make non-production use of the Licensed Work. The
        Licensor may make an Additional Use Grant, above, permitting limited
        production use.
        
        Effective on the Change Date, or the fourth anniversary of the first publicly
        available distribution of a specific version of the Licensed Work under this
        License, whichever comes first, the Licensor hereby grants you rights under
        the terms of the Change License, and the rights granted in the paragraph
        above terminate.
        
        If your use of the Licensed Work does not comply with the requirements
        currently in effect as described in this License, you must purchase a
        commercial license from the Licensor, its affiliated entities, or authorized
        resellers, or you must refrain from using the Licensed Work.
        
        All copies of the original and modified Licensed Work, and derivative works
        of the Licensed Work, are subject to this License. This License applies
        separately for each version of the Licensed Work and the Change Date may vary
        for each version of the Licensed Work released by Licensor.
        
        You must conspicuously display this License on each original or modified copy
        of the Licensed Work. If you receive the Licensed Work in original or
        modified form from a third party, the terms and conditions set forth in this
        License apply to your use of that work.
        
        Any use of the Licensed Work in violation of this License will automatically
        terminate your rights under this License for the current and all other
        versions of the Licensed Work.
        
        This License does not grant you any right in any trademark or logo of
        Licensor or its affiliates (provided that you may use a trademark or logo of
        Licensor as expressly required by this License).
        
        TO THE EXTENT PERMITTED BY APPLICABLE LAW, THE LICENSED WORK IS PROVIDED ON
        AN "AS IS" BASIS. LICENSOR HEREBY DISCLAIMS ALL WARRANTIES AND CONDITIONS,
        EXPRESS OR IMPLIED, INCLUDING (WITHOUT LIMITATION) WARRANTIES OF
        MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, AND
        TITLE.
        
        MariaDB hereby grants you permission to use this License's text to license
        your works, and to refer to it using the trademark "Business Source License",
        as long as you comply with the Covenants of Licensor below.
        
        Covenants of Licensor
        
        In consideration of the right to use this License's text and the "Business
        Source License" name and trademark, Licensor covenants to MariaDB, and to all
        other recipients of the licensed work to be provided by Licensor:
        
        1. To specify as the Change License the GPL Version 2.0 or any later version,
           or a license that is compatible with GPL Version 2.0 or a later version,
           where "compatible" means that software provided under the Change License can
           be included in a program with software provided under GPL Version 2.0 or a
           later version. Licensor may specify additional Change Licenses without
           limitation.
        
        2. To either: (a) specify an additional grant of rights to use that does not
           impose any additional restriction on the right granted in this License, as
           the Additional Use Grant; or (b) insert the text "None".
        
        3. To specify a Change Date.
        
        4. Not to modify this License in any other way.
License-File: LICENSE
Keywords: ai,anthropic,budget-tracking,cost-management,energy-efficiency,finops,gpu,llm,openai
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: 3.15
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.9
Requires-Dist: click<9.0.0,>=8.1.0
Requires-Dist: httpx<1.0.0,>=0.25.0
Requires-Dist: rich<16.0.0,>=13.0.0
Requires-Dist: tiktoken<1.0.0,>=0.5.0
Provides-Extra: all
Requires-Dist: anthropic<2.0.0,>=1.0.0; extra == 'all'
Requires-Dist: google-generativeai<1.0.0,>=0.8.0; extra == 'all'
Requires-Dist: openai<4.0.0,>=1.0.0; extra == 'all'
Provides-Extra: anthropic
Requires-Dist: anthropic<2.0.0,>=1.0.0; extra == 'anthropic'
Provides-Extra: dev
Requires-Dist: mypy>=1.0.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.21.0; extra == 'dev'
Requires-Dist: pytest-cov>=4.0.0; extra == 'dev'
Requires-Dist: pytest>=7.0.0; extra == 'dev'
Requires-Dist: ruff>=0.1.0; extra == 'dev'
Provides-Extra: gemini
Requires-Dist: google-generativeai<1.0.0,>=0.8.0; extra == 'gemini'
Provides-Extra: openai
Requires-Dist: openai<4.0.0,>=1.0.0; extra == 'openai'
Description-Content-Type: text/markdown

# Lutflow

**Measure vLLM throughput and allocated token cost—including idle time.**

Lutflow reads the metrics of a vLLM server you already run and writes a local JSON
report. Missing data stays explicit. No Lutflow account, cluster agent or GPU
sensor is required. The Python SDK also tracks provider spend inside your app.

## Start with a running vLLM server

Requires Python 3.9+ and [uv](https://docs.astral.sh/uv/). Replace the URL with your
**engine's** base URL, not a router. The server must already expose `/metrics`.

```bash
uvx --from 'lutflow>=0.3.3,<0.4' lutflow check --endpoint http://localhost:8000 &&
uvx --from 'lutflow>=0.3.3,<0.4' lutflow measure --endpoint http://localhost:8000 --duration 60 --interval 5 --json measurement.json &&
python3 -m json.tool measurement.json
```

These commands check the telemetry, observe a 60-second window, then display the
report. `measure` does not generate traffic. If the server is idle, use `probe`
with permission to send requests (below).

Without your USD hourly rate, monetary fields are `unavailable`; token counts
and throughput can still be reported. An idle window has no defined cost per
token. Useful tokens mean **prompt + generated tokens**, not accepted answers.

## What the report tells you

| Field | Meaning |
|---|---|
| Tokens | Counter differences over the observation window. |
| Allocated cost | Your hourly rate applied to the entire window, idle time included. |
| Active throughput and cost | Tokens and cost over intervals that produced tokens. |
| Productive duty cycle | Share of sampled time with token production. |
| Latency | Window statistics from the histograms the engine publishes. |

Each metric in `tier0.measure.v1` carries its value, unit, status and source.
Missing values are `null`, not zero; warnings describe uncertainty and limits.
The report stays on your machine. Live commands do not measure GPU utilization,
power, energy or VRAM; these fields remain unavailable. Throughput and
productive duty cycle measure neither physical GPU utilization nor answer quality.

## Commands

### `check`: before spending a measurement window

**Measurable** means the metrics expose one engine-process identity, the required
typed metric families and counter creation markers. `check` judges one snapshot;
it does not run a benchmark or certify GPU readiness or model quality.

```bash
uvx --from 'lutflow>=0.3.3,<0.4' lutflow check --endpoint http://localhost:8000 --json check.json
```

Exit codes: **0** pass, **1** demonstrated failure, **3** unreadable/no verdict,
**2** invalid usage. Read the reported reasons before continuing after a failure.
Add `--model "$MODEL_ID"` to require that model's metrics and `/v1/models` entry.
Use `--metrics-file saved.prom` instead of `--endpoint` to check a saved snapshot.

### `measure`: observe existing traffic

To include monetary fields, supply a sourced quote and the allocation it covers.
For an **instance** quote, set `USD_PER_HOUR`, `GPUS_IN_INSTANCE` and `GPU_COUNT`
from your price source and deployment record; keep the quote's date and source.

```bash
uvx --from 'lutflow>=0.3.3,<0.4' lutflow measure --endpoint http://localhost:8000 \
  --duration 60 --interval 5 --json measurement.json \
  --gpu-hourly-rate "$USD_PER_HOUR" --rate-basis instance \
  --gpus-in-basis "$GPUS_IN_INSTANCE" --gpu-count "$GPU_COUNT"
```

For a per-GPU quote, use `--rate-basis per_gpu` and omit `--gpus-in-basis`.
No rate or GPU count is guessed.
Cost is calculated from your inputs; it is not a reconciled cloud invoice.

A process-identity change, counter reset or token-counter gap stops the window
with exit **1** and no report. An unreadable endpoint returns **2**. For a URL
with a path, give `measure` the full metrics URL; `check` instead appends `/metrics`
to its base URL. `--model` selects a model when the endpoint exposes several.

`lutflow measure --from-telemetry FILE` reads an existing `.jsonl` or `.jsonl.gz`
export offline and prints a table by concurrency: read-only, no network, no live window.

### `probe`: generate explicitly confirmed load

```bash
uvx --from 'lutflow>=0.3.3,<0.4' lutflow probe --endpoint http://localhost:8000 \
  --model "$MODEL_ID" --concurrency 4 --duration 60 --json probe.json
```

This example requests four concurrent streams for 60 seconds. `probe` shows its
plan and requires you to type **`yes`** before sending synthetic requests.
There is no unattended confirmation flag. Use a bare engine URL: path-bearing
proxy URLs are not supported for `probe`. Reports mark the load as induced.

## Python SDK: budgets inside your process

Wrap an OpenAI, Anthropic or Gemini client with `Client.wrap(provider_client)`
to track spend and enforce a local budget. Wrapped calls still go to your provider.
Install `lutflow[openai]>=0.3.3,<0.4` and set `OPENAI_API_KEY`; the call uses your key.
lutflow ships no price table: declare the price you pay for each exact model id,
where you read it and the date it applied. The figures below are placeholders.

```python
import openai
from lutflow import BudgetStrategy, Client, TokenPrice

client = Client(
    tenant_id="my-app",
    budget_usd=10.0,  # an example limit you choose
    on_budget_exceeded=BudgetStrategy.RAISE_ERROR,
    token_prices={
        "gpt-4o-mini": TokenPrice(
            input_usd_per_million=0.15,  # replace with the price you pay
            output_usd_per_million=0.60,
            source="<the price page you read>",
            effective_at="2026-10-01",
        ),
    },
)
wrapped = client.wrap(openai.OpenAI())
wrapped.chat.completions.create(model="gpt-4o-mini", messages=[{"role": "user", "content": "Hi!"}])
print(client.accumulated_cost_usd)
print(client.remaining_budget_usd)
```

The budget is checked before each wrapped call. The call that crosses it returns
its response and its spend is recorded; the next wrapped call raises
`BudgetExceededError` before it reaches the provider (`WARN_ONLY` lets it through,
`CALLBACK` is notified). Spend already made is not reverted: this is a local guard,
not a cap on your provider account, and it does not stop cluster workloads.
`SELF_KILL` no longer terminates the process from a wrapped call.

Metered: `chat.completions.create` and `embeddings.create` (OpenAI),
`messages.create` (Anthropic), `generate_content` and chat `send_message`
(Gemini), synchronous and without `stream=True`, priced from the usage the
provider returns. Other methods, streaming and async clients raise
`UnsupportedMethodError` without calling the provider. A model without a declared
price raises `PriceUnavailableError`. A response without usage is returned, and
later wrapped calls raise `SpendUnverifiableError`; so do the calls after a provider
call that raised, because that request may have run.

## What's new in 0.3.x

- **0.3.3:** wrapper integrity: no built-in prices or substituted rates, the
  response that crosses the budget is kept and later calls stop, and methods the
  wrappers do not meter are refused.
- **0.3.2:** window-mean TPOT from recent vLLM inter-token histograms; a warning
  identifies the histogram bucket used to interpolate p95.
- **0.3.1:** `check` evaluates telemetry structure before a measurement window.
- **0.3.0:** resets, identity changes and token gaps reject unsafe windows.

## Limitations

- **vLLM only** for live endpoint measurement. The `check` descriptor targets
  vLLM v0.30.0 metrics; other layouts can fail preflight. Multi-process exporters
  can omit the process identity and creation markers this check requires.
- **Identity must be published.** Without it, `measure` warns and cannot detect
  every restart whose counters keep rising. Successful completion alone does not
  establish process continuity.
- **Sampling limits resolution.** Productive duty cycle reflects scrape intervals.
  Histogram p95 is interpolated inside a bucket, not an exact tail measurement.
  TPOT is a per-token-gap mean, not a per-request mean.
- **Wrapper costs come from your prices.** A cost is the provider-reported tokens
  times the price you declare; it is not reconciled with an invoice. OpenAI and
  Gemini cached input tokens are priced at the full input rate; Anthropic cache
  tokens leave the call unmetered. The deprecated `price_per_1k_usd` applies one
  rate to input and output.
- **Endpoint access uses your environment.** Proxy and certificate variables apply.
  JSON from `check` includes the endpoint path, which can identify a namespace or
  pod; review it before sharing. No report is uploaded to Lutflow.

## Installation and extras

For a persistent CLI:

```bash
uv tool install 'lutflow>=0.3.3,<0.4'
```

For Python code, install inside your virtual environment:

```bash
python -m pip install 'lutflow>=0.3.3,<0.4'
```

Provider extras: `'lutflow[openai]>=0.3.3,<0.4'`,
`'lutflow[anthropic]>=0.3.3,<0.4'`, `'lutflow[gemini]>=0.3.3,<0.4'`,
or `'lutflow[all]>=0.3.3,<0.4'`. They add provider SDKs; the vLLM CLI needs no extra.
The `anthropic` and `all` extras require Python 3.10+; the CLI and other extras support Python 3.9+.

## License

Source available under **BSL 1.1**, licensed by **Lutflow EAS**. The Change Date is
**2030-03-29**, when the Change License becomes **Apache-2.0**. The packaged
`LICENSE` contains the full terms, including commercial-use restrictions.

## Links

- [Website](https://lutflow.dev)
- [Package and source distributions](https://pypi.org/project/lutflow/#files)
- [Contact: oscar@lutflow.com](mailto:oscar@lutflow.com)
