Metadata-Version: 2.1
Name: flow-compute
Version: 3.25.3
Summary: CLI and SDK for submitting and managing GPU workloads
Keywords: gpu,compute,cloud,foundry,ml,ai
Author-Email: "Foundry Technologies, Inc." <support@mithril.ai>
License: Apache-2.0
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Project-URL: Homepage, https://mithril.ai
Project-URL: Documentation, https://github.com/mithrilcompute/flow#readme
Project-URL: Repository, https://github.com/mithrilcompute/flow
Project-URL: Issues, https://github.com/mithrilcompute/flow/issues
Requires-Python: >=3.10
Requires-Dist: pydantic<3.0,>=2.0
Requires-Dist: pyyaml<7.0,>=6.0
Requires-Dist: httpx<1.0,>=0.24.0
Requires-Dist: jinja2<4.0,>=3.0
Requires-Dist: rich<14.0,>=13.0
Requires-Dist: cloudpickle<4.0,>=2.0
Requires-Dist: click<8.2.0,>=8.1.8
Requires-Dist: prompt-toolkit<4.0,>=3.0
Requires-Dist: hishel<1.0.0,>=0.1.3
Provides-Extra: s3
Requires-Dist: boto3<2.0,>=1.26; extra == "s3"
Provides-Extra: recommended
Requires-Dist: numpy>=1.20; extra == "recommended"
Requires-Dist: trogon<1.0,>=0.6; extra == "recommended"
Provides-Extra: jupyter
Requires-Dist: ipykernel<7,>=6.29; extra == "jupyter"
Requires-Dist: jupyter-client<9,>=8.6; extra == "jupyter"
Provides-Extra: serverless
Requires-Dist: temporalio<2.0,>=1.7.0; extra == "serverless"
Requires-Dist: starlette<2.0,>=1.0.1; extra == "serverless"
Requires-Dist: uvicorn<1.0,>=0.23.0; extra == "serverless"
Requires-Dist: kubernetes<35.0,>=29.0; extra == "serverless"
Provides-Extra: skypilot
Requires-Dist: skypilot[aws,gcp]<1.0,>=0.11.0; extra == "skypilot"
Description-Content-Type: text/markdown

# Flow CLI & SDK

**Run GPU workloads on Mithril from the shell or from Python.**
Flow procures GPUs through Mithril, spins up InfiniBand-connected instances, and runs your
workloads. It is one engine with two projections: the CLI is what you meet first, and
`import flow` is the same engine for programs. Every CLI verb runs on the SDK.

![PyPI - Version](https://img.shields.io/pypi/v/flow-compute) [Public repo](https://github.com/mithrilcompute/flow)

## First hour

The same three commands `flow --help` teaches:

```bash
uv tool install flow-compute   # or: pipx install flow-compute
flow setup                     # Mithril account, API key and project
flow tutorial                  # Guided first GPU run with health checks
flow run 'nvidia-smi'          # Run a command on a GPU
```

`flow run` without `-i` bids on the cheapest NVIDIA GPU instance type with live capacity,
prints the choice with its family and hourly price, and asks before any bid above $20/hour.
The implicit default never picks a TPU or an accelerator family Flow does not know, since
the first command is `nvidia-smi`; name other hardware with `-i` (`-i v6e-1`, `-i cpu`).
`flow tutorial` prices its GPU test the same way and asks before launching it; `--yes` is
the only scripted consent and `--skip-example` leaves the test out. Add `--dry-run` to see
the configuration without bidding:

```
$ flow run 'nvidia-smi' --dry-run
╭─ Task Configuration ─────────────────────────────────────────────╮
│ Name           run-0972ae                                        │
│ Command        nvidia-smi                                        │
│ Instance Type  a100                                              │
│ Market Price   $8.00/hr (a100, cheapest live NVIDIA GPU)         │
│ Priority       Med                                               │
│ Max Bid        $16.00/hr                                         │
│ Code Upload    auto (timeout 600s) → /workspace (host+container) │
╰──────────────────────────────────────────────────────────────────╯

✓ Configuration is valid
```

Then watch, inspect and stop what you started:

```bash
flow status              # List and monitor tasks
flow logs <task> -f      # Stream task logs
flow ssh <task>          # SSH into a running task
flow cancel <task>       # Cancel a task
```

```
$ flow status
╭─ ❊ Flow • 9 completed ────────────────────────────────────────────╮
│                                                                   │
│     #  Status        Task                          Type           │
│     1  ✓ completed   cli-welcome                   1×A100         │
│     2  ✓ completed   post-training-sanity-1        1×A100·40G     │
│     3  ✓ completed   pretraining-pending-a100-40g  1×A100·40G     │
│     4  ✓ completed   post-training-fine-tune       8×H100         │
│     5  ✓ completed   inference-serving             1×H100         │
│     6  ✓ completed   pretraining-train-8xa100      8×A100         │
│     7  ✓ completed   post-training-sanity-2        1×A100·40G     │
│     8  ✓ completed   reasoning-rollout-b200        8×B200         │
│     9  ✓ completed   pretraining-2048-h100         256×8×H100     │
│                                                                   │
╰───────────────────────────────────────────────────────────────────╯
```

The same first hour from Python:

```python
import flow

task = flow.run("nvidia-smi", instance_type="a100")
for line in task.log_stream():
    print(line, end="")
```

Optional assistant integrations:

```bash
flow claude  # Install Flow skills for Claude Code
flow codex   # Install Flow skills for Codex
```

---

## Background

> **There's a paradox in GPU infrastructure today:**
> Massive GPU capacity sits idle, even as AI teams wait in queues—starved for compute.
> Mithril, the AI-compute **omnicloud**, dynamically allocates GPU resources from a global pool (spanning Mithril's first-party resources and 3rd-party partner cloud capacity) using efficient two-sided auctions, maximizing surplus and reducing costs. Mithril seamlessly supports both reserved-in-advance and just-in-time workloads—maximizing utilization, ensuring availability, and significantly reducing costs.

---

## Why choose Flow

Status quo GPU provisioning involves quotas, complex setups, and queue delays, even as GPUs sit idle elsewhere or in recovery processes. Flow addresses this:

**Dynamic Market Allocation** – Efficient two-sided auctions ensure you pay the lowest market-driven prices rather than inflated rates.

**Simplified Batch Execution** – An intuitive interface designed for cost-effective, high-performance batch workloads without complex infrastructure management.

Provision from 1 to thousands of GPUs for long-term reservations, short-term "micro-reservations" (minutes to weeks), or spot/on-demand needs—all interconnected via InfiniBand. High-performance persistent storage and built-in Docker support further streamline workloads, ensuring rapid data access and reproducibility.

---

## Why Flow + Mithril?

| Pillar                                              | Outcome                                                                      | How                                                                                  |
| --------------------------------------------------- | ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
| **Iteration Velocity and Ease**                     | Fresh containers quickly once capacity is allocated.                         | `flow dev` for a GPU workstation or `flow run` to launch tasks                      |
| **Best price-performance via market-based pricing** | Preemptible secure jobs for **\$1/h100-hr**                                  | Blind two-sided second-price auction; client-side bid capping                        |
| **Availability and Elasticity**                     | Self-serve access to spot and reserved capacity, subject to account access and market availability. | Spot capacity plus overflow capacity from partner clouds                             |
| **Abstraction and Simplification**                  | InfiniBand VMs, CUDA drivers, auto-managed healing buffer—all pre-arranged.  | Mithril virtualization and base images preconfigured + Mithril capacity management.  |

> *"The tremendous demand for AI compute and the large fraction of idle time makes sharing a perfect solution, and Mithril's innovative market is the right approach."* — **Paul Milgrom**, Nobel Laureate (Auction Theory and Mechanism Design)

---

## Workflows

### Research mode

* `flow run` → reproducible batch jobs from a command, a YAML file or a SLURM script.
* `flow dev` → interactive development on an allocated GPU workstation.
* `import flow` → the same engine for pipelines and orchestration.

```bash
# Launch a batch job on discounted H100s
flow run 'python train.py' -i 8xh100

# Run a saved configuration
flow run task.yaml

# Frictionlessly leverage an existing SLURM script
flow run job.slurm
```

```python
# Serverless-style decorator
@app.function(instance_type="a100")
def train(lr: float) -> dict:
    ...
```

### Infrastructure mode

* `flow instance create -i 8xh100 -N 20` → request a 20-node GPU cluster
* `flow volume create -s 10000 --interface file` → provision 10 TB of persistent, high-speed storage
* `flow ssh instance -- nvidia-smi` → run across all nodes in parallel

```
$ flow instance create -i 8xh100 -N 20 --dry-run
╭─ Instance Configuration ────────────────────────────────╮
│ Name         interactive-47fa95                         │
│ Instances    20 × 8xh100                                │
│ Max Bid      $64.00/hr per instance ($1280.00/hr total) │
│ Code Upload  disabled                                   │
╰─────────────────────────────────────────────────────────╯

✓ Configuration is valid
```

---

## Ideal Use Cases

* **Rapid Experimentation** – Quick iterations for research sprints.
* **Instant Elasticity** – Scale rapidly from one to thousands of GPUs.
* **Collaborative Research** – Shared dev environments with per-task cost controls.

Flow is not yet ideal for: always‑on ≤100 ms inference, strictly on‑prem regulated data, or models that fit on laptop or consumer-grade GPUs.

---

## Architecture (30‑s view)

```
Your intent ⟶ Flow Execution Layer ⟶ Global GPU Fabric
```

*Flow abstracts complex GPU auctions, InfiniBand clusters, and multi-cloud management into a single seamless and unified developer interface.*

---

## Installation

### Requirements

- Python 3.10 or later
- Recommended: use `uv` to auto-manage a compatible Python when installing the CLI
- Linux or macOS for native CLI workflows. On Windows, use WSL2; native Windows is not fully validated.

### 1) Install uv — optional but recommended
Installation guide: [docs.astral.sh/uv/getting-started/installation](https://docs.astral.sh/uv/getting-started/installation/)

- macOS/Linux:
  ```bash
  curl -LsSf https://astral.sh/uv/install.sh | sh
  ```
- Windows: use WSL2 for the Flow CLI, then run the macOS/Linux command inside WSL.
  ```bash
  curl -LsSf https://astral.sh/uv/install.sh | sh
  ```

### 2) Install Flow

- Global CLI (uv):
  ```bash
  uv tool install flow-compute
  flow setup
  ```

- Global CLI (pipx):
  ```bash
  pipx install flow-compute
  flow setup
  ```

### Providers

Flow talks to one provider per configuration. Mithril is the default and the only provider
installed by the base package; `flow setup --provider` (or `FLOW_PROVIDER`) selects the active
one. The SkyPilot provider is an optional extra, `pip install 'flow-compute[skypilot]'`, and
is not installed by default. Switching providers means switching configurations; a single
config does not span providers.

---

## Under the Hood (Advanced)

* **Bid Caps** – Protect budgets automatically.
* **Self-Healing** – Spot nodes dynamically migrate tasks.
* **Docker/Conda** – Pre-built images or dynamic install.
* **Multi-cloud Ready** – Mithril (with Oracle, Nebius integrations internal to Mithril), and more coming
* **SLURM Compatible** – Run `#SBATCH` scripts directly.

---

## Python SDK

### Advanced Task Configuration

```python
# Distributed training example (32 GPUs, Mithril groups for InfiniBand connectivity by default)
task = flow.run(
    "torchrun --nproc_per_node=8 train.py",
    instance_type="8xa100",
    num_instances=4,  # Total of 32 GPUs (4 nodes × 8 GPUs each)
    env={"NCCL_DEBUG": "INFO"}
)

# Mount S3 data + persistent volumes
task = flow.run(
    "python analyze.py",
    instance_type="a100",
    data_mounts={
        "/datasets": "s3://ml-bucket/imagenet",  # S3 via s3fs
        "/models": "volume://pretrained-models"   # Persistent storage
    }
)
```

The vocabulary is the same on every surface (`instance_type`, `env`, `data_mounts`, ...);
see [docs/vocabulary.md](docs/vocabulary.md).

### Key Features Summary

* **Distributed Training** – Multi-node InfiniBand clusters auto-configured
* **Code Upload** – Automatic with `.flowignore` (or `.gitignore` fallback)
* **Live Debugging** – SSH into running instances (`flow ssh`)
* **Cost Protection** – Built-in `max_price_per_hour` safeguards
* **Jupyter Integration** – Connect notebooks to GPU instances

**Documentation**: https://docs.mithril.ai/cli-and-sdk/quickstart

## Further Reading

* [Restoring the Promise of Public Cloud for AI](https://mithril.ai/blog/restoring-the-promise-of-the-public-cloud-for-ai)
* [Introducing Mithril](https://mithril.ai/blog/introducing-foundry)
* [Spot Auction Mechanics](https://docs.mithril.ai/compute-and-storage/spot-bids#spot-auction-mechanics)
