Metadata-Version: 2.5
Name: workflowio
Version: 0.1.0
Summary: Draw a pipeline as a live factory floor in your terminal.
Project-URL: Homepage, https://github.com/bobgdickson/workflowio
Project-URL: Issues, https://github.com/bobgdickson/workflowio/issues
Author: Bob Dickson
License: MIT License
        
        Copyright (c) 2026 Bob Dickson
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Keywords: dashboard,mermaid,observability,pipeline,terminal,tui,workflow
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: System :: Monitoring
Classifier: Topic :: Terminals
Requires-Python: >=3.10
Requires-Dist: pyyaml>=6.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == 'dev'
Description-Content-Type: text/markdown

# workflowio

Draw a pipeline as a live factory floor in your terminal.

Define the shape of a workflow once — in YAML, or by importing a mermaid diagram
you already have — then feed it measurements and watch items move along the
belts between its stages, sized by whatever they are worth.

```
┌─ ACCOUNTS PAYABLE ─────────────────────────────────────────── DEMO · 1s≈4m · 5s ─┐
│ VOUCHERS 722   VALUE $837K   OUT 3.72/h   ALERTS 441   DARK 1                    │
│ frame 20:24:57 (0s ago)   8/9 nodes reporting · gathered in 137ms                │
└──────────────────────────────────────────────────────────────────────────────────┘

 ┌───────┐    ┌───────┐    ┌───────┐    ┌───────┐    ┌───────┐    ┌───────┐
 │░░░░░░░│    │·······│    │███████│    │███░░░░│    │████░░░│    │█░░░░░░│
 │NO DATA│∘∘∘∘│   27.1│┄∘∘∘│    411│◆∘∘∘│    200│┄∘∘▶│    241│┄┄◆◆│     74│
 │░░░░░░░│    │     /h│    │  $825K│    │    $2K│    │  $1.3M│    │  $167K│
 └───────┘    └───────┘    └───────┘    └───────┘    └───────┘    └───────┘
 STAGING      BUILD        ENTERED      MATCH        BUDGET CHK   APPROVAL
 PS_VCHR_HDR… AP_VCHRBLD   PS_VOUCHER   MATCH_STATU… BUDGET_HDR_… APPR_STATUS
```

MIT licensed. One dependency (PyYAML). The renderer is standard library only —
no curses, so it behaves the same on Windows, macOS and Linux.

## Install

```sh
pip install workflowio
```

## Sixty seconds

```sh
workflowio demo                                # a built-in example, no file needed
workflowio import docs/pipeline.mmd -o deck.yaml
workflowio run deck.yaml --demo                # synthetic data, no backend
workflowio run deck.yaml                       # your data
```

A deck can be four lines:

```yaml
name: Nightly ETL
flow: extract > transform > load > verify
```

`workflowio run that.yaml --demo` already draws it. Everything else is
annotation you add when you want it:

```yaml
name: Nightly ETL
units: { value: bytes, item_noun: files, rate_per: minute }
flow: extract > transform > load > verify
nodes:
  - { key: extract,   label: EXTRACT,   source: s3://raw }
  - { key: transform, label: TRANSFORM, source: dbt, kind: process }
  - { key: load,      label: LOAD,      source: warehouse }
  - { key: verify,    label: VERIFY,    source: checks, kind: process }
source:
  kind: command
  command: ["./scripts/etl-frame.py"]
  interval: 30
```

## Feeding it data

A source prints one JSON object per frame. That is the entire integration
surface — anything that can print JSON can drive the display. Run
`workflowio schema` for the annotated contract; the short version:

```json
{
  "nodes": {
    "extract":   { "count": 1840, "value": 3362485, "oldest_seconds": 372816, "rate": 9.7 },
    "transform": { "rate": 6.68 },
    "verify":    { "status": "no_data", "error": "connection refused" }
  },
  "items": [
    { "node": "extract", "id": "job-8871", "label": "nightly-etl",
      "value": 1830.0, "age_seconds": 420, "alert": true, "note": "retried twice" }
  ]
}
```

Rates are always per hour on the wire, whatever you display. Durations are
always seconds. Every field is optional except `nodes` — **omit what you cannot
measure rather than sending a zero.**

Four ways to deliver frames:

| | |
|---|---|
| `kind: command` | run a command on an interval, read its stdout |
| `kind: stream` | JSON-lines on stdin: `./watch.sh \| workflowio run deck.yaml --stdin` |
| `kind: replay` | walk a recorded file — `--record` writes one |
| `kind: demo` | synthesised from the deck, for when you have no backend yet |

In-process, skip JSON entirely:

```python
import workflowio

deck = workflowio.load("deck.yaml")
workflowio.run(deck, workflowio.CallableSource(my_probe, interval=30))
```

## Importing a mermaid diagram

Most people proposing a pipeline have already drawn it. Retyping that graph is
exactly the chore that stops a tool being used, so:

```sh
workflowio import docs/pipeline.mmd -o deck.yaml
```

reads `flowchart`/`graph` sources — including a ```` ```mermaid ```` block
inside a markdown file — and gives you the topology. Node shapes become kinds
(`[(cylinder)]` and `[box]` are buffers; `(rounded)`, `{decision}` and
`([stadium])` are processes), and a `<br/>` in a label splits into the node's
label and its source.

What mermaid cannot carry is what the YAML is for: where numbers come from,
which queue is trouble, which stage is a filtered view of another. Lines the
parser does not understand are reported, never silently dropped.

## Three promises

These are the reason this exists rather than being a box-drawing snippet, and
each one was paid for by a display that lied.

**Unknown is never zero.** A stage nobody could measure is drawn hatched and
labelled `NO DATA`, with the reason in the table. A quiet pipeline and a blind
sensor are opposite situations. This is why every measurement is optional and
why `status: "no_data"` exists — a display that writes `0` because a probe
failed is worse than one that says nothing, because zero looks like an answer.

**Inferred is never presented as measured.** Only some stages can report their
own throughput cheaply. An edge without one inherits the nearest known rate
along the chain, and is marked `derived`. It gets animated; it does not get to
claim it was measured.

**Identity is never invented.** Items you send are real things with real ids —
send a capped sample: the largest, the oldest, the ones in trouble. To make belt
density match the measured rate the display adds anonymous marks (`∘`, hollow,
no size). A synthesised packet never borrows an id it does not have.

Two smaller rules follow from the same instinct:

- **`subset_of`** — some stages are status flags on the same item rather than
  places it moves between. Declaring `subset_of: entered` excludes a stage from
  totals and draws its bar against its parent. Without it, one item queued in
  three flag-stages is counted three times; that is not hypothetical, it is
  where the rule came from.
- **`alert`** — an exception queue, where a rising count is bad news rather than
  busy news. Nothing leaves one until a person clears it, so the belt into it is
  drawn as backpressure by definition.

## Reading the display

| | |
|---|---|
| `▪ ◆ █` | an identified item, sized by value. Thresholds come from the frame's own items, so the ramp means something whether you move dollars, bytes or rows |
| `∘` | an anonymous item — makes density match the rate, carries no identity |
| `┄┄┄` | a belt with a known rate · `┈┈┈` one without |
| `▶` | backpressure: the queue ahead is growing faster than it drains |
| `█████░░` | queue depth, relative to the largest queue on screen |
| `░░░░░░░` | no data |
| `⊂parent` | a subset, excluded from totals |

Keys: `q` quit · `space` pause · `r` refresh now · `+`/`-` speed.

**About speed.** Belts carry real rates, and a pipeline moving nine items an
hour has motionless belts at 1×. `--speed` compresses time so the belts read —
the default is 240 (one second ≈ four simulated minutes), and the header always
states the compression in force. It changes packet *density* only, never a
number reported anywhere on screen.

## Still frames

```sh
workflowio run deck.yaml --once --no-color     # plain text, for CI logs or a README
```

```python
canvas = workflowio.render_once(deck, frame=frame, width=100, height=30)
print(canvas)
```

## Limitations in 0.1

- The spine is one-dimensional: it draws belts between neighbouring nodes.
  Branches and loops are kept in the deck and shown in the node table, but have
  no belt drawn. Fine for pipelines, lossy for a graph with real forks.
- Truecolor is assumed. Use `--theme slate`, `--no-color` or `NO_COLOR=1` where
  it is not available.
- Themes: `tron` (default), `amber`, `matrix`, `slate`.

## Examples

`examples/` has a CI pipeline (imported from mermaid, with a working source
script in `frame_ci.py`), and an accounts-payable deck that exercises `alert`
and `subset_of`.

```sh
workflowio run examples/accounts-payable.yaml --demo --dark staging
```

`--dark NODE` forces a node to report nothing, which is how you check what an
unmeasured stage looks like before one happens for real.
