Metadata-Version: 2.5
Name: fuaran-ui-data
Version: 0.0.2
Summary: Data-science companion to the Fuaran UI Python host — dataframe adapters (pandas / polars / Arrow), a one-call self-contained HTML export, and in-place notebook output.
Project-URL: Homepage, https://github.com/Fuaran-Data/fuaran-py-data
Project-URL: Repository, https://github.com/Fuaran-Data/fuaran-py-data
Author-email: "Andrew J. Willshire" <andrew@fuaran.com>
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: arrow,dataframe,fuaran,notebook,pandas,polars,wire-format
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: Jupyter
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Visualization
Classifier: Topic :: Software Development :: Libraries
Classifier: Typing :: Typed
Requires-Python: >=3.12
Requires-Dist: fuaran-ui<0.11,>=0.10.0
Provides-Extra: arrow
Requires-Dist: pyarrow>=16.0; extra == 'arrow'
Provides-Extra: dev
Requires-Dist: mypy>=1.11; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff<0.17,>=0.16; extra == 'dev'
Provides-Extra: duckdb
Requires-Dist: duckdb>=1.0; extra == 'duckdb'
Requires-Dist: pyarrow>=16.0; extra == 'duckdb'
Provides-Extra: live-host
Requires-Dist: fastapi>=0.110; extra == 'live-host'
Requires-Dist: httpx>=0.27; extra == 'live-host'
Requires-Dist: uvicorn>=0.29; extra == 'live-host'
Provides-Extra: locale
Requires-Dist: babel>=2.18; extra == 'locale'
Provides-Extra: notebook
Requires-Dist: anywidget>=0.9; extra == 'notebook'
Requires-Dist: ipython>=8.0; extra == 'notebook'
Provides-Extra: pandas
Requires-Dist: pandas>=2.0; extra == 'pandas'
Provides-Extra: polars
Requires-Dist: polars<2,>=1.0; extra == 'polars'
Provides-Extra: test
Requires-Dist: pytest>=8; extra == 'test'
Description-Content-Type: text/markdown

# fuaran-ui-data

The **data-science companion** to [`fuaran-py`](https://github.com/fuaran-ui/fuaran-py), the Python
host of the Fuaran UI wire format.

`fuaran-ui` is deliberately standard-library-only: no third-party runtime dependency, importable
anywhere CPython runs — including in the browser, under WebAssembly. That constraint is what makes
it a dependable host, and it is also why it cannot carry a pandas adapter, a browser bundle, or a
Jupyter integration. This package carries those, as a **second distribution** rather than an extra
of the host, so the two can move on their own release rhythms.

> **Status: early.** Everything below is shipped: the adapters, the self-contained HTML export,
> the markdown and email-safe projections, the interactive notebook output, in-place notebook
> patching, the notebook-side ask-for-a-change, and the Streamlit-compatible shim.

## The adapters

**The one line from the DataFrame you already have to a source a grid or a chart can bind.**

```python
import fuaran_ui_data as fd
from fuaran_ui.ui import col

fr = fd.frame(df).filter(col("amount") > 0).sort("amount", descending=True)
binding = fr.to_transform_binding()  # pass as a node source
preview = fr.collect()  # the host's reference evaluator, locally
```

`df` may be a pandas or polars `DataFrame`, an Arrow table, anything else implementing the
dataframe interchange protocol, a mapping of column name to values, or a source you already have.
`from_pandas`, `from_polars`, `from_arrow` and `from_dataframe` are the same conversions under
their own names, and `to_source` is the dispatcher.

**What crosses the boundary is the source, and only the source.** The pipeline stays what it
already was: a serializable `Transform` the host's evaluator runs, identically, wherever it lands.
Nothing here evaluates anything — `.collect()` is the host's own reference evaluator, previewing
locally what a browser will compute later.

Three things about the conversion are worth knowing before you meet them.

**A column type the wire cannot carry is refused by name.** A timezone-aware timestamp silently
converted to UTC is a lost declaration; an object column holding both numbers and strings
stringified without asking is a schema invented on your behalf. Both raise, naming the column, what
it holds, and the one edit that resolves it — and `schema={"col": "string"}` pins a column
explicitly when that is what you meant.

**Nulls travel as a validity mask, never as a sentinel.** `None`, NaN, `pandas.NA`, `NaT` and an
unset Arrow validity bit are five spellings of one thing, and all five arrive as the wire's own
null. No value is ever chosen to stand for "missing".

**A numpy integer column that acquires a null arrives as `float`, and a masked one does not.**
pandas widened it to `float64` before this package saw it — the null is a NaN and NaN is a float —
so that is what its dtype says and what the evaluator's own widening rule gives. polars and Arrow
carry a real validity mask, so the same logical column stays `int` there. Use pandas' `Int64` dtype,
or pin the column, to keep `int` with nulls. The full per-library inference table is in each
adapter module's docstring.

## Start from a query

Most notebook data starts as a query, not a file. A DuckDB connection hands back an Arrow table,
which is a source the adapters above already read, so the query result becomes a `DataSource` in
one call:

```python
import duckdb
import fuaran_ui_data as fd

con = duckdb.connect()
source = fd.from_duckdb(con, "SELECT region, SUM(amount) AS revenue FROM sales GROUP BY 1")
```

`from_sql` is the same for any DB-API 2 connection — sqlite3, psycopg2, whatever your driver is. It
takes the driver's **Arrow fetch when the driver has one** and reads `cursor.description` plus
`fetchall()` otherwise, and either way the column types are decided by the same inference the
adapters above use. There is one inference table in this package, and the tests run the two paths
over identical data and compare the emitted bytes rather than trusting that sentence.

```python
import sqlite3

con = sqlite3.connect("sales.db")
source = fd.from_sql(con, "SELECT region, amount FROM sales WHERE region = ?", ("EMEA",))
```

Pass your driver's parameters rather than formatting values into the SQL — they go through
untouched, and letting the driver bind a value is what keeps it from being spliced into the
statement.

**Two column types a query commonly produces need a cast, and the refusal names it.** A `DECIMAL`
cannot cross because the wire's scalar set has no decimal and widening one to float would silently
lose the precision the type was chosen for. A `DATE` cannot cross the Arrow path because pyarrow's
interchange implementation declines to export it, though the wire carries `date` perfectly well.
Both are one edit: `SELECT CAST(amount AS DOUBLE) AS amount`, or `CAST(d AS VARCHAR)` with
`schema={"d": "date"}` to put the type back.

**Your connection is consumed, never carried.** It is read inside the call and nothing that comes
back can reach it, so no DSN, password or token can appear in an emitted document — which the tests
assert against the exported bytes, since that is the only place it could show.

## How many rows an artefact may carry

A frame becomes one JSON array per column, and the wire bounds a single array at **100 000
elements** (`MAX_ARRAY_LENGTH`, WIRE_FORMAT.md §21.1). Every adapter here refuses a frame past that
bound, by name, at the moment you convert:

```python
fd.from_pandas(df)  # 200_000 rows
# ValueError: this frame carries 200000 rows, above the wire limit MAX_ARRAY_LENGTH = 100000:
# every column becomes one JSON array, and a conformant decoder refuses an array past that length
# (WIRE_FORMAT.md §21.1). Filter or aggregate before converting, or carry a POINTER to the data
# instead of the rows — `ref_parquet(url, schema=...)` is the host-resolved form, and the grid
# fetches it where it is rendered.
```

It refuses **at the adapter** rather than letting the document reach a decoder, because the two
failures are not comparable. The refusal here names your frame, its size and the bound; the one
downstream arrives from whatever host eventually reads the artefact, after `patch` has already
advanced the tree, about an array it can only describe by position.

`max_rows` is a ceiling of your own, below that one:

```python
fd.from_pandas(df, max_rows=5_000)  # every adapter takes it, keyword-only
fd.from_sql(con, "SELECT ...", max_rows=5_000)
```

It is **refused, never truncated**. A chart of the first 5 000 rows presented as a chart of your
data is worse than an error, because nothing in the artefact says which it is — so an over-budget
frame stops, and trimming it stays your decision and reads as one. A `max_rows` above the wire limit
is itself refused: a cap above the format's own ceiling would claim a permission the format does not
grant.

When the answer is "these rows genuinely belong in the artefact", the next section is the form that
carries them.

## Point at the data instead of carrying it

Sometimes the right artefact is a **pointer**. `ref_parquet` returns the `ref` form of
`DataSource`: the host resolves the URL when it renders — a browser reading a remote Parquet by
HTTP range request, for instance — so the document stays kilobytes whatever the Parquet weighs, and
the rows never travel through the notebook that authored it.

```python
from fuaran_ui.ui import col

ref = fd.ref_parquet(
    "https://data.example.org/sales.parquet",
    schema={"region": "string", "revenue": "float"},
)

fr = ref.frame().filter(col("region").eq("EMEA")).sort("revenue", descending=True)
binding = fr.to_transform_binding()  # refuses col("revenu") HERE, not in the browser
```

**The schema is declared up front because nothing else can type the pipeline.** A ref carries no
rows, so without a declaration nothing downstream can tell a typo from a column the host will
supply, and the mistake surfaces as an empty grid in somebody else's browser. `ref.frame()` returns
the host's own `Frame` with one method overridden — every verb, every rename, every `bind` is
unchanged — and the check runs where the host already refuses an unbound parameter, at
`to_transform_binding()`. It follows the columns through the pipeline, so a derived name is
readable afterwards and a column a `group_by` dropped is not; where the resulting columns are
genuinely unknowable, past a `pivot` or a `join`, it stops checking rather than guessing.
`fd.check_binding(binding)` is the same check as a plain function, for a binding built some other
way.

**A URL in a document is published, so one carrying a credential is refused.** Userinfo
(`https://user:pass@host/…`) is refused outright. A query string carrying a signature or token — a
presigned S3, GCS or SAS link — is refused too, but that one is a real and deliberate pattern, so
it is an opt-in rather than a ban: `ref_parquet(..., allow_signed_url=True)`. The point is that a
presigned link ends up in a shared file because someone decided it should.

`examples/sql_sources.py` is the whole thing end to end: DuckDB over a public Parquet by HTTP range
request, and the same dashboard exported twice — once with the rows embedded for offline reading,
once carrying the ref for a host to resolve.

## Write polars, ship data

The adapters above carry the **source**. This carries the **pipeline**: a polars `LazyFrame`'s
logical plan lowered into the same serializable `Transform` the algebra runs everywhere, so a
pipeline you already have travels as data instead of being rewritten.

```python
import polars as pl
import fuaran_ui_data as fd

lf = (
    pl.LazyFrame(rows)
    .filter(pl.col("amount") > 0)
    .with_columns((pl.col("amount") * pl.col("units")).alias("value"))
    .group_by("region", maintain_order=True)
    .agg(pl.col("value").sum().alias("total"))
    .sort("total", descending=True, nulls_last=True)
)

fr = fd.frame_from_polars(lf)  # source + lowered pipeline, as a host Frame
binding = fr.to_transform_binding()  # the node source, wire-ready
fr.collect()  # the same reference evaluator, for a preview
```

`lower_polars(lf)` returns the pipeline alone, when you have a source of your own.

`filter`, `select`, `with_columns`, `group_by().agg()`, `sort`, `unique`, `head` / `slice` and
equi-`join` lower, over the algebra's scalar expression set — arithmetic, comparison, logic,
`cast`, `when`/`then`/`otherwise`, `fill_null` / `coalesce`, `abs` / `round` / `floor` / `ceil`,
the string length and case functions, `str.slice`, and `dt.year` / `month` / `day`.

**Anything else is refused by name.** Not approximated, not dropped: a wrong lowering would produce
a `Transform` that is valid on the wire, runs everywhere, and computes something you never wrote,
and nothing downstream would ever say so. Each refusal names the polars node it found, why the
algebra cannot carry it, and — where you can express what you meant in a form that does lower — the
one edit that does it. So `sort()` asks for `nulls_last=True` (the algebra sorts nulls last in both
directions), `join()` asks for `coalesce=False` (polars drops the right-hand key and the algebra
keeps both), and `std()` asks for `ddof=0` (the algebra's is the population deviation).

**polars' serialised plan is not a stable public contract**, so the lowering declares the version
range it is tested against — currently `>=1.33,<2.0` — and refuses outside it rather than reading a
shape it has never seen. The test suite asserts the installed polars is inside that range, so moving
a bound is a test run rather than a judgement.

Three differences belong to the algebra rather than to the lowering, and no edit resolves them:
summing a group whose values are **all** null is `0` in polars and null in the algebra; division by
zero is null in the algebra and infinity in polars; and row order out of an unordered step is
nobody's promise, so pass `maintain_order=True` if you want the two previews to line up. The
subpackage docstring carries the full mapping and refusal tables.

## One file, and a live renderer in the cell

```python
fd.export_html(dashboard, "revenue.html", title="Revenue by region")  # one self-contained file
fd.notebook.display(dashboard)  # interactive, in the cell
```

Both render with the **real browser renderer**, which this package embeds: the built standalone
bundle and the reference stylesheet ship inside the wheel. *Maintainers run Node to produce that
bundle; consumers never do* — there is no build step, no toolchain and no server anywhere on the
install side.

**The exported file is self-contained, and a browser enforces it.** The tree, the renderer and the
stylesheet are inlined, and the document declares `default-src 'none'`, so it renders from `file://`
with no network at all — verified as zero subresource requests, not merely intended. Mail it, attach
it to a ticket, put it in an artefact store; sharing a result means sending a file.

**It costs about 0.6 MB.** The worked example in [`examples/dashboard.ipynb`](examples/dashboard.ipynb)
exports at 612,898 bytes, of which 508,828 is the renderer and 101,093 the stylesheet — so the figure
is near-constant, and a larger dashboard is barely a larger file. Every interactive notebook output
carries the same copy, inside a sandboxed frame, which is why a notebook saved with its outputs still
renders on a machine with no Python and no kernel. For outputs that only need to be *read*, the
host's own static display is a fraction of the size.

`fuaran_ui_data.notebook.interactive(tree)` returns the same output as a value and needs **no
notebook library at all** — the display protocol is a method name, not an import. Only
`display()`, the convenience that pushes an output from anywhere in a cell, needs the `notebook`
extra.

## Re-run the cell, patch the output

```python
view = fd.notebook.patch(dashboard, show_ops=True)  # display it, and keep the handle
view  # the cell's last expression

view.update(rebuilt_dashboard)  # …later: PATCHED, not re-rendered
```

`update` does not redraw anything. It diffs the tree it is holding against the one you passed and
sends the resulting `TreeOp` script to the renderer **already in the output**, which applies it in
place — so scroll position, an open disclosure, a sorted grid, focus, everything the browser owns
and the tree does not, survives. Editing the aggregation behind the dashboard in
[`examples/patching.ipynb`](examples/patching.ipynb) costs **four ops**; `update` returns their
summaries, and `show_ops=True` lists them under the output.

**Every op is re-decoded on the way in.** What crosses the channel is canonical wire JSON, and the
renderer's own decoder runs on it at the frame's edge: the kernel is not trusted to have produced a
well-formed op merely because it produced one. A refusal stops the batch with the displayed tree
untouched, surfaces the diagnostic, and falls back to mounting the tree whole — with no round trip,
because the current tree is synced state.

**A patched output is live state, and that is the trade.** Patching needs a channel from the kernel
to the output, which is the Jupyter widget protocol; this path therefore needs the `notebook` extra
(`anywidget` carries this package's own front-end code as widget state, which is how a pure-Python
distribution ships JavaScript without publishing a front-end extension). It renders where a kernel
is running, or where the front end saved and restored widget state. When you want an output that
survives anywhere — a mailed notebook, a machine with no Python — that is `interactive`, whose
output is a whole self-contained document. Both are worth having.

## Three projections, one tree

```python
fd.export_html(dashboard, "gallery.html")  # the live page
fd.export_markdown(dashboard, "gallery.md", title="Weekly")  # the crawlable document
fd.export_email_html(dashboard, "digest.html", subject="Weekly")  # the Monday-morning digest
```

Nothing is authored three times. There is one tree; each file is a **projection** of it, and each
projection is a pure function of that tree and its resolved data — so re-running produces
byte-identical files, and all three resolve their text and figures through the same host
functions. A document, a digest and the page cannot disagree about what a number is.

[`examples/three_projections.py`](examples/three_projections.py) is the worked example: one
dashboard, three files, runnable on a bare install. The suite runs it and asserts the outputs, so
it fails if it rots.

**What differs between them is declared, not incidental.** The document and the digest render for
targets that execute nothing, so each wire kind needs an answer to a question the live page never
asks: *what does this become when nothing runs?* The host declares one per kind, in a scope table
next to the code — `rendered`, `structural`, `openLive` (a labelled "open live" affordance, never
a half-working control), or `omitted`. Three of those are visible in the example:

| | live page | document | digest |
|---|---|---|---|
| a **chart** | drawn by the browser renderer | inline SVG, or its rows as a table — your choice | an "open live" link: Outlook's Word engine draws no SVG |
| a **button** | a control | a labelled affordance | a labelled affordance: a button that cannot fire is worse than none |
| a **disclosure** | collapsible | always expanded | always expanded: a section a reader cannot open is content they never learn exists |

The tables are checked rather than asserted — completeness is measured against the wire format's
own per-kind fidelity manifest, so a new node kind cannot arrive with no declared posture, and a
kind that manifest marks as gaining its behaviour at hydration *must* project to an affordance in
both. A new interactive kind therefore reddens a test instead of shipping a dead button to an
inbox.

**The chart choice is yours, and it is a real one.** `MarkdownOptions(charts=...)` takes `"svg"`
(the picture, lowered through the host's own chart lowering and inlined) or `"table"` (the chart's
resolved rows as GFM). Most markdown renderers — including the one this project ships — escape raw
HTML by construction, so an inlined `<svg>` prints as visible angle brackets through them. Neither
mode is the safe one and neither is a fallback: they are two honest readings of a picture, and you
know which reader is downstream.

**`EmailOptions(live_url=...)` is what turns an affordance into a link.** Without it the digest
says "available in the live view", which is honest; with it, each affordance links back to its own
node in the page. Declare it when you have somewhere to point.

**The projections live in the host, not here.** `fuaran_ui.renderer.render_markdown` and
`render_email_document` sit beside the HTML renderer they have to agree with, which is where the
scope tables, the destination-policy consultation and the email-hostile-construct lint belong.
What this package adds is the gesture it already offered for the live page: tree in, one file out,
written as UTF-8 with `
` endings on every platform — because an artefact whose bytes depend on
the machine that produced it cannot be diffed against one produced elsewhere.

## Numbers and dates for a locale

The host renders `Binding.Format`'s `Number`, `Currency`, `Percent` and `DateTime` in a
**locale-free** form — `1234.50`, `GBP 1234.50`, `42.0%`, `2023-11-14` — because it is
standard-library-only and those four cases are a locale database's answer. The `locale` extra is
that database: a formatter backed by [Babel](https://babel.pocoo.org/), which carries the Unicode
CLDR, plugged into the host's locale formatter seam.

```bash
pip install "fuaran-ui-data[locale]"
```

```python
from fuaran_ui.renderer import render_html
from fuaran_ui.renderer.bindings import BindingSources
from fuaran_ui_data.locale import BabelLocaleFormatter, install_babel_formatter

# Per render: this pass formats for each binding's own locale.
html = render_html(tree, BindingSources(locale="en-GB", formatter=BabelLocaleFormatter()))

# Or process-wide, once at start-up: every pass that names no formatter picks it up.
previous = install_babel_formatter()
```

A `Currency` bound for `en-GB` then renders `£1,234.50`, a `Percent` for `fr-FR` `42 %`, a `Medium`
date for `fr-FR` `14 nov. 2023` — in the page, the markdown document, the email digest and the
speech projection alike, because all four resolve text through the same seam. A per-render choice
wins over the installed one. Without the extra, `BabelLocaleFormatter()` and
`install_babel_formatter()` raise `LocaleFormattingUnavailableError` naming the install line above;
importing `fuaran_ui_data.locale` imports no Babel.

**It honours the tag the binding resolves** — `LocaleSource.Explicit`'s own, or the pass's ambient
`locale` — and each case's declared options: the decimals, the ISO currency code (with that
currency's own minor-unit digits), the date and time style pair. The option shapes and the
rounding (half away from zero, on the shortest decimal spelling of the value) follow the browser
host's `Intl` calls, and instants render in UTC, since a server-side render has no viewer's zone.

**It does not guess.** A tag Babel cannot parse, or a locale CLDR does not have, gets the host's
locale-free form rather than a neighbouring locale's text, which would read as correct. An empty
ambient tag means the runtime default: `BabelLocaleFormatter(default_locale="en-GB")` when you name
one, else the process locale from the environment, else the locale-free form.

**A `Metric` is not one of these sites.** Its figure is a numeric slot whose `format` carries no
locale, so the host formats it through its locale-free cell path whichever formatter is installed —
as the browser host's server renderer does too. Put the figure in a text slot through
`Binding.Format` to localise it.

**Locale output follows the installed Babel's CLDR version.** CLDR revises separators, symbols and
patterns between releases, so a Babel upgrade can change the text for the same input — which is why
the wire format excludes these four cases from byte parity between hosts. The suite measures this
formatter against the browser host's `Intl` output for `en-US`, `en-GB`, `fr-FR`, `de-DE` and
`ja-JP` over the same CLDR release: every corpus site agrees, and the few differences it records —
the narrow no-break space V8 swaps out of English times, the "at" joiner ICU uses for long dates —
are asserted with their causes rather than smoothed over.

## Or keep the tree in Python and stream the ops

The export bakes a tree into a file and the notebook output patches a cell. There is a third place the tree
can live, and it is a **worked sample** rather than package surface:
[`samples/live-host/`](samples/live-host/README.md) holds one tree per session in a FastAPI process,
streams typed `TreeOp` scripts to a browser over Server-Sent Events, and folds interactions
dispatched back from that browser through the host's own `fuaran_ui.ops.apply` — every accepted edit
and every refused op landing in a hash-chained stream `verify_chain` can check.

```bash
pip install "fuaran-ui-data[live-host]"
cd samples/live-host && uvicorn app:app --port 14150
```

It is for the workloads the other two cannot carry: data too large to inline in a document, a
computation too heavy for a browser, or a source behind a credential the browser must never hold.
The tree's *inputs* never leave the server; what crosses is the tree, and then the ops that edit it.

| Subpackage | Role |
|---|---|
| `fuaran_ui_data.adapters` | **Shipped.** pandas, polars, and Arrow into the Compute layer's columnar model. Arrow arrives through the dataframe interchange protocol (`__dataframe__`), so anything that speaks the protocol is supported without a fourth code path. |
| `fuaran_ui_data.export` | **Shipped.** The embedded standalone browser renderer, behind a one-call self-contained HTML export — one file, openable offline, no build step and no server. Plus the same tree's other two projections: a markdown document and an email-safe digest, one call and one file each. |
| `fuaran_ui_data.notebook` | **Shipped.** Interactive output in a sandboxed frame, and in-place patching — a re-executed cell applies typed tree operations to the output already on screen rather than replacing it, so scroll position, selection and interaction state survive the edit. |
| `fuaran_ui_data.controls` | **Shipped.** The state store and write-back port both documents mount through, which is what makes a control in them recompute rather than merely render. |
| `fuaran_ui_data.share` | **Shipped.** The same tree as a link — the host's teleport bundle in a URL fragment, opening live in the public receiver. Nothing is uploaded and nothing is stored: the link is the artefact. |
| `fuaran_ui_data.testing` | **Shipped.** Structural assertions and canonical-wire snapshots over a tree, as a pytest plugin — the UI is data, so it is tested by reading its structure rather than its pixels. |
| `fuaran_ui_data.capabilities` | **Shipped.** The other half of the compute split — a Python body, declared with typed argument holes and a return kind, that a host resolves an `Invoke` to by id. The artefact carries the id and the args and nothing else. |

## The part that cannot travel

Most notebook work turns out to be liftable: a filter, a sort, a group-by is a `Transform` that travels as
data, and whatever renders the tree evaluates it. The remainder is not — model inference, scipy, a numpy
kernel written for one dataset — and no amount of algebra will make it travel. `fuaran_ui_data.capabilities`
is the seam for that remainder: a **Python body, registered under an id**, that the wire references by name.

```python
from fuaran_ui.ui import fuaran as F
from fuaran_ui.ui.capability import enum, int_range, invoke
from fuaran_ui_data.capabilities import Capabilities, Realised

caps = Capabilities()


@caps.capability(
    "forecast.revenue",
    args={"horizon": int_range(1, 12), "scenario": enum("base", "high", "low")},
    returns="columns",
)
def forecast(horizon: int, scenario: str) -> dict[str, list[object]]:
    """Whatever cannot travel — a model, scipy, a kernel of your own."""
    ...


chart = F.chart(
    "forecast-chart",
    source=invoke("forecast.revenue", horizon=6, scenario="base"),
    x_field="period",
    y_fields=["revenue"],
)
```

**Encode that chart and its whole source is `{"$type":"Invoke","capabilityId":"forecast.revenue","args":[…]}`.**
No code, no model, no data, and nothing about how the number was arrived at. That is what the seam buys: a
dashboard stays *data* even when part of what it shows is not, so it exports, shares as a link, snapshots and
diffs exactly as one with no capability in it. A reader without a host gets a chart whose source has not been
realised — an honest thing to be — rather than a chart that quietly stopped being data.

Both boundaries are checked, and in opposite directions. The arguments arrive from outside the process, so they
are validated against the closed `HoleSpace` vocabulary *before* the body runs; the return goes into a tree that
will be encoded, so it is checked against the declared kind *after*. A parameter with no declared hole is refused
at declaration — at import, where the mistake is — rather than at whichever invocation first happens to reach it.

Resolving is **total**: it returns a value or a named refusal, never an exception.

```python
outcome = caps.resolve_json(document)  # or resolve(id, args) / resolve_invocation(parsed)
outcome.value if isinstance(outcome, Realised) else (outcome.code, outcome.reason)
```

An unregistered id, an argument outside its space, a body that raised, a body that ran past its declared bound
and a return the wire cannot carry are all *answers* a host can put in the slot the invocation was bound to. A
host that let those raise would have to decide, at every call site, which exceptions are a bad request and which
are a broken server — and the usual result of that decision is a 500 for both.

One thing about the bound is worth knowing before you rely on it: **it bounds the wait, not the work.** CPython
cannot interrupt a running thread, so a body that exceeds its bound is abandoned rather than stopped — the caller
is told it timed out and the session carries on, while the thread runs to completion. That is the property a host
needs, and "timeout" ordinarily implies cancellation, so it is said plainly.

[`samples/live-host/`](samples/live-host/README.md) works it through end to end: a forecast chart bound to an
invocation, realised server-side from data the browser cannot reach, with every refusal written into the page as
a named failure and recorded.

## Move a control, and it recomputes

```python
from fuaran_ui.ui import col, control, param

region = control.select("region", options=col("region").unique(), source=fd.frame(df))
scoped = fd.frame(df).filter(col("region").eq(param("region"))).bind(region)

fd.export_html(quick.dashboard("Revenue", region, chart_over(scoped)), "revenue.html")
```

Open that file with no server and no Python and the chart regroups when the select moves. The
control declares a state slot; the pipeline reads it through `param("region")`; the embedded
renderer re-derives the `Transform` from the current store — the same evaluator, on the same
parameters, that the host runs. `select` / `multi_select` / `range` / `date_range` are the host's
(`fuaran_ui.ui.control`); what this package adds is the **store** the renderer resolves against and
the **port** a control writes through, because whose state that is and where it lives is not a
renderer's question. Both are ~30 lines of JavaScript shipped as package data, and both documents —
the exported file and the notebook output — mount through them.

Three behaviours worth knowing, because each is a decision rather than a detail:

- **The declared default seeds the slot** (wire format §24.4), so an untouched document already
  shows the default's view rather than an empty one.
- **A cleared control restores the unwritten slot**, so an unseeded parameter prunes its filter and
  the table is unfiltered — "nothing chosen" is the absence of a constraint, not a constraint that
  nothing satisfies.
- **A parameter no control fills is refused where it is authored**, by name, rather than silently
  dropping a filter in someone else's browser.

## Ask for a change

```python
from fuaran_ui.op_stream import InMemorySink
from fuaran_ui_data.ask import AskSession, HostedProvider

view = fd.notebook.patch(dashboard)  # the cell's last expression: the output appears
stream = InMemorySink()
session = AskSession(view, provider=HostedProvider.from_env(), sink=stream)

session.ask("make region a filter")
# applied 2 ops from hosted: InsertChild  revenue-dashboard <- region-filter; ReplaceBinding  chart.source
```

The output is **patched, not redrawn** — the sort you set and the disclosure you opened survive,
because the ops travel over the channel `patch` already opened.

**What comes back is ops, never a re-rendered tree, and that is the whole design.** A model that
hands you a tree can only be trusted as far as you inspect the tree. A model that hands you ops
against a tree you are holding can be checked one op at a time, by the host's own machinery, before
anything moves: each op re-decodes through the wire codec, every node it carries goes through the
pre-emit validator, every action shape it wires up meets the default-deny dispatch gate, and the
script folds through the apply engine **all or nothing**. A refusal names the stage, the code and
the path, changes nothing, and is fed back to the provider **once** as a repair before you see it —
once, not until it works, because a model that has been told what is wrong and got it wrong again
is not converging. Every applied op is recorded in the op stream you pass, hash-chained and
attributed to an agent actor carrying the provider's own model label.

The default dispatch gate permits local state edits and **denies every outward effect**, so an ask
cannot wire up a navigation or a capability call nobody asked for. Permit shapes one at a time,
deliberately: `gate=DispatchGate.permissive_inert().with_permitted("Navigate")`.

### Which path is the paid one

There are two providers, and the honest way to describe the difference is who pays.

- **`HostedProvider` is the paid path.** It reaches the Fuaran generation endpoint through the
  host's public client, and that endpoint's URL and access token are the commercial gate. What the
  money buys is a service that already knows the wire format, already emits canonical ops, and asks
  nothing of you in the way of prompt engineering. Read the credentials from the environment
  (`HostedProvider.from_env()`) — never type one into a cell, which is saved with the notebook and
  rendered in every copy of the output.
- **`ByokProvider` is bring-your-own-key, and it exists so this package is runnable without buying
  anything.** An Apache-2.0 distribution whose only path runs through a paid endpoint is a
  brochure. So this one posts to whichever model API you already hold a key to, with the op
  vocabulary supplied as a system prompt from here; you supply the key, the model and — for an API
  that does not speak the chat-completions shape — the request/reply pair. The trade is plain: the
  hosted path is tuned for this and supported; the BYOK path is a general model being asked to emit
  a format it has been told about in one prompt, so expect the repair turn to earn its keep.

**The BYOK key is held in memory and nowhere else.** This package never reads it from a file and
never writes it to one. It goes into the `Authorization` header of the one request, is kept out of
the request body, out of the provider's `repr`, and out of every message the provider returns —
each one passes through a redaction step first, so a transport error that echoed a header cannot
carry a credential into a saved notebook. The tests assert each of those absences by name.

A third provider, `ReplayProvider`, replays recorded emissions for a test or a worked example. It
announces itself as a replay in the op-stream record, so a recorded ask can never be mistaken for a
live one by someone reading the stream later.
## Send it as a link

```python
fd.share(dashboard)  # -> https://fuaran-ui.gallery/receiver.html#t=FT1.…
```

"Send me that dashboard" needs no hosting on either side. The link opens the app **live** in the
public receiver — the real browser renderer, no account, no sign-up — and it is a plain string, so
it prints, pastes and hands to `webbrowser.open` with nothing to unwrap. In a notebook the cell
renders it as a clickable anchor with its size beside it.

**What travels is everything in the tree, including the data.** The host encodes the tree, any
state you seed, and every embedded source's rows into one hash-sealed
[teleport bundle](https://github.com/fuaran-ui/fuaran-py) (canonical JSON, deflated, base64url)
and that bundle *is* the link. A `Ref` source is the exception, and it is the exception on
purpose: a `Ref` carries a name the receiving host resolves, so its rows never enter the link at
all.

**What does not travel is anything to a server of ours, because there is no upload step to
travel by.** The bundle rides after the `#`, and a browser never puts a URL fragment in an HTTP
request — so the receiver's static host serves a page and learns nothing about what you shared.
Nothing is stored server-side, there is no record to delete and no link to expire: **the link is
the artefact**. Open the network tab and watch, which is the point of the design — it makes
"nothing was uploaded" a checkable fact rather than a policy claim.

**Over budget is refused, never truncated.** A bundle is sealed over its whole envelope, so a
shortened one does not carry a smaller app — it carries nothing, and fails its digest check on
arrival. The refusal names the size, the budget, the embedded rows and the two ways out:

```
the teleport bundle is 27,255 characters, 19,255 over the 8,000-character budget. The tree
embeds 4,000 rows across 2 columns, in one source. Two remedies, and truncating is neither:
replace the embedded rows with a Ref source, whose name the receiving host resolves so the rows
never enter the link at all; or export the whole thing, data included, with
fuaran_ui_data.export_html(tree, path) — one self-contained file, under no budget because it is
not a URL.
```

The default budget is the host's practical URL ceiling (8,000 characters); pass
`budget=fuaran_ui.teleport.BUDGET_QR_HARD` when the link has to survive being photographed as a
QR code, and `origin=` to point at your own deployment of the receiver.

> The default origin is the **showcase** (`fuaran-ui.gallery`), not the authoring playground
> (`fuaran-ui.live`). They are deliberately separate origins so the showcase's zero-key-egress
> posture is inspectable per origin, and the teleport receiver ships in the showcase artefact.
## Unit-test your UI

**A Fuaran UI is data, so it is tested the way data is tested: by reading its structure.**

```python
from fuaran_ui_data.testing import assert_tree


def test_the_dashboard(fuaran_snapshot):
    dashboard = build_dashboard(df)

    assert_tree(dashboard).valid().kind("kpi-revenue", "Metric").children_count("kpis", 3).binding(
        "revenue-chart", "source"
    ).is_transform().text("title").contains("Revenue")

    fuaran_snapshot(dashboard)
```

`pip install "fuaran-ui-data[test]"` and the fixture is there — the plugin registers itself, so
there is no `conftest.py` line to remember.

**Restyle the dashboard and every one of those assertions stays green.** Not because they are
weak: because not one of them ever looked at a style, a class name, a colour or a geometry. That
is the difference between testing a UI and testing a picture of one, and it is what makes such a
suite worth keeping — a test that fails whenever anything moves is a test people delete. The
worked example in [`tests/test_gallery_dashboard.py`](tests/test_gallery_dashboard.py) proves it
both ways round: the same assertions are replayed against a restyled tree, *and* the canonical
bytes are asserted to have genuinely changed, so the demonstration cannot pass by the restyle
having done nothing.

**Every failure names three things**, because a message missing any one of them sends the reader
back to the tree it was meant to save them walking:

```
node 'kpi-revenue' is not a Chart.

  node      kpi-revenue
  path      gallery > kpis > kpi-revenue
  expected  kind 'Chart'
  found     kind 'Metric'
```

The path is the chain of node ids from the root — the way an author locates a node, and stable
under an edit that inserts a sibling.

`valid()` runs the host's pre-emit validator and reports **every** finding, not the first: a tree
with four duplicate ids has four things to fix, and a message naming one costs four runs to learn
what one run already knew. `text()` reads what a node *says* through the host's own document
projection, so it carries a metric's label and a chart's rows and no styling at all.

### The snapshot is the other half

`fuaran_snapshot(tree)` records the tree's **canonical wire bytes** under `__snapshots__/` beside
the test, one file per test, and `--snapshot-update` re-records them. Canonical is what makes that
worth doing: the wire format guarantees two structurally equal trees encode to identical bytes on
every host, so a byte difference here is a *structural* difference and can never be a formatting
one. A snapshot of rendered HTML would fail on a whitespace change; this one cannot.

The two halves answer different questions and neither replaces the other. The assertions say what
must stay true and are indifferent to everything else; the snapshot says nothing may change at
all, and is what notices a change nobody described.

Two behaviours are deliberate rather than incidental:

- **A missing snapshot fails**, naming `--snapshot-update`, rather than recording itself and
  passing. A snapshot that writes itself on first run makes a brand-new test green on a run that
  asserted nothing — including in CI, where nobody looks.
- **The stored bytes are compared; an indented view of them is diffed.** Canonical JSON is one
  long line, so the failure message re-indents both sides mechanically — inserting newlines at
  structural punctuation and touching nothing else — and shows a unified line diff. It never
  re-serialises: parsing and re-printing would re-lay-out every number, and the reader would be
  shown a diff that disagrees with the comparison that actually failed.

`assert_tree` itself imports no pytest and is usable in any runner, or in none.

## Every re-run, as a history you can walk back

```python
from fuaran_ui_data.history import History

hist = History()  # .fuaran/history.jsonl, beside the notebook
hist.record(dashboard)  # the last line of the cell, on every re-run
```

That is the whole of the authoring side. Each re-run's diff is appended to a hash-chained op
stream, so *"what changed since the version I sent on Monday"* stops being a question about
memory:

```python
hist.history()  # [Version(number=1, ops=1, actor='ann', …), Version(number=2, ops=3, …)]
monday = hist.checkout(1)  # that tree back, byte-for-byte
hist.verify()  # Intact(records=3, head='9c1f…')
```

**One version per re-run, one record per version, and no bookkeeping beside the chain.** A version
number *is* a chain sequence and an op count *is* the size of that re-run's diff, so there is
nothing here that can disagree with the file. The first record is a diff against an empty tree, so
the file alone reproduces every version — `checkout` needs no snapshot next to it that could rot.

**Attribution is recorded and it comes from somewhere you can point at**: the `actor=` argument,
else `$FUARAN_ACTOR`, else the account the kernel is running as. That last one is a documented
default rather than a guess — it is the only identity a process can observe without being told,
and it is what a version-control system writes when nothing configures it — but it does mean an
account name lands in a file you may go on to share.

**What the chain proves, and what it does not.** Every record's hash folds its predecessor, the op,
the actor, the timestamp and the outcome, so an edit in the file's interior, a removed record or a
reordered one is caught and named *by its position in the file*. What no hash chain can see on its
own is a clean truncation of its tail — what is left is a shorter, perfectly valid chain — so pass
the head hash you kept and the length is proved too: `hist.verify(head=…)`.

### Two analysts, one dashboard

```python
report = hist.merge(colleague)  # three-way, over the version you both forked from
```

The base is found from the chain itself: two histories that forked share a byte-identical prefix,
and the first differing hash is the fork. Anything only one of you touched merges silently.
Anything you both touched comes back as a conflict carrying **all three values** — and nothing is
recorded until a person settles it:

```
1 conflicted cell(s) across 1 node(s), forked at version 2:
  title  [unpicked]
    kind
      base   {"id":"title","kind":{"$type":"Heading","level":1,"text":"Units by region",…}}
      ours   {"id":"title","kind":{"$type":"Heading","level":1,"text":"Units by region — Q3",…}}
      theirs {"id":"title","kind":{"$type":"Heading","level":1,"text":"Q3 units by region",…}}
```

```python
report.pick("title", "theirs")
report.resolve()  # merges again with that choice and records the result as a version
```

`pick` takes a side per node — `"ours"`, `"theirs"` or `"base"` — and picking a node picks its
subtree, so a conflict reported at a leaf costs a leaf. `resolve` refuses while any conflicted node
is unpicked, and names the ones outstanding; a merge that quietly chose for you would be worse than
one that failed, because you would go on to publish the result. The resolution is recorded as an
ordinary version, so the fact that a merge happened — and what it settled on — is in the chain like
any other edit.

The diff, the chain, the replay and the three-way merge are all the host's, unchanged. What this
package adds is somewhere durable to put the stream and a merge surface a person can settle.
[`examples/history.ipynb`](examples/history.ipynb) is the worked version: three re-runs, a
checkout, and a merge with one conflict.

## Already have a Streamlit script?

```python
import fuaran_ui_data as fd
import fuaran_ui_data.st as st

st.title("Regional revenue")
st.metric("Total", "£27,530", "8.4%")
st.dataframe(df)
st.bar_chart(df, x="region", y="revenue")

fd.export_html(st.fuaran_tree(), "revenue.html")
```

A script whose calls are within the covered set runs **unchanged** under
`import fuaran_ui_data.st as st` and produces an artefact instead of a server — a file to export,
a link to send, or live output in the cell.

**This is a migration aid, not a compatibility promise**, and it is a deliberate one: a
half-compatible surface invites the wrong comparison, so the shape of what it does *not* do is
stated here rather than discovered.

**The covered set is twelve calls**, and `fuaran_ui_data.st.COVERED` is the table — each call
against the constructor it becomes, including the places a mapping loses something. `title` /
`header` / `markdown` / `metric` / `dataframe` / `table` / `bar_chart` / `line_chart` /
`selectbox` / `multiselect` / `slider` / `columns`. Columns work both ways a script writes them
(`col.metric(…)` and `with col:`), and they nest.

**Everything else raises, by name.** `st.button` says that the artefact has no Python behind it and
names the covered controls; `st.session_state` says there is no session and points at the
parameter a control declares; an unknown call still refuses rather than arriving as an
`AttributeError` a `getattr` probe would swallow. The same rule reaches into a covered call's
arguments: one that would change *what the artefact says* — `column_config`, which can hide a
column; `on_change`, which names a callback nothing will call — is refused, while one that only
addresses a Streamlit server's own page chrome (`key`, `help`, `width`, `use_container_width`, …)
is accepted and dropped. That second set is closed and enumerable as
`fuaran_ui_data.st.IGNORED_ARGS`. Nothing is ever silently no-opped, because an artefact quietly
missing a section the script plainly wrote is a defect nothing downstream would ever report.

### The execution model is the difference, not the size of the table

Streamlit **re-executes the whole script** on every interaction: a widget call returns what the
reader last chose, and every line after it runs again with that value. There is no re-execution
here. The script runs **once**, top to bottom, and what it leaves behind is a tree.

So a control call returns its **default** — once. The consequence to hold on to:

```python
region = st.selectbox("Region", ["EMEA", "APAC"])  # returns "EMEA", always
if region == "EMEA":
    st.markdown("EMEA led the quarter.")  # this is what the artefact contains
```

**A script that branches on a widget value takes the branch its default selects, and bakes that
branch into the artefact.** Moving the select in the exported file does not re-run the `if` —
there is no Python there to run it. The same is true of a filter written in pandas over the
returned value: it is evaluated once, at authoring time, and the rows it produced are the rows the
artefact holds.

That is not the end of interactivity, it is a different kind of it. Each control call declares a
**parameter** (`selectbox` and `multiselect` one, `slider` two — `<name>_min` and `<name>_max`), and
a pipeline that reads one recomputes **client-side** when the reader moves the control, with no
Python anywhere. Making a migrated script respond is therefore one deliberate edit: turn the branch
into data.

```python
from fuaran_ui.ui import col, param

region = st.selectbox("Region", sorted(df["region"].unique()), key="region")
scoped = (
    fd.frame(df).filter(col("region").eq(param("region"))).bind(*st.fuaran_page().controls)
)  # the controls the script declared, in order
```

`key=` is read rather than dropped, because it is already exactly the fact needed: the name of the
parameter the control declares. Without one the label supplies the name, slugged.

## Install

```bash
pip install fuaran-ui-data                    # the host, and nothing else
pip install "fuaran-ui-data[pandas]"          # + pandas
pip install "fuaran-ui-data[polars]"          # + polars
pip install "fuaran-ui-data[arrow]"           # + pyarrow, via the interchange protocol
pip install "fuaran-ui-data[notebook]"        # + IPython, anywidget
pip install "fuaran-ui-data[live-host]"       # + FastAPI, uvicorn, httpx (the live-host sample)
pip install "fuaran-ui-data[duckdb]"          # + DuckDB and pyarrow, for from_duckdb
pip install "fuaran-ui-data[test]"            # + pytest (the structural-assertion plugin)
pip install "fuaran-ui-data[locale]"          # + Babel, for locale-aware Binding.Format text
```

`from_sql` and `ref_parquet` need **nothing** from that list: the first reads any DB-API 2
connection your own driver already provides, and the second is a pointer with a declared schema, so
neither has a dependency to install.

Requires CPython **3.12+**.

**Every dataframe and notebook integration is optional, and that is the shape of the package rather
than a convenience.** A base install pulls in `fuaran-ui` and nothing more; each adapter reads its
library through that library's own public surface, inside the call that needs it. So importing
`fuaran_ui_data` in an environment holding none of them succeeds — which is what lets this package
be a dependency of something that does not know which dataframe its caller uses. The tests assert
it in subprocesses that make the other libraries genuinely unimportable, because "we do not import
pandas at import time" is exactly the kind of property that decays through a single convenient
top-level import that breaks nothing for the person who adds it. CI runs the suite once per extra
installed alone, and once with none of them.

## The host pin, and why it has two ends

```toml
dependencies = ["fuaran-ui>=0.10.0,<0.11"]
```

The **lower** bound names the current host release, 0.10.0. Nothing here needs a surface 0.10.0
added: the raise is an adoption, so that this package's first release installs beside the host a
user gets today, and the suite runs green against it. What the package needs from the host was set
by 0.8.0, the first release carrying the locale formatter seam the `locale` extra plugs into, and
the `Expr` `concat` that `concat_str` lowers to — against 0.7.0 the first is missing and the second would fail on first
preview. Beneath that, 0.7.0 was the first release under the `fuaran-ui` name, and what the older
floors bought still stands — the decoder that routes every container's children through the node path,
which the `testing` plugin's assertions rely on to see inside a disclosure, and whose op-stream
replay skips recorded refusals by default; and the markdown and email-safe projections
`export_markdown` and `export_email_html` call, plus the parameter-bound controls the `controls`
runtime wires. The pin is the contract, and nothing here feature-detects: a package that probed for
either surface would install cleanly against a host that cannot serve it and fail later — a
projection call would fail outright, and a control authored against an older host would render
correctly and inertly, which is worse, because nothing says so.

The rule the floor follows is that a version declared in a source tree is not a released one —
pinning a version that exists only on a maintainer's machine makes a package uninstallable for
everyone else while resolving perfectly at home. So the floor names a release that is on the index,
and a raise waits for the host's tag. Tag the host first.

The **upper** bound makes a host bump a deliberate adoption. The host tracks parity with the
language-neutral wire specification; this package tracks the dataframe and notebook ecosystem. Those
are different clocks, and a floating upper bound lets one drag the other — quietly, between two
installs, with nothing recorded here to attribute the change to.

## Develop

```powershell
pwsh ./run.ps1                 # provision .venv (first run) + lint + format + types + tests
                               #   + embedded-renderer drift + boundary
pwsh ./run.ps1 -SkipInstall    # run the gate against an already-provisioned .venv
```

Or drive the tools directly inside the venv: `ruff check .`, `ruff format .`, `mypy`, `pytest`.

### The embedded renderer (maintainers)

The two files under `src/fuaran_ui_data/_bundle/` are **generated, never hand-placed**:

```powershell
pwsh ./scripts/sync-renderer-bundle.ps1 -Sync   # rebuild, copy, re-fingerprint
pwsh ./scripts/sync-renderer-bundle.ps1 -Check  # the drift gate, also run by run.ps1 and CI
```

The sync needs a `fuaran-ts` checkout (adjacent, or named by `FUARAN_TS_REPO`) and Node. The
**check** does not: the committed bytes are always hashed against the fingerprint recorded beside
them, and the questions that need the renderer source are reported **NOT CHECKED** where it is
absent rather than passed over quietly — because "nothing to check here" and "everything checked"
must not read alike.

The last stage of that gate is a **publication-boundary sweep**
([`gates/check-publication-boundary.ps1`](gates/check-publication-boundary.ps1)): every tracked file
is checked for vocabulary a reader outside this repository could not look up. It prints the residue
it knowingly carries on every run, green included — an exception that makes a gate silent is the
failure such a gate exists to prevent. `-ListPatterns` prints the catalogue.

## Licence

Apache-2.0. See [`LICENSE`](LICENSE) and [`NOTICE`](NOTICE).
