Metadata-Version: 2.4
Name: forktex-grid
Version: 0.1.0
Summary: Postgres-native dynamic tabular tier — tables, columns, relations and rows declared at runtime, built on the forktex substrate.
License-Expression: AGPL-3.0-or-later OR LicenseRef-ForkTex-Commercial
License-File: LICENSE
License-File: NOTICE
Author: FORKTEX
Author-email: info@forktex.com
Requires-Python: >=3.14,<4.0
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Database
Classifier: Topic :: Database :: Database Engines/Servers
Classifier: Topic :: Software Development
Classifier: Typing :: Typed
Requires-Dist: asyncpg (>=0.31)
Requires-Dist: forktex (>=0.11,<1)
Requires-Dist: pydantic (>=2.0)
Requires-Dist: sqlalchemy[asyncio] (>=2.0)
Project-URL: Bug Tracker, https://github.com/forktex/forktex-grid/issues
Project-URL: Changelog, https://github.com/forktex/forktex-grid/blob/master/CHANGELOG.md
Project-URL: Documentation, https://github.com/forktex/forktex-grid/tree/master/docs
Project-URL: Homepage, https://forktex.com
Project-URL: Repository, https://github.com/forktex/forktex-grid
Description-Content-Type: text/markdown

# forktex-grid

Tables, columns, relations and rows declared at **runtime**, stored in Postgres.
When your tables are known at deploy time you write DDL; when your tenants
define them, you need this.

```bash
pip install forktex-grid
```

Python 3.14, PostgreSQL 14+. Built on the
[`forktex`](https://github.com/forktex/forktex-py) substrate, which pip resolves
for you.

## What it does

You declare the shape you want as data. Grid converges the schema toward it, and
gives you a typed, filterable, paginated table on the other side.

```python
from forktex_grid import ColumnSpec, Namespace, Schema, TableSpec

ns = Namespace(session, str(org_id))

await ns.apply(
    Schema(
        tables=[
            TableSpec(
                slug="invoice",
                label="Invoice",
                columns=[
                    ColumnSpec(key="number", label="Number", type_id="text"),
                    ColumnSpec(key="total", label="Total", type_id="decimal"),
                    ColumnSpec(key="issued_at", label="Issued at", type_id="date"),
                ],
            )
        ]
    )
)

invoices = await ns.table("invoice")
await invoices.create({"number": "INV-001", "total": 250, "issued_at": "2026-03-01"})

page = await invoices.query(
    filter={"column": "total", "op": "gte", "value": 100},
    sort=[{"column": "issued_at", "direction": "desc"}],
    limit=50,
)
for row in page.rows:
    print(row.values["number"])
```

Nothing above was in a migration. `apply` is idempotent — call it again with a
fourth column and grid adds exactly that column.

Use it when the shape is not known up front: per-tenant custom fields,
user-defined record types, agent-authored state. **When your tables are known,
use `forktex.database` and real DDL.** Grid is not a replacement for a schema
you can write down.

## Installing

```bash
pip install forktex-grid
```

**Install with Python 3.14.** `forktex` and `forktex-grid` both declare
`requires-python = ">=3.14"`, and pip hides a release whose floor your
interpreter does not meet rather than saying why — so an older interpreter
reports an unrelated older lineage and the dependency looks unsatisfiable. It is
not; the interpreter is.

Developing against a local checkout of the substrate:

```bash
pip install -e ../forktex-py
pip install -e .
```

## Wiring it into your app

There is no global state. Migrate the substrate once at startup, then construct
one `Namespace` per request over the session you already have.

```python
from contextlib import asynccontextmanager

from fastapi import Depends, FastAPI
from sqlalchemy.ext.asyncio import AsyncSession

from forktex.database import close_engine, connection, init_engine, session_scope
from forktex_grid import Namespace, apply_migrations


@asynccontextmanager
async def lifespan(app: FastAPI):
    init_engine(settings.db_url)
    await apply_migrations(connection.engine)   # idempotent, concurrency-safe
    yield
    await close_engine()


app = FastAPI(lifespan=lifespan)


@app.get("/invoices")
async def list_invoices(
    org_id: str,
    session: AsyncSession = Depends(session_scope),
):
    grid = await Namespace(session, org_id).table("invoice")
    return (await grid.query(limit=50)).rows
```

`apply_migrations(engine, *, schema=...)` brings up the `forktex_grid` schema. It
is safe to run on every replica at once.

`namespace` is a plain string and defaults to `""`. Tenant isolation is by
convention — enforced by passing the right value. There is no ambient context
that will do it for you.

## How consumers use it

One example and one sentence each; **[`docs/grid.md`](docs/grid.md) has the exact
semantics.**

### Evolving a schema

`apply` takes a typed `Schema` or a plain JSON dict, so a schema can be built at
runtime or loaded from a file.

```python
report = await ns.apply(schema, prune=True, allow_destructive=True, dry_run=True)
```

`prune=True` makes the declaration authoritative — anything absent is removed.
`allow_destructive=True` is required before any drop or type-tightening.
`dry_run=True` plans without writing. The return value is a reconcile report as a
JSON dict; read it before you run the same call for real.

### Schema and data in one transaction

```python
from forktex_grid import RowOp

await ns.batch(schema=schema, rows=[RowOp(op="insert", table="invoice", values={...})])
```

Either the whole batch lands or none of it does.

### Filtering and sorting

`filter` and `sort` accept the plain-dict wire forms — so they can arrive
straight from a request body — or the typed `FilterNode` and `SortKey` from
`forktex.database.filters`, which is where the operator vocabulary lives. Grid
does not define its own: `FilterOp` on grid's surface is re-exported from the
substrate, so a filter written against one consumer reads the same against
another.

### Custom fields on tables you already own

An **extension** is a grid table whose rows link 1:1 to a host row by
`external_ref`. Your table is never touched.

```python
from forktex_grid import Extension

clients = await ns.declare(
    TableSpec(
        slug="client_ext",
        label="Client extension",
        binding=Extension(physical_relation="public.client_record", primary_key="id"),
        columns=[ColumnSpec(key="account_manager", label="Account manager", type_id="text")],
    )
)

await clients.create({"account_manager": "IP"}, external_ref=host_id)
row = await clients.get_by_external_ref(host_id)
```

`external_ref` accepts any host primary key — `int`, `str` or `UUID` — stored as
canonical text, so a table keyed by `bigserial` does not have to mint a
surrogate.

### Querying a table you already own

An **overlay** projects an existing physical table through the grid query API.
It is read-only, offset-paginated, and accepts plain column projections only.

```python
from forktex_grid import Overlay

await ns.declare(
    TableSpec(
        slug="client",
        label="Client",
        binding=Overlay(
            physical_relation="public.client_record",
            primary_key="id",
            namespace_column="org_id",
            column_map={"name": "display_name"},
        ),
        columns=[ColumnSpec(key="name", label="Name", type_id="text")],
    )
)
```

Reach for it when you need existing rows *inside* a grid query, not merely
annotated. [`docs/grid-binding-design.md`](docs/grid-binding-design.md) covers
both bindings and why the seam is data rather than a mixin.

### Relations

```python
await grid.relate("owner", source_id=invoice_id, target_id=client_id)
clients = await grid.related("owner", source_id=invoice_id)
```

Everything past the first argument is keyword-only. `relate` is the reason:
three same-typed positionals mean swapping two is a silent semantic bug no type
checker can catch.

### Registering your own field type

`type_id` is a validated `VARCHAR(64)`, not a closed enum — the built-ins are a
seed. Adding one is a code-only change, with no enum edit and no migration.

```python
import ipaddress

import sqlalchemy as sa

from forktex_grid import Capabilities, FieldTypeHandler, FilterOp, register_field_type


class IPv4Type(FieldTypeHandler):
    type_id = "ipv4"                     # a string, not an enum member — that is the point
    capabilities = Capabilities(
        filterable=True,
        sortable=True,
        filter_ops=frozenset({FilterOp.eq, FilterOp.ne, FilterOp.in_, FilterOp.is_null}),
        index_kinds=frozenset({"btree"}),
        default_index_kind="btree",
    )

    def normalize(self, value, *, config):      # wire value → stored value; validates here
        return None if value is None else str(ipaddress.IPv4Address(str(value)))

    def to_cell(self, value, *, config):        # stored value → sidecar column
        return None if value is None else str(value)

    def from_cell(self, cell, *, config):       # sidecar column → stored value
        return None if cell is None else self.normalize(cell, config=config)

    def promoted_type(self, *, config):         # the SQL type when promoted to a sidecar
        return sa.String(15)


register_field_type(IPv4Type())
```

Those four methods are the whole contract, and
[`docs/grid.md`](docs/grid.md) works the example through in full. A handler
signals a bad value with `ValueError` and the write path turns it into
`BadRequestError`, so a handler never imports grid's error vocabulary.

### Driving it from an LLM

`forktex_grid.ops` exposes the operations as declarative, JSON-serialisable
commands, so a model emits a command object rather than Python.

```python
from forktex_grid.ops import TOOLS, run, tool_schemas

schemas = tool_schemas()
result = await run(ns, "query", {"table": "invoice", "limit": 10})
```

`forktex_grid.declare` is the other opt-in front door: a decorator DSL for
declaring a schema as classes. Both are importable but deliberately outside
`__all__`, so the root surface stays lean.

## Errors

| Class | Raised when |
|---|---|
| `NotFoundError` | Table, row or relation does not exist |
| `BadRequestError` | Invalid spec, filter, or value for a column's type |
| `AlreadyExistsError` | A slug or key collides within the namespace |
| `ReadOnlyStorage` | A write was attempted against a read-only overlay |

All four are exported from the package root and all derive from
`forktex.error.AppError`, so one `AppError` handler in the consuming service
renders them with the right code alongside its own rather than a masked 500.

## Stability

**The public surface is `forktex_grid.__all__`** — 31 names, asserted as an exact
set by `tests/test_architecture/test_public_surface.py`, so neither an addition
nor a removal can happen by accident. Anything reachable but not in `__all__` is
internal and may change in any release. `ops` and `declare` are deliberately in
that category: importable front doors, not part of the frozen surface.

**This is 0.x, but it is not a free-for-all.** SemVer permits 0.x to break
anything at any time; this project does not use that licence. A public name is
removed only after one minor release in which it still works and warns.
`warnings.deprecated` marks it, `_compat.DEPRECATED` schedules the removal, and
`tests/test_deprecations.py` asserts a marked name still imports, still warns,
and appears in the ledger. Behaviour a correct consumer would notice is called
out under *Changed* in `CHANGELOG.md`.

**`FieldType` members are append-only.** Each is a string persisted in
`grid_column.type_id`, and two are named in SQL CHECK constraints, so members are
added and never renamed or removed. `type_id` itself is an open `VARCHAR(64)`: a
type this package does not implement is registered by whoever does, and the enum
never has to grow for it. **A `type_id` string is the stable identifier; a
handler class is not** — the class may be renamed, moved or restructured.

**Migrations are forward-only, and an applied one is never edited.** A schema
change is a new `v000N` file. `tests/test_migration_immutability.py` pins the
sha256 of every migration that has shipped, so editing one fails CI rather than
quietly leaving already-migrated databases describing a schema this code no
longer expects. Fixing a bad migration means writing the next one.

**`forktex_grid` is a Postgres schema name in deployed databases** as well as
this distribution's import name. It is never renamed. Tests map over it with
`schema_translate_map` rather than changing it.

**Docs are tested.** `tests/test_docs_accuracy.py` imports every name these pages
claim is importable, constructs every spec literal they show, and resolves every
relative link — so a broken example fails CI instead of a reader's terminal.

## Documentation

[`docs/grid.md`](docs/grid.md) — the consumer reference: every public name, the
full field-type contract, the agent and decorator surfaces, and the gotchas.
[`docs/grid-binding-design.md`](docs/grid-binding-design.md) — host binding, in
depth. [`docs/development.md`](docs/development.md) — contributing, the layering
rules, and the release checklist.

## Licence

Dual-licensed: AGPL-3.0-or-later, or a commercial licence from FORKTEX S.R.L.
See [LICENSE](LICENSE) and [NOTICE](NOTICE); commercial enquiries to
info@forktex.com.

