Metadata-Version: 2.4
Name: xyzdb
Version: 1.1.1
Summary: Official Python driver for xyzDB — a thin, zero-dependency TCP client.
Author-email: Tunolabs <ivan@tuno.bar>
License-Expression: Apache-2.0
Project-URL: Homepage, https://xyzdb.bar
Project-URL: Repository, https://github.com/Tunolabs/xyzdb
Project-URL: Documentation, https://github.com/Tunolabs/xyzdb
Project-URL: Issues, https://github.com/Tunolabs/xyzdb/issues
Keywords: xyzdb,database,driver,client,vector,gravity,mcp,agent-memory
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Database
Classifier: Topic :: Database :: Front-Ends
Classifier: Typing :: Typed
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: test
Requires-Dist: pytest>=7; extra == "test"
Dynamic: license-file

# xyzDB — Python driver

A thin, **zero-dependency** Python client for xyzDB. It speaks the binary wire
protocol directly (pure standard library), with a fluent query API, parameter
binding, and bearer-token auth.

- **Requires:** Python ≥ 3.9 and a running `xyzdb-server` (default `localhost:2505`).
- **Dependencies:** none at runtime.

The engine itself is developed at the official xyzDB repo:
[github.com/Tunolabs/xyzdb](https://github.com/Tunolabs/xyzdb).

> **Source & repository.** This driver's source ships inside the published
> package — the sdist and wheel contain it, so `pip download xyzdb` gives you
> the code. The clients monorepo is not public; the engine, the wire spec
> [`PROTOCOL.md`](https://github.com/Tunolabs/xyzdb/blob/main/PROTOCOL.md), and
> the reference client live at
> [github.com/Tunolabs/xyzdb](https://github.com/Tunolabs/xyzdb).

## Install

```bash
pip install xyzdb
```

## Quickstart

```python
import xyzdb

with xyzdb.connect("127.0.0.1", 2505) as db:
    # Schema (optional — lobes auto-create on first PUT)
    db.create_lobe("clients", hint="Customer records")
    db.create_anchor("rfc", unique_in="clients")

    # Write
    db.put("clients", {"rfc": "ACME-001", "name": "Acme Corp", "region": "US-West"})

    # Read one (anchor lookup, O(1))
    acme = db.find("clients", where={"rfc": "ACME-001"}).one()
    print(acme.name)                      # -> "Acme Corp"

    # Read many (top-N)
    top = list(
        db.scan("clients", where={"region": "US-West"})
          .order_by("revenue", descending=True)
          .limit(10)
    )
```

`connect()` returns a `Client`. Use it as a context manager (auto-closes) or
call `db.close()` yourself. A client owns one TCP connection and is **not**
thread-safe — use one per thread.

## Connecting & auth

```python
db = xyzdb.connect(host="127.0.0.1", port=2505)
```

If the server runs with `--auth-token`, set `XYZDB_TOKEN` in the environment;
the driver sends the bearer token automatically on connect:

```bash
export XYZDB_TOKEN="my-secret-token"
```

The token travels **in the clear**: the auth frame carries the bytes themselves —
no challenge, no hash — so anything that can read the connection can replay it.
Over an untrusted network the server has to be running with TLS
(`--tls-cert`/`--tls-key`); this is a property of the protocol, not of this
driver (`PROTOCOL.md` §2, §4).

**Chunked formats are not implemented here.** The wire defines two
(`JSON_CHUNKED`, `BINARY_CHUNKED`) and this driver requests neither, so it never
receives one: the server enters that path only on the format byte you send, never
by its own decision about response size. Page large results with a cursor instead.

Prefer `127.0.0.1` over `localhost` under high concurrency (avoids the dual
IPv4/IPv6 connection budget).

### Connection lifecycle

The rules, in one place, because everything below follows from them:

1. **One client owns one TCP socket and is not thread-safe.** Concurrency comes
   from more clients, not from sharing one.
2. **A client never reconnects.** The first transport failure ends it —
   permanently, and later calls say so rather than writing into a dead socket.
   `db.is_alive` reports it without a round trip; a `False` client is replaced.
3. **A timeout is not a cancellation.** The client stops waiting; the server was
   never told. The statement may have executed, so a timed-out **write** is
   indeterminate until you read it back. That is why nothing here retries for you.
4. **The token is checked on the first statement, not on connect.** The protocol
   has no positive auth acknowledgement — a good token is answered with silence —
   so `connect()` succeeding says nothing about the token. A rejected one arrives
   as `AUTH_FAILED` on the first statement. Issue any cheap statement after
   connecting if you want to find out immediately.

```python
db = xyzdb.connect("127.0.0.1", 2505)

# Per-statement deadline, overriding the client default for this call only.
rows = db.scan("creditos").timeout(2.0).execute()

if not db.is_alive:
    ...  # discard it and build another
```

**Pooling.** The driver ships no pool, because the right size and validation
policy are properties of your application, not of the protocol. A pool that works
is small: hold N independent clients, lend one at a time, check `is_alive` before
lending, and **discard** — never re-run the statement — on a transport failure.
Validate a long-idle connection with a read-only statement (`SHOW LOBES`) *before*
handing it out: that is not a retry, it happens before the caller's statement and
it is idempotent.

**Idempotency instead of retries.** The safe answer to an ambiguous write is an
anchored `PUT` with `on_conflict="update"`: the anchor makes the record's identity
the caller's, so running it twice converges instead of duplicating. Retrying a
plain `put` after a timeout is how you get two records under a `UNIQUE` anchor.
See also [`KNOWN-ISSUES.md`](../KNOWN-ISSUES.md).

## Writing

```python
# Single record. Prefix a field with * to make it a gravity field
# (records sharing the value are co-located on disk).
db.put("creditos", {"*rfc": "ACME-001", "_type": "Credit", "monto": 50000})

# Linked write (relationship to a parent record).
db.put("creditos",
       {"*rfc": "ACME-001", "_type": "Payment", "amount": 1000},
       link_to={"lobe": "clients", "where": {"rfc": "ACME-001"}, "as": "owner"})

# Upsert on the anchor.
db.put("clients", {"rfc": "ACME-001", "name": "Acme Inc"}, on_conflict="update")

# Atomic batch (≤ 10,000 records per call).
db.put_batch("clients", [
    {"rfc": "A-1", "name": "One"},
    {"rfc": "A-2", "name": "Two"},
])
```

`on_conflict` only supports `"update"`.

### Co-location: gravity and its sub-axis

> **Needs engine ≥ 1.1.0.** `SATELLITE BY` does not parse on a 1.0.x engine — it comes
> back as a parse error, which does not name the version. Everything else on this page
> works against engine ≥ 1.0; see [Versioning & license](#versioning--license) for the full statement.

Gravity decides which records land next to each other on disk. A **satellite**
splits each gravity bucket into sub-buckets, so a query pinning both fields reads
only the matching rows instead of the whole bucket. It is a pure optimisation —
same rows, same order — and the lobe must be **empty** when you declare it,
because existing rows would stay in the default sub-bucket and a bounded query
would never reach them.

```python
db.create_lobe("events")
db.gravity_by("conv", "events")       # GRAVITY BY conv IN "events"
db.satellite_by("kind", "events")     # SATELLITE BY kind IN "events"

# Reads only the "message" rows of conversation c1, not the whole bucket.
msgs = db.scan("events", where={"conv": "c1", "kind": "message"}).execute()
```

## Reading

`find()` and `scan()` return a `QueryBuilder`. Chain trailing clauses and
pipeline steps, then terminate with `.execute()`, `.one()`, iteration, or a
terminal verb. Clauses are assembled in the order the engine requires
(`WHERE → ORDER BY → LIMIT → CURSOR`, then `| steps`) regardless of call order.

`.execute()` returns a **`RecordSet`**: it iterates and indexes like a list of
`Record`, and also carries `budget_stop`, `has_more` and `cursor`. Use `.one()`
when you expect at most one row.

```python
# Graph fetch: a record plus its linked descendants.
tree = db.find("clients", where={"rfc": "ACME-001"}).pull(depth=2).execute()

# Update in place.
db.find("clients", where={"rfc": "ACME-001"}).set(status="inactive")

# Delete matching records.
db.scan("creditos", where={"status": "cancelled"}).delete()

# Grouped aggregate.
r = (db.scan("creditos", where={"_type": "Credit"})
       .group_by("status")
       .aggregate("count()", "sum(monto)"))
print(r.count, r.sum_monto)

# Semantic top-k. The engine never embeds — declare the vector field, store
# embeddings you produced, then NEAREST over them (pass a query vector from the
# same model, or a REF to a stored record's vector).
db.create_lobe("mem")
db.create_vector("emb", "mem")            # VECTOR emb IN "mem"
db.put("mem", {"id": "a", "conv": "c1", "emb": [0.1, 0.2, 0.3]})
hits = (db.scan("mem", where={"conv": "c1"})
          .limit(10000)
          .nearest("emb", query_vector, k=10, metric="cosine")
          .execute())
```

Metrics for `nearest()`: `"cosine"`, `"dot"`, `"l2"`.


**Pin the satellite on a `NEAREST` too.** The axis bounds semantic search the same
way it bounds a scan: with both the gravity and satellite fields pinned, the engine
scores only that sub-range and returns the exact top-k **of the filtered set**
rather than the top-k of the whole bucket. Declaring the axis and then not pinning
it in the query buys nothing.
```python
# Same declarations as above (gravity `conv`, satellite `kind`).
# Scores only the "message" sub-range of conversation c1.
hits = (db.scan("events", where={"conv": "c1", "kind": "message"})
          .nearest("emb", query_vector, k=10, metric="cosine")
          .execute())
```

### When `NEAREST` comes back short

> **Needs engine ≥ 1.1.0.** A 1.0.x engine truncates without saying so: the field is
> simply absent, so the check below is a no-op rather than an error — which is the
> failure mode this field exists to remove. See [Versioning & license](#versioning--license).

The server puts a wall-clock budget on scoring. If it expires mid-scan you get a
**partial**, not an error: the highest-scoring rows found so far, plus a
`budget_stop` describing the cut.

```python
hits = db.scan("mem", where={"conv": "c1"}).nearest("emb", qv, k=10).execute()

if hits.budget_stop:
    bs = hits.budget_stop
    print(f"partial: {bs.found} of {bs.candidates} candidates, "
          f"{bs.examined} checked, {bs.strategy}")
```

`strategy` says *what kind* of partial it is, and the two are not the same claim.
`"score_order"` means the rows are a true **prefix** — nothing unexamined scores
better. `"key_order"` means they are the best of a contiguous **key region**, so
the unwalked part may hold *better* rows, not merely more; do not extrapolate the
observed pass rate to the tail.

Consuming a truncated result without reading `.budget_stop` emits a
`TruncatedResultWarning` once — a partial should not be readable as a complete
answer by accident. Reading the field silences it. To make it fatal instead:

```python
import warnings
warnings.simplefilter("error", xyzdb.TruncatedResultWarning)
```

### Filters (`where=`)

`where=` takes a dict (an AND conjunction) or a raw predicate string:

```python
# Equality (AND-combined)
db.scan("fintech", where={"status": "overdue", "region": "US-West"})

# List → IN; operator dicts → comparisons / set / null tests
db.scan("fintech", where={"status": ["overdue", "defaulted"]})   # status IN [...]
db.scan("fintech", where={"amount": {">": 5000}})
db.scan("clients", where={"tags": {"contains": "tech"}})
db.scan("data", where={"score": {"is_not_null": True}})

# Raw string → the full OR / NOT / parenthesised tree (build from trusted text;
# for untrusted values use execute() with $params)
db.scan("fintech", where='(status = "overdue" OR status = "defaulted") AND amount > 5000')
```

`FIND` is AND-only (anchor/gravity fast path); use `SCAN` for `OR`/`NOT`.

### More reads: FETCH, FOLLOW, SHAPE, TAKE

```python
# FETCH — several co-located lobes in ONE call (shared key). Returns one
# envelope whose fields are a named section per lobe.
env = db.fetch(["clientes", "creditos", "operaciones"],
               where={"rfc": "ACME-001"},
               as_names=["cliente", "creditos", "operaciones"])

# FOLLOW — jump across lobes: each message's cited_doc resolved in "documents".
docs = (db.find("messages", where={"thread": "T-1"})
          .follow("cited_doc", to="documents", on="doc_id").execute())

# SHAPE — project each record down to the fields you need.
rows = (db.scan("clients", where={"tier": "gold"})
          .order_by("score", descending=True).limit(10)
          .shape("name", "score").execute())

# TAKE — top-N over an aggregate (O(N) when a metric-ordered ghost exists),
# or a plain stream truncation.
from xyzdb import metric
top = (db.scan("creditos", where={"_type": "Credit", "status": ["active", "overdue"]})
         .group_by("rfc")
         .agg(metric("sum(monto)", as_="exposicion"), "count()")
         .take(100, by="exposicion").execute())

top5 = db.scan("clients").limit(1000).take(5).execute()   # truncate the stream
```

Use the non-terminal `.agg(...)` step (not the `.aggregate(...)` terminal) when a
`TAKE` follows the aggregate.

### Conditional aggregates (per-metric filter)

Each metric may carry its own alias and `WHERE`, computing several conditional
aggregates in one pass. Build them with `metric()`:

```python
from xyzdb import metric

r = (db.scan("credits").group_by("empresa_id").aggregate(
        metric("count()", as_="total"),
        metric("count()", as_="active", where={"status": "active"}),
        metric("sum(amount)", as_="overdue_amount", where={"status": "overdue"}),
     ))
```

### Grouped vs ungrouped: read the right side

The two answer with different shapes and they are **not** interchangeable:

```python
total = db.scan("credits").aggregate(metric("count()", as_="total"))
total.total                       # 1234

by_company = db.scan("credits").group_by("empresa_id").aggregate(
    metric("count()", as_="total"))
for row in by_company.groups:     # one row per group, grouping field included
    print(row["empresa_id"], row["total"])
```

`groups` is `None` for an ungrouped pipeline, and the flat accessors are empty for
a grouped one — reading a grouped result as flat raises `GroupedResultWarning`,
because returning nothing silently is how a dashboard ends up showing every
customer zero orders.

### The catalog as data

`show_lobes()` / `show_anchors()` return the engine's display lines verbatim
(`{"info": [...]}`). For values, use the parsed pair:

```python
for l in db.list_lobes():
    print(l.name, l.anchors, l.hint)
for a in db.list_anchors("creditos"):
    print(a.field, a.unique)
```

### Deleting & purging

`DELETE` requires a selection — a `where=` (or a pipeline that produced records).
Calling `.delete()` on an unfiltered scan raises `InvalidUsageError` (still a
`ValueError`) and points you to `purge()`, so you never empty a lobe by accident.
**There is no cascade**: linked children are never removed for you.

```python
db.scan("creditos", where={"status": "cancelled"}).delete()   # matched rows only
db.delete_where("creditos", where={"status": "cancelled"})    # same, standalone form
db.purge("scratch")                                           # empty the whole lobe (deliberate)
```

### Linking

`put(..., link_to=...)` co-locates a child into its parent's bucket **within one
lobe**, and that is the form `PULL` can traverse. The standalone `link()` writes
the relation field without re-bucketing, so the engine confirms a link that `PULL`
never returns — for parent/child reads, declare `GRAVITY BY <parent_field>` on the
child lobe and read the bucket:

```python
db.gravity_by("customer_id", "orders")      # orders live with their customer
db.scan("orders", where={"customer_id": "ACME-001"}).execute()
```

### Pagination

`SCAN` without `.limit()` caps at 1,000 rows; the hard ceiling is 10,000. Page
beyond it by round-tripping the opaque cursor token:

```python
page = db.scan("creditos", where={"rfc": "ACME-001"}).limit(1000).execute()
rows = list(page)
while page.has_more:
    page = (db.scan("creditos", where={"rfc": "ACME-001"})
              .limit(1000).cursor(page.cursor).execute())
    rows.extend(page)
```

Cursor pagination is not valid together with `order_by`, `aggregate`, or ghost
routing.

## Parameter binding (injection-safe)

Never interpolate untrusted input into a statement. Pass it through bound
`$name` placeholders — substituted server-side, so the text never becomes
syntax:

```python
db.execute('FIND "clients" WHERE name = $n', {"n": user_input})
```

`execute()` is also the escape hatch for any xyTalk the fluent API doesn't
cover.

## Result types

- **`RecordSet`** — what `.execute()` returns for row results. Iterates and
  indexes like a list of `Record`, plus `.budget_stop`, `.has_more`, `.cursor`,
  `.records`, `.to_dicts()` and `.raw()` (the undecorated server response).
- **`Record`** — one row. Access fields by attribute (`rec.name`) or key
  (`rec["name"]`). `.to_dict()`, `.to_json()`, `.to_compact()`, `.get(k, default)`.
  For `PULL` results, iterate the record to walk linked children.
- **`BudgetStop`** — on a truncated `NEAREST`: `.examined`, `.candidates`,
  `.found`, `.strategy`. `None` when the result is complete.
- **`AggregateResult`** — metrics by attribute or key; parenthesised keys are
  normalised (`sum(monto)` → `result.sum_monto`).

`PULL` is the one row query that does not return a `RecordSet`: it returns a
`Record` tree, because it has no row set and no cut to report.

## Errors

```python
from xyzdb import XyzDBError, ConnectionError, InvalidUsageError, TimeoutError

try:
    db.put("clients", {"rfc": "dupe"})
except InvalidUsageError as e:
    print("bad call:", e)           # never reached the wire
except TimeoutError as e:
    print("may have run:", e)       # the write is indeterminate
except ConnectionError as e:
    print("transport:", e)          # connect/send/recv failure
except XyzDBError as e:
    print(e.code, e.message)        # server-side rejection
```

Every code the driver can raise is in one of two exported tuples —
`SERVER_ERROR_CODES` (the wire contract, `PROTOCOL.md` §8) and
`CLIENT_ERROR_CODES` — so branching on `.code` is exhaustive:

| Code | Class | What it means |
|---|---|---|
| `INVALID_USAGE` | `InvalidUsageError` | the call is wrong before any I/O (an unrenderable value, a `DELETE` with no selection) |
| `AUTH_FAILED` | `XyzDBError` | the server rejected the token; set `XYZDB_TOKEN` |
| `TIMEOUT` | `TimeoutError` | the deadline expired — the statement **may have run** |
| `CONNECTION_ERROR` | `ConnectionError` | transport failure; the client is now dead |
| `PROTOCOL_ERROR` | `XyzDBError` | a frame or body this driver cannot decode |
| `FRAME_TOO_LARGE` | `XyzDBError` | the statement or announced response exceeds 16 MiB |

The hierarchy is built for both styles of handler: `ConnectionError` and
`TimeoutError` are `XyzDBError` subclasses (and `TimeoutError` is also a builtin
`TimeoutError`), and `InvalidUsageError` is both an `XyzDBError` **and** a
`ValueError` — which is what it used to be, so old handlers keep working.

### Values the driver will not guess at

`None` is written as a null, lists and dicts are rendered structurally, and a
value with no xyTalk representation raises `InvalidUsageError` naming the field
rather than being interpolated as its `str()`. Before 1.1.1 an unrecognised value
was stored as text — `None` became the word `"None"`.

## Tests

Integration tests need a running server on `localhost:2505` (override with
`XYZDB_PORT`):

```bash
pip install -e './python[test]'
pytest python/tests
```

The truncation test needs an engine whose `NEAREST` airbag fires on a test-sized
corpus; against the 3,000 ms default it skips with a reason rather than passing
vacuously. To exercise it, start the server with `--nearest-budget-ms 5`. The
result wrapper itself is covered offline, with no server, in
`tests/test_truncation.py` against the shared golden vector.

## Versioning & license

Released independently under a `py-X.Y.Z` SemVer tag; the tip always speaks the
current **xyTalk v1** wire surface. SemVer is from the client API's point of
view — MAJOR breaks the client API, MINOR is additive, PATCH is fixes.

**The client version is not the engine version.** They move independently: a
client `1.1.1` and an engine `1.0.0` are a normal, supported pair, and a client
`2.x` would not imply an engine `2.x` exists. This client reads `1.1.1` and the
engine `1.1.0`: the two numbering lines are unrelated, and for one week they
happened to match — **coincidence, not a contract**.

What a release supports is stated by the `Compatibility:` line in
[`CHANGELOG.md`](CHANGELOG.md), not by the version number. For `1.1.0`: it works
against **engine ≥ 1.0**; only the `SATELLITE BY` helper and seeing `budget_stop`
require **engine ≥ 1.1.0**.

This package is **Apache-2.0**. The linked repository,
[github.com/Tunolabs/xyzdb](https://github.com/Tunolabs/xyzdb), is the xyzDB
**engine**, published under the **Business Source License 1.1** — a different
license from this client. The clients are permissive; only the engine is BUSL-1.1.

The wire protocol,
[`PROTOCOL.md`](https://github.com/Tunolabs/xyzdb/blob/main/PROTOCOL.md), may be
implemented by anyone, under any license: if the engine's license gives you
pause, you can write your own client without asking.

For the full language reference and operator docs, see the engine repo's
[`docs/`](https://github.com/Tunolabs/xyzdb/tree/main/docs) and the
[MCP integration guide](https://github.com/Tunolabs/xyzdb/blob/main/docs/mcp-integration.md).
