Metadata-Version: 2.5
Name: utic-invocation-settings
Version: 0.5.0
Summary: Public library for consuming encrypted Unstructured plugin invocation settings (AES-256-GCM + RSA-OAEP-256 envelopes, whole-object or per-field, decrypt + cache).
Project-URL: Homepage, https://github.com/Unstructured-IO/utic-public-libs
Project-URL: Repository, https://github.com/Unstructured-IO/utic-public-libs
Project-URL: Documentation, https://github.com/Unstructured-IO/utic-public-libs/tree/main/libs/utic-invocation-settings#readme
Project-URL: Threat model, https://github.com/Unstructured-IO/utic-public-libs/blob/main/libs/utic-invocation-settings/docs/threat-model.md
Project-URL: Changelog, https://github.com/Unstructured-IO/utic-public-libs/blob/main/libs/utic-invocation-settings/CHANGELOG.md
Project-URL: Issues, https://github.com/Unstructured-IO/utic-public-libs/issues
Requires-Python: >=3.11
Requires-Dist: cryptography>=43.0.1
Requires-Dist: pydantic<3.0.0,>=2.12.5
Description-Content-Type: text/markdown

# utic-invocation-settings

Public library for **consuming encrypted Unstructured plugin invocation settings** — the plugin-side
half of the cellular-dataplane "settings in the invoke payload" design. It reads the v1 settings
envelope (RSA-OAEP-256-wrapped **AES-256-GCM**), decrypts **inside the plugin** at invoke time, and
caches only previously authenticated envelopes.

It is deliberately **self-contained on `cryptography` + `pydantic`** — no dependency on any
private-feed package — so it can be published to public PyPI and imported by external plugin authors.

Ships [PEP 561](https://peps.python.org/pep-0561/) type information (`py.typed`): the result type of
every call below is inferred, not `Any`.

## Why

Under the cellular dataplane, a shared pod may serve multiple tenants, so a plugin identity decrypts
settings routed to that plugin rather than a shared service handing out plaintext. Settings arrive as an
opaque ciphertext envelope; this library turns that envelope into a plain settings object, verifying
integrity and never logging secrets.

The wire format is the `Envelope` model in `utic_invocation_settings/envelope.py`: its field
constraints are the contract, and `Envelope.model_json_schema()` exports them as JSON Schema. The
producer (Secrets Provider / operator) emits exactly that shape; this library is the reference
consumer. What that format does and does not guarantee is written down in
[the threat model](https://github.com/Unstructured-IO/utic-public-libs/blob/main/libs/utic-invocation-settings/docs/threat-model.md)
— read it before you rely on an envelope for anything.

## See it work

[`docs/walkthroughs/envelope-cryptography/`](https://github.com/Unstructured-IO/utic-public-libs/blob/main/libs/utic-invocation-settings/docs/walkthroughs/envelope-cryptography/README.md) holds three runnable walkthroughs —
executable documentation, not production code. No cluster, no fixtures:

```bash
cd libs/utic-invocation-settings
uv run --no-sync python docs/walkthroughs/envelope-cryptography/round_trip.py
```

| | |
|---|---|
| [`round_trip.py`](https://github.com/Unstructured-IO/utic-public-libs/blob/main/libs/utic-invocation-settings/docs/walkthroughs/envelope-cryptography/round_trip.py) | Both sides in one file: settings in → sealed → settings out → refusals. |
| [`producer_side.py`](https://github.com/Unstructured-IO/utic-public-libs/blob/main/libs/utic-invocation-settings/docs/walkthroughs/envelope-cryptography/producer_side.py) | Builds an envelope with only `cryptography` and the standard library, then opens it with `resolve_settings`. A producer in another language depends on exactly this working. |
| [`consumer_side.py`](https://github.com/Unstructured-IO/utic-public-libs/blob/main/libs/utic-invocation-settings/docs/walkthroughs/envelope-cryptography/consumer_side.py) | Seals with the library, then walks the **core cryptographic path** by hand — not every check `resolve_settings` makes; the walkthrough README lists what it leaves out. |

They print the fixture settings at both ends with a digest beside them, and the envelope as JSON,
so the round trip is visible rather than asserted at you. The digest is the part that matters: it
shows the same *bytes* came back, not merely equal-looking values. Where an envelope is abridged for
width it is labelled as a display copy that will not open. The private key is never printed, and
neither is any value the library echoed out of a rejected settings document.

`tests/unit/test_walkthroughs.py` runs each file and checks it exits `OK` — each self-verifies, so a
walkthrough that stopped matching the library fails rather than misleading you. `tests/packaging`
asserts the directory is absent from the built wheel.

## Usage

Declare the settings your plugin needs, then resolve them from the envelope:

```python
from typing import Any

import pydantic
from utic_invocation_settings import resolve_settings


class MySettings(pydantic.BaseModel):
    api_key: str
    timeout_seconds: int = 30


def on_invoke(body: dict[str, Any]) -> dict[str, Any]:
    envelope = body["invocation_settings"]["dag_node_settings"]
    settings = resolve_settings(envelope, MySettings)  # raises if unusable
    return fetch(settings.api_key, timeout=settings.timeout_seconds)
```

`settings` is a `MySettings`, so `settings.api_key` type-checks and a typo does not.

That one call loads this pod's mounted workload key, decrypts, caches, and validates into
`MySettings`. Pass a pydantic model class, a `TypeAdapter`, any callable taking the settings mapping,
or nothing at all for the plain mapping (`resolve_settings(envelope)`). Each of those four is a
separate typed overload, so the inferred result is the model, the adapter's type, the callable's
return type, or `dict[str, Any]`. The envelope itself may be a raw JSON mapping or an already-parsed
`Envelope`.

### The Unstructured `/invoke` boundary

`resolve_settings` takes one envelope or document and knows nothing about a request body. The
Unstructured plane's placement and fallback policy is the separately named
`resolve_invocation_settings` helper: a v2 document is accepted only under
`body["invocation_settings"]["dag_node_settings"]`; plain mappings pass through only on
compatibility pods; and native pods set `FF_INVOCATION_SETTINGS=true` so
absence or plaintext fails closed.

```python
from collections.abc import Mapping
from typing import Any

from utic_invocation_settings import (
    RESERVED_ENVELOPE_KEY,
    resolve_invocation_settings,
)

_MISSING = object()  # `null` is a value that arrived, not absence


def invocation_settings(body: Mapping[str, Any]) -> dict[str, Any] | None:
    raw = body.get(RESERVED_ENVELOPE_KEY, _MISSING)
    if raw is _MISSING:
        return resolve_invocation_settings(None)
    if raw is None:
        raise ValueError("invocation_settings cannot be null")
    return resolve_invocation_settings(raw)

settings = invocation_settings(body)
```

The additive metadata capability is `invoke_with_sealed_dag_node_settings_v2`. A controller forwards
a v2 document only to a plugin advertising that exact capability; the whole-object-v1 capability
keeps its existing meaning during mixed-version rollout.

**Only a genuinely absent field may reach a compatibility fallback.** `null`, `{}`, a malformed
document, one sealed to another recipient, one that fails authentication, and one whose plaintext
your model rejects are all values that arrived. Every one raises; never catch an
`InvocationSettingsError` and fall back to boot settings.

`dag_node_settings_identity(document)` exposes the v2 document's non-decrypting equality token for
routers that batch records. It covers the plain skeleton and sealed `(pointer, value_digest)` pairs;
fresh ciphertext for the same values compares equal, while a value or position change does not.

### Async handlers

**There is no async API, by design.** This library is CPU-bound: a resolve is an RSA unwrap, an
AES-GCM open, two SHA-256 digests, a JSON parse, and your model's validation. It opens no sockets and
makes no network calls. The only IO anywhere in the package is reading the workload-identity mount
(`tls.key`/`tls.crt`), and `WorkloadIdentity.load()` memoizes the result — so that is one pair of
small file reads per process per mount directory, not per request.

So an `async def` entry point would have nothing to await. Offering one would imply that awaiting is
natural because something blocks on IO, when the honest description is "this burns CPU for a couple
of milliseconds". Whether to move CPU work off your event loop is a judgement about *your* latency
budget, and `asyncio.to_thread` is the stdlib primitive for exactly that:

```python
async def on_invoke(envelope: dict):
    settings = await asyncio.to_thread(resolve_settings, envelope, MySettings)
```

This is not a formality — it is worth doing. A cold resolve stalls every other task on the loop for
~2.2 ms, and OpenSSL releases the GIL for the RSA operation, so a worker thread genuinely absorbs it
rather than merely relocating the stall. The technique is sound; it was the *interface* that was
wrong, since a wrapper around `to_thread` adds no capability a caller does not already have.

It **keeps the inferred result type**: `to_thread` is generic over a `ParamSpec`, so `mypy --strict`
infers `MySettings` above, including through bound methods (`resolver.resolve`). The typing-contract
test asserts this for every overload shape.

One note on cost: the thread hop is ~36 µs, and a warm resolve is ~51 µs — so if you resolve the same
envelope repeatedly, offloading buys less than the cold-path number suggests.

### Provenance, and configuring a resolver

`SettingsResolver(...)` is the configurable form (same relationship as `requests.get` to
`requests.Session`) when the cache TTLs and budgets, the clock, or the key loader need supplying. A
resolver constructs and privately owns its caches — configured by validated scalars
(`settings_ttl_seconds`, `key_ttl_seconds`, `settings_cache_max_bytes`, or the `DISABLED` sentinel),
never by injected cache instances — and identity is bound per resolver with no per-call override,
because a resolver's caches derive their authority from its identity.

`SettingsResolver.resolve_detailed(envelope)` returns the envelope's metadata alongside the plaintext:
`expires_at`, `credential_version`, and the authenticated `settings_digest` as a ready-made
`cache_key` for caller-side derived objects. There is no module-level `resolve_detailed`; reach the
shared instance with `default_resolver().resolve_detailed(envelope)`.

### Errors

Every failure (unknown format, missing key, RSA/GCM failure, digest mismatch, model rejection)
raises a subclass of `InvocationSettingsError` — it never returns partial or unverified plaintext.
Each class carries three class attributes so a host can map an outcome onto its own transport
without matching on messages:

- **`reason`** — a stable machine-readable code, unique across the taxonomy. The only part safe to
  switch on.
- **`blame`** — which participant to investigate (`CALLER`, `RECIPIENT`, `CONTENT`, `ROUTING`), and
  the single input to the HTTP mapping below.
- **`retry`** — a `RetryDisposition`: `NEVER`, `SAME_REQUEST` (replay these identical bytes; the
  fault is a local transient such as a Secret that has not been projected yet), or `NEW_INVOCATION`
  (replaying is futile, but the *platform* can compose or re-address an invocation that succeeds —
  explicitly **not** a client retry).

`retryable` remains available as a strictly narrower derived shim: it is `True` only for
`SAME_REQUEST`, and it is computed from `retry` so the two cannot drift.

**HTTP class comes from `blame`, by one rule:** `blame is Blame.CALLER` → **422**, anything else →
**5xx**. The line it draws is whether a different request would work. Settings that were required
and not supplied, or that arrived in a shape this contract does not allow, are the caller's to fix;
settings that were supplied and could not be *processed* are orchestration — the producer, this
pod's mounted identity, or the routing between them — which no request the caller composes can
repair.

| error | `reason` | `blame` | `retry` | HTTP |
|---|---|---|---|---|
| `MalformedEnvelopeError` | `malformed_envelope` | `CALLER` | `NEVER` | 422 |
| `UnsupportedFormatError` | `unsupported_format` | `CALLER` | `NEVER` | 422 |
| `SealedDagNodeSettingsRequiredError` | `sealed_dag_node_settings_required` | `RECIPIENT` | `NEVER` | 5xx |
| `MalformedDagNodeSettingsError` | `malformed_dag_node_settings` | `ROUTING` | `NEW_INVOCATION` | 5xx |
| `KeyNotFoundError` | `recipient_mismatch` | `ROUTING` | `NEW_INVOCATION` | 5xx |
| `DecryptionError` | `decryption_failed` | `CONTENT` | `NEVER` | 5xx |
| `IntegrityError` | `integrity_mismatch` | `CONTENT` | `NEVER` | 5xx |
| `SettingsValidationError` | `settings_validation_failed` | `CONTENT` | `NEVER` | 5xx |
| `IdentityNotMountedError` | `identity_not_mounted` | `RECIPIENT` | `SAME_REQUEST` | 5xx |
| `IdentityUnreadableError` | `identity_unreadable` | `RECIPIENT` | `SAME_REQUEST` | 5xx |
| `IdentityMaterialError` | `identity_material_invalid` | `RECIPIENT` | `NEVER` | 5xx |
| `CertificateRequiredError` | `certificate_required` | `RECIPIENT` | `SAME_REQUEST` | 5xx |
| `IdentityConfigurationError` | `identity_configuration_invalid` | `RECIPIENT` | `NEVER` | 5xx |

The identity rows are where `retry` earns its keep: a Secret that has not been projected yet is
`SAME_REQUEST` (replay these identical bytes), while a mount that is present and *wrong* is `NEVER`
and pages someone. Flattening those together fails pods permanently on an ordinary startup race.

The classification rule, so a new class has an obvious answer rather than a judgement call: presence
faults are transient (`SAME_REQUEST`), content faults are permanent (`NEVER`), and addressing faults
need a new invocation.

`SettingsValidationError` additionally carries `issues`: a bounded tuple of `SettingsIssue`, each a
`code` drawn from pydantic's own closed error vocabulary and a `loc` whose string components must be
declared by your model's schema (anything else — a mapping key, a discriminator value — is
`<redacted>`, because for a settings payload those come from the input and the input is the secret).
The validator's own message is never reproduced: a custom validator is free to interpolate the
rejected value into it, and callers do.

### Where the key comes from

`WorkloadIdentity` reads `tls.key` (and `tls.crt` when present) from `$WORKLOAD_IDENTITY_DIR`, else
`$INVOCATION_SETTINGS_KEY_DIR`, else `/var/run/workload-identity`. When a certificate is mounted it
is the **anchor**: the `kid` comes from the certificate and the mounted key must match it. A `kid`
this pod does not hold yields `KeyNotFoundError` (investigate routing); a mount that is absent or
self-inconsistent yields `IdentityConfigurationError` (investigate this pod's Secret).

Where certificates are projected fleet-wide, make anchoring mandatory — with
`WorkloadIdentity(require_certificate=True)` or `$WORKLOAD_IDENTITY_REQUIRE_CERTIFICATE=true`, which
reaches pods that only ever call the module-level `resolve_settings`. A key-only mount then raises
`CertificateRequiredError` instead of silently self-anchoring on the key, which would make this pod
answer for a `kid` the producer never sealed to.

`IdentityConfigurationError` now has transient/permanent subclasses (see the table above): "not
projected yet" and "present and unreadable" are `SAME_REQUEST`, while "present and wrong" —
unparseable PEM, non-RSA key, key below the RSA-3072 floor, a certificate that does not match the
key — is `NEVER`, and is a paging condition rather than a backoff condition.

### Field-level documents (v2)

`resolve_settings` also reads `u10d.invocation-settings.v2` documents — the field-level format
(`envelope-contract-v2.md`) in which the settings structure travels in plaintext and each secret
field is sealed as its own envelope at its position. The call and the result are identical to v1;
which format arrives is the producer's decision, negotiated out-of-band (a plugin advertises the
`invoke_with_sealed_dag_node_settings_v2` capability on `/metadata` to receive v2). The format
exists so one rotated credential can be resealed and swapped into a document without touching the
rest: each field is cached under its own fingerprint, so the swap re-decrypts exactly one field.

`ResolvedSettings.fields` (via `resolve_detailed`) lists each sealed field's RFC 6901 pointer,
`value_digest`, and advisory metadata. The single-field door is
`decrypt_field_value(envelope, pointer, ...)` / `open_field_envelope(envelope, pointer, ...)` —
what a rotation delivery's ciphertext goes through on its own. **The pointer is required**: it is
bound into the field's AAD, so an envelope presented at any position other than the one it was
sealed for fails authentication. Note the
property trade the format makes: sealed fields keep v1's confidentiality and integrity per field,
while the plain structure has neither — see the threat model, §1c, before relying on either.

On the v2 path, `fields` also records what this resolve proved: possession of the recipient key
for exactly those positions, and nothing when it is empty. Resolving a v1 envelope always unwrapped
against the recipient's private key, so a v2 document with no sealed fields — which consults the
key loader not at all — is a real change for a caller that was reading resolution success as an
implicit key-possession signal. It was never an authorization signal (§1a), but where that side
effect was being relied on, `fields` is what replaces it. Note the asymmetry: a v1 envelope reports
`fields == ()` while always requiring the key, so the tuple only discriminates within v2 — which
is enough, because the caller knows which format it passed in.

### Lower-level primitives

Still public, for a consumer that already holds an envelope or wants to own the orchestration:

```python
settings = decrypt_settings(               # decrypt + parse; the primitive itself never caches
    envelope,
    private_key_loader=load_private_key,
    limits=DEFAULT_LIMITS,                 # optional: tighten (never loosen) the size / JSON ceilings
)
```

Caching is not a property of the primitives — `open_envelope` and `decrypt_settings` retain nothing
and take no cache. A process that must bound or reuse recovered plaintext constructs a
`SettingsResolver`, which owns an authenticated, identity-scoped cache privately and re-authorizes
every hit; no cache instance crosses the API boundary in either direction.

The root API stops there on purpose. Cache-key derivation (`envelope.envelope_fingerprint`), the
authenticated-header byte layout (`envelope.protected_aad`) and envelope *production*
(`crypto.seal_settings`, `crypto.seal_field`) are supported but live in their own submodules: re-exporting them would make
each one's representation a root-level compatibility contract.

## Security and cache notes

See [`docs/threat-model.md`](https://github.com/Unstructured-IO/utic-public-libs/blob/main/libs/utic-invocation-settings/docs/threat-model.md)
for the full threat model,
[`docs/adr-0001-envelope-v2.md`](https://github.com/Unstructured-IO/utic-public-libs/blob/main/libs/utic-invocation-settings/docs/adr-0001-envelope-v2.md)
for the proposal that closes the gaps it documents (now re-targeting a future format id), and
[`docs/adr-0002-field-level-encipherment.md`](https://github.com/Unstructured-IO/utic-public-libs/blob/main/libs/utic-invocation-settings/docs/adr-0002-field-level-encipherment.md)
for the field-level v2 document.

- This is encryption and integrity for the recipient, not producer authentication. A holder of the
  recipient's public key can create an envelope, so deployments must deliver envelopes over an
  authenticated control-plane path.
- `expires_at` and `credential_version` are plaintext advisory metadata outside the authenticated
  header. Use them for scheduling/freshness hints, never authorization decisions. `expires_at` may
  only ever *shorten* a cache lifetime, so the worst a party who rewrites it can achieve is making
  this process decrypt more often.
- `settings_digest` is a public, unsalted SHA-256 value and can link identical settings or support
  low-entropy guess confirmation. Treat envelopes as sensitive metadata.
- **A cache is never authority.** Every hit — plaintext or content key — is reauthorized against the
  envelope's recipient before it is served, so sharing a cache between key loaders cannot leak
  across identities.
- The decrypted-settings cache is keyed by the full authenticated-envelope fingerprint; the AES-key
  cache is keyed by `(kid, encryption_key_digest, sha256(encrypted_key))`. Both keys are namespaced
  by the recipient `kid`. `clear()` is not a revocation barrier for work already decrypting
  concurrently.
- The plaintext cache stores a **deeply-immutable parsed snapshot**, and every hit rebuilds a fresh
  mutable tree from it, so two concurrent resolves can never alias one settings object. Its byte
  charge is an estimate of that parsed structure, measured per document — a parsed mapping runs
  2.6x-23.7x its JSON size depending on shape, so a constant multiplier would silently under-count.
- There is a **256 KiB cache-admission ceiling** (`MAX_CACHEABLE_SETTINGS_BYTES`) and an **8 MiB
  byte budget** (`DEFAULT_SETTINGS_CACHE_MAX_BYTES`). Settings larger than the admission ceiling
  still resolve — they simply pay the full cold cost every time and are never cached; nothing is
  rejected merely for failing *cache admission*, which is not a correctness property. This is
  distinct from the envelope's hard **1 MiB plaintext wire ceiling** (`MAX_PLAINTEXT_BYTES`, 4x the
  admission ceiling, with `MAX_CIPHERTEXT_CHARS` derived from it): an envelope whose plaintext would
  exceed *that* is refused at the boundary as a wire-contract violation the caller can act on. The
  two thresholds are deliberately separate decisions — "too big to retain" versus "too big to
  accept".
- For a genuine no-cache deployment, pass the `DISABLED` sentinel:
  `SettingsResolver(settings_ttl_seconds=DISABLED, key_ttl_seconds=DISABLED)` retains nothing between
  calls, at the price of the full ~2.2 ms cold cost on every invoke. It is a real no-cache, not a
  short TTL, and is compared by identity so no falsy `0`/`None`/`""` reaches the disabled branch by
  accident.

**Reporting a vulnerability.** Do not open a public issue. Report privately via the
[repository's Security tab](https://github.com/Unstructured-IO/utic-public-libs/security) (GitHub
private vulnerability reporting), or to the Unstructured maintainers through your existing support
channel. Include the package version and the envelope *shape* — never real ciphertext, settings
values, or key material.

## Develop

```bash
make install          # uv sync --locked
make test             # unit tests + coverage
make test-packaging   # build the wheel, install it clean, check metadata + consumer types
make check            # ruff + version consistency
```

`make test-packaging` builds the distribution once and tests that exact artifact in a fresh virtual
environment: version agreement across `pyproject.toml` / `uv.lock` / wheel metadata / installed
metadata, `py.typed` presence, the declared dependency set, the root public API, and a mypy
consumer-contract check that the documented result types are what a call site actually infers.

**Note:** published to public PyPI on merge to main (see the repository README).
