Metadata-Version: 2.4
Name: keble-data-infra-contract
Version: 0.11.0
Summary: Framework-free raw commerce contracts shared by Keble data providers and the raw API.
Author-email: zhenhao-ma <bob0103779@gmail.com>
Requires-Python: <3.14,>=3.13
Requires-Dist: annotated-types<1,>=0.7
Requires-Dist: keble-helpers<2.0.0,>=1.58.0
Requires-Dist: pydantic<3,>=2.10
Description-Content-Type: text/markdown

# keble-data-infra-contract

Install the provider-neutral public contract with
`pip install "keble-data-infra-contract>=0.10.0,<1"`.

Framework-free, provider-neutral raw-commerce contracts. This distribution owns
validated request/response models, provider protocols, operation keys, manifests,
and typed domain errors. It intentionally has no FastAPI, database, HTTP client,
provider, platform, or idea dependency.

The distribution declares only registry-resolvable runtime dependencies. Do not
add workspace-relative `tool.uv.sources` here: downstream Git/subdirectory installs
must build this contract without inheriting paths outside the repository checkout.

Provider packages own implementations and native DTOs. The raw API consumes
provider manifests and constructs HTTP paths from their typed operation keys.

`Page[T]` and `RankingSnapshot[T]` carry a provider-neutral `usage` list of
`keble-helpers.UsageAccountingEvent`. Providers measure physical tokens,
requests, items, media, or bytes; this contract only transports those facts.
It must never contain prices, currency conversion, allocation policy, or
deployment secrets. Cache hits return an empty list so old upstream spend is
not replayed to consumers.

Version `0.2.0` is the exact-usage release. It requires
`keble-helpers>=1.58.0` and preserves media/storage/compute quantities plus
fully failed video-operation attempts without reconstructing prices. The
distribution has no Git or workspace source override, so consumers resolve the
same public contract in clean build and runtime environments.

Version `0.3.0` adds first-class evidence-bearing `TRENDING` operations without
aliasing search or rank. Products, creators, sellers, and videos have distinct
typed query contracts; every result carries stable provider identity, optional
product-family identity, structured standout evidence, source exhaustion, and
shrink diagnostics. Providers report raw source truth and never apply cross-page
dedupe. The central raw API alone applies the request's `none`, `entity`, or
`product_family` policy through an opaque continuation session.

Version `0.4.0` adds the provider-neutral indexed-commerce boundary required by
Shopify without reusing TikTok board or Amazon ASIN contracts:

- `StoreSearchQuery`, `StoreIdentityLookupQuery`, `StoreProductsQuery`, and
  `ProductIdentityLookupQuery`;
- optional product-search cursors that cannot be combined with numbered pages;
- `PageInfo.end_cursor` while preserving numbered-provider metadata;
- `CommerceStoreObservation` with typed popularity values and stable identities;
- provider-neutral `CommerceCatalogCoverage` with immutable full/window scope,
  terminal completeness, absolute cutoff, exhaustion, and an exact one-outcome-
  per-discovered-product accounting invariant;
- Store Leads product and image counts retained as separate sizing signals;
- indexed query provenance separated from ordered Store Leads/storefront source
  provenance and immutable artifact ids.

Provider-specific lifecycle values are normalized into closed universal store
and listing state enums. Unknown fields, untyped extension bags, inferred
identities, and source-provider collapse remain forbidden.

```python
request = ProductTrendingQuery(
    query="red clothing",
    market="US",
    dedupe_mode=TrendDedupeMode.PRODUCT_FAMILY,
)
```

Blank product intent is valid category browsing. A filtered/dedupe-empty page
may remain continuable; only definitive upstream evidence can set
`TrendExhaustion.EXHAUSTED`. `TrendEvidence.summary` is display-ready but must
also carry an auditable metric, event time, or provider-board position.

Version `0.5.0` replaces string-only upstream errors with a single immutable
`DataInfraFailure` payload and `DataInfraFailureError`. Provider manifests now
require a `ProviderFailureAdapterProtocol`; the raw API registers the declared
native exception classes without importing Keepa, Amazon, TikTok, FastMoss, or
EchoTik error types. Retry/payment/operator combinations are validated, payment
requires reviewed evidence, and physical usage/capacity observations remain
exact and sanitized.
The minimum helper line is `1.52.2`, which preserves negative provider token
debt and a separate exact refill-reduction rate instead of rejecting, clamping,
or misclassifying those capacity facts.
`parse_retry_after(...)` is the single shared parser for positive delta-seconds
and canonical IMF-fixdate headers. It ignores malformed, past, zero, and
overflowing hints, returns timezone-aware UTC, and does not apply Platform's
versioned maximum-hint policy.

`IdempotencyKeyConflictError` is the provider-neutral signal for a caller key
already bound to another operation/request fingerprint. Its safe message never
contains the caller key. Data Infra API owns Redis hashing, claim/replay, HTTP
`409`, and retention; the contract deliberately owns no datastore policy.

Version `0.6.0` makes the expected non-success HTTP body unambiguous through
`DataInfraFailureEnvelope`. The envelope contains exactly one validated
`failure` and rejects parallel `detail` or extension bags; Data Infra API and
Data Platform must therefore evolve and release their serializers/parsers in
the same wave. A source-aware CI guard compares the current contract `src/`
tree with the newest package-scoped annotated tag and rejects changed source
whose package version was not advanced. Package metadata regressions also fail
closed.

Version `0.7.0` carries the same canonical `DataInfraFailure` inside failed
video operation outcomes. Job-attempt and Result projections therefore retain
the producer-owned disposition, operator action, retry time, sanitized native
identifiers, usage completeness, and exact usage events. The normalized
operation and nested failure must contain the same usage list; divergent copies
fail validation rather than creating two financial truths.

Version `0.8.0` extends the video-ingestion repository protocol with one
bounded `list_stalled(now, limit)` read. Implementations return only
nonterminal rows whose owner is absent or expired; recovery callers must pass
each returned completion event back through the existing atomic `claim` method.
No reset, force-unlock, provider response, price, or database shape enters the
framework-free contract.

Version `0.8.1` makes failure transport self-identifying and centralizes local
contract-error conversion. Every canonical non-success response advertises
`X-Keble-Data-Infra-Failure-Schema: 1` beside the existing
`DataInfraFailureEnvelope`. Consumers must require both this exact header and a
valid envelope; legacy `detail`, malformed JSON, unknown schema versions, and
wrong media types are untrusted responses rather than guessed failures.
`failure_from_contract_error(...)` is the one framework-free conversion for
operation/provider/request/idempotency rejections and can retain exact usage
captured after provider I/O. `ProviderRequestValidationError.after_provider_io`
distinguishes those post-I/O validations from preflight known-zero rejection.

Version `0.9.0` makes catalog coverage financially exact: every discovered
product must resolve to exactly one terminal processed, unchanged, skipped, or
failed outcome. Partial windows remain representable, but an omitted or
double-counted product can no longer pass validation and silently distort
media, provider-unit, or per-product forecasts. This is a first-release
contract step; pre-release invalid projections use the guarded clean break
rather than a second permissive wire schema.

Version `0.11.0` adds exact-route source-health authority: `SourceHealth`
contains configured state, `HEALTHY`/`UNHEALTHY`/`STALE`/`UNKNOWN`, durable
success/failure times, and one bounded 30-second freshness window. It removes
the misleading `liveVerified` probe claim; only real provider invocations may
produce observed health.

Version `0.10.0` prepares the accepted intelligence source-contract surface for
publication. It retains the provider-neutral capability, evidence, identity,
relationship, revocation, and exact-route admission contracts as one releaseable
package. `SourceOperationAdmissionProjection` exposes only selected stable cost
profile keys, one typed positive finite attempt bound per key, the exact quota
selector, and positive workload quantities that exclude concurrency leases. Its
schema revision and fingerprint are deterministic. Data Infra policy/catalog
remains authoritative for keys, bounds, quantities, rates, money, endpoints,
secrets, recovery policy, and live quota values. The release gate verifies the
package-scoped version bump and imports every contract symbol used by the
accepted G2 Platform source from an isolated built artifact; it does not bump,
tag, publish, or otherwise release the composed API.

Side effects if changes:

- Data Infra API must attach the header to every expected non-success response.
- Data Platform must use one total versioned parser and sanitize untrusted bodies.
- Invocation scopes must retain captured usage before converting post-I/O errors.
- Provider packages still own native failures; this factory accepts only contract
  errors and does not classify provider-native exceptions.

Side effects if changes:

- Data Infra API and repository test doubles must implement the same stalled
  eligibility boundary.
- Active claims are authoritative and cannot be stolen by an operator scan.
- Adding another nonterminal ingestion status requires a coordinated protocol,
  index, implementation, and recovery-test update.

The minimum helper line is `1.58.0`. Currency amounts retain the stable
`{"amount":{"value":...},"currency":...}` wire shape, while `amount.value`
now has one explicit meaning: ISO minor units at
`Currency.minor_unit_exponent`. Use `Money.from_minor_units` for provider
integer units and `Money.from_major_units(Decimal(...))` for human/provider
major-unit decimals; do not reintroduce cents or fixed-two-decimal conversion.

Side effects if changes:
- provider packages must adapt every declared native exception to this model;
- Data Infra API owns HTTP/Retry-After serialization of the payload;
- Idea/Platform own their separate durable job/recovery persistence models.
- provider adapters must use `parse_retry_after` rather than redeclaring HTTP
  date or numeric parsing.

Side effects if changes:

- provider packages and the raw API serialize these exact camel-case events;
- the Ideas platform deduplicates producer attempt keys and prices quantities
  with its own immutable profile effective at the event time.

```bash
uv run --package keble-data-infra-contract pytest -q keble-data-infra-contract/tests
uv run npx --yes pyright .
uv run python keble-data-infra-contract/src/keble_data_infra_contract/release_guard.py
uv build --package keble-data-infra-contract
```

Contract values inherit `ContractModel`, which accepts snake-case Python names
and camel-case wire aliases, rejects undeclared fields, and is frozen. Persisted
Mongo shapes do not belong here; provider/API packages own `SchemaBase` /
`MongoObjectBase` aggregates when durability is required.
`ContractModel` and the deliberately producer-tolerant Job/Result projection
both configure Pydantic through `keble-helpers.PydanticModelConfig`; a local
`ConfigDict` convention would split alias and validation ownership again.

`Money`, `Currency`, and `AmazonMarketplace` are imported from
`keble-helpers`. Provider-native response details remain inside the provider
package instead of crossing this boundary through an untyped extension bag.

Video enrichment uses the same framework-free boundary: immutable asset,
submission, job, completion-event, result, and normalized-projection models plus
one `VideoEnrichmentGateway` Protocol. HTTP submission and Mongo correlation
remain implementations owned by `keble-data-infra-api`.

Each video operation outcome carries canonical `UsageAccountingEvent` rows and
an opaque producer price-profile identity. The normalized projection preserves
both the per-operation status (including partial failure) and a flattened usage
view. It deliberately omits producer estimates and monetary values: ASR/OCR
route policy stays in Video Enrichment, while the Ideas platform resolves its
own immutable rates.

The job read also retains `analysisOperationAttempts` for every whole-workflow
retry. This is the accounting path for fully failed jobs and for failed provider
calls that precede a later successful result. Consumers must deduplicate by the
producer attempt key carried by each usage event; they must not infer zero cost
from the absence of a result.

Authorized provider transcripts use one canonical `VideoTranscript`/
`VideoTranscriptSegment` shape across submission, authoritative result reads, and
the local projection. Reuse evidence binds the normalized transcript checksum to
the data-infra source reference and media SHA-256; a title/description such as
`ShortVideo.caption` is not a spoken transcript and cannot bypass ASR.

## Portable commerce metadata

`CommerceProduct` may carry provider-neutral description, feature bullets,
product type, full category path, source-language hint, and a precision-aware
release-date interval. `RECENTLY_RELEASED` is the canonical trend reason;
`RECENTLY_LISTED` is accepted only as a deprecated input alias and never
emitted. Consumers must not infer absent metadata or replace release time with
offer-history time.

Listing-authoritative variant candidates remain bounded assertions over listings
embedded in the same `CommerceProduct`. Every `VARIANT_OF` assertion must
include the candidate's own listing identity, but it need not include the
response-authoritative listing: a child-ASIN response may therefore retain an
observed sibling-to-parent representative assertion. External endpoints and
assertions filed under a candidate they do not mention are rejected.

## Shopify listing authority v2

One normalized Shopify product page is one listing-authoritative
`CommerceProduct`. Its `source_identity` and only embedded `CommerceListing`
use `PRODUCT_LISTING` with the normalized `keble_shopify/shopify_product_key`;
the Product GID remains `channel_product_id` plus a store-scoped
`shopify_storefront/shopify_product_id` alias. Product handles and page URLs
remain mutable locator/display evidence and are not canonical identities.

Sellable variants remain bounded `CommerceProductVariant` children. Each child
uses `PRODUCT_VARIANT` with `keble_shopify/shopify_variant_key` and retains its
store-scoped `shopify_storefront/shopify_variant_id` alias. Neither the Product
GID provider nor the former variant-key listing namespace can create a second
canonical Product/ProductListing/ProductVariant identity.

Every normalized product aggregate contains exactly one occurrence-and-checksum
bound `PRODUCT_LISTING_SOLD_BY_STOREFRONT` assertion. The registered mapping now
uses `shopify-product-projection-v2` and `keble-shopify-index-v2`; provider/API
cache revisions must treat v1 per-variant listing and relationship identities
as incompatible rather than adapt or alias them.

Side effects if changes:

- `keble-data-infra-api.integrations.shopify.mapper` must emit one listing and
  one `SOLD_BY` assertion per product page.
- Shopify provider projections must retain normalized product/variant/store
  keys and original Product/Variant GIDs separately.
- Platform publication, store-expansion metrics, and relationship ingestion
  count product pages/listings independently from their variant children.

## Source intelligence evidence

Every provider operation carries a `SourceOperationCapability` validated
against its registered Pydantic request model. `ProviderManifest` derives a
stable semantic revision and a provider-affine `SourceOperationSelection`;
runtime health and catalog fingerprints are separate values.

`SourceOperationCapabilitySnapshot` is the one secret-free, exact-`RouteKey`
projection for Campaign forecast/revalidation. It retains the registered
request/result schema revisions, manifest capability revision, exact selection
and typed capability, fingerprints, and one exact bounded admission projection.
Its `freshUntil` cannot exceed 30 seconds after `healthObservedAt`; `UNKNOWN`
health remains an admission failure, not an implicit provider probe or
availability claim.

`MetricObservation` now supports precise measured/interval time, raw source
period labels, coverage, confidence, and strict value/unit pairing.
`SourceRelationshipAssertion` binds provider-namespaced subject/object IDs to a
reviewed mapping, relation metrics, time, and provenance without accepting a
canonical edge. Exact-byte `SourceArtifactRef` and trusted
`SourceShareabilityEvidence` compose `SourceResultEvidence` at the provider
durability boundary. Evidence `sourceService` is the literal `data_infra` on
both direct Python construction and camel-case JSON validation, so provider
payloads and cache documents cannot impersonate another evidence authority.
The open `SourceServiceCode` remains limited to non-evidence revocation and
authority transport. Public pages, rankings, trending results, and failure
envelopes carry bounded exact-response evidence; item and relationship
associations distinguish direct normalization bytes from selection/traversal
bytes. A provider cannot publish a result until its parser input is durably
captured, checksum-bound, and authorized for the current consumption.
