Metadata-Version: 2.5
Name: dq-export-connector
Version: 0.1.0.dev1
Summary: Worker logic for the Export Connector DCR.
Author: Decentriq
Requires-Python: >=3.10
Requires-Dist: obstore<0.12,>=0.11.1
Requires-Dist: pydantic<3,>=2.7
Description-Content-Type: text/markdown

# dq-export-connector

Worker logic for the **Export Connector DCR**: one package for every
Connector Kind, each in its own sub-package. The **bucket** kind uploads each
Input's dataset to S3, Azure Blob or GCS through
[`obstore`](https://developmentseed.org/obstore/).

ddc pins this package by hash in a fully locked `requirements.txt` and runs
`dq_export_connector.runner.main` in the task's Python environment. It is
published to PyPI, since the compiled graph installs it from there.

- `dq_export_connector.runner`: the entry point. Reads `/input/config.json`
  and `/input/credentials.json`, sets up logging, and dispatches on the
  Connector Kind and the action.
- `dq_export_connector.redact`: keeps every value from the sealed half, and
  URL query strings, out of log lines and result messages.
- `dq_export_connector.bucket`: the bucket kind (`ValidateConnection` and
  `Export`).
- `dq_export_connector._models`: pydantic models generated from ddc's JSON
  schemas, so the config the worker reads and the receipt and validation
  result it writes match ddc's Rust types. Regenerate them with
  `scripts/codegen-models.sh` after the types change and
  `decentriq_dcr_compiler` has been rebuilt; a test fails while they are
  stale.

```sh
uv run pytest            # unit tests, against obstore's in-memory store
uv run pytest -m live    # against real buckets, credentials from the environment
uv run mypy dq_export_connector tests
```

## Releasing

Bump `version` in `pyproject.toml`, then push a `dq-export-connector-v<version>`
tag; the `Publish dq-export-connector to PyPI` workflow tests, builds and uploads
it. Use `.devN` versions while developing; PyPI releases can't be replaced.
ddc then pins the new version in a new `export_connector/v0/worker/<version>/`.
