Metadata-Version: 2.5
Name: pmdb_utils
Version: 0.5.63
Summary: A utilities package for poorman-dataplatform
Author-email: saebod <saebod@hotmail.com>
Requires-Python: >=3.10
Requires-Dist: azure-identity>=1.25.0
Requires-Dist: azure-keyvault>=4.2.0
Requires-Dist: bump-my-version>=1.2.5
Requires-Dist: minio>=7.2.16
Requires-Dist: polars[rtcompat]>=1.36.1
Requires-Dist: psycopg2-binary>=2.9.12
Requires-Dist: python-dotenv>=1.1.1
Requires-Dist: requests>=2.32.5
Description-Content-Type: text/markdown

# pmdb_utils

Shared utilities for the Poorman Data Platform.

## Starting archive processing

After every dataset for a source has landed successfully, start its archive
flow through FastAPI and retain the returned Kestra execution ID:

```python
from pmdb_utils import start_archive

execution_id = start_archive(
    branch="master",
    source_system="boliga",
)
```

Set `FASTAPI_URL` to the FastAPI base URL, for example
`http://p-ubuntu:1000`. The request is fire-and-forget with respect to the
archive flow: it returns after Kestra accepts the execution, without waiting
for the flow to finish. Request failures are raised to the caller and are not
automatically retried, avoiding duplicate archive executions after ambiguous
timeouts.

## PostgreSQL writes

`PostgresDB` reads the application connection from `DATABASE_URL` by default.
Select `DATABASE_URL_LOG` explicitly when writing to the separate log database.

```python
from pmdb_utils.dataacceslayer import PostgresDB

with PostgresDB.from_env("DATABASE_URL_LOG") as db:
    db.create_table(
        "public.worker_log",
        {
            "id": "BIGSERIAL PRIMARY KEY",
            "job_id": "TEXT NOT NULL",
            "status": "TEXT NOT NULL",
            "details": "JSONB",
        },
    )

    db.insert(
        "public.worker_log",
        {"job_id": "job-123", "status": "started", "details": {}},
    )
    changed = db.update(
        "public.worker_log",
        values={"status": "complete"},
        where={"job_id": "job-123"},
    )
    deleted = db.delete(
        "public.worker_log",
        where={"job_id": "job-123"},
    )
```

The `update` and `delete` methods require at least one equality condition and
return the affected row count. Values are parameterized, table and column names
are quoted, and dictionaries/lists are adapted for PostgreSQL JSON columns.
Use `bulk_insert` for multiple rows and `execute` or `transaction` for complex
SQL.

## Publishing

Pull requests that change this package test it on Python 3.10 and 3.13, build
both the source distribution and wheel, and validate their metadata. After the
change is merged into `master`, the same artifacts are published to PyPI using
Trusted Publishing.

Every published change must use a new version in `pyproject.toml`. After
changing the version or dependencies, refresh and commit the lockfile:

```bash
uv lock
```

The PyPI project must trust this GitHub publisher:

- Owner: `saebod`
- Repository: `poorman-dataplatform`
- Workflow: `publish-pmdb-utils.yml`
- Environment: `pypi`
