Metadata-Version: 2.4
Name: opensanctions-pravda
Version: 0.8.0
Summary: Durable web evidence capture with a remote browser, Postgres, and content-addressed storage
Keywords: evidence,web-archiving,web-capture,playwright,har,forensics
Author: OpenSanctions
License-Expression: MIT
License-File: LICENSE
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Internet :: WWW/HTTP :: Browsers
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Dist: aiosqlite>=0.22.1
Requires-Dist: fsspec[gcs]>=2026.4.0
Requires-Dist: playwright==1.64.0
Requires-Dist: psycopg[binary]>=3.3.4
Requires-Dist: sqlalchemy[asyncio]>=2.0.50
Requires-Dist: unmhtml>=0.4.0
Requires-Python: >=3.12
Project-URL: Homepage, https://github.com/opensanctions/pravda
Project-URL: Documentation, https://github.com/opensanctions/pravda#readme
Project-URL: Repository, https://github.com/opensanctions/pravda
Project-URL: Issues, https://github.com/opensanctions/pravda/issues
Description-Content-Type: text/markdown

# Pravda

Pravda is a Python library for durable web evidence capture. It drives a
remote Playwright browser to preserve rendered HTML, full-page
screenshots, standalone HTML, metadata, and HAR recordings with response
bodies. Snapshots are
recorded in a SQL database (PostgreSQL or SQLite) and on any fsspec-compatible
backend for later inspection or comparison.

Applications own the browser, database, and storage backend.

- **Browser**: a remote Playwright Chromium WebSocket endpoint (e.g. headed
  Chrome under xvfb).
- **Database**: PostgreSQL or SQLite.
- **Storage**: any fsspec URL (local path, `s3://`, `gs://`, …) for
  content-addressed artifacts.

## Installation

**Python** 3.12+

```bash
pip install opensanctions-pravda
```

## Quick start

The application owns the SQLAlchemy engine and an `async_sessionmaker`;
construct a
[`PravdaConfig`](#configuration) for the remaining settings and build a
long-lived `Pravda`. Reuse a single instance across captures — each capture
opens its own browser connection and database session — and dispose the engine
on shutdown.

```python
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine

from pravda import Pravda, PravdaConfig

engine = create_async_engine("postgresql+psycopg://pravda:pravda@localhost:5432/pravda")
sessionmaker = async_sessionmaker(engine)
config = PravdaConfig(
    browser_ws_url="ws://localhost:3000",
    storage_base_path="./data",
)


async def capture_example():
    pravda = Pravda(config, sessionmaker)
    snapshot = await pravda.snapshot("https://example.com")
    print(snapshot.id, snapshot.http_status, snapshot.rendered_html)
```

## Configuration

- `browser_ws_url` — remote Playwright WebSocket URL.
- `storage_base_path` — fsspec storage URL, such as `./data`, `s3://bucket`,
  or `gs://bucket`.
- `timeouts` — a frozen `Timeouts` of wall-clock budgets for the captured
  website, defaulting to sensible values; see its docstring for what each
  budget does on expiry. Override individual fields with
  `dataclasses.replace(Timeouts(), load=60)`.
- `liveness` — a frozen `Liveness` of deadlines for Pravda's own browser
  machinery. `setup` bounds connecting and preparing the recording context;
  `close` bounds closing it, which finalizes the recording. Both are
  seconds, default 30.

## Usage

### Capture a page

By default, Pravda navigates to the URL, waits for the normal `load` state,
captures the evidence, and persists the result. The target-facing capture is
bounded by one wall-clock budget; a wedged-but-alive storage backend or
database hangs until the caller cancels. Browser machinery — connecting,
preparing, or closing the recording context — is Pravda's infrastructure,
not the target: its failures raise `InfrastructureError` and persist
nothing, with a wedged-but-alive endpoint failing at the `liveness`
deadlines.

For custom navigation or interaction, pass an async `drive(page, url)`
callback. The callback owns the initial navigation; Pravda owns capture,
persistence, and cleanup, and records the state it leaves behind.

```python
async def drive(page, url):
    await page.goto(url, wait_until="commit")
    await page.wait_for_selector(".results")


async def capture_results():
    pravda = Pravda(config, sessionmaker)
    snapshot = await pravda.snapshot("https://example.com", drive=drive)
```

`page` is a real `playwright.async_api.Page`. Playwright errors and timeouts
from `drive` are persisted as failed snapshots. Other callback exceptions
propagate and persist nothing.

When the browser endpoint sends PDFs as downloads, `page.goto()` in custom
callbacks may raise `Download is starting`; catch that Playwright error if
the download is expected. Pravda recovers the downloaded body into the HAR.

## Storage

Artifacts are content-addressed files organized under the captured page's
hostname. The public `Snapshot` resolves its artifact fields to full storage
paths: `rendered_html`, `screenshot`, and `standalone_html` point at stored
files, and each HAR `response.content._file` resolves to its stored response
body. `standalone_html` is a self-contained rendering of the page —
stylesheets inlined, images embedded as data URIs, active content sanitized —
derived from the rendered DOM and the recorded HAR bodies for direct
inspection; derivation failures drop only that artifact.
Persisted database fields and HAR values keep their relative, content-addressed
names. Other storage failures propagate, and no snapshot record is persisted.

## Development

Requires [uv](https://docs.astral.sh/uv/) and a remote Playwright browser
endpoint. Tests run against an in-memory SQLite database, so no database
service is needed. Environment variables live in `.env`; `uv` does not read
it automatically:

```bash
# Install dependencies
uv sync

# Local environment (once)
cp .env.example .env

# Validate
uv run --env-file .env pytest
uv run ruff check .
uv run ruff format --check .
```

## License

MIT — see [LICENSE](LICENSE).
