Metadata-Version: 2.5
Name: blockweaver
Version: 0.3.2
Summary: Download and verify immutable EVM block datasets.
Project-URL: Repository, https://github.com/edoski/blockweaver
Project-URL: Issues, https://github.com/edoski/blockweaver/issues
Author: Edoardo S.
License-Expression: MIT
License-File: LICENSE
Classifier: Operating System :: MacOS
Classifier: Operating System :: POSIX :: Linux
Requires-Python: >=3.11
Requires-Dist: aiohttp<4,>=3.12
Requires-Dist: polars<2,>=1.31
Requires-Dist: typer<1,>=0.16
Provides-Extra: bigquery
Requires-Dist: google-cloud-bigquery<4,>=3.38; extra == 'bigquery'
Description-Content-Type: text/markdown

# Blockweaver

Blockweaver downloads and verifies immutable, feature-selected EVM block datasets. Chains and JSON-RPC providers are configuration, not code. Each successful request publishes one data file and one canonical manifest.

Python 3.11 or newer is required. Dataset writing is supported on Linux and macOS.

```console
uv tool install blockweaver
blockweaver init
```

`init` writes a complete executable example with environment-backed RPC URLs and a local chain. Configuration selection is explicit `--config`, then `BLOCKWEAVER_CONFIG`, then the platform user config path. The command never overwrites a file.

Edit that generated file for user-specific profiles. For example:

```toml
[chains.ethereum]
chain_id = 1
finality_tag = "finalized"
provider = "primary"
verifier = "verifier"
```

Configuration is strict: unknown keys, unknown profiles, invalid URLs, ambiguous `url`/`url_env` pairs, and non-finite or greater-than-one-hour timeouts fail before network access. Chain profiles may override `provider` and `verifier`. CLI values override the selected chain and provider profiles, which override global defaults.

Inspect configuration and the closed feature catalog without exposing endpoints. `chains` reports `available_sources`; `features` reports top-level `available_sources` and per-feature `supported_sources`.

```console
blockweaver chains
blockweaver features --chain local
```

## Download

Bounds are inclusive. Supply exactly one complete range form:

```console
blockweaver download --from-block 19000000 --to-block 19000999

blockweaver download \
  --from-time 2026-01-01T10:30+01:00 \
  --to-time 2026-01-01T10:45:30+01:00 \
  --feature timestamp \
  --feature block_hash \
  --feature effective_priority_fee_per_gas_p50 \
  --format csv
```

Dates mean the full UTC day. Datetimes accept timezone-aware hour, minute, or second precision; reduced-precision end bounds include the final second of that period. Blockweaver resolves time bounds against the finalized chain and rejects pre-genesis, empty, future, and partly unfinalized requests instead of clipping them.

`block_number` is always the first column. Other columns are selected explicitly or inherited from `defaults.features`. Header features share `eth_getBlockByNumber`; selected priority-fee percentiles share one `eth_feeHistory` request per acquisition chunk with only the requested percentiles.

An omitted `--id` mints a UUID4 and emits it on stderr before acquisition. Reusing an explicit `--id` resumes only an exact binding of chain, requested and resolved range, features, format, source, and provider profile names. Batch size, concurrency, timeout, and endpoint credentials may change between attempts.

The output is exactly:

```text
ROOT/<uuid4>/
  manifest.json
  blocks.parquet | blocks.csv
```

The directory UUID must equal the manifest's dataset UUID. Parquet is the typed default. CSV uses canonical decimal integers and UTF-8 strings; `manifest.json` is its type authority. The unversioned manifest records the tool version, request, resolved range, ordered schema and units, acquisition plan, chain identity, non-secret provider profile names, finality proof, verification samples, byte size, and SHA-256 digest. JSON is sorted, compact, UTF-8, and newline-terminated.

Work is checkpointed under a hidden directory. Complete chunks are digest-bound and validated before reuse. The fully assembled candidate is validated, synced, and atomically renamed without replacement; existing destinations are never overwritten.

Python consumers use the same strict local validator as `verify`:

```python
from blockweaver import open_dataset

dataset = open_dataset("downloads/11111111-1111-4111-8111-111111111111")
print(dataset.chain_id, dataset.first_block, dataset.last_block, dataset.schema, dataset.data_path)
```

## Verify

Local verification is strict and needs no provider:

```console
blockweaver verify ./downloads/11111111-1111-4111-8111-111111111111
```

RPC verification uses a configured profile or a direct URL. It checks deterministic samples and refreshes the stored finality proof. `--full-rpc` checks every row in bounded chunks, including ancestry across chunk boundaries.

```console
blockweaver verify DATASET --provider verifier
blockweaver verify DATASET --rpc-url http://127.0.0.1:8545 --full-rpc
```

RPC-only tuning requires `--provider` or `--rpc-url`. An explicit `--config` requires `--provider`; `--rpc-url` can override that named profile's endpoint.

Progress and errors are JSON Lines on stderr. Errors include stable `code` and `message` fields. Success is one JSON receipt on stdout. URLs and environment values are excluded from manifests, receipts, and intentional logs.

Providers must implement EVM JSON-RPC batch requests, historical `eth_getBlockByNumber`, the configured `finalized` or `safe` tag, and `eth_feeHistory` when priority-fee features are selected. Independent verification consumes quota. Blockweaver checks provider agreement, numbered ancestry to the tagged anchor, a numbered anchor reread, and deterministic row samples; this is strong operational verification, not a trustless consensus client.

## BigQuery source

Google Blockchain Analytics is an optional acquisition route for history that an RPC provider cannot serve. Install it explicitly; ordinary RPC installs do not include Google libraries:

```console
uv tool install 'blockweaver[bigquery]'
```

Configure a strictly validated `project.dataset` identifier on the chain and one billing project or environment reference. The byte cap is mandatory.
When BigQuery is the default source, configuration may omit `defaults.provider` and the unused primary profile; the verifier profile remains mandatory.
Set `chains.<name>.bigquery_dataset`, then configure `[bigquery]` with exactly one billing `project` or `project_env` and `maximum_bytes_billed`.

Select it globally with `defaults.source = "bigquery"` or per request:

```console
blockweaver download --source bigquery --from-block 1000000 --to-block 1000999
```

The same ranges, features, formats, resume state, receipts, and two-file artifacts apply to both sources. Time ranges are resolved against the configured verifier RPC before BigQuery planning. Blockweaver reads the required `blocks`, `transactions`, and `receipts` schemas, rejects unavailable selected features, then performs a dry run and checks its result schema and estimated bytes. Only then does it execute the fixed whitelisted query with `maximum_bytes_billed` enforced again by BigQuery. Results stream through bounded pages into the normal checkpoints.

BigQuery rows are not trusted as chain truth. The verifier RPC checks chain ID, resolved edges, target hash, deterministic row samples, numbered ancestry, and a reread finalized or safe anchor before publication. The manifest records the dataset identifier and verifier profile, never the billing project, environment value, credentials, or SQL text. The configured dataset must expose the recognized common Google Blockchain Analytics schema; arbitrary SQL and field mappings are not supported.

For development:

```console
uv sync --locked --dev
uv run pytest
uv run --extra bigquery python -c 'from google.cloud import bigquery'
uv run ruff check src tests
uv run ruff format --check src tests
uv run pyright
uv run vulture src tests --min-confidence 80
```

Licensed under the [MIT License](LICENSE).
