Metadata-Version: 2.4
Name: graphsense-lib
Version: 2.15.4
Summary: Graphsense backend lib and automation cli
Author-email: Iknaio Cryptoasset Analytics GmbH <contact@iknaio.com>
Project-URL: Homepage, https://graphsense.org/
Project-URL: Source, https://github.com/graphsense/graphsense-lib
Project-URL: Changelog, https://github.com/graphsense/graphsense-lib/blob/master/CHANGELOG.md
Project-URL: Tracker, https://github.com/graphsense/graphsense-lib/issues
Project-URL: Download, https://github.com/graphsense
Keywords: graphsense
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Intended Audience :: Developers
Classifier: Topic :: Utilities
Requires-Python: <3.14,>=3.10
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
License-File: LICENSE
Requires-Dist: filelock>=3.16
Requires-Dist: click>=8.1
Requires-Dist: pandas>=2.3.3
Requires-Dist: methodtools>=0.4.7
Requires-Dist: simplejson>=3.19
Requires-Dist: goodconf[yaml]>=6.0
Requires-Dist: pydantic>=2.10
Requires-Dist: pydantic-settings>=2.7
Requires-Dist: requests>=2.32.5
Requires-Dist: parsy<3.0,>=2.0
Requires-Dist: rich>=13.9
Requires-Dist: base58>=2.1
Requires-Dist: bitarray>=3.0
Requires-Dist: eth-event>=1.4
Requires-Dist: eth-hash>=0.3.0
Requires-Dist: cassandra-driver>=3.29
Requires-Dist: async-lru>=2.0.4
Requires-Dist: tenacity>=9.0
Requires-Dist: coincurve>=21.0.0
Requires-Dist: bech32>=1.2.0
Requires-Dist: httpx>=0.28.1
Requires-Dist: orjson>=3.10
Requires-Dist: ecdsa>=0.19.2
Requires-Dist: eth-account>=0.13.7
Requires-Dist: msgpack>=1.2.0
Provides-Extra: ingest
Requires-Dist: web3>=7.0; extra == "ingest"
Requires-Dist: lz4>=4.3.2; extra == "ingest"
Requires-Dist: pyarrow>=23.0.0; extra == "ingest"
Requires-Dist: deltalake>=1.5.1; extra == "ingest"
Requires-Dist: duckdb>=1.0.0; extra == "ingest"
Requires-Dist: grpcio>=1.49.0; extra == "ingest"
Requires-Dist: protobuf>=4.21.0; extra == "ingest"
Requires-Dist: redis>=5.0.0; extra == "ingest"
Provides-Extra: conversions
Requires-Dist: networkx>=3.2.1; extra == "conversions"
Requires-Dist: matplotlib; extra == "conversions"
Provides-Extra: tagstore
Requires-Dist: sqlmodel>=0.0.22; extra == "tagstore"
Requires-Dist: asyncpg>=0.30.0; extra == "tagstore"
Requires-Dist: psycopg2-binary>=2.9.3; extra == "tagstore"
Requires-Dist: colorama>=0.4.6; extra == "tagstore"
Requires-Dist: fastapi[all]>=0.115.0; extra == "tagstore"
Requires-Dist: httpx<1.0.0; extra == "tagstore"
Requires-Dist: gitpython>=3.1; extra == "tagstore"
Requires-Dist: giturlparse>=0.10; extra == "tagstore"
Requires-Dist: lz4>=4.3.2; extra == "tagstore"
Requires-Dist: python-dotenv>=1.0.1; extra == "tagstore"
Requires-Dist: pyyaml-include<3,>=2.0; extra == "tagstore"
Requires-Dist: pyyaml>=6.0; extra == "tagstore"
Requires-Dist: tabulate>=0.9.0; extra == "tagstore"
Requires-Dist: anytree>=2.9.0; extra == "tagstore"
Provides-Extra: tagpacks
Requires-Dist: graphsense-lib[tagstore]; extra == "tagpacks"
Requires-Dist: rapidyaml>=0.10.0; extra == "tagpacks"
Requires-Dist: rapidfuzz>=3.13.0; extra == "tagpacks"
Provides-Extra: web
Requires-Dist: graphsense-lib[conversions,tagstore]; extra == "web"
Requires-Dist: asgi-lifespan>=2.1.0; extra == "web"
Requires-Dist: fastapi>=0.115.0; extra == "web"
Requires-Dist: python-dateutil>=2.9.0; extra == "web"
Requires-Dist: redis>=5.0.0; extra == "web"
Requires-Dist: uvicorn[standard]>=0.32.0; extra == "web"
Requires-Dist: gunicorn>=23.0.0; extra == "web"
Provides-Extra: transformation
Requires-Dist: pyspark<4.0.0,>=3.5.0; extra == "transformation"
Requires-Dist: setuptools>=80.0.0; extra == "transformation"
Provides-Extra: clustering
Requires-Dist: graphsense-clustering>=0.1.0; extra == "clustering"
Provides-Extra: mcp
Requires-Dist: graphsense-lib[web]; extra == "mcp"
Requires-Dist: fastmcp<4.0,>=3.2; extra == "mcp"
Requires-Dist: pyyaml>=6.0; extra == "mcp"
Provides-Extra: all
Requires-Dist: graphsense-lib[ingest]; extra == "all"
Requires-Dist: graphsense-lib[conversions]; extra == "all"
Requires-Dist: graphsense-lib[tagpacks]; extra == "all"
Requires-Dist: graphsense-lib[web]; extra == "all"
Requires-Dist: graphsense-lib[transformation]; extra == "all"
Requires-Dist: graphsense-lib[clustering]; extra == "all"
Requires-Dist: graphsense-lib[mcp]; extra == "all"
Dynamic: license-file

# GraphSense Library

[![Test and Build Status](https://github.com/graphsense/graphsense-lib/actions/workflows/run_tests.yaml/badge.svg)](https://github.com/graphsense/graphsense-lib/actions) [![PyPI version](https://badge.fury.io/py/graphsense-lib.svg)](https://badge.fury.io/py/graphsense-lib) [![Python](https://img.shields.io/pypi/pyversions/graphsense-lib)](https://pypi.org/project/graphsense-lib/) [![Downloads](https://static.pepy.tech/badge/graphsense-lib)](https://pepy.tech/project/graphsense-lib)

A comprehensive Python library for the GraphSense crypto-analytics platform. It provides database access, data ingestion, maintenance tools, and analysis capabilities for cryptocurrency transactions and networks.

> **Note:** This library uses optional dependencies. Use `graphsense-lib[all]` to install all features.

## Quick Start

### Installation

```bash
# Install with all features
uv add graphsense-lib[all]

# Install from source
git clone https://github.com/graphsense/graphsense-lib.git
cd graphsense-lib
make install
```

### Serving the REST API locally

The web API requires two backend connections: a **Cassandra** cluster (blockchain data) and a **TagStore** (PostgreSQL). You can configure them via environment variables or a YAML config file.

#### Option A: Environment variables only

```bash
GS_CASSANDRA_ASYNC_NODES='["<cassandra-host>"]' \
GRAPHSENSE_TAGSTORE_READ_URL='postgresql+asyncpg://<user>:<password>@<host>:<port>/tagstore' \
GS_CASSANDRA_ASYNC_CURRENCIES='{"btc":{"raw": "btc_raw", "transformed": "btc_transformed"},"eth":{}}' \
uv run --extra web uvicorn graphsenselib.web.app:create_app --factory --host localhost --port 9000 --reload
```

#### Option B: YAML config file

Point `CONFIG_FILE` to a REST-specific config (see `instance/config.yaml` for a full example):

```bash
CONFIG_FILE=./instance/config.yaml make serve-web
```

Or without Make:

```bash
CONFIG_FILE=./instance/config.yaml \
uv run --extra web uvicorn graphsenselib.web.app:create_app --factory --host localhost --port 9000 --reload
```

#### Option C: `.graphsense.yaml` with a `web` key

If you already have a `.graphsense.yaml` (or `~/.graphsense.yaml`) for the CLI, you can add a `web` key containing the REST config. The app will pick it up automatically without setting `CONFIG_FILE`:

```yaml
# .graphsense.yaml
environments:
  # ... your existing CLI config ...

web:
  database:
    nodes: ["<cassandra-host>"]
    currencies:
      btc:
      eth:
  gs-tagstore:
    url: "postgresql+asyncpg://<user>:<password>@<host>:<port>/tagstore"
```

```bash
make serve-web
```

**Config resolution order:** explicit `config_file` param > `CONFIG_FILE` env var > `./instance/config.yaml` > `.graphsense.yaml` `web` key > env vars only.

#### Environment variable substitution in YAML

Any string value in a config file may reference environment variables with `${VAR}`. This works in both the CLI config (`.graphsense.yaml`) and the REST config, and applies to nested values (lists, maps) too. Useful for keeping secrets out of the file:

```yaml
gs-tagstore:
  url: ${TAGSTORE_URL}
database:
  nodes:
    - ${CASSANDRA_HOST}
```

- **Defaults:** `${VAR:-fallback}` uses `fallback` when `VAR` is unset or empty. Use `${VAR:-}` to allow an empty value.
- **Errors:** a `${VAR}` referencing an unset variable with no default fails loudly at load time.
- **Escaping:** write `$${VAR}` to emit a literal `${VAR}` without substitution. A bare `$VAR` (no braces) is never substituted.

#### Optional REST settings (env vars)

| Variable | Default | Description |
|---|---|---|
| `GSREST_DISABLE_AUTH` | `false` | Disable API key authentication |
| `GSREST_ENSURE_TAGSTORE_SCHEMA_ON_STARTUP` | `false` | Auto-initialize TagStore tables/views at startup when missing |
| `GRAPHSENSE_TAGSTORE_READ_URL` | — | TagStore database URL (e.g., `postgresql://user:password@host:5432/tagstore`) |
| `GSREST_ALLOWED_ORIGINS` | `*` | CORS allowed origins |
| `GSREST_LOGGING_LEVEL` | — | Logging level (DEBUG, INFO, …) |
| `GS_CASSANDRA_ASYNC_PORT` | `9042` | Cassandra port |
| `GS_CASSANDRA_ASYNC_USERNAME` | — | Cassandra username |
| `GS_CASSANDRA_ASYNC_PASSWORD` | — | Cassandra password |

> **Precedence: the YAML config file wins over these environment variables.**
> This is the reverse of the more common convention, so it is easy to get caught
> out by. The variables above are a *fallback* for keys the config file does not
> set — they do not override it.
>
> Precedence is decided **per key**, not per block. Given
> `database: {nodes: [...], username: gs_user}`, a `GS_CASSANDRA_ASYNC_USERNAME`
> is ignored (the file sets it), while `GS_CASSANDRA_ASYNC_PASSWORD` still
> applies (the file does not). The same rule covers the `GSREST_*` variables:
> `GSREST_DISABLE_AUTH=true` has no effect if the file already sets
> `disable_auth: false`.
>
> To make an environment variable authoritative for a key the config file sets —
> the usual case for rotating a Cassandra password without editing the file —
> reference it from the file instead of shadowing it:
>
> ```yaml
> database:
>   username: gs_user
>   password: ${CASSANDRA_PASSWORD}
> ```
>
> See [Environment variable substitution in YAML](#environment-variable-substitution-in-yaml)
> above. Note that `${VAR}` and `GS_CASSANDRA_ASYNC_PASSWORD` are independent
> mechanisms: the placeholder is resolved when the file is read, so the variable
> name it references is yours to choose and need not match any name in this table.
>
> This ordering applies to the **REST config only**. The CLI config
> (`.graphsense.yaml`, `GRAPHSENSE_*`) resolves the two the other way round —
> there a `GRAPHSENSE_`-prefixed variable *does* override the file. `${VAR}`
> substitution works the same in both, which is why it is the portable way to
> keep a secret out of either config file.

When enabling `GSREST_ENSURE_TAGSTORE_SCHEMA_ON_STARTUP=true`, keep in mind:

- The DB user must have DDL privileges (create tables/views/indexes/extensions/procedures).
- Startup may be slower because schema checks and potential initialization run before the app serves traffic.
- In multi-replica deployments, initialize schema once (migration/init job) to avoid startup races.

If TagStore is not configured (`gs-tagstore` missing) or the TagStore URL is unreachable, the REST app now falls back to a mock TagStore so endpoints still work. In this mode, tag-specific responses (labels, actors, taxonomies, tag counts) are empty.

### MCP (Model Context Protocol) interface

The web stack also serves a curated MCP endpoint at `/mcp` (path configurable via `GS_MCP_PATH`) so LLM clients — Claude Code, Claude Desktop, Cursor, custom agents — can query GraphSense directly without writing glue code. It is **mounted inside the main FastAPI app**: one `uvicorn` process serves both REST and MCP.

```bash
# Install with the mcp extra (pulls in fastmcp)
uv add 'graphsense-lib[mcp]'

# Start the stack — MCP comes up automatically at /mcp
uv run uvicorn graphsenselib.web.app:create_app --factory --port 9000

# Drift gate (suitable for CI; no DB needed):
uv run graphsense-cli mcp validate-curation
```

The tool surface is strictly curated — **not** every REST endpoint becomes a tool. Consolidations merge endpoints that LLMs always chain (e.g. one `lookup_address` replaces four address-level calls). See [`src/graphsenselib/mcp/README.md`](src/graphsenselib/mcp/README.md) for the design principles, the full tool catalog, how to add or modify tools, deployment details (transport, lifespan composition), and the proprietary `search_neighbors` forward.

### REST API evolution and deprecation policy

The REST API follows semantic versioning via `info.version` in the OpenAPI spec
and the `__api_version__` field in the library:

- **Patch** (`2.10.x`): bug fixes, no schema changes.
- **Minor** (`2.y.0`): additive changes — new endpoints, new fields, new
  optional parameters. Deprecations may be introduced here but deprecated
  surfaces continue to work.
- **Major** (`x.0.0`): removal of deprecated surfaces and other breaking
  changes. Major bumps are rare and announced in advance.

Deprecated endpoints and fields remain fully functional for **at least two
minor releases or six months**, whichever is longer. Deprecations are announced
through three mechanisms, listed from most to least machine-readable:

#### OpenAPI schema (`deprecated: true`)

Deprecated paths and response fields carry `deprecated: true` in
`/openapi.json`, which renders as a strikethrough in Swagger UI (`/docs`) and
is propagated to the generated Python client's docstrings. Check the spec at
build time to fail CI if you depend on a deprecated surface.

#### HTTP response headers (RFC 9745 / RFC 8594)

Responses from deprecated routes carry:

- **`Deprecation: true`** — [RFC 9745](https://www.rfc-editor.org/rfc/rfc9745).
  Signals that this specific endpoint is deprecated. Currently emitted as the
  literal string `true`; future releases may upgrade it to an `@<epoch>`
  timestamp indicating when deprecation took effect.
- **`Link: </docs#section/Deprecation-policy>; rel="deprecation"; type="text/html"`** —
  points clients at the authoritative policy page.
- **`Sunset: <HTTP-date>`** — [RFC 8594](https://www.rfc-editor.org/rfc/rfc8594).
  Announces the committed removal date for the endpoint. For the
  `/entities/...` endpoints (superseded by `/clusters/...`), the sunset is
  set to `Sat, 31 Oct 2026 00:00:00 GMT`. After that date the deprecated
  endpoints may be removed without further notice. Other deprecations may
  be introduced with different sunset dates in future releases.

To detect deprecation in your own client code, inspect the `Deprecation` and
`Sunset` response headers and log a warning (or fail CI) when you hit a
deprecated surface. Example with the generated Python client:

```python
from graphsense import ApiClient, Configuration
from graphsense.api import ClustersApi

cfg = Configuration(host="https://api.iknaio.com", api_key={"api_key": "..."})
with ApiClient(cfg) as api_client:
    clusters = ClustersApi(api_client)
    response = clusters.get_cluster_with_http_info(currency="btc", cluster=264711)
    headers = response.headers
    if headers.get("Deprecation"):
        sunset = headers.get("Sunset", "no sunset date set")
        print(f"WARNING: endpoint deprecated (sunset: {sunset})")
```

#### CHANGELOG

Every deprecation is recorded in [`CHANGELOG.md`](CHANGELOG.md) under the
release that introduced it, and every removal is recorded in the major release
that applies it. Use the changelog as the audit trail when planning client
upgrades.

### Basic Usage

#### Database Access with Configuration File

```python
from graphsenselib.db import DbFactory

# Using GraphSense config file (default: ~/.graphsense.yaml)
with DbFactory().from_config("development", "btc") as db:
    highest_block = db.transformed.get_highest_block()
    print(f"Highest BTC block: {highest_block}")

    # Get block details
    block = db.transformed.get_block(100000)
    print(f"Block 100000: {block.block_hash}")
```

#### Direct Database Connection

```python
from graphsenselib.db import DbFactory

# Direct connection without config file
with DbFactory().from_name(
    raw_keyspace_name="eth_raw",
    transformed_keyspace_name="eth_transformed",
    schema_type="account",
    cassandra_nodes=["localhost"],
    currency="eth"
) as db:
    print(f"Highest block: {db.transformed.get_highest_block()}")
```

#### Async Database Services

The async services are used internally by the REST API and can also be used standalone. `AddressesService` depends on several other services:

```python
from graphsenselib.db.asynchronous.services import (
    BlocksService, AddressesService, TagsService,
    EntitiesService, RatesService,
)

# Services are initialized with their dependencies
blocks_service = BlocksService(db, rates_service, config, logger)
addresses_service = AddressesService(
    db, tags_service, entities_service, blocks_service, rates_service, logger
)

address_info = await addresses_service.get_address("btc", "1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa")
txs = await addresses_service.list_address_txs("btc", "1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa")
```

## Command Line Interface

GraphSense-lib exposes a comprehensive CLI tool: `graphsense-cli`

### Basic Commands

```bash
# Show help and available commands
graphsense-cli --help

# Check version
graphsense-cli version

# Show current configuration
graphsense-cli config show

# Generate config template
graphsense-cli config template > ~/.graphsense.yaml

# Show config file path
graphsense-cli config path
```

## Modules

### Database Management

Query and manage the GraphSense database state.

```bash
# Show database management options
graphsense-cli db --help

# Check database state/summary
graphsense-cli db state -e development

# Get block information
graphsense-cli db block info -e development -c btc --height 100000

# Query logs (for Ethereum-based chains)
graphsense-cli db logs -e development -c eth --from-block 1000000 --to-block 1000100
```

### Schema Operations

Create and validate database schemas.

```bash
# Show schema options
graphsense-cli schema --help

# Create database schema for a currency
graphsense-cli schema create -e dev -c btc

# Validate existing schema
graphsense-cli schema validate -e dev -c btc

# Show expected schema for currency
graphsense-cli schema show-by-currency btc

# Show schema by type (utxo/account)
graphsense-cli schema show-by-schema-type utxo
```

### Data Ingestion

Ingest raw cryptocurrency data from nodes.

```bash
# Show ingestion options
graphsense-cli ingest --help

# Ingest blocks from cryptocurrency node
graphsense-cli ingest from-node \
    -e dev \
    -c btc \
    --start-block 0 \
    --end-block 1000 \
    --create-schema

# Ingest with custom batch size
graphsense-cli ingest from-node \
    -e dev \
    -c eth \
    --start-block 1000000 \
    --end-block 1001000 \
    --batch-size 100
```

### Full Transform (raw → transformed)

Run the raw → transformed "full transform" by driving the external
[graphsense-spark](https://github.com/graphsense/graphsense-spark) Scala job via
`spark-submit`. The release jar is downloaded from a public GitHub Release asset
(token-free) and cached under `<cache_directory>/spark-jars`. The command creates
a fresh dated transformed keyspace, resolves Spark properties from a
`spark_config` profile, and launches the job.

```bash
# Show options
graphsense-cli transformation raw-to-transformed --help

# Run against a pinned release, creating a fresh dated transformed keyspace
graphsense-cli transformation raw-to-transformed -e prod -c btc --version v26.06.0

# Use the latest stable release (also the default when no version is configured)
graphsense-cli transformation raw-to-transformed -e prod -c btc --version latest

# Preview only: print the resolved Spark config + spark-submit command, no side effects
graphsense-cli transformation raw-to-transformed -e prod -c btc --dry-run

# Run Spark locally for testing
graphsense-cli transformation raw-to-transformed -e dev -c btc --local

# Write into an existing transformed keyspace instead of a fresh dated one
graphsense-cli transformation raw-to-transformed -e prod -c btc \
    --target-keyspace btc_transformed

# Pass extra args through to the graphsense-spark job after `--`
graphsense-cli transformation raw-to-transformed -e prod -c btc -- --debug 1
```

Configured via a `full_transform_args` section in the config file:

```yaml
full_transform_args:
  # Release tag to run. Omit or set to "latest" to resolve the newest stable
  # (non-prerelease) release from the GitHub API at run time.
  version: v26.06.0
  version_overrides: # optional, per-currency
    eth: v26.06.1
  artifact: fat # "fat" (self-contained assembly, default) or "slim" (+ Maven packages)
  spark_profile: # selects a spark_config profile per currency
    btc: utxo
  jar_args: # optional, per-currency extra job args
    btc: ["--bech32-prefix", "bc", "--bucket-size", "5000"]
  # Optional Cassandra Sidecar bulk-write path
  sidecar:
    enabled: false
    contact_points: ["cassandra-1:9043", "cassandra-2:9043"]
    local_dc: dc1
    consistency_level: LOCAL_QUORUM
```

Spark properties (including `spark.master`) come from the selected `spark_config`
profile; Cassandra coordinates (host/port/credentials) are taken from the
environment config. The command is backend-neutral (`backend: scala` today,
`pyspark` reserved) so a future native implementation can be selected without
changing how it is invoked.

#### Java runtime when submitting from the Docker image

The Docker image defaults to Java 17 but also ships a secondary Temurin 11 JRE
at `/opt/java11`. If you submit against a standalone cluster whose executors run
Java 11 (the prod cluster — its hosts are shared with Cassandra 4.x, which caps
them there) and the profile uses `spark.serializer=KryoSerializer` (the full
transform profiles do), the driver must run Java 11 too: Kryo serializes
JDK-internal field layouts that differ between major Java versions, so a
Java-17 driver fails to deserialize task results from Java-11 executors
(`java.io.EOFException` in `TaskResultGetter`). Opt in per run by setting
`JAVA_HOME` — `spark-submit` picks it up, nothing else changes:

```bash
# From the host
docker run -e JAVA_HOME=/opt/java11 ... graphsense-cli transformation raw-to-transformed -e prod -c trx ...

# Or inside the container
JAVA_HOME=/opt/java11 graphsense-cli transformation raw-to-transformed -e prod -c trx ...
```

Drop the override once the cluster JVM moves to 17+ (needs Cassandra 5.x on the
shared hosts first).

### Delta Updates

Update transformed keyspace from raw keyspace.

```bash
# Show delta update options
graphsense-cli delta-update --help

# Check update status
graphsense-cli delta-update status -e dev -c btc

# Perform delta update
graphsense-cli delta-update update -e dev -c btc

# Validate delta update consistency
graphsense-cli delta-update validate -e dev -c btc

# Patch exchange rates for specific blocks
graphsense-cli delta-update patch-exchange-rates \
    -e dev \
    -c btc \
    --start-block 100000 \
    --end-block 200000
```

### Exchange Rates

Fetch and ingest exchange rates from various sources.

```bash
# Show exchange rate options
graphsense-cli exchange-rates --help

# Fetch from CoinDesk
graphsense-cli exchange-rates coindesk -e dev -c btc

# Fetch from CoinMarketCap (requires API key in config)
graphsense-cli exchange-rates coinmarketcap -e dev -c btc

# Ingest from CryptoCompare (requires cryptocompare_api_key in config,
# see https://developers.coindesk.com/)
graphsense-cli exchange-rates cryptocompare ingest -e dev -c btc --abort-on-gaps
```

### Monitoring

Monitor GraphSense infrastructure health and state.

```bash
# Show monitoring options
graphsense-cli monitoring --help

# Get database summary
graphsense-cli monitoring get-summary -e dev

# Get summary for specific currency
graphsense-cli monitoring get-summary -e dev -c btc

# Send notifications to configured handlers
graphsense-cli monitoring notify \
    --topic "database-update" \
    --message "BTC ingestion completed"
```

### Event Watching (Alpha)

Watch for cryptocurrency events and generate notifications.

```bash
# Show watch options
graphsense-cli watch --help

# Watch for money flows on specific addresses
graphsense-cli watch money-flows \
    -e dev \
    -c btc \
    --address 1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa \
    --threshold 1000000  # satoshis
```

### File Conversion Tools

Convert between different file formats.

```bash
# Show conversion options
graphsense-cli convert --help
```

## Configuration

GraphSense-lib uses a YAML configuration file that defines database connections and environment settings. Default locations: `./.graphsense.yaml`, `~/.graphsense.yaml`.

### Generate Configuration Template

```bash
graphsense-cli config template > ~/.graphsense.yaml
```

### Example Configuration Structure

```yaml
# Optional: default environment to use
default_environment: dev

environments:
  dev:
    # Cassandra cluster configuration
    # Default port is 9042. To use a non-default port, embed it in the host
    # string ("host:port"); all nodes should use the same port (a warning is
    # logged and the first port wins if they differ).
    cassandra_nodes: ["localhost"]
    # Optional authentication
    # username: "cassandra"
    # password: "cassandra"

    # Consistency levels for the synchronous (ingest/write) connection.
    # consistency_level applies to the whole ExecutionProfile (reads + writes);
    # serial_consistency_level only affects lightweight transactions
    # (IF NOT EXISTS upserts). Defaults shown below.
    # consistency_level: "LOCAL_QUORUM"
    # serial_consistency_level: "LOCAL_SERIAL"

    # Currency/keyspace configurations
    keyspaces:
      btc:
        raw_keyspace_name: "btc_raw"
        transformed_keyspace_name: "btc_transformed"
        schema_type: "utxo"

        # Node connection for ingestion
        ingest_config:
          node_reference: "http://localhost:8332"
          # Optional authentication for node
          # username: "rpcuser"
          # password: "rpcpassword"
          # Optional: staleness tolerance in hours. After every
          # `ingest from-node` run the timestamp of the highest ingested raw
          # block is checked against this threshold and a notification is sent
          # if the data is older (same check as `monitoring monitor-raw-ingest`).
          # raw_ingest_staleness_threshold: 10
          # Optional: ingest the latest exchange rates from this provider
          # before every `ingest from-node` run (same as `exchange-rates
          # <provider> ingest --abort-on-gaps`). Supported: coingecko,
          # coinmarketcap, cryptocompare (each needs its API key below).
          # exchange_rates_provider: "coingecko"

        # Keyspace setup for schema creation
        keyspace_setup_config:
          raw:
            replication_config: "{'class': 'SimpleStrategy', 'replication_factor': 1}"
          transformed:
            replication_config: "{'class': 'SimpleStrategy', 'replication_factor': 1}"

      eth:
        raw_keyspace_name: "eth_raw"
        transformed_keyspace_name: "eth_transformed"
        schema_type: "account"

        ingest_config:
          node_reference: "http://localhost:8545"

        keyspace_setup_config:
          raw:
            replication_config: "{'class': 'SimpleStrategy', 'replication_factor': 1}"
          transformed:
            replication_config: "{'class': 'SimpleStrategy', 'replication_factor': 1}"

  prod:
    cassandra_nodes: ["cassandra1.prod", "cassandra2.prod", "cassandra3.prod"]
    username: "gs_user"
    password: "secure_password"

    keyspaces:
      btc:
        raw_keyspace_name: "btc_raw"
        transformed_keyspace_name: "btc_transformed"
        schema_type: "utxo"

        ingest_config:
          node_reference: "http://bitcoin-node.internal:8332"

        keyspace_setup_config:
          raw:
            replication_config: "{'class': 'NetworkTopologyStrategy', 'datacenter1': 3}"
          transformed:
            replication_config: "{'class': 'NetworkTopologyStrategy', 'datacenter1': 3}"

# Optional: Slack notification configuration
slack_topics:
  database-update:
    hooks: ["https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK"]

  payment_flow_notifications:
    hooks: ["https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK"]

# Optional: API keys for external services
coingecko_api_key: ""
coinmarketcap_api_key: "YOUR_CMC_API_KEY"
# Required for the cryptocompare exchange-rates provider since June 2026
# (https://developers.coindesk.com/)
cryptocompare_api_key: ""

# Optional: cache directory for temporary files
cache_directory: "~/.graphsense/cache"
```

## Advanced Features

### Tagpack Management

GraphSense-lib includes comprehensive tagpack management tools (formerly standalone tagpack-tool). For detailed documentation, see [Tagpack README](tagpack/docs/README.md).

```bash
# Validate tagpacks
graphsense-cli tagpack-tool tagpack validate /path/to/tagpack

# Insert tagpack into tagstore
graphsense-cli tagpack-tool insert \
    --url "postgresql://user:pass@localhost/tagstore" \
    /path/to/tagpack

# Show quality measures
graphsense-cli tagpack-tool quality show-measures \
    --url "postgresql://user:pass@localhost/tagstore"
```

### Tagstore Operations

```bash
# Initialize tagstore database
graphsense-cli tagstore init

# Initialize with custom database URL
graphsense-cli tagstore init --db-url "postgresql://user:pass@localhost/tagstore"

# Get DDL SQL for manual setup
graphsense-cli tagstore get-create-sql
```

### Cross-chain Analysis

```python
# Using an initialized AddressesService (see above for setup)
related = await addresses_service.get_cross_chain_pubkey_related_addresses(
    "1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa"
)

for addr in related:
    print(f"Network: {addr.network}, Address: {addr.address}")
```

### Function Call Parsing

```python
from graphsenselib.utils.function_call_parser import parse_function_call

# Parse Ethereum function calls
function_signatures = {
    "0xa9059cbb": [{
        "name": "transfer",
        "inputs": [
            {"name": "to", "type": "address"},
            {"name": "value", "type": "uint256"}
        ]
    }]
}

parsed = parse_function_call(tx_input_bytes, function_signatures)
if parsed:
    print(f"Function: {parsed['name']}")
    print(f"Parameters: {parsed['parameters']}")
```

## Development

**Important:** Requires Python >=3.10, <3.13.

### Setup Development Environment

```bash
# Initialize development environment (installs deps + pre-commit hooks)
make dev

# Or install dev dependencies only
make install-dev
```

### Code Quality and Testing

Before committing, please format, lint, and test your code:

```bash
# Format code
make format

# Lint code
make lint

# Run fast tests
make test

# Or run all steps at once
make pre-commit
```

For comprehensive testing:

```bash
# Run complete test suite (including slow tests)
make test
```

#### Podman Notes

If you run the test suite with Podman, make sure your shell points at the Podman socket:

```bash
export DOCKER_HOST="unix://${XDG_RUNTIME_DIR}/podman/podman.sock"
```

The test fixtures automatically disable Ryuk when `DOCKER_HOST` contains `podman.sock` and rely on explicit fixture cleanup instead.

### Release Process

This repository uses two source-of-truth versions in the root `Makefile`:

- **Library version**: `RELEASESEM` (released with `vX.Y.Z`, `vX.Y.Z-rc.N`, or `vX.Y.Z-dev.N` tags)
- **OpenAPI/API version**: `WEBAPISEM` (written to `src/graphsenselib/web/version.py`)

The Python client package version is derived from the API version and should match it.

Library package versioning is dynamic via `setuptools_scm` (`pyproject.toml`):

- Git tag `v2.9.8` -> package version `2.9.8`
- Git tag `v2.9.8-rc.1` -> package version `2.9.8rc1`
- Git tag `v2.9.8-dev.1` -> package version `2.9.8.dev1`
- Commits after a tag append local metadata, for example `2.9.8.dev1+g<sha>.d<date>`

Use the root Makefile helpers:

```bash
# Show all current versions
make show-versions

# Update and validate OpenAPI contract version
make update-api-version WEBAPISEM=v2.10.0
make check-api-version WEBAPISEM=v2.10.0

# Sync client version from API version and validate
make sync-client-version WEBAPISEM=v2.10.0
make check-client-version WEBAPISEM=v2.10.0

# Generate Python client (package version = OpenAPI info.version)
make generate-python-client

# Create both release tags from Makefile versions
make tag-version
```

Tagging behavior:

- Library release tag: `vX.Y.Z`, `vX.Y.Z-rc.N`, or `vX.Y.Z-dev.N` (from `RELEASESEM`)
- Client release tag: `webapi-vA.B.C` (from `WEBAPISEM`)

Recommended library versioning routine:

1. For development prereleases, set `RELEASESEM` to `vX.Y.Z-dev.N` (for example `v2.10.0-dev.1`)
2. For release candidates, set `RELEASESEM` to `vX.Y.Z-rc.N`
3. For stable releases, set `RELEASESEM` to `vX.Y.Z`
4. Create tags with `make tag-version`
5. Push tags with `git push origin --tags`

CI trigger background:

- Stable library tags (`vX.Y.Z`) trigger:
  - GitHub Release creation
  - Python library package build/publish (`graphsense-lib`)
  - Docker image build/publish
- Client tags (`webapi-vA.B.C`) trigger Python client package build/publish (`clients/python`)
- Other library tags (`vX.Y.Z-rc.N`, `vX.Y.Z-dev.N`) do not trigger GitHub Release or Python package publish; they only trigger Docker image build/publish

1. Update CHANGELOG.md with new features and fixes
2. Update relevant versions (library/API/client) based on what changed
3. Sync API/client versions if needed (`make update-api-version` + `make sync-client-version`)
4. Create and push tags:

```bash
make tag-version
git push origin --tags
```

## Troubleshooting

### OpenSSL Errors

Some components use OpenSSL hash functions that aren't available by default in OpenSSL 3.0+ (e.g., ripemd160). This can cause test suite failures. To fix this, enable legacy providers in your OpenSSL configuration. See the "fix openssl legacy mode" step in `.github/workflows/run_tests.yaml` for an example.

### Common Issues

1. **Connection Refused**: Verify Cassandra is running and accessible
2. **Schema Validation Errors**: Ensure database schema matches expected version
3. **Import Errors**: Install with `[all]` option for complete feature set
4. **Python Version**: Requires Python >=3.10, <3.13

### Getting Help

- Check [GitHub Issues](https://github.com/graphsense/graphsense-lib/issues)
- Review [GraphSense Documentation](https://graphsense.github.io/)
- Use `--help` with any CLI command for detailed usage information
- For tagpack-specific issues, see [Tagpack Documentation](tagpack/docs/README.md)

## License

See LICENSE file for licensing details.

## Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Run `make pre-commit` to ensure code quality
5. Submit a pull request

---

**GraphSense** - Open Source Crypto Analytics Platform
Website: https://graphsense.github.io/
