Metadata-Version: 2.4
Name: arrakis-admin
Version: 0.1.0
Summary: Admin CLI for the Arrakis low-latency timeseries data distribution platform
Project-URL: Homepage, https://git.ligo.org/ngdd/arrakis-admin
Project-URL: Repository, https://git.ligo.org/ngdd/arrakis-admin.git
Project-URL: Issues, https://git.ligo.org/ngdd/arrakis-admin/issues
Author-email: Olivia Godwin <olivia.godwin@ligo.org>
Maintainer-email: Olivia Godwin <olivia.godwin@ligo.org>
License-Expression: GPL-3.0-or-later
License-File: LICENSE
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Astronomy
Classifier: Topic :: Scientific/Engineering :: Physics
Requires-Python: >=3.11
Requires-Dist: arrakis-backend-kafka>=0.10
Requires-Dist: arrakis-server>=0.14
Requires-Dist: arrakis>=0.15
Requires-Dist: confluent-kafka>=1.8
Requires-Dist: gpstime
Requires-Dist: rich>=13.0
Requires-Dist: typer>=0.12
Provides-Extra: dev
Requires-Dist: mypy; extra == 'dev'
Requires-Dist: mypy-extensions; extra == 'dev'
Requires-Dist: pip; extra == 'dev'
Requires-Dist: pytest; extra == 'dev'
Requires-Dist: pytest-cov; extra == 'dev'
Requires-Dist: ruff; extra == 'dev'
Provides-Extra: lint
Requires-Dist: mypy; extra == 'lint'
Requires-Dist: mypy-extensions; extra == 'lint'
Requires-Dist: pip; extra == 'lint'
Requires-Dist: ruff; extra == 'lint'
Provides-Extra: test
Requires-Dist: pytest; extra == 'test'
Requires-Dist: pytest-cov; extra == 'test'
Description-Content-Type: text/markdown

<h1 align="center">arrakis-admin</h1>

<p align="center">Admin CLI for the Arrakis low-latency timeseries data distribution platform</p>

<p align="center">
  <a href="https://git.ligo.org/ngdd/arrakis-admin/-/pipelines/latest">
    <img alt="ci" src="https://git.ligo.org/ngdd/arrakis-admin/badges/main/pipeline.svg" />
  </a>
  <a href="https://git.ligo.org/ngdd/arrakis-admin/-/pipelines/latest">
    <img alt="coverage" src="https://git.ligo.org/ngdd/arrakis-admin/badges/main/coverage.svg" />
  </a>
  <a href="https://ngdd.docs.ligo.org/arrakis-admin/">
    <img alt="documentation" src="https://img.shields.io/badge/docs-mkdocs%20material-blue.svg?style=flat" />
  </a>
  <a href="https://pypi.org/project/arrakis-admin/">
    <img alt="pypi version" src="https://img.shields.io/pypi/v/arrakis-admin.svg" />
  </a>
</p>

---

Administrative tooling for operating Arrakis deployments: manage the
Kafka topics backing data partitions, probe data topics for debugging,
and migrate server metadata caches.

## Resources

* [Documentation](https://docs.ligo.org/ngdd/arrakis-admin)
* [Source Code](https://git.ligo.org/ngdd/arrakis-admin)
* [Issue Tracker](https://git.ligo.org/ngdd/arrakis-admin/-/issues)

## Installation

With `pip`:

```
pip install arrakis-admin
```

## Quickstart

Every command supports `--help`. Commands that modify the broker show
what they will do and prompt for confirmation, and support `--dry-run`
to preview changes without applying them.

### Manage topics

Create Kafka topics for a publisher's data partitions, with partition
assignments queried from the Arrakis server:

```
➜ arrakis-admin topics create H1-lldd -b kafka1:9092 -s grpc://arrakis:31206

topics to create (2):
  broker: kafka1:9092
  retention: 7d (604_800_000ms), replication factor: 1

  + arrakis-H1-lldd_float32
  + arrakis-H1-lldd_float64

metadata topic (compact, indefinite retention):
  + arrakis-metadata-H1-lldd

Is this ok [y/N]:
```

Partition assignments can instead be read from a publisher TOML config
with `--config`, and replay-context publishers are supported via
`--replay-id`.

Sync data topics against the channel metadata published on the broker,
creating missing topics and removing stale ones:

```
➜ arrakis-admin topics sync -b kafka1:9092 --dry-run
```

Remove a publisher's topics (or all Arrakis topics with `--all`):

```
➜ arrakis-admin topics remove -b kafka1:9092 -p H1-lldd
```

### Probe data topics

A debugging tool that consumes a partition's data topic directly from
Kafka and unpacks each message's Arrow payload, reporting block times,
per-channel content, and latencies:

```
➜ arrakis-admin probe -p H1-lldd_float64 -b kafka1:9092 -v
probing topic 'arrakis-H1-lldd_float64' on kafka1:9092 (Ctrl-C to stop)

offset=5231 time=1418063158.000000000 latency=0.412s transport=0.198s size=125.4kB channels=42 samples=131072
  [   0]  float64  16384 samples
  [   1]  float64  16384 samples
  ...
```

`latency` is the delay between the GPS block time and receipt;
`transport` is the delay relative to the Kafka message timestamp,
separating publisher lag from broker lag.

A channel name can be given instead of a partition ID, in which case
its partition is resolved through the Arrakis server and the channel's
rows are highlighted in the output:

```
➜ arrakis-admin probe -c H1:GDS-CALIB_STRAIN -s grpc://arrakis:31206 -b kafka1:9092
```

### Migrate metadata caches

Convert an arrakis-server TOML metadata cache to SQLite, or upgrade an
existing SQLite cache to the current schema:

```
➜ arrakis-admin cache migrate metadata.toml
migrated 12483 channels from metadata.toml to metadata.db

➜ arrakis-admin cache migrate metadata.db
upgraded metadata.db from schema v2 to v3
```

Clear all data from a cache while preserving its schema:

```
➜ arrakis-admin cache clear metadata.db
```
