Metadata-Version: 2.4
Name: zarranalyzer
Version: 1.0.0b1
Summary: Analyze Zarr stores for completeness, sizing, and archival distribution workflows
Keywords: zarr,analysis,data-archiving,cloud-storage,scientific-data
Author: Kameswarrao Modali
Author-email: Kameswarrao Modali <modali@dkrz.de>
License-Expression: GPL-3.0-only
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: System :: Archiving
Requires-Dist: fsspec>=2026.2.0
Requires-Dist: matplotlib>=3.10.8
Requires-Dist: plotly>=6.6.0
Requires-Dist: pyslk>=2.3.0
Requires-Dist: rich>=14.3.3
Requires-Dist: validators>=0.35.0
Requires-Dist: zarr>=3.1.5
Requires-Dist: build>=1.4.0 ; extra == 'dev'
Requires-Dist: codespell>=2.4.1 ; extra == 'dev'
Requires-Dist: mypy>=1.19.1 ; extra == 'dev'
Requires-Dist: pre-commit>=4.3.0 ; extra == 'dev'
Requires-Dist: pytest>=9.0.2 ; extra == 'dev'
Requires-Dist: ruff>=0.15.4 ; extra == 'dev'
Requires-Dist: twine>=6.2.0 ; extra == 'dev'
Requires-Dist: sphinx>=9.1.0 ; extra == 'docs'
Requires-Dist: sphinx-rtd-theme>=3.1.0 ; extra == 'docs'
Requires-Dist: myst-parser>=5.1.0 ; extra == 'docs'
Requires-Dist: rst2pdf>=0.105 ; extra == 'docs'
Requires-Python: >=3.12
Project-URL: Homepage, https://github.com/kmodali/zarranalyzer
Project-URL: Issues, https://github.com/kmodali/zarranalyzer/issues
Project-URL: Documentation, https://zarranalyzer.readthedocs.io
Project-URL: Changelog, https://github.com/kmodali/zarranalyzer/blob/main/CHANGELOG.md
Provides-Extra: dev
Provides-Extra: docs
Description-Content-Type: text/markdown

# zarranalyzer

zarranalyzer analyzes Zarr stores for completeness and sizing, supports visualization, and can package selected content into tar archives for downstream archival workflows.

## Repository Layout

- `src/zarranalyzer`: package source code
- `tests`: repository-level test suite

## Contributing

Contribution workflow, coding rules, testing expectations, and PR guidelines are documented in `CONTRIBUTING.md`.

## Installation

zarranalyzer requires Python 3.12 or newer.

Install from a local clone (recommended for development):

```bash
cd /path/to/zarranalyzer
python -m pip install -e .
```

Install as a regular package from a local clone:

```bash
cd /path/to/zarranalyzer
python -m pip install .
```

Build wheel/sdist and install the wheel:

```bash
cd /path/to/zarranalyzer
python -m pip install build
python -m build
python -m pip install dist/*.whl
```

Optional: install development tooling used in this repository:

```bash
cd /path/to/zarranalyzer
python -m pip install -e .[dev]
```

Optional: install documentation tooling:

```bash
cd /path/to/zarranalyzer
python -m pip install -e .[docs]
```

## Development Commands

Run unit test suite:

```bash
python -m tests.test_all_modules
python -m tests.regression_test
```

Run focused config/object-store tests:

```bash
python -m unittest tests.test_all_modules.TestConfigAndSubgroup
```

Generate API documentation from source docstrings:

```bash
python scripts/generate_api_docstrings.py
```

Build Read the Docs style documentation (HTML and PDF):

```bash
python -m sphinx -b html docs docs/_build/html
python -m sphinx -b pdf docs docs/_build/pdf
```

## Releases

Repository release process and checklist are documented in `RELEASE.md`.

Bump package version (single-source in `pyproject.toml`):

```bash
python scripts/release.py --bump patch
```

Automated publishing is configured via GitHub Actions Trusted Publishing:

- `.github/workflows/publish-pypi.yml`
- tag push `v*` publishes to PyPI
- manual `workflow_dispatch` supports TestPyPI and PyPI targets

Version consistency CI is configured via:

- `.github/workflows/version-consistency.yml`

Run pre-commit quality checks (lint, type checks, spell checks, and hygiene checks):

```bash
python -m pip install pre-commit
pre-commit install
pre-commit run --all-files
```

Run benchmark harness:

```bash
python -m zarranalyzer.benchmark --help
python -m zarranalyzer.benchmark --synthetic --phase diagnose
python -m zarranalyzer.benchmark --synthetic --phase both
```

Create a synthetic Zarr experiment to test diagnose/distribute/dispatch flows:

```bash
cd src
python -m zarranalyzer.synthetic_store --output-dir ../tests/data/synthetic_experiment --clean --zarr-formats 2,3 --run-smoke
```

## Object Store Access

For supported S3 endpoints, zarranalyzer now uses this access order:

1. Try anonymous access first.
2. If anonymous access fails, try configured credentials.
3. If credentials are missing and anonymous access fails, exit with a clear error message.

You can provide credentials in three supported ways:

1. Use the new utility helper (recommended for notebooks/scripts).
2. Use config setters with environment-variable names.
3. Export environment variables directly.

Using the utility helper:

```python
import zarranalyzer

# Accepts raw values OR env-var names containing the values.
zarranalyzer.setObjectStoreCredentials(
	accessKey="AWS_ACCESS_KEY_ID",
	secretKey="AWS_SECRET_ACCESS_KEY",
)
```

You can also pass credentials directly on store-facing utility calls:

```python
from zarranalyzer.utils import diagnoseStore, findMissingChunksInStore

diagnoseStore(
	"https://s3.eu-dkrz-3.dkrz.cloud/mybucket/my-store.zarr",
	accessKey="AWS_ACCESS_KEY_ID",
	secretKey="AWS_SECRET_ACCESS_KEY",
)

findMissingChunksInStore(
	"https://s3.eu-dkrz-3.dkrz.cloud/mybucket/my-store.zarr",
	accessKey="AKIA...",
	secretKey="...",
)
```

Using config setters:

```python
from zarranalyzer.config import configObj

# Pass environment variable names.
configObj.setaccessKey("AWS_ACCESS_KEY_ID")
configObj.setsecretKey("AWS_SECRET_ACCESS_KEY")
```

Using environment variables:

```bash
export AWS_ACCESS_KEY_ID="<ACCESS_KEY>"
export AWS_SECRET_ACCESS_KEY="<SECRET_KEY>"
```

Alternative variable names are also recognized:

- `S3_ACCESS_KEY_ID`
- `S3_SECRET_ACCESS_KEY`

## Logging

CLI logging level can be set via:

- argument: `--log-level DEBUG`
- environment variable: `ZARRANALYZER_LOG_LEVEL=DEBUG`

During CLI usage, logs are also written to a timestamped file:

- default directory: `logs/`
- filename pattern: `zarranalyzer_YYYYMMDD_HHMMSS.log`

You can override the log directory with:

- environment variable: `ZARRANALYZER_LOG_DIR=/path/to/logs`

For notebook/library usage (non-CLI), enable the same logging with:

```python
import zarranalyzer

zarranalyzer.enable_logging("INFO")
```
