Metadata-Version: 2.4
Name: bdv-playground-deconvolution
Version: 0.21.0.2
Summary: Tiled, lazy, multi-GPU Richardson-Lucy deconvolution for large 5D microscopy images
Project-URL: Homepage, https://github.com/unige-biochem/bdv-playground-deconvolution
Project-URL: Repository, https://github.com/unige-biochem/bdv-playground-deconvolution
Project-URL: Issues, https://github.com/unige-biochem/bdv-playground-deconvolution/issues
Author-email: Nicolas Chiaruttini <nicolas.chiaruttini@unige.ch>
License-Expression: MIT
License-File: LICENSE
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Classifier: Topic :: Scientific/Engineering :: Image Processing
Requires-Python: >=3.10
Requires-Dist: pyimagej>=1.8.0
Requires-Dist: scyjava>=1.12.0
Provides-Extra: notebook
Requires-Dist: jupyterlab>=4.0; extra == 'notebook'
Description-Content-Type: text/markdown

# BDV-Playground Deconvolution

Tiled, lazy, multi-GPU Richardson–Lucy deconvolution for large **5D** microscopy
images (XYZ + channels + timepoints), in Python.

```
pip install bdv-playground-deconvolution   # import bdvpg_deconvolution
```

It handles images far bigger than GPU memory by working **tiled** and **lazily**:
each volume is split into overlapping blocks, each block is deconvolved on the
GPU, and nothing is computed until you actually browse or export the result.
Multiple GPUs (or several contexts on one GPU) can be used in parallel.

All channels and timepoints are processed and written out by default, in the
original order, using a single PSF.

Under the hood it drives [BigDataViewer-Playground](https://bigdataviewer-playground-documentation.readthedocs.io/en/latest/processing_images/deconvolution.html) and [CLIJ2](https://github.com/clij/clij2-fft) through [PyImageJ](https://github.com/imagej/pyimagej). Python is the orchestration layer.

## Why deconvolution

The axial (Z) view is where widefield blur is worst and where deconvolution
helps most:

| Raw | Deconvolved |
|-----|-------------|
| ![Cross-section — raw](assets/CrossSection-Raw.png) | ![Cross-section — deconvolved](assets/CrossSection-Deconvolved.png) |

## Install

```bash
pip install bdv-playground-deconvolution                 # core
pip install "bdv-playground-deconvolution[notebook]"     # + JupyterLab
```

Works in any Python ≥3.10 environment — venv, conda, or `uv pip`. This puts
`bdvpg-deconvolve`, `bdvpg-gpu-pool` and `bdvpg-smoke-test` on your PATH; call
them directly, no
`uv run` prefix. (If you are working from a clone instead, see
[Development](#development).)

**No conda required, and you do not need to install Java or Maven yourself.**
`scyjava`/`jgo` provision both automatically via
[`cjdk`](https://pypi.org/project/cjdk) on first use, into a user-level cache
(`%LOCALAPPDATA%\cjdk` on Windows, `~/.cache/cjdk` elsewhere).

The only real prerequisite is an **OpenCL-capable GPU** with vendor drivers
installed — that part is not pip-installable.

> **First run is heavy.** Installing is a few MB, but the first *run* downloads
> a JDK (~190 MB), Maven, and the ImageJ2/BIOP Maven tree — several hundred MB,
> once, then cached. It needs `maven.scijava.org` reachable.

Verify your setup without a GPU or any data:

```bash
bdvpg-smoke-test    # boots the JVM, resolves every Java class used
bdvpg-gpu-pool      # shows the GPUs and the configured pool
```

## Quick start (CLI)

Headless and save-only — the intended batch / pipeline interface:

```bash
bdvpg-deconvolve \
  --image  /path/to/image.czi \
  --psf    /path/to/psf.tif \
  --out    /path/to/output_folder \
  --iterations 120 \
  --threads 10
```

Writes `<image>.ome.tiff` to the output folder, preserving channel order.
`bdvpg-deconvolve --help` lists every option.

## Notebook

The notebook is not shipped in the wheel — grab it from the repo:

```bash
curl -LO https://raw.githubusercontent.com/unige-biochem/bdv-playground-deconvolution/main/notebooks/Deconvolve.ipynb
jupyter lab
```

[`notebooks/Deconvolve.ipynb`](notebooks/Deconvolve.ipynb) does interactive
parameter tuning and views raw + deconvolved side by side in BigDataViewer.
Use `mode="interactive"` (needs a display).

## Library

```python
from bdvpg_deconvolution import DeconvolveParams, init_imagej, run

ij = init_imagej(mode="headless", max_heap="32g")
run(DeconvolveParams(
    image_file="image.czi",
    psf_file="psf.tif",
    output_folder="out/",
    num_iterations=120,
), ij=ij)
```

A JVM starts **once per process**, so `init_imagej()` must be called before any
work and its `mode` cannot change afterwards.

`DeconvolveParams` takes `series` and `series_naming` alongside the CLI flags.
To inspect a file's series without running anything, open it and ask the
source service — `describe_series()` returns `(index, name, n_channels)` tuples:

```python
from bdvpg_deconvolution.pipeline import describe_series
```

Library users keep control of the process: `run()` and `init_imagej()` never
terminate it. Only the console-script entry points do (see
[Nextflow](#nextflow)), via `pipeline.hard_exit()`.

Note that reusing one gateway for many files keeps every opened source
registered until `run()` cleans them up, which it only does when
`show_in_bdv=False`. In a long notebook session, re-opening a file whose
dataset name was already used can leave stale nodes behind.

## Point Spread Function

One **single-channel PSF** is supplied per image and reused for all channels.
If no empirical PSF (e.g. from sub-resolution beads) is available, a theoretical
one can be generated with the
[PSF Generator](https://bigwww.epfl.ch/algorithms/psfgenerator/) Fiji plugin.

![Theoretical PSF](assets/PSF-Theoretical-Generated.png)

## Parameters

| Flag | Default | Notes |
|------|---------|-------|
| `--iterations` | 120 | Richardson–Lucy steps |
| `--regularization` | 0.0 | 0 = none; increase to tame noise/ringing |
| `--no-non-circulant` | (on) | disable non-circulant edge handling |
| `--block-size-x/y/z` | 256/256/64 | tiling — lower if you run out of GPU memory |
| `--overlap-size` | 16 | tile overlap, avoids seams |
| `--threads` | 10 | CPU-side workers feeding the GPU pool |
| `--gpu-pool` | (persisted) | GPU workers per device, see [Multi-GPU configuration](#multi-gpu-configuration) |
| `--output-pixel-type` | keep original | or `Float` |
| `--compression` | LZW | OME-TIFF compression |
| `--resolution-levels` | 1 | OME-TIFF pyramid levels |
| `--series` | — | which image of a [multi-series file](#multi-series-files) to process |
| `--series-naming` | name | `name` or `index`, suffix for multi-series output |
| `--range-channels` | all | subset of channels to export, see [Selecting a sub-range](#selecting-a-sub-range) |
| `--range-slices` | all | subset of Z slices to export |
| `--range-frames` | all | subset of timepoints to export |
| `--unit` | MICROMETER | coordinate unit |
| `--overwrite` | off | refuse to clobber existing output unless set |
| `--mode` | headless | escape hatch if a command misbehaves headless |
| `--max-heap` | — | JVM heap, e.g. `32g` |

### Multi-series files

Many formats (CZI, LIF, ND2…) hold several images in one file — typically one
per stage position. **Single-series files need no extra flag** and behave
exactly as before, writing `<image>.ome.tiff`.

A multi-series file is refused unless you say which image you mean, because the
alternative would be to deconvolve unrelated positions together as if they were
channels of one image. The error lists what is inside:

```
$ bdvpg-deconvolve --image day4to5.czi --psf psf.tif --out ./out
ERROR: 'day4to5.czi' contains 4 series; choose one with series=<index> (CLI: --series <index>):
  0  Day4to5 - Position 5  (2 channels)
  1  Day4to5 - Position 6  (2 channels)
  2  Day4to5 - Position 7  (2 channels)
  3  Day4to5 - Position 8  (2 channels)

Each series is written to its own file. The name defaults to
<image>_<series name>.ome.tiff; set series_naming='index'
(CLI: --series-naming index) for <image>_<index>.ome.tiff instead.
```

Pick one with `--series`, which is zero-based and indexes that listing:

```bash
bdvpg-deconvolve --image day4to5.czi --psf psf.tif --out ./out --series 2
# -> out/day4to5_Day4to5_-_Position_7.ome.tiff
```

Output naming for a multi-series file follows `--series-naming`:

| `--series-naming` | Output for series 2 above |
|-------------------|---------------------------|
| `name` (default) | `day4to5_Day4to5_-_Position_7.ome.tiff` |
| `index` | `day4to5_2.ome.tiff` |

`name` keeps the acquisition's own labels, which survive a re-export in a
different order; `index` gives short, predictable names that are easier to glob
in a pipeline. Series names are sanitised for the filesystem — spaces become
underscores and `<>:"/\|?*` are replaced.

Since only one series is processed per run, a whole file is covered by looping
over the indices, each run producing its own OME-TIFF:

```bash
for i in 0 1 2 3; do
  bdvpg-deconvolve --image day4to5.czi --psf psf.tif --out ./out --series $i
done
```

Note this pays the JVM startup cost per series. From Python you can instead call
`run()` repeatedly against a single `init_imagej()` gateway.

The PSF is treated differently on purpose: its first source is always used, as
before, so a multi-series PSF is not an error.

### Selecting a sub-range

`--range-channels`, `--range-slices` and `--range-frames` restrict what gets
written to the OME-TIFF. Because the deconvolution is lazy, blocks outside the
selection are never computed — a narrow range is genuinely cheaper, which makes
these flags the natural way to test parameters on one channel or a few slices
before committing to a full run:

```bash
bdvpg-deconvolve --image raw.czi --psf psf.tif --out ./test \
  --range-channels 0 --range-slices 20:30 --iterations 40
```

The syntax is [Kheops' `IntRangeParser`](https://github.com/BIOP/ijp-kheops/blob/master/src/main/java/ch/epfl/biop/kheops/IntRangeParser.java):

| Expression | Selects |
|------------|---------|
| *(blank)* | everything — the default |
| `2` | index 2 only |
| `0,2,5` | indices 0, 2 and 5 |
| `0:4` | 0, 1, 2, 3, 4 — **both bounds inclusive** |
| `0:2:8` | 0, 2, 4, 6, 8 — `start:step:end` |
| `-1` | the last index |
| `0:end` | everything, written out |
| `end:-1:0` | every index, reversed |
| `0:3,end` | blocks combine — 0, 1, 2, 3 and the last one |

Indices are **zero-based**, `end` is the last valid index, and negative values
count backwards from the end. Ranges are selections only — there is no syntax
for *removing* indices. An out-of-bounds index is an error, so `0:end` is the
safe way to say "all of them" when you are also composing other blocks.

The CLI is **save-only** by design — it deconvolves and writes an OME-TIFF.
Viewing results is the notebook's job: a CLI process exits as soon as the work
is done, which tears down the JVM and any BigDataViewer window with it.

## Multi-GPU configuration

Deconvolution runs on a pool of CLIJ contexts spread across the available GPUs.
The pool is described by a string of `device:workers` pairs — `0:2, 1:4` means
2 contexts on GPU 0 and 4 on GPU 1, i.e. **6 GPU workers**.

### Inspecting the setup

`bdvpg-gpu-pool` reports the devices and the configured pool. With no argument
it changes nothing, so it is safe to run any time:

```
$ bdvpg-gpu-pool
Available OpenCL devices (2):
  0  NVIDIA RTX PRO 4500 Blackwell
  1  NVIDIA RTX PRO 2000 Blackwell

Configured pool: 0:4, 1:2
  device 0  4 workers  NVIDIA RTX PRO 4500 Blackwell
  device 1  2 workers  NVIDIA RTX PRO 2000 Blackwell
  total GPU workers: 6
```

Device indices in a pool spec are the indices in that listing. Enumerating
devices does not allocate anything; add `--probe` to actually build the pool
and print its details, which is a real test that the configuration works:

```
$ bdvpg-gpu-pool --probe
...
CLIJxPool [size:6 idle:6]:
	- [IDLE] NVIDIA RTX PRO 4500 Blackwell
		- Img Support [true]  OpenCL [v1.2]
```

### Setting the pool

Either pass a spec to `bdvpg-gpu-pool`, or use `--gpu-pool` on a deconvolution
run:

```bash
bdvpg-gpu-pool "0:2, 1:4"                         # set it once
bdvpg-deconvolve --image raw.czi ... --gpu-pool "0:2, 1:4"   # set it per run
```

Both do the same thing, and two properties of that thing are worth knowing:

- **The setting is persistent and global.** It is written to the ImageJ
  preferences (the same key the Fiji *Pool Configuration* dialog uses), so it
  outlives the process, applies to later runs, and is shared with any other
  ImageJ tool on the machine. Omitting `--gpu-pool` leaves whatever is already
  configured in place — it does not reset to a default.
- **It is read once per JVM.** The pool is a lazy singleton built on first use,
  so `--gpu-pool` is applied before any GPU work starts. Changing the setting
  from inside a process that has already built its pool only affects the next
  process, and `bdvpg-gpu-pool` warns when that happens.

A spec naming a device that does not exist is rejected before anything is
written, listing the devices that do.

> **Pool workers vs `--threads`.** The pool config sets the number of **GPU-side**
> workers. `--threads` is the number of **CPU-side** workers feeding that pool
> (load, convert, hand to GPU, retrieve, write). Keep `--threads` a bit higher
> than the total GPU workers so the GPUs are never left waiting.

## Nextflow

The CLI is the intended Nextflow interface — one image per task, headless:

```groovy
process deconvolve {
    input:
      tuple val(sample), path(image), path(psf)
    output:
      path "${image.baseName}.ome.tiff"
    script:
      """
      bdvpg-deconvolve --image ${image} --psf ${psf} --out . \\
                 --iterations ${params.iterations} --threads ${params.threads}
      """
}
```

One JVM boots per invocation, so one-image-per-task is the right granularity.

> **The CLI terminates the process itself.** ImageJ starts AWT even headless,
> leaving non-daemon threads (`AWT-EventQueue-0`, `AWT-Shutdown`) that keep the
> JVM alive after the work is done — the command would otherwise write its
> OME-TIFF and then hang forever, holding a Nextflow slot with nothing left to
> do. `scyjava.shutdown_jvm()` clears this only some of the time, so the entry
> points end with `os._exit` instead. Exit codes are preserved. The consequence
> is that JVM shutdown hooks do not run, so anything that must reach disk is
> flushed explicitly — which is why setting the GPU pool also saves the ImageJ
> preferences rather than trusting them to be written at exit.
For reproducible runs, containerise with the OpenCL runtime, a pre-warmed cjdk
cache, and a pre-resolved `.jgo` env so tasks don't each re-download.

## Reproducibility

Two package managers are in play. `uv.lock` pins the Python side; the Java side
is pinned by the coordinates in
[`bdvpg_deconvolution/pipeline.py`](bdvpg_deconvolution/pipeline.py):

```python
DEFAULT_ENDPOINTS = [
    "net.imagej:imagej:2.16.0",
    "ch.epfl.biop:bigdataviewer-biop-tools:0.21.0",
]
```

Bump those and cut a release when you want to move the Java side.

The JVM itself is *not* pinned by default — cjdk prefers a suitable system JDK
and downloads one otherwise. To pin it, before the first `init_imagej()`:

```python
from scyjava import config
config.set_java_constraints(fetch="always", vendor="zulu", version="21")
```

## Status

The pipeline is a faithful transcription of a production Fiji/Groovy workflow,
and the interop layer is verified (`bdvpg-smoke-test` passes: JVM boots, all Java
classes, the `SourceService` tree and the GPU enumeration resolve). A full GPU
run has **not** been exercised end-to-end here — validate against a known
dataset first.

The multi-series selection has not been exercised against a real multi-series
file either: if the source tree layout is not the expected
`dataset > ImageName > series`, the code falls back to treating the file as a
single series, which would look like a file with one image.

Not yet implemented:

- `--prefetch` — warm the JDK/Maven/jgo caches ahead of first use.

## Development

From a clone, `uv` manages the environment and `uv.lock` pins it:

```bash
git clone https://github.com/unige-biochem/bdv-playground-deconvolution
cd bdv-playground-deconvolution
uv sync                      # core
uv sync --extra notebook     # + JupyterLab
uv run bdvpg-smoke-test      # verify the Java interop
```

> `uv run` uses the project's own `.venv` and **ignores an activated conda
> environment**. Either use `uv run` from the clone, or `pip install` into your
> conda env and call the commands directly — don't mix the two.

## Credits

Built on the BigDataViewer-Playground / Kheops / CLIJ2 stack.

## License

MIT — see [`LICENSE`](LICENSE). © Nicolas Chiaruttini, Department of
Biochemistry, University of Geneva.

That covers this package's own source, which is pure Python orchestration and
ships no Java code. The Java stack it drives is resolved from Maven on **your**
machine at first run, and parts of it are GPL — notably Bio-Formats
`formats-gpl`, which supplies the readers for proprietary formats such as
`.czi`. Simply installing and running this package does not put you under those
terms; the GPL restricts copying, distribution and modification, not use.

If you **redistribute a bundle** that contains those jars — most likely the
container image suggested in [Nextflow](#nextflow) — you are distributing a
combined work, and the bundle as a whole must go out under GPL terms. The
sources here remain MIT for anyone who takes them on their own.
