Metadata-Version: 2.5
Name: probixi
Version: 0.8.3
Summary: Self-Calibrating Probabilistic Peak Finding for Serial X-Ray Crystallographic Data
Project-URL: Homepage, https://github.com/ryan-odea/probixi
Project-URL: Repository, https://github.com/ryan-odea/probixi
Project-URL: Bug Tracker, https://github.com/ryan-odea/probixi/issues
Project-URL: Ryan O'Dea (ORCID), https://orcid.org/0009-0000-0103-9546
Author-email: Ryan O'Dea <ryan.odea@psi.ch>, Michal Kepa <michal.kepa@epfl.ch>, Tobias Weinert <tobias.weinert@psi.ch>
Maintainer-email: Ryan O'Dea <ryan.odea@psi.ch>
License: MIT
License-File: LICENSE
Keywords: peak finding,probabilistic,x-ray crystallography
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: 3.15
Requires-Python: >=3.10
Requires-Dist: click
Requires-Dist: deflate
Requires-Dist: duckdb
Requires-Dist: h5py
Requires-Dist: hdf5plugin
Requires-Dist: matplotlib
Requires-Dist: numpy
Requires-Dist: pillow
Requires-Dist: torch
Provides-Extra: dev
Requires-Dist: black; extra == 'dev'
Requires-Dist: isort; extra == 'dev'
Requires-Dist: myst-parser; extra == 'dev'
Requires-Dist: piccolo-theme; extra == 'dev'
Requires-Dist: pytest; extra == 'dev'
Requires-Dist: ruff; extra == 'dev'
Requires-Dist: sphinx; extra == 'dev'
Requires-Dist: sphinx-autodoc-typehints; extra == 'dev'
Requires-Dist: sphinx-copybutton; extra == 'dev'
Provides-Extra: docs
Requires-Dist: myst-parser; extra == 'docs'
Requires-Dist: piccolo-theme; extra == 'docs'
Requires-Dist: sphinx; extra == 'docs'
Requires-Dist: sphinx-autodoc-typehints; extra == 'docs'
Requires-Dist: sphinx-copybutton; extra == 'docs'
Provides-Extra: gpu
Requires-Dist: nvidia-libnvcomp-cu12==5.3.0.16; extra == 'gpu'
Requires-Dist: triton>=3.5; extra == 'gpu'
Description-Content-Type: text/markdown

# probixi - Self-Calibrating (PROB)ab(I)listic Peak Detection for Serial (X)-Ray Crystallograph(I)c Data <a href="https://github.com/ryan-odea/probixi">
<img src="https://raw.githubusercontent.com/ryan-odea/Probixi/main/docs/assets/sticker.png" align="right" height="138" alt="Probixi logo with link to website." /></a>
[![Lifecycle:
experimental](https://img.shields.io/badge/lifecycle-experimental-orange.svg)](https://lifecycle.r-lib.org/articles/stages.html#experimental)
[![PyPI version](https://badge.fury.io/py/probixi.svg)](https://pypi.org/project/probixi) 
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/probixi)
[![PyTorch](https://img.shields.io/badge/PyTorch-ee4c2c.svg)](https://pytorch.org/)
[![codecov](https://codecov.io/gh/ryan-odea/Probixi/graph/badge.svg?token=DMOVJJUWXP)](https://codecov.io/gh/ryan-odea/Probixi)
[![CUDA](https://img.shields.io/badge/CUDA-supported-76b900.svg)](https://developer.nvidia.com/cuda-zone)
[![Apple Silicon MPS](https://img.shields.io/badge/Apple%20Silicon-MPS-000000.svg?logo=apple)](https://developer.apple.com/metal/pytorch/)
[![Downloads](https://static.pepy.tech/badge/probixi)](https://pepy.tech/project/probixi)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
[![Documentation Status](https://readthedocs.org/projects/probixi/badge/?version=latest)](https://probixi.readthedocs.io)
[![Code Style](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)

`probixi` treats peakfinding as anomaly detection. Bragg signal is sparse, and the stream of frames is therefore mostly a sample of the detector's background. `probixi` learns that background at every pixel as frames arrive, whitens each frame against it, and reports the pixels the background does not explain. The peak contrast, peak prior, background blend, and detection threshold are calibrated on the data, and no parameter requires tuning. Each frame is then indexed, predicted, and integrated, with every peak weighted by its detection posterior.

![image](https://raw.githubusercontent.com/ryan-odea/Probixi/main/docs/assets/br_fel.gif)

## Installing the Package

You can install via Pypi with pip:

```bash
pip install probixi
```

Or the latest development version with

```bash
pip install git+https://github.com/ryan-odea/probixi.git
```

## Using `probixi`

The command line goes from raw frames to integrated reflections in one call. The Python API returns lazy streams whose results remain on the GPU until collected, and any stage can feed further torch processing.

`probixi` calibrates on a set of seed frames before detection. `--gif` writes an animation of the noise model over those frames.

Via the CLI:

```bash
probixi -i files.lst -g myGeometry.geom -p myCell.cell -o stream.stream --device cuda --gif myNoiseModel.gif
```

Or with python:

```python
import torch

from probixi import Probixi, DataOffloader

pipeline = Probixi(
    list_file="files.lst",
    geometry_file="myGeometry.geom",
    cell_file="myCell.cell",
    device=torch.device("cuda"),
)

pipeline.noise_diagnostics("myNoiseModel.gif", stop=32)
cal = pipeline.calibrate(n_seed=1636)
print(f"kappa={cal.kappa:.2f}  prior_peak={cal.prior_peak:.4f}  "
      f"threshold={pipeline.threshold_calibration.threshold:.2f}")

# Stream every frame through detect -> index -> predict + integrate. The stream
# is lazy and each result stays on the GPU until you touch it, so you can branch
# off any downstream processing with torch
with DataOffloader(
    "stream.stream",
    geometry=pipeline.geometry,
    cell=pipeline.target_cell,
    geometry_file="myGeometry.geom",
    files=pipeline.metadata.files,
) as off:
    for result in pipeline.index_stream(pipeline.frames(), batch_size=8):
        off.write(result)  # or: pipeline.index_stream(...).to_stream(off)
        print(f"frame {result.frame_index}: "
              f"{result.n_indexed}/{result.n_peaks} indexed (rmsd {result.rmsd:.4f})")
```

### DuckDB output

The `.stream` format is convenient for interop with `partialator`, but querying a
run means re-parsing a large text file. `probixi` can instead write a
[DuckDB](https://duckdb.org) database. Please note this may be the default in the future.

```bash
probixi -i files.lst -g myGeometry.geom -p myCell.cell -o run.duckdb --device cuda
```

Or with python:

```python
stream = pipeline.index_stream(pipeline.frames(), batch_size=8)
stream.to_db(
    "run.duckdb",
    geometry=pipeline.geometry,
    cell=pipeline.target_cell,
    geometry_file="myGeometry.geom",
    files=pipeline.metadata.files,
)
```

The database holds run metadata as small tables (`geometry`, `panels`, `cell`) plus:

- **`frames`**, keyed by `frame_id` (a hash of `filename//event`) with additional per-frame information
- **`crystals`**, one row per accepted lattice, with its cell, orientation, and fit statistics
- **`reflections`**, the integrated Miller indices (`h k l`, `I`, `sigma`, `peak`,
  `background`, `fs/ss`, panel, resolution)
- **`peaks`**, the peak-search results per frame


### Resolving indexing ambiguity

When the lattice symmetry is higher than the structure's point group, each crystal can
be indexed in more than one way. `probixi-resolve` clusters the crystals in a database
into the two choices and reindexes the ones on the wrong side. It is a torch/Triton implementation of
CrystFEL's `ambigator`, by Thomas White.

```bash
probixi-resolve -i run.duckdb -y 3 -w 321 -o detwinned.duckdb
```

Without `-o` the database is rewritten in place. Reindexing transforms `reflections`
`h k l` along with the matching `crystals` reciprocal axes and cell parameters, and
records every assignment in an `ambiguity` table. The same thing from python:

```python
from probixi import Ambigator

amb = Ambigator("run.duckdb", symmetry="3", apparent="321")
result = amb.resolve()
print(f"{result.n_reindexed}/{len(result)} crystals reindexed")
amb.reindex(result, output="detwinned.duckdb")
```


### Using `probixi` as only a peakfinder

If you prefer your own indexer, use `probixi` as a peakfinder only, through the CLI's `--peaks-only` flag or the Python API's `peak_stream`. The exported peaks also improve conventional indexers on strong data (see the [vignettes](https://probixi.readthedocs.io)).

Via the CLI:

```bash
probixi -i files.lst -g myGeometry.geom -o peaks.stream --peaks-only --device cuda
```

Or with python:

```python
import torch

from probixi import Probixi, PeakOffloader

pipeline = Probixi(
    list_file="files.lst",
    geometry_file="myGeometry.geom",
    device=torch.device("cuda"),
)

# Calibrate the noise model + detection threshold on the seed frames, as usual.
pipeline.calibrate(n_seed=1636)

peaks = pipeline.peak_stream(pipeline.frames(), estimate_scale=False)
with PeakOffloader(
    "peaks.stream",
    geometry=pipeline.geometry,
    geometry_file="myGeometry.geom",
    files=pipeline.metadata.files,
) as off:
    for result in peaks:
        if len(result):       # skip blanks; export only frames with peaks
            off.write(result)
```

## Dependencies

- python >= 3.9
  - click
  - h5py
  - hdf5plugin
  - numpy
  - torch
  - matplotlib
  - pillow
  - duckdb

## Contributing

There are many different ways to contribute to further development of this tool. If you experience a bug or would like an additional feature, please open up a [ticket](https://github.com/ryan-odea/probixi/issues). 

If you would like to contribute actively by merging code, please open a PR with the following:

1. Code is formatted with `isort`, then `black`, followed by a `ruff --check`. This will initiate on PR, so it might be best to check beforehand.
2. Docstrings are minimally on user-facing functions in [`numpy` style](https://numpydoc.readthedocs.io/en/latest/format.html). 
3. Comments, or some explanation (in PR) for the additions, limited to the scope of the project. For a bug fix, the explanation belongs in the PR rather than in the code.
