Metadata-Version: 2.4
Name: particleflow
Version: 3.2.0
Summary: Machine Learning for Particle Flow Reconstruction
Author: Farouk Mokhtar, Eric Wulff, Javier Duarte
Author-email: Joosep Pata <joosep.pata@cern.ch>
License: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Physics
Requires-Python: ==3.11.*
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: array-record~=0.8.3
Requires-Dist: autopep8
Requires-Dist: awkward
Requires-Dist: boost_histogram
Requires-Dist: click
Requires-Dist: comet-ml
Requires-Dist: fastjet
Requires-Dist: fsspec
Requires-Dist: importlib-resources
Requires-Dist: jupyter
Requires-Dist: jupyter-book>=2.1.6
Requires-Dist: keras
Requires-Dist: keras-tuner
Requires-Dist: matplotlib
Requires-Dist: mlcroissant
Requires-Dist: mplhep
Requires-Dist: networkx
Requires-Dist: notebook
Requires-Dist: numba
Requires-Dist: numpy
Requires-Dist: onnx
Requires-Dist: onnxscript
Requires-Dist: pandas
Requires-Dist: papermill
Requires-Dist: plotly
Requires-Dist: pre-commit
Requires-Dist: protobuf~=5.28.3
Requires-Dist: psutil
Requires-Dist: pyarrow~=18.1.0; sys_platform == "linux"
Requires-Dist: pyarrow~=22.0.0; sys_platform == "darwin"
Requires-Dist: pydantic
Requires-Dist: pytest
Requires-Dist: ray[tune]
Requires-Dist: scikit-learn
Requires-Dist: scikit-optimize
Requires-Dist: scipy
Requires-Dist: seaborn
Requires-Dist: setGPU
Requires-Dist: snakemake
Requires-Dist: tabulate
Requires-Dist: tensorboard
Requires-Dist: tensorflow~=2.20.0
Requires-Dist: tensorflow-datasets~=4.9.9
Requires-Dist: tensorflow-metadata~=1.17.3
Requires-Dist: torch==2.11.0
Requires-Dist: torchvision
Requires-Dist: torch_runstats
Requires-Dist: torch-scatter
Requires-Dist: tqdm
Requires-Dist: uproot
Requires-Dist: umap-learn
Requires-Dist: vector
Requires-Dist: zenodo_get
Requires-Dist: huggingface_hub
Requires-Dist: apache_beam~=2.71.0; sys_platform == "linux"
Requires-Dist: spconv-cu124; sys_platform == "linux"
Requires-Dist: addict
Requires-Dist: timm
Requires-Dist: colorhash
Requires-Dist: flash-attn; sys_platform == "linux"
Requires-Dist: colliderml>=0.4.2
Dynamic: license-file

### **Summary**

**ML-based particle flow (MLPF)** focuses on developing full event reconstruction for particle detectors using computationally scalable and flexible machine learning models. The project aims to improve particle flow reconstruction across various detector environments, including CMS, as well as future detectors via Key4HEP.
We build on existing, open-source simulation software by the experimental collaborations.

**Documentation: [jpata.github.io/particleflow](https://jpata.github.io/particleflow/)**

Read the documentation for the physics overview, installation and workflow guides, current capabilities, publications, and development priorities. The Markdown source is in [`docs/`](docs/index.md).

<p float="left">
  <img src="images/diagram.svg" alt="High-level overview" width="800"/>
</p>

---

### **TLDR; I just want to run the code**
You can use `uv` to set up the repo and test that everything works:
```
git clone --recurse-submodules https://github.com/jpata/particleflow.git
uv sync --project envs/ort-gpu
uv run ./scripts/local_test_cld.sh
uv run ./scripts/local_test_cms.sh
```

Before opening a pull request, run the lightweight quality checks with
`uvx --python 3.12 pre-commit run --all-files`. The cyclomatic-complexity check covers the
first-party `mlpf/` package (not the vendored detector configuration under
`mlpf/data/key4hep/gen/`) and rejects new functions above the transitional
complexity ceiling. Prefer small, single-purpose helpers when generating or
refactoring code; a local `# noqa: C901` should be a reviewed exception, not an
automatic workaround.

To run the same all-files check before every local commit, enable the repository
hook with `git config core.hooksPath .githooks`. The hook uses the same pre-commit
flags and Python version as GitHub Actions. It requires `uv`, which installs the
lint tool and Python 3.12 if needed.

Alternatively, you can use a prepared container:
```
apptainer exec --nv https://jpata.web.cern.ch/jpata/pytorch-20260305-08d6950.sif ./scripts/local_test_cld.sh
apptainer exec --nv https://jpata.web.cern.ch/jpata/pytorch-20260305-08d6950.sif ./scripts/local_test_cms.sh
```



### **Datasets**

See the user documentation to [choose a dataset](https://jpata.github.io/particleflow/datasets/catalog), [download and verify published TFDS data](https://jpata.github.io/particleflow/datasets/download), or [produce and publish a dataset](https://jpata.github.io/particleflow/datasets/generate).

### **Training**

The [training guide](https://jpata.github.io/particleflow/training/train) provides a verified CPU check, a bounded GPU example, expected outputs, and checkpoint continuation.

For CLD hit-based reconstruction, the recommended model is `pyg-cld-hits-heptv2-set-v1`
(grouped HEPTv2 with tracker-neighborhood features), using the
[`cld_pf_heptv2_default_50k` training scenario](configs/training/scenarios/cld_pf_heptv2_default_50k.yaml).
See the [scenario guide](configs/training/README.md) for launch commands.

### **Model Upload**

To upload a trained model to the Hugging Face Hub:
```bash
uv run python3 scripts/upload_model_hf.py experiments/pyg-clic-hits-v1_clic_20260328_144021_479374 --version v3.1.0
```

### **Model Download & Evaluation**

The [Key4HEP evaluation guide](https://jpata.github.io/particleflow/validation/key4hep) downloads a published checkpoint and runs it on an example EDM4hep ROOT file. The [validation overview](https://jpata.github.io/particleflow/validation/overview) explains which checks are needed before interpreting physics performance.

## **End-to-end workflow**

The [dataset-generation guide](https://jpata.github.io/particleflow/datasets/generate) documents the Pixi/Snakemake pipeline from detector simulation through validated TFDS output. Training, evaluation, physics validation, and ONNX validation are covered in the [user documentation](https://jpata.github.io/particleflow/).

---

### **Publications**

The following publications trace the development of MLPF from early proofs of concept to full detector simulations and fine-tuning studies across detectors.

* [2021] First full-event GNN demonstration of MLPF: [Paper](https://doi.org/10.1140/epjc/s10052-021-09158-w) [Code](https://zenodo.org/records/4559587) [Dataset](https://doi.org/10.5281/zenodo.4559324)
* [2021] First demonstration in CMS Run 3: [Paper](http://dx.doi.org/10.1088/1742-6596/2438/1/012100) [CMS-DP](https://cds.cern.ch/record/2792320)
* [2022] Improved performance in CMS Run 3: [CMS-DP](http://cds.cern.ch/record/2842375)
* [2024] Improved performance with full simulation for future colliders: [Paper](https://doi.org/10.1038/s42005-024-01599-5) [Code](https://zenodo.org/records/10928968) [Results](https://doi.org/10.5281/zenodo.10567397)
* [2025] Fine-tuning across detectors: [Paper](https://doi.org/10.1103/PhysRevD.111.092015) [Code](https://zenodo.org/records/14930299)
* [2026] CMS Run 3 full results: [Paper](https://arxiv.org/abs/2601.17554) [CMS-DP](https://cds.cern.ch/record/2937578) [Code](https://zenodo.org/records/15573658)

---

### **Citations and Reuse**

You are welcome to reuse the code in accordance with the [LICENSE](https://github.com/jpata/particleflow/blob/main/LICENSE).

**How to Cite**

1. **Academic Work:** Please cite the specific papers listed in the **Publications** section above relevant to the method you are using (e.g., initial GNN idea, fine-tuning, or specific detector studies).
2. **Code Usage:** If you use the code significantly for research, please cite the specific [tagged version from Zenodo](https://zenodo.org/search?q=parent.id%3A4452541&f=allversions%3Atrue&l=list&p=1&s=10&sort=version).
3. **Dataset Usage:** Cite the [appropriate dataset](https://zenodo.org/search?q=mlpf&f=allversions%3Atrue&f=resource_type%3Adataset&l=list&p=1&s=10&sort=version) via the Zenodo link and the corresponding paper.

**Contact**

For other collaboration ideas, please [get in touch via GitHub Discussions](https://github.com/jpata/particleflow/discussions/categories/general).
