Metadata-Version: 2.3
Name: aplusml
Version: 0.3.1
Summary: Conduct usefulness simulations of ML models embedded in workflows
License: Apache 2.0
Author: Michael Wornow
Requires-Python: >=3.10
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Provides-Extra: dev
Requires-Dist: contourpy (==1.0.7)
Requires-Dist: cycler (==0.11.0)
Requires-Dist: dill (==0.3.6)
Requires-Dist: fast-histogram (==0.11)
Requires-Dist: fonttools (==4.38.0)
Requires-Dist: joblib (==1.2.0)
Requires-Dist: jupyter (==1.0.0)
Requires-Dist: jupyterlab (==3.5.2)
Requires-Dist: kiwisolver (==1.4.4)
Requires-Dist: matplotlib (==3.8.0)
Requires-Dist: mizani (==0.13.0)
Requires-Dist: mpire (==2.6.0)
Requires-Dist: mpl-scatter-density (==0.7)
Requires-Dist: multiprocess (==0.70.14)
Requires-Dist: numpy (<2.0)
Requires-Dist: oyaml (==1.0)
Requires-Dist: packaging (==25.0)
Requires-Dist: palettable (==3.3.0)
Requires-Dist: pandas (==2.2.0)
Requires-Dist: patsy (==0.5.3)
Requires-Dist: pillow (==9.4.0)
Requires-Dist: plotnine (==0.14.5)
Requires-Dist: pydot (==1.4.2)
Requires-Dist: pygments (==2.17)
Requires-Dist: pyparsing (==3.0.9)
Requires-Dist: python-dateutil (==2.8.2)
Requires-Dist: pytz (==2022.7.1)
Requires-Dist: pyyaml (==6.0)
Requires-Dist: ruamel-yaml (==0.17.21)
Requires-Dist: ruamel-yaml-clib (==0.2.7)
Requires-Dist: scikit-learn (==1.2.0)
Requires-Dist: seaborn (==0.12.2)
Requires-Dist: simpy (==4.0.1)
Requires-Dist: six (==1.16.0)
Requires-Dist: sphinx (==8.1.3) ; extra == "dev"
Requires-Dist: sphinx-rtd-theme (==3.0.2) ; extra == "dev"
Requires-Dist: statsmodels (==0.14)
Requires-Dist: threadpoolctl (==3.1.0)
Requires-Dist: tqdm (==4.64.1)
Project-URL: Bug Tracker, https://github.com/som-shahlab/aplusml/issues
Project-URL: Homepage, https://github.com/som-shahlab/aplusml
Description-Content-Type: text/markdown

<div align="center">
  <h1>APLUS ML</h1>
  <div>
    <strong>A</strong> <strong>P</strong>ython <strong>L</strong>ibrary for <strong>U</strong>sefulness <strong>S</strong>imulations of <strong>M</strong>achine <strong>L</strong>earning Models
  </div>
  <div>
    <h4>
      <a href="https://aplusml.readthedocs.io/en/latest/" target="_blank">⚙ Documentation</a> • <a href="https://pypi.org/project/aplusml/" target="_blank">🐍 PyPi</a> • <a href="https://www.sciencedirect.com/science/article/pii/S1532046423000400?via%3Dihub" target="_blank">📝 Paper</a>
    </h4>
    <h4>
      <code>pip install aplusml</code>
    </h4>
  </div>
  <img src="https://github.com/som-shahlab/aplusml/blob/bc3fba49257092f038a862e870b4cae86f9eb659/assets/graphical%20abstract.png" height="400" />
</div>

## Installation

1. Run the following commands to install **APLUS ML**:

```bash
pip install aplusml
```

2. Install **graphviz** by [downloading it here](https://graphviz.org/download/). If you're on Mac with `homebrew`, simply run:
```
brew install graphviz
```

## Usage

Please see the <a href="https://aplusml.readthedocs.io/en/latest/">documentation</a> for usage examples.

Run `tutorials/synthetic_pad.ipynb` to try an example notebook which works out-of-the-box.

This simulates a utility analysis of PAD referral pathways for synthetic PAD patients.

## Motivation

APLUS ML is a simulation framework for conducting usefulness assessments of machine learning models in workflows.

It aims to quantitatively answer the question: *If I use this ML model within this workflow, will the benefits outweigh the costs, and by how much?*

APLUS was originally developed for clinical workflows in healthcare settings, thus all of our examples are healthcare workflows. However, APLUS ML is a broadly applicable library to any workflow that involves a machine learning model making decisions on a stream of datapoints, and we encourage contributors from any domain to use and extend APLUS ML.

## Tutorials

We showcase APLUS on two clinical workflows: 

1. Early detection of peripheral artery disease (PAD)
2. Triaging patients for advanced care planning (ACP) consults

Jupyter notebooks for these use cases can be found in the `tutorials/` folder.

### Early Detection of PAD

The code used to generate the figures in our paper is located in the `tutorials/` directory in `pad.ipynb`. This notebook loads de-identified patient data from Stanford Hospital, which can be provided upon request.

The workflows analyzed can be found in the `workflows/` folder. The doctor-driven workflow is in `pad_doctor.yaml` while the nurse-driven workflow is in `pad_nurse.yaml`

This `tutorials/pad.ipynb` was used to generate the following figures from the APLUS paper:

![PAD Figure 1](assets/pad%20figure%201.png)

![PAD Figure 2](assets/pad%20heatmap.png)

### Triaging Patients for ACP Consults

The code used to replicate the findings of [Jung et al. 2021](https://pubmed.ncbi.nlm.nih.gov/33355350/) can be found in the `tutorials/` directory in `acp_jung_replication.ipynb`. This notebook loads de-identified patient data from Stanford Hospital, which can be provided upon request.

The workflows analyzed can be found in the `workflows/` folder in `acp_jung_replication.yaml`

![ACP Figure](assets/acp%20figure.png)

## Plot Gallery

Some additional example plots that can be generated by APLUS are included below:

![Additional Plots](assets/additional%20plots.png)

## Development

### Installation

```bash
# Download repo
git clone https://github.com/som-shahlab/aplus.git
cd aplus

# Create environment
conda create -n aplus python=3.10 -y
conda activate aplus
pip install poetry && poetry install
```

### Tests

The file `tests/run_tests.py` runs all of the `test[d].py` files in the `tests/` directory. Each `test[d].py` file has a corresponding `test[d].yaml` file that serves as its input.

To run tests:
```
cd tests
python3 run_tests.py
```

### Documentation

We use [Sphinx](https://www.sphinx-doc.org/en/master/) to build the documentation, and host it on [Read the Docs](https://readthedocs.org/).

To build the docs, run:

```bash
# View server
sphinx-autobuild docs/source docs/build/html

# Build for dist
make html
```

### Deploy to PyPi

```bash
poetry build
poetry publish
```


## Citation

```
@article{wornow2023aplus,
  title={APLUS: A Python Library for Usefulness Simulations of Machine Learning Models in Healthcare},
  author={Wornow, Michael and Ross, Elsie Gyang and Callahan, Alison and Shah, Nigam H},
  journal={Journal of Biomedical Informatics},
  pages={104319},
  year={2023},
  publisher={Elsevier}
}
```

