Metadata-Version: 2.3
Name: mlcompare
Version: 1.2.0
Summary: Quickly compare machine learning models across libraries and datasets.
Project-URL: Documentation, https://mlcompare.readthedocs.io/en/latest/api_reference
Project-URL: Changelog, https://mlcompare.readthedocs.io/en/latest/release_notes
Project-URL: Repository, https://github.com/MitchMedeiros/MLCompare
Author: Mitchell Medeiros
License-Expression: MIT
License-File: LICENSE.txt
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: Unix
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Topic :: Scientific/Engineering
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: huggingface-hub>=0.21.0
Requires-Dist: kaggle>=1.0.0
Requires-Dist: lightgbm>=4.0.0
Requires-Dist: numpy<2.0.0,>=1.23.5
Requires-Dist: openml>=0.13.0
Requires-Dist: pandas[pyarrow]>=2.0.0
Requires-Dist: plotly>=4.0.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: scikit-learn>=1.4.0
Requires-Dist: torch>=2.0.0
Requires-Dist: xgboost>=2.1.0
Provides-Extra: dev
Requires-Dist: black>=24.0.0; extra == 'dev'
Requires-Dist: ipykernel>=6.0.0; extra == 'dev'
Requires-Dist: pytest-cov>=5.0.0; extra == 'dev'
Requires-Dist: pytest>=8.0.0; extra == 'dev'
Requires-Dist: ruff>=0.5.0; extra == 'dev'
Provides-Extra: docs
Requires-Dist: matplotlib>=3.9.1; extra == 'docs'
Requires-Dist: pydata-sphinx-theme==0.15.4; extra == 'docs'
Requires-Dist: python-dotenv==1.0.1; extra == 'docs'
Requires-Dist: sphinx-autodoc-typehints==2.2.3; extra == 'docs'
Requires-Dist: sphinx-copybutton==0.5.2; extra == 'docs'
Requires-Dist: sphinx-design==0.6.0; extra == 'docs'
Requires-Dist: sphinx-docsearch==0.0.7; extra == 'docs'
Requires-Dist: sphinx-sitemap==2.6.0; extra == 'docs'
Requires-Dist: sphinx==7.3.7; extra == 'docs'
Requires-Dist: sphinxext-opengraph==0.9.1; extra == 'docs'
Description-Content-Type: text/markdown

<p align="center">
    <img src="https://d1nheu3uhuz51e.cloudfront.net/mlcompare/logo_text_1k.png" width="500" alt="MLCompare Logo">
</p>


<div align="center">
<a href="https://pypi.org/project/mlcompare"><img 
    alt="Supported Python Versions"
    src="https://img.shields.io/pypi/pyversions/mlcompare?logo=data%3Aimage%2Fsvg%2Bxml%3Bbase64%2CPHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIxLjAxZW0iIGhlaWdodD0iMWVtIiB2aWV3Qm94PSIwIDAgMjU2IDI1NSI%2BPGRlZnM%2BPGxpbmVhckdyYWRpZW50IGlkPSJsb2dvc1B5dGhvbjAiIHgxPSIxMi45NTklIiB4Mj0iNzkuNjM5JSIgeTE9IjEyLjAzOSUiIHkyPSI3OC4yMDElIj48c3RvcCBvZmZzZXQ9IjAlIiBzdG9wLWNvbG9yPSIjMzg3ZWI4Ii8%2BPHN0b3Agb2Zmc2V0PSIxMDAlIiBzdG9wLWNvbG9yPSIjMzY2OTk0Ii8%2BPC9saW5lYXJHcmFkaWVudD48bGluZWFyR3JhZGllbnQgaWQ9ImxvZ29zUHl0aG9uMSIgeDE9IjE5LjEyOCUiIHgyPSI5MC43NDIlIiB5MT0iMjAuNTc5JSIgeTI9Ijg4LjQyOSUiPjxzdG9wIG9mZnNldD0iMCUiIHN0b3AtY29sb3I9IiNmZmUwNTIiLz48c3RvcCBvZmZzZXQ9IjEwMCUiIHN0b3AtY29sb3I9IiNmZmMzMzEiLz48L2xpbmVhckdyYWRpZW50PjwvZGVmcz48cGF0aCBmaWxsPSJ1cmwoI2xvZ29zUHl0aG9uMCkiIGQ9Ik0xMjYuOTE2LjA3MmMtNjQuODMyIDAtNjAuNzg0IDI4LjExNS02MC43ODQgMjguMTE1bC4wNzIgMjkuMTI4aDYxLjg2OHY4Ljc0NUg0MS42MzFTLjE0NSA2MS4zNTUuMTQ1IDEyNi43N2MwIDY1LjQxNyAzNi4yMSA2My4wOTcgMzYuMjEgNjMuMDk3aDIxLjYxdi0zMC4zNTZzLTEuMTY1LTM2LjIxIDM1LjYzMi0zNi4yMWg2MS4zNjJzMzQuNDc1LjU1NyAzNC40NzUtMzMuMzE5VjMzLjk3UzE5NC42Ny4wNzIgMTI2LjkxNi4wNzJNOTIuODAyIDE5LjY2YTExLjEyIDExLjEyIDAgMCAxIDExLjEzIDExLjEzYTExLjEyIDExLjEyIDAgMCAxLTExLjEzIDExLjEzYTExLjEyIDExLjEyIDAgMCAxLTExLjEzLTExLjEzYTExLjEyIDExLjEyIDAgMCAxIDExLjEzLTExLjEzIi8%2BPHBhdGggZmlsbD0idXJsKCNsb2dvc1B5dGhvbjEpIiBkPSJNMTI4Ljc1NyAyNTQuMTI2YzY0LjgzMiAwIDYwLjc4NC0yOC4xMTUgNjAuNzg0LTI4LjExNWwtLjA3Mi0yOS4xMjdIMTI3LjZ2LTguNzQ1aDg2LjQ0MXM0MS40ODYgNC43MDUgNDEuNDg2LTYwLjcxMmMwLTY1LjQxNi0zNi4yMS02My4wOTYtMzYuMjEtNjMuMDk2aC0yMS42MXYzMC4zNTVzMS4xNjUgMzYuMjEtMzUuNjMyIDM2LjIxaC02MS4zNjJzLTM0LjQ3NS0uNTU3LTM0LjQ3NSAzMy4zMnY1Ni4wMTNzLTUuMjM1IDMzLjg5NyA2Mi41MTggMzMuODk3bTM0LjExNC0xOS41ODZhMTEuMTIgMTEuMTIgMCAwIDEtMTEuMTMtMTEuMTNhMTEuMTIgMTEuMTIgMCAwIDEgMTEuMTMtMTEuMTMxYTExLjEyIDExLjEyIDAgMCAxIDExLjEzIDExLjEzYTExLjEyIDExLjEyIDAgMCAxLTExLjEzIDExLjEzIi8%2BPC9zdmc%2B&labelColor=blue&color=yellow&label=Python"></a>
<a href="https://pypi.org/project/mlcompare/"><img 
    alt="PyPI - Version"
    src="https://img.shields.io/pypi/v/mlcompare?logo=pypi&label=PyPi&labelColor=white&color=blue"></a>
<a href="https://pypi.org/project/mlcompare/"><img 
    alt="PyPI - License"
    src="https://img.shields.io/pypi/l/mlcompare?logo=data%3Aimage%2Fsvg%2Bxml%3Bbase64%2CPHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIxZW0iIGhlaWdodD0iMWVtIiB2aWV3Qm94PSIwIDAgMTYgMTYiPjxwYXRoIGZpbGw9Im5vbmUiIHN0cm9rZT0iIzk3OTc5NyIgc3Ryb2tlLWxpbmVjYXA9InJvdW5kIiBzdHJva2UtbGluZWpvaW49InJvdW5kIiBkPSJNNC41IDEzLjVoN004LjAxIDF2MTIuMDZNMS41IDMuNWgzbDEuNS0xaDRsMS41IDFoM00uNSAxMEwzIDQuNDhMNS41IDEwQzQgMTEgMiAxMSAuNSAxMG0xMCAwTDEzIDQuNDhMMTUuNSAxMGMtMS41IDEtMy41IDEtNSAwIi8%2BPC9zdmc%2B&labelColor=darkred&color=lightgrey&label=License"></a>
<a href="https://www.pepy.tech/projects/MLCompare"><img 
    alt="Pepy Total Downlods"
    src="https://img.shields.io/pepy/dt/MLCompare?logo=data%3Aimage%2Fsvg%2Bxml%3Bbase64%2CPHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIyNTYiIGhlaWdodD0iMjU2IiB2aWV3Qm94PSIwIDAgNTAgNTAiPjxnIGZpbGw9Im5vbmUiIHN0cm9rZS1saW5lY2FwPSJyb3VuZCIgc3Ryb2tlLWxpbmVqb2luPSJyb3VuZCIgc3Ryb2tlLXdpZHRoPSIyIj48cGF0aCBzdHJva2U9IiMxQUZGOUYiIGQ9Ik0zMS4yNSAyOS4xNjdMMjUgMzUuNDE3bC02LjI1LTYuMjVtNi4yNSA2LjI1VjYuMjUiLz48cGF0aCBzdHJva2U9IiMxQUZGOUYiIGQ9Ik04LjMzMyAzNS40MTd2Ni4yNWEyLjA4MyAyLjA4MyAwIDAgMCAyLjA4NCAyLjA4M2gyOS4xNjZhMi4wODMgMi4wODMgMCAwIDAgMi4wODQtMi4wODN2LTYuMjUiLz48L2c%2BPC9zdmc%2B&label=Downloads&labelColor=grey&color=lightgreen"></a>
<br>
<a href="https://mlcompare.readthedocs.io/en/latest/api_reference"><img
    alt="Read the Docs"
    src="https://img.shields.io/readthedocs/mlcompare?logo=data%3Aimage%2Fsvg%2Bxml%3Bbase64%2CPHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIyNTYiIGhlaWdodD0iMjU2IiB2aWV3Qm94PSIwIDAgNDggNDgiPjxwYXRoIGZpbGw9IiM5MGNhZjkiIGQ9Ik00MCA0NUg4VjNoMjJsMTAgMTB6Ii8%2BPHBhdGggZmlsbD0iI2UxZjVmZSIgZD0iTTM4LjUgMTRIMjlWNC41eiIvPjxwYXRoIGZpbGw9IiMxOTc2ZDIiIGQ9Ik0xNiAyMWgxN3YySDE2em0wIDRoMTN2MkgxNnptMCA0aDE3djJIMTZ6bTAgNGgxM3YySDE2eiIvPjwvc3ZnPg%3D%3D&labelColor=white&label=Docs"></a>
<a href="https://github.com/MitchMedeiros/MLCompare/actions/workflows/lint.yml"><img
    alt="GitHub Actions Workflow Status"
    src="https://img.shields.io/github/actions/workflow/status/MitchMedeiros/MLCompare/lint.yml?logo=data%3Aimage%2Fsvg%2Bxml%3Bbase64%2CPHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIyNTYiIGhlaWdodD0iMjU2IiB2aWV3Qm94PSIwIDAgMjQgMjQiPjxwYXRoIGZpbGw9IiNlMzI0MDAiIGQ9Im0xMS4wNzUgMTQuOTVsMS44NS0yLjUyNWwxLjgyNSAyLjUyNUwxOCAxMC41MjVMMTYuNCA5LjM1bC0xLjY1IDIuMjI1TDEyLjkgOS4wNWwtMS44MjUgMi41MjVsLTEuODUtMi41MjVMNiAxMy40NzVsMS42IDEuMTc1bDEuNjI1LTIuMjI1ek0xMiAyMnEtMi4wNzUgMC0zLjktLjc4OHQtMy4xNzUtMi4xMzdUMi43ODggMTUuOVQyIDEydC43ODgtMy45dDIuMTM3LTMuMTc1VDguMSAyLjc4OFQxMiAydDMuOS43ODh0My4xNzUgMi4xMzdUMjEuMjEzIDguMVQyMiAxMnQtLjc4OCAzLjl0LTIuMTM3IDMuMTc1dC0zLjE3NSAyLjEzOFQxMiAyMm0wLTJxMy4zNSAwIDUuNjc1LTIuMzI1VDIwIDEydC0yLjMyNS01LjY3NVQxMiA0VDYuMzI1IDYuMzI1VDQgMTJ0Mi4zMjUgNS42NzVUMTIgMjBtMC04Ii8%2BPC9zdmc%2B&label=Lint&labelColor=lightgreen"></a>
<a href="https://github.com/MitchMedeiros/MLCompare/actions/workflows/unit-tests.yml"><img
    alt="GitHub Actions status (MacOS Unit Tests)"
    src="https://img.shields.io/github/actions/workflow/status/MitchMedeiros/MLCompare/unit-tests.yml?logo=data%3Aimage%2Fsvg%2Bxml%3Bbase64%2CPHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIxZW0iIGhlaWdodD0iMWVtIiB2aWV3Qm94PSIwIDAgMTI4IDEyOCI%2BPHBhdGggZmlsbD0iIzY5Njk2OSIgZD0iTTMxLjUxMiAzMC4zOThoNjEuMzA0YTMuMDA2IDMuMDA2IDAgMCAxIDAgNi4wMTJIMzEuNTEyYTMuMDA3IDMuMDA3IDAgMCAxLTMuMDA0LTMuMDA0YTMuMDEgMy4wMSAwIDAgMSAzLjAwNC0zLjAwOG0wIDAiLz48cGF0aCBmaWxsPSIjMDA5ZmUzIiBkPSJNMzIuMDQ3IDI0LjMySDQ0LjM3djIuODQ0SDMyLjA0N3ptMCAwIi8%2BPHBhdGggZmlsbD0iI2M3ZDMwMiIgZD0iTTQ4LjE2OCAyNC4zMmgxMi4zMjR2Mi44NDRINDguMTY4em0wIDAiLz48cGF0aCBmaWxsPSIjZjA3ZTE2IiBkPSJNNjQuMDcgMjQuMzJoMTIuMzI4djIuODQ0SDY0LjA3em0wIDAiLz48cGF0aCBmaWxsPSIjZGYyODE1IiBkPSJNNzkuOTEgMjQuMzJoMTIuMzI0djIuODQ0SDc5Ljkxem0wIDE1LjIyaDEyLjMyNHYyMC44MzVINzkuOTF6bTAgMCIvPjxwYXRoIGZpbGw9IiNmMDdlMTYiIGQ9Ik02NC4wNyAzOS41NGgxMi4zNTJ2MzMuODQ3SDY0LjA3em0wIDAiLz48cGF0aCBmaWxsPSIjYzdkMzAyIiBkPSJNNDguMTY4IDM5LjU0aDEyLjMyNHY1MC42OThINDguMTY4em0wIDAiLz48cGF0aCBmaWxsPSIjMDA5ZmUzIiBkPSJNMzIuMDQ3IDM5LjU0SDQ0LjM3djYxLjc5MkgzMi4wNDd6bTAgMCIvPjwvc3ZnPg%3D%3D&label=Unit%20Tests&labelColor=darkgreen"></a>
<a href="https://app.codecov.io/gh/MitchMedeiros/MLCompare"><img
    alt="Code Coverage"
    src="https://img.shields.io/codecov/c/github/MitchMedeiros/MLCompare?logo=codecov&labelColor=black&label=Coverage"></a>
</div>

<br>

MLCompare is a Python package for running model comparison pipelines, with the aim of being both simple and flexible. It supports multiple popular ML libraries, retrieval from multiple online dataset repositories, common data processing steps, and results visualization. Additionally, it allows for using your own models and datasets within the pipelines.

<table align="center">
    <tr>
        <th><div align="center">Libraries</div></th>
        <th><div align="center">Datasets</div></th>
        <th><div align="center">Data Processing</div></th>
    </tr>
    <tr>
        <td valign="top">
            <ul>
                <li>Scikit-learn</li>
                <li>XGBoost</li>
            </ul>
        </td>
        <td valign="top">
            <ul>
                <li>Kaggle</li>
                <li>OpenML</li>
                <li>Hugging Face</li>
                <li>locally saved</li>
            </ul>
        </td>
        <td valign="top">
            <ul>
                <li>train-test split</li>
                <li>drop columns</li>
                <li>handle NaNs: <i>drop</i> | <i>forward-fill</i> | <i>backward-fill</i></li>
                <li>encoders: <i>OneHot</i> | <i>Ordinal</i> | <i>Target</i> | <i>Label</i></li>
                <li>scalers: <i>Standard</i> | <i>MinMax</i> | <i>MaxAbs</i> | <i>Robust</i></li>
                <li>transformers: <i>Quantile</i> | <i>Power</i> | <i>Normalizer</i></li>
            </ul>
        </td>
    </tr>
</table>

<h2>Installing</h2>

It is recommended to create a new virtual environment. Example with Conda:

```console
conda create -n compare_env python==3.11.9
conda activate compare_env
```

Install this library with pip:

```console
pip install mlcompare
```

Note that for MacOS, both XGBoost and LightGBM require `libomp`. It can be installed with <a href="https://brew.sh">Homebrew</a>:

```console
brew install libomp
```

<h2>A Simple Example</h2>

Running a pipeline with multiple datasets and models is done by creating a list of dictionaries for each and providing them to a pipeline function.

The below example downloads a dataset from OpenML and Kaggle, one-hot encodes some of the columns in the Kaggle dataset, and trains and evaluates a Random Forest and XGBoost model on them.

```python
import mlcompare

datasets = [
    {
        "type": "openml",
        "id": 8,
        "target": "drinks",
    },
    {
        "type": "kaggle",
        "user": "gorororororo23",
        "dataset": "plant-growth-data-classification",
        "file": "plant_growth_data.csv",
        "target": "Growth_Milestone",
        "oneHotEncode": ["Soil_Type", "Water_Frequency", "Fertilizer_Type"],
    }
]

models = [
    {
        "library": "sklearn",
        "name": "RandomForestRegressor",
    },
    {
        "library": "xgboost",
        "name": "XGBRegressor",
        "params": {"num_leaves": 40, "n_estimators": 200}
    }
]

mlcompare.full_pipeline(datasets, models, "regression")
```

In the case of the XGBoost model some non-default parameter values were used.

<h2>Planned Additions</h2>

<h3>Version 1.3</h3>
<ul>
    <li>LightGBM support</li>
    <li>CatBoost support</li>
    <li>Model results graphing and visualization</li>
    <li>Improved documentation</li>
    <li>Support for presplit data</li>
</ul>

<h3>Version 1.4</h3>
<ul>
    <li>PyTorch support</li>
    <li>TensorFlow support</li>
    <li>Additional dataset sources</li>
    <li>Built-in model and dataset collections for quick testing of similar model types/datasets</li>
    <li>Optional pipeline caching</li>
    <li>Optional trained model saving</li>
</ul>

<h3>Version 1.5</h3>
<ul>
    <li>S3 Support</li>
</ul>
