Metadata-Version: 2.5
Name: melite
Version: 0.2.5
Summary: Tabular classification benchmarking toolkit for classifier selection, repeated stratified cross-validation, final model export, and artifact-based inference.
Project-URL: Homepage, https://github.com/NanoBiostructuresRG/melite
Project-URL: Repository, https://github.com/NanoBiostructuresRG/melite
Project-URL: Issues, https://github.com/NanoBiostructuresRG/melite/issues
Project-URL: Documentation, https://nanobiostructuresrg.github.io/melite/
Author-email: "Flavio F. Contreras-Torres" <contreras.flavio@tec.mx>, "Ana C. Murrieta" <ana.murrieta@tec.mx>
License-Expression: LGPL-3.0-or-later
License-File: COPYING
License-File: COPYING.LESSER
License-File: LICENSE
Keywords: benchmarking,classification,classifier selection,cross-validation,csv,inference,machine learning,scikit-learn,tabular data,xgboost
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: GNU Lesser General Public License v3 or later (LGPLv3+)
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Requires-Dist: joblib>=1.2
Requires-Dist: matplotlib>=3.6
Requires-Dist: numpy>=1.23
Requires-Dist: pandas>=1.5
Requires-Dist: scikit-learn>=1.2
Requires-Dist: xgboost>=1.7
Provides-Extra: dev
Requires-Dist: build>=1.0; extra == 'dev'
Requires-Dist: hatchling>=1.18; extra == 'dev'
Requires-Dist: mypy==2.3.1; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: ruff==0.16.4; extra == 'dev'
Requires-Dist: twine>=4.0; extra == 'dev'
Provides-Extra: docs
Requires-Dist: mkdocs-material>=9.5; extra == 'docs'
Requires-Dist: mkdocs>=1.6; extra == 'docs'
Requires-Dist: mkdocstrings[python]>=0.25; extra == 'docs'
Description-Content-Type: text/markdown

# MELITE — Multi-Model Classifier Evaluator

[![License: LGPL v3](https://img.shields.io/badge/License-LGPL_v3-blue.svg)](LICENSE)
[![Version](https://img.shields.io/badge/version-v0.2.5-blue.svg)]()
[![PyPI](https://img.shields.io/pypi/v/melite.svg)](https://pypi.org/project/melite/)
[![Python](https://img.shields.io/pypi/pyversions/melite.svg)](https://pypi.org/project/melite/)
[![CI](https://github.com/NanoBiostructuresRG/melite/actions/workflows/ci.yml/badge.svg)](https://github.com/NanoBiostructuresRG/melite/actions/workflows/ci.yml)
[![Docs](https://img.shields.io/badge/docs-GitHub%20Pages-teal.svg)](https://nanobiostructuresrg.github.io/melite/)


## Description

**MELITE** is a Python package and command-line tool for evaluating and
comparing classifiers on numeric tabular datasets. It separates hyperparameter
tuning from classifier evaluation, preserves the evidence used for selection, and
exports the selected model as a reusable artifact for downstream inference.

MELITE operates at the tabular modeling level. Its learning algorithms consume
numeric feature matrices (`X`) and target labels (`y`), regardless of how those
features were produced. Inputs may therefore originate from fingerprints,
descriptors, dimensionality-reduction methods, clinical variables, experimental
measurements, industrial features, or other numeric representations.

## Purpose

MELITE is designed to make classifier comparison and selection explicit,
reproducible, and auditable. Its workflow separates stages
that are often mixed together in small classification workflows:

- hyperparameter tuning;
- classifier evaluation;
- comparison and selection;
- final fitting on all available data;
- model export and inference.

This separation ensures that, within each outer cross-validation split, the
data used to evaluate a tuned classifier are held out from the hyperparameter
search that produced it, while preserving the evidence needed to understand
how the competing classifiers performed.


## Why Use MELITE?

- **Controlled evaluation.** Hyperparameter tuning is kept separate from the
  evidence used to compare classifiers.
- **Evidence preservation.** Aggregate and fold-level evaluation results are
  retained for every evaluated classifier, not only for the selected one.
- **Explicit selection.** Classifier selection follows a predefined criterion
  based on cross-validation evidence rather than an informal choice after
  training.
- **Domain-agnostic inputs.** MELITE works with numeric tabular data without
  assuming how the features were generated.
- **Reusable artifacts.** After selection, the chosen classifier can be fitted
  on all available data and saved as a model artifact for prediction.
- **CLI and Python interfaces.** MELITE can be used through its command-line
  workflow and through a focused public Python API.



## What MELITE Does

| MELITE does | MELITE does not |
|---|---|
| Evaluate multiple classifiers on prepared numeric `X` and `y`. | Generate domain-specific features or descriptors. |
| Tune supported classifiers within the evaluation design. | Act as a general AutoML framework. |
| Preserve aggregate and fold-level evaluation evidence. | Generate PCA, UMAP, fingerprints, or other feature representations. |
| Select the best active classifier by mean outer-CV F1-macro. | Process raw domain-specific inputs. |
| Fit and export the selected model as a `.pkl` artifact. | Perform automatic feature engineering or feature selection. |
| Run inference from exported model artifacts. | Guarantee a stable 1.0 API yet. |


## Evaluation Contract

For a registered dataset, MELITE follows the contract below:

1. `X` is a two-dimensional numeric feature matrix and `y` provides the target
   labels for the same samples.
2. Each active classifier is evaluated under the configured outer
   cross-validation design.
3. For tunable classifiers, hyperparameter search occurs only within the
   training portion of each outer split.
4. Evaluation evidence is obtained from the held-out folds of repeated
   stratified outer cross-validation.
5. Mean outer-CV F1-macro is used to select the best active classifier for each
   dataset.
6. Aggregate and per-fold evidence are preserved for every evaluated
   classifier.
7. After selection, the chosen classifier is fitted using all available data.
   If it is tunable, MELITE performs a final full-data hyperparameter search to
   determine the exported configuration.
8. `melite export` does not run a second post-selection evaluation. It fits the
   selected classifier on all available data and serializes the final model
   artifact.
9. Smoke mode is intended for fast execution checks, not final classifier selection.


## Installation

### Package Users


Install **MELITE** in a supported Python environment:

```bash
python -m pip install melite
```

Verify the installation:

```bash
melite --version
```

### Contributors and Developers

Clone the repository and install in editable mode with development dependencies:

```bash
git clone https://github.com/NanoBiostructuresRG/melite.git
cd melite
conda create -n melite_env python=3.11
conda activate melite_env
python -m pip install -e ".[dev]"
```

To build the documentation locally, install the `docs` extra as well:

```bash
python -m pip install -e ".[dev,docs]"
mkdocs serve
```

## Quick Start

The bundled example creates a ready-to-run synthetic numeric CSV dataset and
example configuration for a short evaluation workflow, so you can verify that
MELITE is installed and working. Cloning the repository is **not** required.

```bash
melite example
melite run --smoke --config melite_example/config.toml
```

`melite example` creates `./melite_example/` in the current directory.

### Next steps

- [Prepare and use your own datasets](docs/usage.md#data-preparation).
- [Configure MELITE](docs/usage.md#configuration).
- [Run a full evaluation](docs/usage.md#melite-run).
- [Export a model](docs/usage.md#melite-export) and use the
  [Python prediction API](docs/usage.md#python-api).

## Workflow


### CLI Workflow

The command-line interface provides the canonical end-to-end MELITE workflow:

1. Register one or more numeric datasets in a TOML configuration file.
2. Choose the active classifiers.
3. Run `melite run` to generate evaluation evidence and selected results.
4. Inspect `results.csv`, `evaluations.csv`, `evaluation_folds.csv`, and the
   dataset-level F1-macro evidence figures.
5. Run `melite export` for the selected result you want to preserve as a model
   artifact.
6. Use the exported `.pkl` artifact through `melite.predict()` for inference.


### Python Workflow

The Python API is intentionally component-oriented. It exposes configuration,
dataset loading, evaluation-evidence plotting, artifact-based prediction, and
version metadata as public symbols.

MELITE does not expose the full evaluation orchestration as a stable
high-level Python workflow API. For reproducible end-to-end execution, use the
CLI and a version-controlled TOML configuration.


## Supported Classifiers

**MELITE** currently supports four classifier keys:

| Key | Classifier | Active by default |
|---|---|---|
| `svc` | Support Vector Classifier | Yes |
| `rf` | Random Forest | Yes |
| `xgb` | XGBoost | Yes |
| `stack` | Stacking classifier | No |

The default configuration is:

```toml
[classifiers]
active = ["svc", "rf", "xgb"]
```

Add `"stack"` to evaluate Stacking alongside the default classifiers.

Standalone SVC is evaluated as a `StandardScaler` -> `SVC` pipeline, with
probability fitting disabled during standalone evaluation. Exported SVC
artifacts retain probability support for inference. Random Forest and XGBoost
remain unscaled. The opt-in Stacking classifier combines a scaled probabilistic SVC
with Random Forest and XGBoost base estimators and uses logistic regression as
the final estimator.

## Input Format

### Dataset Registry

Datasets are registered under user-defined `[datasets.<dataset_id>]` entries.
For example:

```toml
[datasets.morgan_r2_2048]
path = "data/morgan_r2_2048.npz"
label_path = "raw/labels.npy"
family = "fingerprints"
method = "Morgan"
variant = "r2_2048"
description = "Morgan fingerprints, radius 2, 2048 bits"
```

Here, `family` is optional **dataset metadata** used to describe the feature
representation. It is unrelated to the classifier selected or evaluated by
MELITE.

Each dataset must define `path` and `label_path`. Optional metadata fields such
as `family`, `method`, `variant`, `level`, and `description` are preserved for
traceability and do not trigger dataset-specific execution logic.

The legacy `[benchmark]` configuration section remains supported for backward
compatibility. New configurations should use the dataset registry.

### Array Requirements

A registered `.npz` dataset must contain an `X` array. MELITE validates that:

- `X` is two-dimensional;
- `X` is numeric;
- the number of rows in `X` matches the number of labels in `y`;
- an embedded `y` array, when present, matches the configured label vector.

A typical input layout is:

```text
raw/
└── labels.npy

data/
├── morgan_r2_2048.npz
├── rdkit_descriptors.npz
├── PCA85.npz
└── UMAP90.npz
```

The filenames and feature families are examples only. MELITE does not require
PCA, UMAP, fingerprints, descriptors, or any other specific feature-generation
method.

## Main Outputs

A standard **MELITE** workflow produces evaluation artifacts and, when requested,
a final exported model:

```text
output/
├── results.txt
├── results.csv
├── evaluations.csv
├── evaluation_folds.csv
├── figures/
│   └── evaluation_f1_macro_<dataset>.png
└── Model_<classifier>_<dataset>.pkl
```

The artifacts have distinct roles:

- `results.txt` — human-readable summary of the selected results.
- `results.csv` — selected classifier result for each dataset.
- `evaluations.csv` — aggregate evaluation evidence for every active classifier.
- `evaluation_folds.csv` — outer-CV evidence for every dataset, classifier, and
  outer split.
- `figures/evaluation_f1_macro_<dataset>.png` — visualization of the outer-CV
  F1-macro evidence used for classifier selection.
- `Model_<classifier>_<dataset>.pkl` — final full-data fitted model created by
  `melite export`.

See the [Output Data Contract](docs/usage.md#output-data-contract) for the
column-level schemas and semantics.

The evaluation figure is generated from already-computed outer-CV evidence. It
does not trigger additional fitting, tuning, cross-validation, or selection.


## Configuration

MELITE uses TOML configuration files to keep execution choices explicit and
reproducible. Configuration controls, among other settings:

- registered datasets and their metadata;
- active classifiers;
- random state;
- inner and outer cross-validation settings;
- input and output paths.

Use `--config` to supply a project-specific configuration:

```bash
melite run --config my_config.toml
melite export --config my_config.toml --row 0
```

Smoke mode can be requested independently from the configuration:

```bash
melite run --smoke --config my_config.toml
```

See the full configuration reference in the project documentation.

## Development

### Running Tests

Run the test suite:

```bash
python -m pytest tests -q
```

Build the documentation in strict mode:

```bash
mkdocs build --strict
```

Build and check the distributions:

```bash
python -m build --no-isolation
python -m twine check dist/*
```

Run the installed-wheel smoke test:

```bash
python scripts/smoke_install_wheel.py
```

## Contributing

Contributions are welcome. Please open an issue before submitting a pull
request. Follow the existing code style: NumPy-style docstrings, type hints,
and SPDX license headers in all source files.

See [CONTRIBUTING.md](CONTRIBUTING.md) for full guidelines, including the
development setup and the pull request target branch.
Please also read our [Code of Conduct](CODE_OF_CONDUCT.md).


## Documentation

The full documentation is published at:

https://nanobiostructuresrg.github.io/melite/


## Citation

If you use MELITE in your research, please cite it using the metadata in
[CITATION.cff](CITATION.cff) or the format below:


```text
Contreras-Torres, F. F., & Murrieta, A. C. (2026). MELITE: Multi-model Evaluation and Learning for Inference-ready Tabular Experiments. Zenodo. https://doi.org/10.5281/zenodo.20382752
```

## Authors

- **Flavio F. Contreras-Torres** — Tecnológico de Monterrey
- **Ana C. Murrieta** — Tecnológico de Monterrey


## License

This project is licensed under the terms of the
[GNU Lesser General Public License v3.0 or later](LICENSE).
SPDX identifier: `LGPL-3.0-or-later`.
