Metadata-Version: 2.4
Name: efficient-classifier
Version: 2.1.0
Summary: Dataset-agnostic ML classification library. Visualization tools, Slack integration, support for multiple-pipelines.
Home-page: https://github.com/javidsegura/efficient-classifier
Author: Javier D. Segura
Author-email: jdominguez.ieu2023@student.ie.edu
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Requires-Python: >=3.6
Description-Content-Type: text/markdown
Requires-Dist: absl-py==2.2.2
Requires-Dist: appnope==0.1.4
Requires-Dist: asttokens==3.0.0
Requires-Dist: astunparse==1.6.3
Requires-Dist: backports.tarfile==1.2.0
Requires-Dist: Boruta==0.4.3
Requires-Dist: catboost==1.2.8
Requires-Dist: certifi==2025.4.26
Requires-Dist: charset-normalizer==3.4.1
Requires-Dist: comm==0.2.2
Requires-Dist: contourpy==1.3.0
Requires-Dist: cycler==0.12.1
Requires-Dist: debugpy==1.8.13
Requires-Dist: decorator==5.2.1
Requires-Dist: docutils==0.21.2
Requires-Dist: dotenv==0.9.9
Requires-Dist: exceptiongroup==1.2.2
Requires-Dist: executing==2.2.0
Requires-Dist: filelock==3.18.0
Requires-Dist: flatbuffers==25.2.10
Requires-Dist: fonttools==4.56.0
Requires-Dist: fsspec==2025.3.2
Requires-Dist: gast==0.6.0
Requires-Dist: google-pasta==0.2.0
Requires-Dist: graphviz==0.20.3
Requires-Dist: grpcio==1.71.0
Requires-Dist: h5py==3.13.0
Requires-Dist: id==1.5.0
Requires-Dist: idna==3.10
Requires-Dist: imageio==2.37.0
Requires-Dist: imbalanced-learn==0.12.4
Requires-Dist: imblearn==0.0
Requires-Dist: importlib_metadata==8.6.1
Requires-Dist: importlib_resources==6.5.2
Requires-Dist: ipykernel==6.29.5
Requires-Dist: ipython==8.18.1
Requires-Dist: jaraco.classes==3.4.0
Requires-Dist: jaraco.context==6.0.1
Requires-Dist: jaraco.functools==4.1.0
Requires-Dist: jedi==0.19.2
Requires-Dist: Jinja2==3.1.6
Requires-Dist: joblib==1.4.2
Requires-Dist: jupyter_client==8.6.3
Requires-Dist: jupyter_core==5.7.2
Requires-Dist: keras==3.9.2
Requires-Dist: keras-tuner==1.4.7
Requires-Dist: keyring==25.6.0
Requires-Dist: kiwisolver==1.4.7
Requires-Dist: kt-legacy==1.0.5
Requires-Dist: lazy_loader==0.4
Requires-Dist: libclang==18.1.1
Requires-Dist: lightgbm==4.6.0
Requires-Dist: lime==0.2.0.1
Requires-Dist: Markdown==3.8
Requires-Dist: markdown-it-py==3.0.0
Requires-Dist: MarkupSafe==3.0.2
Requires-Dist: matplotlib==3.9.4
Requires-Dist: matplotlib-inline==0.1.7
Requires-Dist: mdurl==0.1.2
Requires-Dist: ml_dtypes==0.5.1
Requires-Dist: more-itertools==10.7.0
Requires-Dist: mpmath==1.3.0
Requires-Dist: namex==0.0.9
Requires-Dist: narwhals==1.41.0
Requires-Dist: nest-asyncio==1.6.0
Requires-Dist: networkx==3.4.2
Requires-Dist: nh3==0.2.21
Requires-Dist: numpy==2.0.2
Requires-Dist: opt_einsum==3.4.0
Requires-Dist: optree==0.15.0
Requires-Dist: packaging==24.2
Requires-Dist: pandas==2.2.3
Requires-Dist: parso==0.8.4
Requires-Dist: patsy==1.0.1
Requires-Dist: pexpect==4.9.0
Requires-Dist: pillow==11.1.0
Requires-Dist: platformdirs==4.3.7
Requires-Dist: plotly==6.1.2
Requires-Dist: prompt_toolkit==3.0.50
Requires-Dist: protobuf==5.29.4
Requires-Dist: psutil==7.0.0
Requires-Dist: ptyprocess==0.7.0
Requires-Dist: pure_eval==0.2.3
Requires-Dist: pyaml==25.1.0
Requires-Dist: Pygments==2.19.1
Requires-Dist: pyparsing==3.2.3
Requires-Dist: python-dateutil==2.9.0.post0
Requires-Dist: python-dotenv==1.1.0
Requires-Dist: pytz==2025.2
Requires-Dist: PyYAML==6.0.2
Requires-Dist: pyzmq==26.3.0
Requires-Dist: readme_renderer==44.0
Requires-Dist: requests==2.32.3
Requires-Dist: requests-toolbelt==1.0.0
Requires-Dist: rfc3986==2.0.0
Requires-Dist: rich==14.0.0
Requires-Dist: scikit-image==0.25.2
Requires-Dist: scikit-learn==1.6.1
Requires-Dist: scikit-optimize==0.10.2
Requires-Dist: scipy==1.13.1
Requires-Dist: seaborn==0.13.2
Requires-Dist: six==1.17.0
Requires-Dist: slack_bolt==1.23.0
Requires-Dist: slack_sdk==3.35.0
Requires-Dist: stack-data==0.6.3
Requires-Dist: statsmodels==0.14.4
Requires-Dist: sympy==1.13.3
Requires-Dist: tensorboard==2.19.0
Requires-Dist: tensorboard-data-server==0.7.2
Requires-Dist: tensorflow==2.19.0
Requires-Dist: tensorflow-io-gcs-filesystem==0.37.1
Requires-Dist: termcolor==3.0.1
Requires-Dist: threadpoolctl==3.6.0
Requires-Dist: tifffile==2025.5.26
Requires-Dist: torch==2.7.0
Requires-Dist: tornado==6.4.2
Requires-Dist: tqdm==4.67.1
Requires-Dist: traitlets==5.14.3
Requires-Dist: twine==6.1.0
Requires-Dist: typeguard==2.13.3
Requires-Dist: typing_extensions==4.13.0
Requires-Dist: tzdata==2025.2
Requires-Dist: urllib3==2.4.0
Requires-Dist: wcwidth==0.2.13
Requires-Dist: Werkzeug==3.1.3
Requires-Dist: wrapt==1.17.2
Requires-Dist: xgboost==3.0.2
Requires-Dist: xlrd==2.0.1
Requires-Dist: zipp==3.21.0
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# EFFICIENT CLASSIFIER, A DATASET-AGNOSTIC ML CLASSIFER LIBRARY

In this library we present a framework to asses multiple classifiers pipelines for any labelled tabular dataset. We apply this library to the CCCS-CIC-AndMal-2020 dataset. A paper for our work is attached in "Project Documentation, Relevant Links"

## Project Set-up / Tutorial
1. Clone this repository
2. Create a .env with the following credentials (if you want to use the slack bot):
 - SLACK_BOT_TOKEN, SLACK_SIGNING_SECRET, SLACK_APP_TOKEN
3. Install libraries from requirements.txt
4. Customize dataset-specific treatment throughout pipeline runners if desired (only if you want to add your own custom logic (e.g, feature engineering, dataset treatment))
5. Customize configurations.yaml with your own preferred settings


## MANTIS: Malware Analysis and Network-based Threat Identification System
As mentioned before, we apply this library to the CCCS-CIC-AndMal-2020 dataset. We conducteed extensive EDA analysis and applied the capture insights for data preprocessing, feature engineering and models optimization.

### Model
Random forest performs the best in pair with stacking model with meta model <i>Logistic Regression</i> and base models <i>Decision Tree, Random Forest, Gaussian Naive Bayes and Feed Forward Neural Network</i>. Them both have a 92% in f1-score. We select random forest as the final model for deployment due to the significant decrease in time for inference and training. 

### Training and tuning
In order to tune the model we used sklearn's bayesian optimzier (keras' implementation for the FFN), tough the library offers the possibilities to use grid search and random forest. In order to adjust the hyperparamter space and the extend of the optimzier search (the number of combinations) please modify modelling_runner->hyperparameters in configurations.yaml

### Inference
Once you have the best model selected out of the assesment after tuning you can serialize your model (either with joblib or pickle). The model serialized contains the results at each stage and the model sklearn object at each stage. In production use the model_sklearn from the assesment attribute in the serialized model. Do model.predict(X) or model.predict_proba(X) if you want soft-predictions.

### Design and Implementation
This library allows for parallel assesment of as many pipelines for different models as provided in configurations.yaml. This library will execute the following phases for each pipeline:

- Dataset load
  - Loads dataset, executes dataset-specific procedures, asses split standard error variation
- Data preprocessing
  - Class imbalance, missing data, outlier detection, bound-checking 
- Feature analysis
  - Feature transformation, feature engineering, feature selection
- Modelling
  - Tuning
  - Post-tuning
      - Callibration (if enabled in configurations.yaml)
- Dev-Ops
  - Send in-real-time logger results to SlackBot (including images results)
  - Store results in CSV
  - Serialize model
  - Pipeline DAG visualization
    
 ![SlackBot](https://github.com/user-attachments/assets/19045a75-32dc-4777-8cfb-e6e39ec4f073)
 <i>Slack Bot</i>
 ![DAG Visualizer](https://github.com/user-attachments/assets/b06781c6-b703-4695-a5c3-ea720809884d)
 <i>DAG Pipeline Visualizer</i>




This library provided us with the scalable framework needed for the complexity of the evaluated pipelines. Through its deeply organized structured, this library allows for short development time and faster debugging. The extensiveness of data stored after a pipeline run allows for efficient auditing as well as a reliable way to revisit the pipeline at retraining time.

### Project Documentation, Relevant links
[Plots results](https://drive.google.com/drive/folders/1Ui2EmIr-5rrXPkab1lGquHp_cQ7w14yA?usp=sharing)

[Paper](https://drive.google.com/drive/folders/1GksAEhtbiqzj-pGVJixrn35E6DRu44gK?usp=drive_link)

[Report](https://docs.google.com/document/d/1yH9gvnJVSH9GLv9ATQ5JQWA2z8Jy4umxxRfMF-y2fiU/edit?usp=drive_link)

[Presentation](https://www.canva.com/design/DAGnoUCnQmQ/VgZLdpPD2IpRFxJj_7TuLg/edit?utm_content=DAGnoUCnQmQ&utm_campaign=designshare&utm_medium=link2&utm_source=sharebutton)

[PyPi library](https://pypi.org/project/efficient-classifier/)


### Project Status
This project is still under development. It is not guaranteed that this library is free of bugs. It currently has been used only for the aforementioned dataset. We have designed the library to be dataset agnostic, however. We are currently working to make this libray more efficient, more robust as well with added features. 

### Contributing
This library encourages external contributions. Please create your own branch at https://github.com/javidsegura/efficient-classifier and make pull request when your contributions are ready. Please, make sure you provide a documented pull request message detailing your contributions.

