Metadata-Version: 2.4
Name: FeatureRank
Version: 0.1.7
Summary: Autoencoder-based feature ranking and selection workflows
Author: Sercan
License: MIT
Requires-Python: <3.15,>=3.10
Description-Content-Type: text/markdown
Requires-Dist: numpy>=2.0
Requires-Dist: pandas>=2.0
Requires-Dist: numexpr>=2.10.2
Requires-Dist: scikit-learn>=1.4
Requires-Dist: matplotlib>=3.8
Requires-Dist: tensorflow<2.22,>=2.21; python_version < "3.14" and (platform_system != "Darwin" or platform_machine != "x86_64")
Requires-Dist: tensorflow<2.17,>=2.16.2; platform_system == "Darwin" and platform_machine == "x86_64" and python_version < "3.13"

# FeatureRank

FeatureRank is a Python project for ranking and selecting features with an
autoencoder. It is intended for tabular datasets in which using every feature
is expensive or makes a model harder to interpret.

The project supports two selection modes:

1. **GLOBAL**: rank the complete feature matrix in one experiment.
2. **Divide & Combine (DC)**: split wide datasets into feature blocks, rank each
   block, combine the selected features, and run a final evaluation.

The selected features can be evaluated with classification, regression, or
unsupervised clustering.

## PyPI installation and GUI (end users)

FeatureRank's GUI can be imported on 64-bit Python 3.10–3.14 desktop systems.
The TensorFlow worker currently runs on Python 3.10–3.13. When the GUI is
started from Python 3.14, it automatically looks for an installed compatible
Python worker. Intel macOS users should use Python 3.12 or earlier because
newer TensorFlow releases do not provide an Intel macOS wheel. Check the
Python version first:

```bash
python3 --version
```

Install the published package with:

```bash
pip install FeatureRank
```

On Windows with Python 3.14, use the same interpreter for installation and
launch:

```powershell
py -3.14 -m pip install --upgrade FeatureRank
py -3.14
```

Then type `import FeatureRank`. Python 3.14 can open the GUI directly. To run
an experiment, also install Python 3.13 (or 3.12/3.11/3.10) with TensorFlow;
the GUI will find that compatible worker automatically.

On Windows with Python 3.14, use the same interpreter for installation and
launch:

```powershell
py -3.14 -m pip install --upgrade FeatureRank
py -3.14
```

Then type `import FeatureRank`. Python 3.14 can open the GUI directly. To run
an experiment, also install Python 3.13 (or 3.12/3.11/3.10) with TensorFlow;
the GUI will find that compatible worker automatically.

![Installing FeatureRank with pip](docs/images/01-pip-install.png)

Open Python:

```bash
python3
```

![Opening Python 3](docs/images/02-python3.png)

Then import FeatureRank at the Python prompt:

```python
>>> import FeatureRank
```

![Importing FeatureRank](docs/images/03-import-featurerank.png)

The import opens the desktop GUI automatically. Do not type `import
FeatureRank` directly in zsh; it is Python code and must be entered after
starting `python3`.

Use the same interpreter for installation and launch. `python` and `python3`
can point to different Python installations, each with its own installed
FeatureRank version and GUI. If you launch with `python3`, install with
`python3 -m pip`; if you launch with `python`, install with `python -m pip`.
Both interpreters receive the same single GUI after installing the current
package version.

Use the same interpreter for installation and launch. `python` and `python3`
can point to different Python installations, each with its own installed
FeatureRank version and GUI. If you launch with `python3`, install with
`python3 -m pip`; if you launch with `python`, install with `python -m pip`.
Both interpreters receive the same single GUI after installing the current
package version.

For local development, run these commands from the repository root instead:

```bash
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip install -e .
```

GPU-specific dependencies are listed in
[`requirements-gpu.txt`](requirements-gpu.txt). Hardware notes are available in
[`GPU_SETUP.md`](GPU_SETUP.md).

The repository metadata is version `0.1.7`. Because PyPI releases cannot be
replaced in place, publish this version (or a newer one) to the existing
`FeatureRank` project when you release the refactor. If an environment still
shows an older FeatureRank version, upgrade it with the command above or use the local
editable installation while developing.

After installation, the package GUI can be opened with:

```bash
FeatureRank
```

`import FeatureRank` is Python code, not a shell command. On a desktop, the
following import opens the GUI automatically:

```python
import FeatureRank
```

For applications that want to decide when the window opens, use:

```python
from FeatureRank import Launch

Launch()
```

On servers or CI without a display, set `FEATURERANK_NO_GUI=1` before
importing the package.

On macOS, some Anaconda installations crash in their native `pythonw`/Tk
bridge. If the import returns without a window, use the Python.org 3.13
interpreter (or update Anaconda's Tk package), then install FeatureRank again
in that interpreter's environment.

If you do not want to install the package, use the equivalent script directly:

```bash
python3 scripts/FeatureRank.py --help
```

## Optional Python API (developers)

The main entry point for regular users is the GUI shown above. The following
Python API is optional and is useful when another Python program needs to start
an experiment. It is not required for the normal `pip install` → `import
FeatureRank` workflow.

The main command-line entry point is:

```text
scripts/FeatureRank.py
```

It uses the same task modules for both GLOBAL and DC experiments. The older
`RunAutoencoder.py` command is kept as a compatibility wrapper.

After the local installation, the workflow can also be called from Python. The
two required arguments are the dataset name and the percentage to select:

```python
import FeatureRank

FeatureRank.run(
    dataset_name="breast_cancer_data.csv",
    feature_percent=20,
    task="classification",
)
```

The same function accepts `mode="dc"` and `block_count=10` for a Divide &
Combine run.

`dataset_name` may be a file name in `data/raw` or a full path to a CSV/TXT
file. A GUI that runs outside the repository should pass the full path to the
user-selected file.

## GUI Workflow

The GUI is the primary end-user interface. Select a dataset from the dropdown
or choose a CSV/TXT file with **Browse…**, select a feature percentage, choose
`GL` or `DC`, and press **START**. `Block Count` is enabled only for `DC`.

The **Feature Selection %** list also contains **Auto**. Auto runs the existing
workflow for 10%, 20%, …, 100% and displays the best result. Accuracy is used
for classification, Pearson correlation for regression, and the lowest cluster
RMSE for clustering.

The **Feature Selection %** list also contains **Auto**. Auto runs the existing
workflow for 10%, 20%, …, 100% and displays the best result. Accuracy is used
for classification, Pearson correlation for regression, and the lowest cluster
RMSE for clustering.

When the GUI opens, the experiment form is ready for these choices:

![FeatureRank GUI](docs/images/04-gui-ready.png)

If the required dataset is not listed, press **Browse…** and select its data
file. For paired datasets, keep the matching label file in the same directory:

![Selecting a dataset](docs/images/05-browse-dataset.png)

While the model runs, the progress bar and log show stages such as loading,
feature ranking, block processing, combining, training, and result creation.
When the run finishes, the summary lists the selected feature count, metric,
execution time, and output directory. **Open Results Folder** opens that
directory in Finder, Explorer, or the Linux file manager.

![Completed FeatureRank result](docs/images/06-results.png)

The selected file can be a normal CSV containing a `target` column, or one of
the project's paired files (`*_data.csv` and `*_label.csv`).

## Local repository command-line mode (developers)

The GUI is the normal user interface. The commands in this section are optional
and are intended for developers, batch experiments, and reproducible research.
They bypass the GUI and run the existing workflow directly in the terminal.
They are available from a repository checkout; a normal PyPI user does not need
them.

Run these shell commands directly in Terminal. Do not first open the Python
prompt and do not type the `>>>` marker. To inspect the CLI options:

```bash
python3 scripts/FeatureRank.py --help
```

A normal classification run is:

```bash
python3 scripts/FeatureRank.py \
  --dataset-name breast_cancer_data.csv \
  --task classification \
  --feature-percent 20 \
  --random-state 42
```

The command can also be run through the compatibility wrapper:

```bash
python3 scripts/RunAutoencoder.py \
  --dataset-name breast_cancer_data.csv \
  --feature-percent 20
```

## GLOBAL Mode

GLOBAL is the default when neither `--global` nor `--dc` is supplied. Its
workflow is:

1. load the dataset and labels,
2. remove the optional ID column and clean the features,
3. encode the target when classification is requested,
4. create the reproducible train/test split and scale the features,
5. train the autoencoder,
6. calculate the FeatureRank score for every feature,
7. select the requested percentage, and
8. evaluate the selected feature set for the requested task.

Run GLOBAL explicitly with:

```bash
python3 scripts/FeatureRank.py \
  --dataset-name carcinom_data.csv \
  --task classification \
  --feature-percent 40 \
  --global
```

The explicit command above and the same command without `--global` produce the
same mode.

## Divide & Combine (DC) Mode

DC is useful for datasets with a very large number of feature columns. Rows and
labels are preserved while only the feature columns are divided.

The steps are:

1. split the feature columns into `N` blocks,
2. run FeatureRank independently on every block,
3. select the requested percentage from each block,
4. translate local feature names to original names with the mapping file,
5. combine the selected original features,
6. remove duplicate selections,
7. create the combined dataset, and
8. run the final task evaluation on that dataset.

Run DC with ten blocks:

```bash
python3 scripts/FeatureRank.py \
  --dataset-name arcene_data.csv \
  --task classification \
  --feature-percent 50 \
  --dc \
  --block-count 10
```

`--global` and `--dc` cannot be used together. If `--block-count` is omitted,
the current default is 10.

The orchestration is implemented in
[`src/DivideCombine.py`](src/DivideCombine.py). Splitting, mapping, and
combining reuse
[`scripts/FeatureBlockDatasetTools.py`](scripts/FeatureBlockDatasetTools.py);
the ranking algorithm is not duplicated for DC.

## Feature Selection Method

For feature `i`, sample `s`, and encoder unit `j`, the contribution score is:

```text
score(i) = max_j mean_s(abs(X[s, i] * W[i, j]))
```

`X` is the scaled input matrix and `W` is the first encoder-layer weight
matrix. Features are sorted by this score. For `p` percent, the selected count
is `ceil(number_of_features * p / 100)`, with at least one feature selected.

The implementation is in
[`src/AutoencoderFeatureSelection.py`](src/AutoencoderFeatureSelection.py).

## Input

The normal input format is a pair of headerless files in `data/raw/`:

```text
data/raw/arcene_data.csv
data/raw/arcene_label.csv
```

Rows represent samples. Columns in the data file represent features. The label
file contains one target value per row. The loader also supports the legacy text
and sparse formats present in the repository.

If the target column has a name other than `target`, use
`--target-column <name>`. If there is no ID column, use `--id-column none`.

Classification targets may be binary or multiclass. Regression targets must be
numeric and continuous. Clustering does not use labels to fit KMeans.

## Tasks

Classification uses a binary classifier for two labels. For multiclass data, the
existing one-vs-rest workflow runs one binary experiment per class and reports
the aggregate metrics.

Regression reports MSE, RMSE, MAE, R², and Pearson correlation where defined.

Clustering evaluates KMeans over the configured `k` range using silhouette
score. A fixed value can be requested with `--cluster-k`.

Examples:

```bash
# Classification
python3 scripts/FeatureRank.py \
  --dataset-name breast_cancer_data.csv \
  --task classification \
  --feature-percent 20

# Regression
python3 scripts/FeatureRank.py \
  --dataset-name air_data.csv \
  --task regression \
  --feature-percent 30

# Clustering
python3 scripts/FeatureRank.py \
  --dataset-name codon_usage_data.csv \
  --task clustering \
  --feature-percent 60
```

## Parameters

The main user-facing parameters are:

| Parameter | Description |
| --- | --- |
| `--dataset-name` | Input dataset name in `data/raw/` |
| `--task` | `classification`, `regression`, or `clustering` |
| `--feature-percent` | Percentage of features to select |
| `--global` | Run GLOBAL mode |
| `--dc` | Run Divide & Combine mode |
| `--block-count` | Number of feature blocks in DC mode |
| `--random-state` | Seed; use `none` for an unseeded run |
| `--target-column` | Target column name |
| `--id-column` | ID column name, or `none` |
| `--cluster-k` | Fixed cluster count for clustering |

For the complete, current list:

```bash
python3 scripts/FeatureRank.py --help
```

Model architecture, epochs, batch size, learning rate, early stopping, and
other defaults are kept in
[`src/Config.py`](src/Config.py), not repeated in every command.

## Outputs

GLOBAL classification output for `arcene_data.csv` and 50% selection is written
under:

```text
outputs/Classification/arcene_data/
├── first_layer_W_list.csv
├── top_50_max_abs_features.csv
├── ORG_*.csv / ORG_*.png
└── metrics/
    ├── ORG_test_metrics.json
    └── top_50_test_metrics.json
```

The selected feature CSV contains the original feature names and their scores.
Metric JSON/CSV files contain task results. Generated plots include prediction,
confusion-matrix, ROC, precision-recall, or clustering figures as applicable.

DC additionally creates:

```text
split_datasets/arcene/
├── arcene_block_*.csv
├── feature_block_mapping.csv
└── split_summary.csv

data/raw/
├── arcene_block_*_data.csv
├── arcene_block_*_label.csv
├── arcene_selected_features_combined_data.csv
└── arcene_selected_features_combined_label.csv
```

The mapping file links every local block feature to its original feature. The
combined dataset is used by the final evaluation and is also saved under
`split_datasets/`.

## Project Structure

```text
Feature_Ranking_Project/
├── FeatureRank/                      installable package and desktop GUI
│   ├── GUI.py                         Tkinter interface and Launch API
│   └── __init__.py                    public package exports
├── data/raw/                         input feature and label files
├── outputs/                          metrics and generated figures
├── split_datasets/                   DC blocks, mapping, and combined data
├── scripts/
│   ├── FeatureRank.py                main command
│   ├── RunAutoencoder.py             compatibility wrapper
│   ├── FeatureBlockDatasetTools.py   split/combine helpers
│   ├── RunBlockFeatureSelection.py   block comparison workflow
│   └── RunFeatureRankCV.py           leakage-free repeated CV
├── src/
│   ├── Classification.py             classification workflow
│   ├── Regression.py                 regression workflow
│   ├── Clustering.py                 clustering workflow
│   ├── DivideCombine.py              DC orchestration
│   ├── AutoencoderFeatureSelection.py ranking and selection
│   ├── DataLoader.py                 dataset loading
│   ├── Preprocessing.py              cleaning, splitting, and scaling
│   ├── Models.py                     Keras model definitions
│   ├── Workflow.py                   shared task entry point
│   └── Config.py                     project defaults
├── tests/
├── pyproject.toml
├── requirements.txt
└── Readme.md
```

All Python script names use PascalCase. `src/__init__.py` remains unchanged
because it is a special Python package filename.

## Requirements

The main technologies are:

- Python
- TensorFlow / Keras
- pandas
- NumPy
- scikit-learn
- Matplotlib

Exact pinned versions are listed in
[`requirements.txt`](requirements.txt). GPU dependencies are listed separately
in [`requirements-gpu.txt`](requirements-gpu.txt).

## Tests

Run the checks from the repository root:

```bash
python3 -m pytest -q
python3 -m pyflakes src scripts tests
python3 -m compileall -q src scripts tests
```

## Citation

Citation information will be added after publication. When reporting a result,
include the dataset source, task, GLOBAL or DC mode, feature percentage, block
count when applicable, random seed, and the generated metric files.
