Metadata-Version: 2.4
Name: aspect-data
Version: 0.0.7
Summary: General purpose, chemistry, and taxonomic featurization of tabular data with caching.
Author-email: Eachan Johnson <eachan.johnson@crick.ac.uk>
License: MIT License
        
        Copyright (c) [year] [fullname]
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
Project-URL: Homepage, https://github.com/scbirlab/aspect
Project-URL: Bug Tracker, https://github.com/scbirlab/aspect/issues
Keywords: ai,data,deep-learning,machine-learning
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: 3.15
Classifier: Programming Language :: Python :: 3 :: Only
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: carabiner-tools[mpl,pd]>=0.0.5.post3
Requires-Dist: datasets>=3.0.0
Requires-Dist: filelock
Requires-Dist: huggingface_hub
Requires-Dist: numpy
Requires-Dist: providence>=0.0.2
Requires-Dist: tqdm
Provides-Extra: chem
Requires-Dist: schemist>=0.0.7; extra == "chem"
Provides-Extra: torch
Requires-Dist: torch>=2.4; extra == "torch"
Provides-Extra: chemprop
Requires-Dist: aspect-data[chem,torch]; extra == "chemprop"
Requires-Dist: chemprop>=2.1.0; extra == "chemprop"
Provides-Extra: deep
Requires-Dist: transformers; extra == "deep"
Provides-Extra: bio
Requires-Dist: vectome>=0.0.8; extra == "bio"
Provides-Extra: dev
Requires-Dist: aspect-data[bio,chemprop]; extra == "dev"
Requires-Dist: pytest>=7.4; extra == "dev"
Requires-Dist: flake8; extra == "dev"
Dynamic: license-file

# 🏔️ aspect

![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/scbirlab/aspect/python-publish.yml)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/aspect-data)
![PyPI](https://img.shields.io/pypi/v/aspect-data)

<img src="docs/source/_static/logo.svg" width="200">

**aspect** is a Python library and command-line tool for reproducible featurisation of tabular machine-learning datasets.

It lets you define independent transformation pipelines for different input columns, including molecular structures, taxonomy, categorical variables, and numerical measurements. Pipelines can be serialised, applied to new datasets, checkpointed, and restored.

Aspect uses [Providence](https://github.com/scbirlab/providence) for dataset loading, source provenance, and checkpointing, and [Hugging Face Datasets](https://huggingface.co/docs/datasets/) for Arrow-backed data processing and caching.

The core idea:

```text
Input dataset
    │
    ├── SMILES ──────→ Molecular fingerprints
    ├── Species ─────→ Taxonomic fingerprints
    ├── Assay ───────→ Categorical encoding
    ├── MW ──────────→ Log transformation
    └── Description ─→ Language-model embeddings
    │
    ▼
Aspect DataPipeline
    │
    ├── Reusable feature specification
    ├── Reproducible dataset checkpoint
    └── Optional source materialisation
    │
    ▼
Providence DataResolver
    │
    ├── Hugging Face Dataset
    ├── Export: CSV / TSV / JSON / Parquet
    └── Model-specific collation / DataLoader
```

Feature representations are stored in Arrow-compatible columns. Model-specific collators can convert these stored values into tensors, molecular graphs, or other runtime representations without embedding model-specific logic in dataset preprocessing.

## Contents

- [Quick start](#quick-start)
- [Installation](#installation)
- [Command-line interface](#command-line-interface)
- [Python API](#python-api)
- [Feature specifications](#feature-specifications)
- [Available transforms](#available-transforms)
- [Saving and loading pipelines](#saving-and-loading-pipelines)
- [Supported input and output formats](#supported-input-and-output-formats)
- [Working with output columns](#working-with-output-columns)
- [Custom transforms](#custom-transforms)
- [Model integration](#model-integration)
- [Development](#development)
- [Issues and documentation](#issues-and-documentation)

---

## Quick start

### 1. Featurise a dataset

Create a small CSV:

```bash
cat > compounds.csv <<'CSV'
compound_id,assay,mwt,pIC50
cmpd-1,MIC,46.07,5.0
cmpd-2,MIC,60.10,6.2
cmpd-3,IC50,78.11,4.8
CSV
```

Generate numeric and categorical features:

```bash
aspect featurize compounds.csv \
    --features mwt:log@log_mwt assay:hash@assay_hash pIC50@target \
    --extras compound_id \
    --output features.parquet
```

The feature specification defines the transformations:

| Specification | Meaning |
|---|---|
| `mwt:log@log_mwt` | Apply natural logarithm to `mwt` |
| `assay:hash@assay_hash` | Hash the assay category into a fixed-length vector |
| `pIC50@target` | Copy `pIC50` into a feature named `target` |
| `--extras compound_id` | Explicitly retain the compound identifier |

By default, Aspect also retains original input columns.

### 2. Save a reproducible checkpoint

The same command can save a checkpoint alongside the exported table:

```bash
aspect featurize compounds.csv \
    --features mwt:log@log_mwt assay:hash@assay_hash \
    --checkpoint features.as \
    --output features.parquet
```

The checkpoint records the feature specification, data source, and information needed to reconstruct the transformed dataset.

### 3. Restore and export

Reconstruct the transformed dataset from its checkpoint:

```bash
aspect write features.as \
    --output restored.parquet
```

Export it in another format:

```bash
aspect write features.as \
    --output restored.csv.gz
```

### 4. Python API

```python
from aspect.data import DataPipeline

data = {
    "compound_id": ["cmpd-1", "cmpd-2", "cmpd-3"],
    "assay": ["MIC", "MIC", "IC50"],
    "mwt": [46.07, 60.10, 78.11],
}

pipe = DataPipeline(
    column_transforms={
        "log_mwt": ("mwt", "log"),
        "assay_hash": (
            "assay",
            {"name": "hash", "kwargs": {"ndim": 8}},
        ),
    },
    columns_to_keep=["compound_id"],
)

resolved = pipe(data, drop_unused_columns=True)

# Access the underlying Hugging Face Dataset
features = resolved.data

print(features.column_names)
print(features[:2])
```

The transformed dataset contains:

```text
['compound_id', 'log_mwt', 'assay_hash']
```

`log_mwt` contains a numeric feature with shape `(n, 1)`, while `assay_hash` contains an `(n, 8)` feature matrix.

Save and reload the pipeline:

```python
pipe.save("features.as")

restored = DataPipeline.load("features.as")
print(restored.data_out.data[:2])
```

---

## Installation

Aspect requires Python 3.11 or later.

### Basic installation

```bash
pip install aspect-data
```

### Optional dependencies

Install only the features you need:

| Extra | Functionality | Installation |
|---|---|---|
| `chem` | Molecular fingerprints and descriptors via Schemist | `pip install "aspect-data[chem]"` |
| `bio` | Taxonomic fingerprints via Vectome | `pip install "aspect-data[bio]"` |
| `torch` | PyTorch integration | `pip install "aspect-data[torch]"` |
| `chemprop` | Chemprop molecular features | `pip install "aspect-data[chemprop]"` |
| `deep` | Transformer-based embeddings | `pip install "aspect-data[deep]"` |

### Install from source

```bash
git clone https://github.com/scbirlab/aspect.git
cd aspect
pip install -e ".[dev]"
```

Aspect depends on [Providence](https://github.com/scbirlab/providence) (`>=0.0.2`), which provides the common dataset-resolution and checkpointing infrastructure.

---

## Command-line interface

Aspect provides three commands:

```bash
aspect serialize
aspect featurize
aspect write
```

View available arguments with:

```bash
aspect --help
aspect featurize --help
aspect serialize --help
aspect write --help
```

### `aspect serialize`

Create a reusable feature specification without processing a dataset:

```bash
aspect serialize \
    --features mwt:log@log_mwt assay:hash@assay_hash \
    --extras compound_id \
    --output feature-spec.as
```

The specification can then be applied to other datasets:

```bash
aspect featurize new-compounds.csv \
    --config feature-spec.as \
    --output new-features.parquet
```

### `aspect featurize`

Apply a feature specification or saved pipeline to a dataset.

```bash
aspect featurize compounds.csv \
    --features mwt:log@log_mwt assay:hash@assay_hash \
    --output features.parquet
```

A saved specification can be supplied with `--config`:

```bash
aspect featurize compounds.csv \
    --config feature-spec.as \
    --output features.parquet
```

#### Checkpoint without exporting

```bash
aspect featurize compounds.csv \
    --config feature-spec.as \
    --checkpoint features.as
```

#### Save source and transformed data

```bash
aspect featurize compounds.csv \
    --config feature-spec.as \
    --checkpoint features.as \
    --force-write-data
```

This requests materialisation of both source and transformed data inside the checkpoint, making it less dependent on the original dataset remaining available.

Without this option, Providence can retain immutable remote datasets by reference, avoiding unnecessary duplication. Local inputs are generally packaged by the current implementation.

#### Process a subset of rows

```bash
aspect featurize large-table.parquet \
    --start 1000 \
    --end 2000 \
    --features mwt:log@log_mwt \
    --checkpoint subset.as
```

`--start` is inclusive and `--end` is exclusive. The example selects rows 1000–1999.

#### Configure the cache

```bash
aspect featurize compounds.csv \
    --features assay:hash@assay_hash \
    --cache .cache \
    --output features.parquet
```

### `aspect write`

Export data from a saved Aspect checkpoint:

```bash
aspect write features.as --output features.csv
aspect write features.as --output features.csv.gz
aspect write features.as --output features.parquet
aspect write features.as --output features.hf
```

The checkpoint is loaded through `DataPipeline.load()` and the output is delegated to Providence.

If the checkpoint contains only a feature specification and no source dataset, there is no transformed table to export.

---

## Python API

The main class is `DataPipeline`.

```python
from aspect.data import DataPipeline
```

A pipeline maps input columns to named output features.

```python
pipe = DataPipeline({
    "log_mwt": ("mwt", "log"),
    "assay_hash": (
        "assay",
        {"name": "hash", "kwargs": {"ndim": 16}},
    ),
})
```

### Input data

Pipelines can be applied to Python mappings, Pandas DataFrames, Hugging Face datasets, local files, or Providence resolvers.

```python
# Dictionary
resolved = pipe({
    "mwt": [46.07, 60.10],
    "assay": ["MIC", "IC50"],
})

# Pandas DataFrame
resolved = pipe(df)

# Local file
resolved = pipe("compounds.csv")

# Hugging Face Hub dataset
resolved = pipe(
    "hf://datasets/scbirlab/fang-2023-biogen-adme~scaffold-split:train"
)
```

### Providence integration

Aspect uses Providence's `DataResolver` to manage source datasets and transformed results.

```python
from providence import DataResolver
from aspect.data import DataPipeline

source = DataResolver("compounds.csv")

pipe = DataPipeline({
    "log_mwt": ("mwt", "log"),
})

result = pipe(source)

# Underlying datasets
input_ds = pipe.data_in.data
output_ds = result.data

# Source metadata
source_info = pipe.data_in.data_source

print(output_ds.column_names)
```

`DataPipeline.__call__()` returns a `DataResolver`, rather than a raw Hugging Face dataset.

The underlying dataset remains available through `.data`, and provenance is tracked through the resolver.

### Exporting results

```python
pipe.write_table("features.parquet")
pipe.write_table("features.csv.gz")
```

This delegates serialization to Providence.

### Selecting rows

For explicit row selection, create and slice the resolver before passing it to Aspect:

```python
source = DataResolver("compounds.csv")
source = source.select_rows(start=100, end=200)

result = pipe(source)
```

### Reusing transformations

The same feature specification can be applied to multiple datasets:

```python
pipe = DataPipeline({
    "log_mwt": ("mwt", "log"),
})

train = pipe("train.csv")
test = pipe("test.csv")
```

Each call returns the corresponding transformed resolver. The pipeline's `data_in` and `data_out` attributes refer to the most recent call.

---

## Feature specifications

Aspect supports both structured Python specifications and compact CLI strings.

### Python specifications

The basic format is:

```python
{
    "output_column": ("input_column", "transform_name")
}
```

Example:

```python
pipe = DataPipeline({
    "log_mwt": ("mwt", "log"),
})
```

### Parameterised transforms

Pass a dictionary containing the transform name and keyword arguments:

```python
pipe = DataPipeline({
    "assay_hash": (
        "assay",
        {
            "name": "hash",
            "kwargs": {
                "ndim": 32,
                "seed": 123,
            },
        },
    ),
})
```

### Chained transforms

Pass a sequence of transformations:

```python
pipe = DataPipeline({
    "scaled_log_mwt": (
        "mwt",
        [
            "log",
            {
                "name": "scale",
                "kwargs": {"factor": 0.1},
            },
        ],
    ),
})
```

The `scale` transform in this example is a custom transform that must be registered first.

Chains execute in the supplied order. Each transform must accept the representation produced by the preceding transform.

### CLI specifications

The CLI format is:

```text
input_column:transform@output_column
```

| Syntax | Behaviour |
|---|---|
| `column` | Retain an input column |
| `column@new_name` | Identity transformation with a new name |
| `column:transform` | Apply a transform and automatically name the output |
| `column:transform@new_name` | Apply a transform with an explicit output name |
| `column:t1:t2@new_name` | Chain transformations |
| `column:transform(arg=value)@name` | Pass simple transform arguments |

Example:

```bash
aspect featurize compounds.csv \
    --features \
        mwt:log@log_mwt \
        assay:hash(ndim=32)@assay_hash \
        pIC50@target \
    --output features.parquet
```

Simple integers, floats, and booleans are parsed automatically. For complex or nested parameters, use the Python API.

---

## Available transforms

### Base transforms

Available with the standard installation:

| Transform | Input | Output |
|---|---|---|
| `identity` | Any supported column | Identity representation |
| `log` | Numeric | Natural logarithm |
| `hash` | Categorical/string | Deterministic fixed-length vector |
| `one-hot` | Categorical | One-hot vector |

Example:

```python
pipe = DataPipeline({
    "log_mwt": ("mwt", "log"),
    "assay_hash": (
        "assay",
        {"name": "hash", "kwargs": {"ndim": 16}},
    ),
    "assay_onehot": (
        "assay",
        {
            "name": "one-hot",
            "kwargs": {"categories": ["MIC", "IC50"]},
        },
    ),
})
```

### Chemistry transforms

Install with:

```bash
pip install "aspect-data[chem]"
```

| Transform | Input | Output |
|---|---|---|
| `morgan-fingerprint` | SMILES | Molecular fingerprint |
| `descriptors-2d` | SMILES | 2D molecular descriptors |
| `descriptors-3d` | SMILES | 3D molecular descriptors |

Example:

```python
data = {
    "compound_id": ["ethanol", "ethylamine", "benzene"],
    "smiles": ["CCO", "CCN", "c1ccccc1"],
}

pipe = DataPipeline(
    {
        "morgan": ("smiles", "morgan-fingerprint"),
        "desc2d": ("smiles", "descriptors-2d"),
    },
    columns_to_keep=["compound_id"],
)

features = pipe(data, drop_unused_columns=True).data
```

### Taxonomic transforms

Install with:

```bash
pip install "aspect-data[bio]"
```

The `vectome-fingerprint` transform encodes species names or taxonomic identifiers as fixed-length taxonomic representations.

```python
data = {
    "species": [
        "Mycobacterium tuberculosis",
        "Escherichia coli",
        "Staphylococcus aureus",
    ],
}

pipe = DataPipeline({
    "species_fp": (
        "species",
        {
            "name": "vectome-fingerprint",
            "kwargs": {"ndim": 128},
        },
    ),
})

features = pipe(data).data
```

### Deep-learning transforms

Install with:

```bash
pip install "aspect-data[deep]"
```

The `hf-bart` transform supports transformer-derived text embeddings.

```python
data = {
    "description": [
        "cell wall inhibitor",
        "protein synthesis inhibitor",
        "DNA gyrase inhibitor",
    ],
}

pipe = DataPipeline({
    "text_embedding": (
        "description",
        {
            "name": "hf-bart",
            "kwargs": {
                "ref": "facebook/bart-base",
                "aggregator": ["mean", "max"],
            },
        },
    ),
})

features = pipe(data).data
```

Model weights may be downloaded on first use.

### Chemprop transforms

Install with:

```bash
pip install "aspect-data[chemprop]"
```

The `chemprop-mol` transform prepares molecular datapoints for Chemprop-style graph models.

Molecular graphs generally require model-specific batching. Aspect separates dataset storage from runtime collation so graph representations can be reconstructed when needed.

---

## Saving and loading pipelines

Aspect supports reusable feature specifications and provenance-aware dataset checkpoints.

### Save a feature specification

Create a pipeline without applying it to any dataset:

```python
pipe = DataPipeline({
    "log_mwt": ("mwt", "log"),
    "assay_hash": (
        "assay",
        {"name": "hash", "kwargs": {"ndim": 16}},
    ),
})

pipe.save("feature-spec.as")
```

Restore and apply it later:

```python
from aspect.data import DataPipeline

pipe = DataPipeline.load("feature-spec.as")

result = pipe({
    "mwt": [46.07, 60.10],
    "assay": ["MIC", "IC50"],
})
```

### Checkpoint a transformed dataset

```python
pipe = DataPipeline({
    "log_mwt": ("mwt", "log"),
})

result = pipe("compounds.csv")
pipe.save("features.as")
```

The checkpoint records the pipeline configuration and a Providence checkpoint describing the input dataset.

Load it again:

```python
restored = DataPipeline.load("features.as")

features = restored.data_out.data
```

By default, when transformed data has not been explicitly saved, loading reconstructs it by applying the feature pipeline to the restored source dataset.

### Materialise transformed data

```python
pipe.save(
    "features.as",
    save_transformed=True,
)
```

This stores a transformed-data checkpoint alongside the source checkpoint.

### Package source data

```python
pipe.save(
    "features.as",
    save_source_data=True,
    save_transformed=True,
)
```

This requests packaging of the input data and transformed data.

For remote datasets with immutable revisions, Providence can instead retain source references, allowing checkpoints to be substantially smaller.

Local sources are normally materialised to preserve reproducibility.

### Load without applying transformations

```python
pipe = DataPipeline.load(
    "features.as",
    skip_call=True,
)
```

This restores the specification and available checkpoint artefacts without automatically running the transformation pipeline.

### Checkpoint layout

A checkpoint may contain:

```text
features.as/
├── config.json
├── data.pv/
└── transformed.pv/
```

`config.json` contains the pipeline specification and checkpoint metadata.

`data.pv` contains the Providence source checkpoint.

`transformed.pv` is included when transformed data is explicitly checkpointed.

The contents of Providence checkpoints depend on whether datasets are retained by reference or materialised.

### Compatibility

The legacy methods `save_checkpoint()` and `load_checkpoint()` remain available as compatibility aliases, but new code should use `save()` and `load()`.

---

## Supported input and output formats

### Input types

Aspect accepts the input types supported by Providence, including:

| Input | Example |
|---|---|
| Python mapping | `{"smiles": ["CCO", "CCN"]}` |
| Pandas DataFrame | `pipe(df)` |
| Hugging Face Dataset | `pipe(dataset)` |
| Hugging Face IterableDataset | `pipe(iterable_dataset)` |
| Local file | `pipe("data.csv")` |
| Hugging Face dataset reference | `pipe("hf://datasets/org/repo~config:split")` |
| Providence DataResolver | `pipe(resolved)` |

### Local files

Commonly supported formats include:

- CSV: `.csv`, `.csv.gz`
- Tab-separated: `.tsv`, `.tsv.gz`, `.txt`, `.txt.gz`
- JSON: `.json`
- Parquet: `.parquet`
- Arrow: `.arrow`
- XML: `.xml`
- Saved Hugging Face datasets: `.hf`

### Output formats

The CLI and `write_table()` use Providence to select an output format from the filename.

| Extension | Format |
|---|---|
| `.csv` | CSV |
| `.csv.gz` | Gzip-compressed CSV |
| `.tsv`, `.txt` | Tab-separated text |
| `.tsv.gz`, `.txt.gz` | Gzip-compressed tab-separated text |
| `.json` | JSON |
| `.parquet` | Parquet |
| `.hf` | Hugging Face dataset directory |
| Unrecognised | Hugging Face dataset directory with `.hf` appended |

### Hugging Face Hub references

Providence supports versioned dataset references:

```text
hf://datasets/<owner>/<repo>~<config>:<split>@<revision>
```

For example:

```text
hf://datasets/scbirlab/fang-2023-biogen-adme~scaffold-split:train
```

The revision, configuration, and split are optional where supported by the resolver.

Pinning a dataset to an immutable commit revision improves reproducibility.

---

## Working with output columns

By default, Aspect retains original columns and adds transformed features.

```python
out = pipe(data)
```

To retain only transformed features and explicitly selected metadata:

```python
pipe = DataPipeline(
    {
        "log_mwt": ("mwt", "log"),
        "assay_hash": ("assay", "hash"),
    },
    columns_to_keep=["compound_id"],
)

out = pipe(
    data,
    drop_unused_columns=True,
)
```

Extra columns can also be specified at call time:

```python
out = pipe(
    data,
    drop_unused_columns=True,
    keep_extra_columns=["compound_id", "target"],
)
```

Access the underlying output dataset using:

```python
dataset = out.data
print(dataset.features)
print(dataset.column_names)
```

---

## Custom transforms

Custom transforms can be registered through Aspect's transform registry.

A transform factory receives configuration arguments and returns a callable that operates on data batches.

```python
import numpy as np

from aspect.data import DataPipeline
from aspect.transform.registry import register_function


@register_function("square")
def Square():
    def _square(data, input_column):
        x = np.asarray(data[input_column], dtype=float)
        return x ** 2

    return _square


pipe = DataPipeline({
    "mwt_squared": ("mwt", "square"),
})

out = pipe({
    "mwt": [46.07, 60.10, 78.11],
}).data
```

### Parameterised transforms

```python
@register_function("scale")
def Scale(factor=1.0):
    def _scale(data, input_column):
        x = np.asarray(data[input_column], dtype=float)
        return x * factor

    return _scale


pipe = DataPipeline({
    "scaled_mwt": (
        "mwt",
        {"name": "scale", "kwargs": {"factor": 0.001}},
    ),
})
```

Custom transforms must be registered in the Python process before loading pipelines that reference them.

---

## Model integration

Aspect separates stored feature representations from model-specific runtime inputs.

The output dataset may contain numeric arrays, chemical fingerprints, taxonomic encodings, or other Arrow-compatible values.

For simple numerical models, extract the dataset:

```python
features = pipe(data).data
```

For PyTorch models, use the built-in DataLoader integration:

```python
loader = pipe.dataloader(
    batch_size=32,
    shuffle=True,
)

for batch in loader:
    # Batched, collated features
    pass
```

This requires the `torch` extra.

Transforms requiring specialised runtime representations can provide collators. These are applied when constructing model batches, allowing features such as molecular graphs to be stored independently from the framework-specific objects consumed by a model.

Custom collators can be provided through the `collators` argument:

```python
loader = pipe.dataloader(
    batch_size=32,
    collators={
        "custom_feature": custom_collator,
    },
)
```

### Mixed molecular and biological features

Aspect can combine multiple representations in one dataset:

```python
data = {
    "compound_id": ["cmpd-1", "cmpd-2", "cmpd-3"],
    "smiles": ["CCO", "CCN", "c1ccccc1"],
    "assay": ["solubility", "clearance", "microsome"],
    "species": [
        "Escherichia coli",
        "Mycobacterium tuberculosis",
        "Staphylococcus aureus",
    ],
    "mwt": [46.07, 45.08, 78.11],
    "label": [0.1, 0.4, 0.8],
}

pipe = DataPipeline(
    {
        "morgan": ("smiles", "morgan-fingerprint"),
        "desc2d": ("smiles", "descriptors-2d"),
        "assay_hash": (
            "assay",
            {"name": "hash", "kwargs": {"ndim": 32}},
        ),
        "species_fp": (
            "species",
            {"name": "vectome-fingerprint", "kwargs": {"ndim": 64}},
        ),
        "log_mwt": ("mwt", "log"),
        "y": ("label", "identity"),
    },
    columns_to_keep=["compound_id"],
)

features = pipe(
    data,
    drop_unused_columns=True,
).data
```

This requires the chemistry and taxonomy extras.

Downstream models can independently consume molecular fingerprints, descriptors, categorical features, and taxonomic encodings, or combine suitable dense features into a common representation.

---

## Development

Install the development dependencies:

```bash
pip install -e ".[dev]"
```

Run the test suite:

```bash
pytest
```

Run module doctests:

```bash
pytest aspect --doctest-modules
```

Run the CLI integration tests:

```bash
bash test/scripts/run-tests.sh
```

Aspect maintains separate GitHub Actions workflows for Python tests and CLI smoke tests.

## Related packages

Aspect is part of a small ecosystem for reproducible molecular machine-learning workflows:

- [Providence](https://github.com/scbirlab/providence) — Dataset loading, source provenance, and checkpointing.
- [Eluent](https://github.com/scbirlab/eluent) — Chemistry-aware grouping, splitting, and cross-validation.
- [Schemist](https://github.com/scbirlab/schemist) — Molecular featurisation and cheminformatics.
- [Vectome](https://github.com/scbirlab/vectome) — Taxonomic and biological representations.

The packages can be used independently or composed into reproducible workflows from source data to model-ready features and evaluation splits.

---

## Issues and documentation

Bug reports, feature requests, and contributions are welcome through the [GitHub issue tracker](https://github.com/scbirlab/aspect/issues).

Documentation: [aspect.readthedocs.org](https://aspect.readthedocs.org)

Source code: [github.com/scbirlab/aspect](https://github.com/scbirlab/aspect)

License: MIT.
