Metadata-Version: 2.4
Name: NextPyper
Version: 0.1.0
Summary: Recovery of homoeologous loci from target capture data in higher ploidy samples
Author-email: Simón Villanueva Corrales <simon.corrales@ibot.cas.cz>, "Yann J.K. Bertrand" <Yann.Bertrand@ibot.cas.cz>
License-Expression: MIT
Project-URL: Homepage, https://github.com/M2BL/NextPyper
Project-URL: Documentation, https://m2bl.github.io/NextPyper
Classifier: Environment :: Console
Classifier: Environment :: MacOS X
Classifier: Intended Audience :: Science/Research
Classifier: Natural Language :: English
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: nextpyper/nextpyper.LICENSE
Requires-Dist: snaketool-utils>=0.0.4
Requires-Dist: snakemake>=9.21
Requires-Dist: pyyaml>=6.0
Requires-Dist: Click>=8.1.3
Requires-Dist: rich-click>=1.9.7
Requires-Dist: pandas>=2.2
Requires-Dist: biopython>=1.83
Requires-Dist: numpy>=1.26
Requires-Dist: scikit-learn>=1.5
Requires-Dist: scipy>=1.14
Requires-Dist: polars>=1.32.3
Requires-Dist: intervaltree>=3.1
Requires-Dist: kmedoids>=0.5.3.1
Requires-Dist: more-itertools>=10.7
Dynamic: license-file

# NextPyper
Recovery of homoeologous loci from target capture data in higher ploidy samples.

## Documentation

Please refer to the [official documentation](https://m2bl.github.io/NextPyper/), which is more complete.

## Table of contents

- [Installation](#installation)
  - [Conda/Mamba](#condamamba)
  - [Pip](#pip)
  - [Manual](#manual)
- [Environments setup (recommended)](#environments-setup-recommended)
- [Usage](#usage)
  - [1. Preparing your probes](#1-preparing-your-probes)
    - [Single-probe sets](#single-probe-sets)
    - [Multi-probe sets](#multi-probe-sets)
  - [2. Preparing your samples](#2-preparing-your-samples)
  - [3. Running NextPyper](#3-running-nextpyper)
- [Output](#output)
  - [Sequence versions](#sequence-versions)
  - [Output structure](#output-structure)


## Installation

Regardless of the installation method, `NextPyper` requires `conda` in your PATH, to manage and deploy workflow environments.

### Conda/Mamba

If you already have [bioconda](https://bioconda.github.io/) set up, you can create a new environment and simply run:

```bash
conda install nextpyper # or mamba
```

### Pip

Alternatively you can use pip:

```
pip install nextpyper
```

### Manual 

You can also install NextPyper manually with pip. We recommend to create a new environment.

```bash
# from your new environment with pip
git clone git@github.com:M2BL/NextPyper.git
cd NextPyper
pip install .
```

## Environments setup (recommended)

NextPyper uses [snakemake](https://snakemake.github.io/) to orchestrate and execute the different steps of the workflow, which uses conda to set up the packages and environments required at each stage.

The first time you run NextPyper, this setup will be done automatically at the start of the workflow.

Alternatively, you can set up these environments beforehand by running:

```bash
nextpyper prepare
```

This command creates the required environments within NextPyper's installation path. You can use the option `--conda-prefix` to specify a different path where to store these environments.

## Usage 

NextPyper requires two inputs, a probes file with the targets of your experiment, and a sample table describing where to find the data of your samples.

### 1. Preparing your probes

NextPyper expects a nucleotide fasta file with the sequences of the targeted loci, what we call the probe set. Probe sets can be divided into two categories:

- **Multi probe sets**: Contains multiple sequences per locus/gene (*e.g.* Angiosperms353).
- **Single probe sets**: Contains a single sequence per locus. Usually, custom probe sets designed from a single reference genome fall in this category.

Sequence names in the probe set are required to be simple (letters, numbers and underscore, characters only). Additionally no duplicated names are allowed.

Currently, only coding regions that can be translated are supported.

#### Single-probe sets

If you are using a single-probe set, you only need to worry about sticking to the naming conventions described above. To check your probes, you can use `nextpyper validate`:

```bash
$ nextpyper validate --probes probes.fasta 

# Probe sequence names comply with naming convention.
# Pattern (.*) yielded no grouping of the sequences.
# Either this is a single-probe set or the pattern is not appropriate for multi-probe mode
```

If you get the message above, your probe set is well formatted to run in single-probes mode.

#### Multi-probe sets

If using a multi-probe set, NextPyper needs to know which sequences target the same locus. The option `--pattern` expects a RegEx (with one capture group) that will inform about that hierarchy based on the name of the probes. 

For instance, for the Angiosperms353 probe set with sequence names:

```
AJFN_probe4471
Ambtr_probe4471
BERS_probe4527
ZENX_probe4527
Arath_probe4691
QUTB_probe4691
```

We can use `--pattern "(\d+)$"`, which will group the sequences by the number at the end of the sequence name (which is the LocusID).

You can use the `validate` subcommand to check if your pattern groups your probes:

```bash
$ nextpyper validate --probes kew_probes.fasta --pattern "(\d+)$"

# Probe sequence names comply with naming convention.
# The pattern yields 353 groups from 4781 probe sequences
```

Use the options `--out_summary` and `--out_hierarchy` to get more details about the grouping.

### 2. Preparing your samples

NextPyper expects a 4-column table that defines the identity and ploidy of your samples, and where to find their associated data.

Knowledge about your samples' ploidy is not required to run NextPyper. Simply set it to 0, if you do not know it, or do not want to take it into account.

The table requires a header with 4 columns (see an example here):

- sample: the name of the sample.
- path_forward: absolute path to the forward reads. It can be .gz compressed.
- path_reverse: absolute path to the reverse reads. It can be .gz compressed.
- ploidy: the ploidy of the sample. If 0, ploidy won't be taken into account to process the sample.

You can use the `gather` subcommand to help you create the sample table. From a data directory ([like this one](https://github.com/M2BL/NextPyper/tree/main/nextpyper/data/mock/reads)) with only the paired-end data (forward and reverse files) you can run:

```bash
nextpyper gather --input data_folder --output sample.tsv
```

To get the input the sample table for your samples. See an example of the result [here](https://github.com/M2BL/NextPyper/blob/main/nextpyper/data/mock/sample.tsv).

The sample names are inferred based on the common prefix between the forward and reverse files. 

### 3. Running NextPyper

Finally, use the `run` subcommand to start the workflow. For single-probes you do not need to specify a pattern, so simply provide the inputs we prepared:

```bash
nextpyper run --threads 16 --single-probes --input input_samples.tsv --probes custom_probes.fasta --output out_single
```

For multi-probes, specify the appropriate pattern as well:

```bash
nextpyper run --threads 16 --input new_hier_samples.tsv --probes kew_probes.fasta --pattern "(\d+)$" --output out_multi
```


## Output

### Sequence versions

The main output of NextPyper are the samples' homologous sequences for each targeted locus. We generate 3 versions of these sequences, depending on the final postprocessing:

- **Exons**: includes only the exonic part of the sequence.
- **Genetigs**: includes exons and introns.
- **Supercontigs**: The full sequence, including exons, introns and flanking regions. 

The exons are the shorter and more conserved version of the output sequences, while the supercontigs are longer and include all discovered variation (which can be a bit noisy). Genetigs usually provide a good balance, containing more signal than the exons without it being overwhelming. You may want to explore which type of sequence suits better your purposes.

### Output structure

NextPyper organizes its main outputs in the following fashion:

- Homologous sequences per sample: `<outdir>/outputs/seqs_per_sample`
- Homologous sequences per locus: `<outdir>/outputs/seqs_per_locus`
- Loci Multiple Sequence Alignments (MSAs): `<outdir>/outputs/alignments`

NextPyper output directory follows the following structure:

```
outdir
├── config.yaml      # Final config with parameters used to run the workflow.
├── execution.log    # Snakemake execution log of all workflow jobs.
├── logs/            # Logs generated by the workflow jobs.
├── nextpyper.log    # Summary of params and snakemake invocation.
├── outputs/         # Final outputs of the workflow.
└── workflow/        # Working folder for all the jobs and stages of the pipeline. It contains intermediate and final files.
```
