Metadata-Version: 2.4
Name: libsize
Version: 0.1.0
Summary: Estimating how many distinct types a library or population contains
Author-email: David Bikard <david.bikard@pasteur.fr>
License-Expression: Apache-2.0
Project-URL: Repository, https://gitlab.pasteur.fr/synbiolab/libsize
Keywords: bioinformatics,library complexity,rarefaction,extrapolation,directed evolution,unseen species
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Classifier: Development Status :: 3 - Alpha
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy
Requires-Dist: scipy
Provides-Extra: dgrec
Requires-Dist: dgrec-tools[lstm]; extra == "dgrec"
Provides-Extra: dev
Requires-Dist: nbdev; extra == "dev"
Dynamic: license-file

# libsize


<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->

A library, a culture or an ecosystem holds some number of distinct types – genotypes,
molecules, species. Sequencing draws a sample, and the sample shows fewer types than exist
because the rare ones are missed. `libsize` estimates **U(n)**, the number of distinct types
that would be seen in a draw of `n`, from a sample much smaller than that.

It was written for DGRec mutagenesis libraries, but nothing in it is specific to them: the
same question comes up for error-prone PCR and site-saturation libraries in directed
evolution, for guide and variant libraries in massively parallel assays, and for species
richness in ecology.

## Install

``` sh
pip install libsize
```

The core needs only numpy and scipy. The DGRec LSTM adapter additionally needs the
[dgrec](https://dbikard.github.io/dgrec/) package and TensorFlow:

``` sh
pip install "libsize[dgrec]"
```

The development version lives at
[gitlab.pasteur.fr/synbiolab/libsize](https://gitlab.pasteur.fr/synbiolab/libsize):

``` sh
pip install git+https://gitlab.pasteur.fr/synbiolab/libsize.git
```

## From a sample

Given counts per type – molecules per genotype, reads per variant – fit the rank-abundance
tail and extrapolate. `G` is the number of types the process can reach; it is a hard ceiling,
so count the positions that actually vary.

``` python
from libsize import estimate

estimate(counts, G=4**14, N=10**8)
# {'library_size': 3114026, 'B': 3, 'alpha': 1.296, ...}
```

Use `method="mle"` (the default). `method="leastsq"` reproduces the fit used for the
published DGRec library sizes and is kept for comparison, not for use – it is the outlier
against every other method here, by a factor of twenty at a 2e4-fold extrapolation.

## From a model of the process

If you can model the process that generates the types, you do not have to assume a shape for
the tail at all – the model scores types it has never seen, and `U(n)` follows by importance
sampling. For a DGRec template repeat:

``` python
from libsize import DgrecLSTM, estimate_from_model

model = DgrecLSTM("TTGTGGCCTTTATCTTCTACGTAGTGAGGATCTCTCAGCGTATGGTTGTCGCCTGAGCTGTAGTTGCCTT")
estimate_from_model(model, n=93_471_023, n_samples=15_000, seed=0)
# {'library_size': 2130948.0, 'se': 181011.0, 'model': 'dgrec-lstm:57585fba7123', ...}
```

`n` counts **mutagenized molecules, not cells**: the model describes the mutational profile
given that conversion happened, not the rate of conversion.

For any other process, implement `sample(k)` and `logprob(types)` and
`estimate_from_model` works unchanged – see [generative](generative.html).

The result carries the model’s identity, and that is not decoration. Retraining the DGRec
LSTM – a change nobody treated as a change of method – moved `U(n)` at the full
extrapolation by a third. **An estimate from a model is not interpretable without naming the
model.**

## Check before you trust

An extrapolated library size is easy to get wrong by orders of magnitude. The
[bounds](bounds.html) module holds cheap checks that rule whole answers out:

``` python
from libsize import good_turing_coverage, chao1, predictable_range, class_floor

good_turing_coverage(counts)   # how much mass the sample actually covers
predictable_range(counts)      # past this depth, no sample-based answer is supported
class_floor(mols, sizes)       # a lower bound needing no extrapolation at all
```

The limit is worth stating plainly: **no estimator reading the tail from the sample can see
beyond about `t ~ log n` further samples per sample drawn**, and that bound is provably
optimal (Orlitsky, Suresh & Wu 2016). `smoothed_good_toulmin` implements the estimator that
attains it; ask it for more and it returns a negative number rather than a wrong one. Past
that range, an answer rests on an assumed functional form or on a model of the process –
not on the data.

## Status

Early. The API may move. The evidence behind the choices here – the comparison against
`preseq`, the limits, the prior art in directed evolution and immune repertoires – lives in
the `complexity-estimation-kb` knowledge base.
