Metadata-Version: 2.4
Name: GenoKit
Version: 1.0.0
Summary: A versatile command-line toolkit for comprehensive genomic feature extraction, analysis, and visualization
Home-page: https://github.com/SitaoZ/GenoKit.git
Author: zhusitao
Author-email: zhusitao1990@163.com
License: MIT
Keywords: genomic feature,extract
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas>=2.2.3
Requires-Dist: setuptools>=72.1.0
Requires-Dist: biopython>=1.86
Requires-Dist: python-louvain>=0.16
Requires-Dist: python-circos>=0.3.0
Requires-Dist: tabulate>=0.9.0
Requires-Dist: tqdm>=4.0
Requires-Dist: Django>=5.2.8
Requires-Dist: pyfaidx>=0.9
Requires-Dist: matplotlib>=3.10.0
Requires-Dist: requests>=2.32.3
Requires-Dist: networkx>=3.6.1
Dynamic: author
Dynamic: author-email
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: keywords
Dynamic: license
Dynamic: license-file
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# GenoKit

GenoKit is a Python command-line toolkit for genome annotation databases,
genomic feature extraction, sequence design, and visualization. This README
describes the **1.0.0 source implementation**.

The main command, `GenoKit`, works with GFF3/GTF annotations and reference FASTA
files. Annotation databases use SQLite by default, with lazy access to gene
models and compatibility with historical pickle databases.

## Installation

Install the published package:

```bash
python -m pip install GenoKit
```

Or install this source checkout:

```bash
git clone https://github.com/SitaoZ/GenoKit.git
cd GenoKit
python -m pip install .
```

For editable development, use `python -m pip install -e .` instead. The published
package may differ from this checkout; inspect `GenoKit --version` and the
installed command's `-h` output when using another release.

### Requirements

The requirements declared in `setup.py` are:

| Dependency | Minimum version |
| --- | --- |
| Python | 3.10 |
| pandas | 2.2.3 |
| setuptools | 72.1.0 |
| biopython | 1.86 |
| python-louvain | 0.16 |
| python-circos | 0.3.0 |
| tabulate | 0.9.0 |
| tqdm | 4.0 |
| Django | 5.2.8 |
| pyfaidx | 0.9 |
| matplotlib | 3.10.0 |
| requests | 2.32.3 |
| networkx | 3.6.1 |

`pip` resolves dependency-specific Python requirements as well as GenoKit's
Python requirement. `gffutils` is no longer a declared dependency.

Jellyfish is an external executable used for genome-scale k-mer counting and
querying `.jf` datasets. Supply its path with `-j` to `GenoKit kmer`, or with
`--jellyfish` to a design command. The Python k-mer engine supports small inputs
up to 250 MiB and k values from 1 to 31.

## Command overview

```bash
GenoKit -h
GenoKit --version
GenoKit create -h
```

Use `GenoKit <command> -h` for the complete options of any command. Subcommand
names and annotation styles are lowercase.

| Group | Command | Purpose |
| --- | --- | --- |
| Database | `create` | Build a GFF3/GTF annotation database |
| Database | `fasta` | Extract selected FASTA records |
| Database | `iupac` | Display nucleotide, amino-acid, or codon tables |
| Database | `kmer` | Build genome/transcriptome k-mer catalogues |
| Database | `stat` | Summarize annotation and genome statistics |
| Extract | `gene` | Extract genomic gene sequences |
| Extract | `transcript` | Extract spliced transcript sequences |
| Extract | `exon` | Extract transcript exon sequences |
| Extract | `intron` | Extract intron sequences |
| Extract | `cds` | Extract coding sequences |
| Extract | `utr` | Extract 5′ and 3′ untranslated regions |
| Extract | `uorf` | Extract upstream open reading frames |
| Extract | `dorf` | Extract downstream open reading frames |
| Extract | `promoter` | Extract regions around the gene's 5′ boundary |
| Extract | `terminator` | Extract regions around the gene's 3′ boundary |
| Extract | `igr` | Extract intergenic regions |
| Design | `barcode` | Generate DNA or RNA barcodes |
| Design | `motif` | Search sequence motifs |
| Design | `primer` | Design primer pairs on genomic DNA or cDNA |
| Design | `sgrna` | Design CRISPR single-guide RNAs |
| Design | `shrna` | Design small hairpin RNAs |
| Design | `sirna` | Design small interfering RNAs |
| Visualize | `view` | Draw annotated gene structures |
| Visualize | `circos` | Draw circular genome plots |
| Visualize | `ppi` | Analyze and draw protein interaction networks |
| Visualize | `hgvs` | Visualize DNA, RNA, and protein variants |

`mrna` is not an active command; use `transcript` for spliced sequences. The
reserved `expp` parser is not exposed by the public command dispatcher.

## Quick start

The examples below assume that `human.gtf` and `human.fa` describe the same
assembly and use matching chromosome identifiers. Replace the example paths
and gene/transcript IDs with those from your annotation.

```bash
mkdir -p results

# Build the annotation database and check/build the FASTA index.
GenoKit create -s gtf -a human.gtf -g human.fa -o results -p human

# Summarize the annotation and genome.
GenoKit stat -d results/human.gtf.sqlite -g human.fa -s gtf -o results/stat.csv

# Extract all CDS sequences.
GenoKit cds -d results/human.gtf.sqlite -g human.fa -s gtf \
  -f fasta -o results/cds.fa

# Extract all mature transcript sequences.
GenoKit transcript -d results/human.gtf.sqlite -g human.fa -s gtf \
  -f fasta -o results/transcripts.fa
```

### Input and output conventions

- Use `-s gtf` for GTF or `-s gff` for GFF3. Supply the matching style explicitly
  in annotation-based extraction and design commands.
- Gene selectors (`-i` on `gene`, `promoter`, `terminator`, and design commands)
  take a **gene ID**. Transcript selectors (`-i` on `transcript`, `cds`, `exon`,
  `intron`, `utr`, `uorf`, and `dorf`) take a **transcript ID**. A reported gene
  symbol is not a replacement for the annotation ID.
- Extraction commands accept `-f csv`, `fasta`, `gff`, or `gtf`. Specify `-f`
  explicitly, especially for `gene` and `terminator`, whose parsers do not set
  a format default. Sequence-design commands advertise `csv` and `fasta`;
  examples here use CSV.
- Supply `-o` for whole-annotation extraction and design, and create its parent
  directory first. Omitting `-i` processes all applicable records.
- For a single selected gene/transcript, use `--print` instead of `-o` to print
  results. Its short form is `-p` for `gene` and `-v` for the other extraction
  commands. Top-level `GenoKit -v` means version.
- The old extraction options `--process` and `--rna_feature` are no longer
  accepted. `-p` and `-r` have other meanings on commands such as `create`,
  `sgrna`, `shrna`, and `kmer`.
- Feature coordinates follow the annotation's 1-based inclusive convention.
  Spliced sequences are assembled from exons and oriented 5′→3′, including
  reverse complementation for negative-strand transcripts. ORF outputs also
  contain transcript-relative coordinates; read the relevant column headers.

## Database commands

### `create`

```bash
# GTF, with SQLite output (default).
GenoKit create -s gtf -a human.gtf -g human.fa -o results -p human
# Output: results/human.gtf.sqlite

# GFF3; gzip-compressed annotation is accepted.
GenoKit create -s gff -a yeast.gff.gz -g yeast.fa -o results -p yeast
# Output: results/yeast.gff.sqlite

# Historical in-memory/pickle backend.
GenoKit create -s gtf -a human.gtf -g human.fa -o results -p human \
  --database-format pickle
# Output: results/human.gtf.pkl
```

Annotation input may be uncompressed or ordinary gzip. Reference FASTA must be
uncompressed or **BGZF/bgzip-compressed**, not ordinary gzip, for indexed random
access. By default, `create` checks/builds the `.fai` index and the `.gzi` index
needed for BGZF FASTA. `--skip-fasta-index` skips this step; subsequent sequence
access still needs a usable index.

The builder checks annotation content for a GFF3/GTF mismatch with `-s`.
Readers detect SQLite by file signature and otherwise load the historical
pickle representation, including existing `.pkl` and `.bin` files. New pickle
output uses `.pkl`. SQLite construction inserts records incrementally, and
readers do not deserialize the complete annotation at startup.

### `stat`, `fasta`, and `iupac`

```bash
# Print statistics, or add -o results/stat.csv to save them.
GenoKit stat -d results/human.gtf.sqlite -g human.fa -s gtf

# chromosomes.txt contains one FASTA record ID per line, without a header.
GenoKit fasta -f human.fa -i chromosomes.txt -o results/selected.fa

# Select one table per invocation.
GenoKit iupac -n
GenoKit iupac -a
GenoKit iupac -c
```

### `kmer`

A single `genokit.kmer.sqlite` catalogue can register multiple k values for
both `genome` and `transcriptome` sources. Its datasets can be SQLite counts or
external Jellyfish `.jf` files. Keep registered `.jf` files available when
moving or sharing a catalogue.

```bash
# Genome catalogue: complete primer and RNA-target lengths.
GenoKit kmer -f human.fa --source-type genome --lengths 18-25 \
  -o results/kmer -r --engine jellyfish -j jellyfish -t 4

# Add spliced-transcript counts to the same catalogue.
GenoKit kmer --source-type transcriptome \
  -d results/human.gtf.sqlite -g human.fa --lengths 18-25 \
  -o results/kmer -r --engine jellyfish -j jellyfish -t 4

# A small FASTA can use the Python engine.
GenoKit kmer -f small.fa --source-type genome -l 20 \
  -o results/small_kmer -r --engine python

# Register an existing Jellyfish dataset without recounting.
GenoKit kmer --register-jf human.fa.kmer.20.jf \
  --source-type genome -o results/kmer -r -j jellyfish
```

`--engine` accepts `auto` (default), `jellyfish`, or `python`. `-l` selects a
single k value (default 18); `--lengths` accepts comma-separated values and
ranges. `-r` enables reverse-complement counting. `--sqlite-db` overrides the
catalogue path, which defaults to `OUTPUT_DIR/genokit.kmer.sqlite`.
For a prebuilt mature-transcript FASTA, use `-f` with
`--source-type transcriptome` instead of `-d` and `-g`.

## Feature extraction

### Genes, transcripts, CDS, exons, introns, and UTRs

```bash
GenoKit gene -d results/human.gtf.sqlite -g human.fa -s gtf \
  -f csv -o results/genes.csv
GenoKit transcript -d results/human.gtf.sqlite -g human.fa -s gtf \
  -f fasta -u -o results/transcripts_marked.fa
GenoKit cds -d results/human.gtf.sqlite -g human.fa -s gtf \
  -f gff -o results/cds.gff
GenoKit exon -d results/human.gtf.sqlite -g human.fa -s gtf \
  -f gtf -o results/exons.gtf
GenoKit intron -d results/human.gtf.sqlite -g human.fa -s gtf \
  -f fasta -o results/introns.fa
GenoKit utr -d results/human.gtf.sqlite -g human.fa -s gtf \
  -f csv -o results/utr.csv

# Print one gene or transcript; replace these IDs with IDs in your database.
GenoKit gene -d results/human.gtf.sqlite -g human.fa -s gtf \
  -i ENSG00000198062 -f fasta --print
GenoKit transcript -d results/human.gtf.sqlite -g human.fa -s gtf \
  -i ENST00000343518 -f fasta --print
```

`transcript -u/--upper` marks CDS in uppercase and UTRs in lowercase. `utr`
extracts both 5′ and 3′ UTRs. CSV output for annotation-backed features includes
gene IDs and gene symbols alongside feature-specific columns.

### Promoters, terminators, and intergenic regions

```bash
GenoKit promoter -d results/human.gtf.sqlite -g human.fa -s gtf \
  -l 1000 -u 10 -f csv -o results/promoters.csv
GenoKit terminator -d results/human.gtf.sqlite -g human.fa -s gtf \
  -l 1000 -u 10 -f fasta -o results/terminators.fa
GenoKit igr -d results/human.gtf.sqlite -g human.fa -s gtf \
  -f gff -o results/intergenic.gff
```

`promoter` uses the gene's strand-aware 5′ boundary: `-l` is the upstream
length (default 100), and `-u` is the length into the gene (default 10).
`terminator` uses the strand-aware 3′ boundary: `-l` extends downstream
(default 100), and `-u` includes the end of the gene (default 10). These commands
operate on gene boundaries, not on each isoform's separate boundary.

### Upstream and downstream ORFs

```bash
GenoKit uorf -d results/human.gtf.sqlite -g human.fa -s gtf \
  -l 6 -f csv -o results/uorf.csv
GenoKit dorf -d results/human.gtf.sqlite -g human.fa -s gtf \
  -l 6 -f gff -o results/dorf.gff
```

`-l/--length` is the ORF length threshold in nucleotides (default 6). The
extraction output filters out ORFs whose length is less than or equal to this
threshold. uORFs are classified as upstream, overlapping, or N-terminal
extension candidates. ORF GFF/GTF output maps spliced coordinates back to the
genome.

For a selected transcript, the parsers also accept
`-m/--schematic_without_intron PREFIX` and
`-n/--schematic_with_intron PREFIX`; these options take string values, not
boolean flags. For gene-level structure plots, use `view` below.

## Sequence design

### `primer`

Choose the actual PCR template with `--template-type genomic|cdna` (default
`cdna`). Template selection controls both sequence extraction and the expected
k-mer reference:

| Template type | Design sequence | Matching `--kmer-source` |
| --- | --- | --- |
| `genomic` | One continuous gene interval, including introns, oriented in the gene's 5′→3′ direction | `genome` |
| `cdna` | Each mature transcript, assembled from exons in 5′→3′ orientation | `transcriptome` |

```bash
# Genomic DNA: search within the gene interval, including introns.
GenoKit primer -d results/human.gtf.sqlite -g human.fa -s gtf \
  --template-type genomic --kmer-source genome \
  -k results/kmer/genokit.kmer.sqlite -x 100 -y 1000 \
  -f csv -o results/genomic_primers.csv

# cDNA: design from mature transcript sequences.
GenoKit primer -d results/human.gtf.sqlite -g human.fa -s gtf \
  --template-type cdna --kmer-source transcriptome \
  -k results/kmer/genokit.kmer.sqlite \
  -a 0.4 -b 0.6 -c 50 -e 70 -n 18 -m 25 -x 100 -y 1000 \
  -f csv -o results/cdna_primers.csv
```

If `--kmer-source` is omitted, it is selected automatically from the template
type, including when both sources coexist in one catalogue. An explicitly
conflicting source is rejected before opening input files. `-k` is optional;
without it, templates are still selected correctly and `Kmer Freq` is `NA`.
When supplied, the catalogue must contain the matching biological source,
canonical reverse-complement counts (build with `GenoKit kmer -r`), and exact
primer-length k values. Candidates are restricted to available primer lengths
and must occur in the selected reference. Raw legacy k-mer files lack source
metadata; use a SQLite catalogue to validate their biological source.

GC bounds (`-a/-b`) are fractions, Tm bounds (`-c/-e`) default to 50–70, primer
lengths (`-n/-m`) to 18–25 nt, and product sizes (`-x/-y`) to 100–1000 bp.
Product size includes both primers and any intervening introns in genomic
mode. Genomic mode searches the full gene interval, and cDNA mode searches the
full mature transcript for an internal amplicon. Add
`-i GENE_ID` to select one gene and display its primer plot. `--print` writes
CSV or FASTA to stdout instead of a file and cannot be combined with `-o`.

CSV output adds `Template Type`; `Transcript ID` is empty for genomic templates.
`Template Seq` replaces the former `Transcript Seq` column. `Start` and `End`
are **0-based, half-open offsets into the oriented template**, not chromosome
coordinates. `Source` identifies the forward or reverse primer; all primer
sequences are written 5′→3′. `Fragment` reports the complete amplicon length.
FASTA descriptions also identify the template type and template-relative
positions.

`--specificity-mode` and its `--seed-size` option have been removed. A seed-only
k=12 dataset is no longer required by primer design. Template choice does not
impose exon-junction targeting or perform mismatch-aware off-target alignment.

### `sgrna`, `sirna`, and `shrna`

Both `sirna` and `shrna` use `--target-mode {all,any}` (default: `all`).
`all` requires each candidate to target every input transcript of the same gene;
`any` requires at least one transcript, not a majority. The modes apply to both
selected-gene (`-i`) and whole-database design. Outputs list actual hit transcript IDs.
The old shRNA `--all` and `--common` CLI options have been removed. Replace `--all`
with `--target-mode all`; `any` is broader than the former `--common` majority filter.

Both commands accept `--region {cds,transcript}`, default `cds`. CDS mode joins
annotated CDS segments and falls back to exons for transcripts without CDS;
transcript mode joins exons including UTRs. Negative-strand sequences are reverse
complemented. For shRNA, pass `--region transcript` to retain the previous
full-transcript behavior. GTF stop_codon records are not separately appended
in CDS mode, matching siRNA. Target mode is applied to the selected sequences.

For siRNA, short or unscannable transcripts remain in the `all` coverage set.
Ambiguous bases are not deleted: windows containing non-ATCG bases are skipped,
so candidates cannot bridge artificial junctions and positions remain valid.
Coverage is evaluated on the selected region (`--region cds` or `transcript`);
noncoding transcripts continue to fall back to exons in CDS mode.

```bash
GenoKit sgrna --pam
GenoKit sgrna -d results/human.gtf.sqlite -g human.fa -s gtf \
  -p NGG -l 20 -k results/kmer/genokit.kmer.sqlite \
  -f csv -o results/sgrna.csv
GenoKit sirna -d results/human.gtf.sqlite -g human.fa -s gtf \
  -l 21 --region cds --target-mode all --top 10 \
  -k results/kmer/genokit.kmer.sqlite -f csv -o results/sirna.csv
GenoKit shrna -d results/human.gtf.sqlite -g human.fa -s gtf \
  -l 21 -p CTCGAG --region cds --target-mode all -k results/kmer/genokit.kmer.sqlite \
  -f csv -o results/shrna.csv
```

| Command | Main options | K-mer source |
| --- | --- | --- |
| `sgrna` | `-p/--pam_pattern` defaults to `NGG`; `-l` defaults to 20; `--pam` prints a reference table | genome |
| `sirna` | `-l` is 19, 20, or 21 (default 21); `--region cds|transcript`; `--target-mode all|any`; optional `--top` per gene | transcriptome |
| `shrna` | `-l` is 19, 20, or 21 (default 21); `-p/--loop` accepts an ATCG sequence or `standard`, `miR30`, `simple`; `--region cds|transcript`; `--target-mode all|any` | transcriptome |

Add `-i GENE_ID` to limit design to one gene. K-mer catalogues are optional;
when supplied, build the matching target length and biological source first.
`--jellyfish PATH` overrides the executable used to query registered `.jf`
files. K-mer frequency screening measures sequence occurrence; it is not a
complete mismatch-aware off-target alignment analysis.

### `barcode` and `motif`

```bash
GenoKit barcode -l 8 -n 20 -s dna -f csv -o results/barcodes.csv
GenoKit barcode -l 10 -n 20 -s rna -f fasta -o results/barcodes.fa

GenoKit motif -i sequences.fa -m CTACGAAAG -t dna -b -o results/motif.csv
GenoKit motif -i sequences.fa -f patterns.txt -t dna -x 2 -o results/motifs.csv
GenoKit motif -i sequences.fa -m 'CTACGAAAG|AAAAAA' -e -o results/regex.csv
```

For `barcode`, supply both `-l` and `-s dna|rna`; `-n` defaults to 20. The CLI
uses GC bounds of 0.4–0.6 and a maximum homopolymer length of 2.

For `motif`, provide `-m` for one pattern or `-f` for a pattern file (one pattern
per line). `-b` searches both strands, `-c` enables case sensitivity, `-d` enables
IUPAC degenerate bases, `-e` enables regular expressions, and `-x` sets the
maximum mismatches (default 0). `-t` selects `dna` (default) or `rna`.

## Visualization

### `view`

```bash
GenoKit view -d results/human.gtf.sqlite -g human.fa -s gtf \
  -i ENSG00000198062 -l 6 --orf-label-limit 20 -f pdf -o results/gene_view
```

Draws transcript structures with exon, CDS, UTR, and ORF features. `-o` is an
output directory. Formats are `pdf` (default), `png`, and `tiff`.
`--orf-label-limit 0` hides individual ORF numbers. Omitting `-i` processes all
genes.

### `circos`

```bash
GenoKit circos -a example/chromosome_length.csv \
  -b example/barplot_data.csv -m example/heatmap_data.csv \
  -k example/link_data.csv -f pdf -o results/circos.pdf
```

The `example/...` paths are resolved against the current directory first, then
against the package's example directory. Track inputs are comma-separated files
with a header row:

| Option | Input columns |
| --- | --- |
| `-a/--chrom` | `chr,start,end` (last column supplies chromosome length) |
| `-c/--cytoband` | chromosome, start, end, band name, stain |
| `-b/--bar`, `-p/--point`, `-l/--line`, `-m/--heatmap` | `chr,start,end,value` |
| `-k/--link` | `chr1,start1,end1,chr2,start2,end2` |

`-t/--track` detects a line or link track from its header. Use an output filename
with the desired extension; advertised formats are `pdf`, `png`, and `tiff`.

### `ppi`

```bash
GenoKit ppi -i interactions.csv -s comma -l auto -o results/ppi
```

Input contains two protein identifiers per line, **without a header**. Set
`-s comma` or `-s tab` explicitly. For example:

```text
TP53,MDM2
TP53,ATM
MDM2,RPL11
```

Layouts are `auto`, `fr`, `kk`, `forceatlas2`, `circle`, `star`, `tree`, `grid`,
`shell`, `spectral`, `spiral`, and `random`. The output directory receives network
and centrality PDFs, a centrality table, a GEXF network, and a log. The
`ppi_centrality.csv` file is currently tab-separated despite its extension.

### `hgvs`

```bash
# variants.txt contains one accession-qualified HGVS expression per line.
GenoKit hgvs -i variants.txt -o results/hgvs

# Use local reference data instead of downloading from NCBI.
GenoKit hgvs -i variants.txt --genbank reference.gb -o results/hgvs_local
```

Use `-s/--string` for a single accession-qualified HGVS expression, or `-i` for
an input file. The file reader also accepts the first column of CSV/TSV input
and skips recognized `hgvs`, `variant`, or `variation` headers. Local GenBank or
GenPept data must match the supplied accession and coordinate type.

The parser accepts DNA coordinate types `g.`, `c.`, `n.`, `m.`, `o.`, RNA `r.`,
and protein `p.`. `--variation` optionally validates the expected variant kind:
`sub`, `del`, `ins`, `delins`, `dup`, `inv`, `rpt`, `alleles`, `ext`, or `fs`.
Supported forms depend on the coordinate type; this is not a claim of complete
HGVS grammar coverage.

References are fetched from NCBI when no local reference is supplied and cached
under `.genokit_cache/hgvs` by default (`--cache-dir` overrides this).
`--email`/`NCBI_EMAIL` and `--api-key`/`NCBI_API_KEY` configure NCBI requests.
Output can be PNG, PDF, or SVG; multiple inputs use an output directory or
prefix. `--context` defaults to 20 and `--dpi` to 200. Diagnostic overrides are
`--allow-reference-mismatch` and `--skip-three-prime-check`; inspect `-h` before
using them.

## Legacy GenBank entry point

`setup.py` also installs **`GenoKitGB`**. It does not install an executable named
`genBankExtract`; that name survives in legacy help text.

The current `genokitgb.py` still imports old module paths such as
`GenoKit.extract_CDS` and `GenoKit.extract_gene`, which do not exist in this
source layout. Its feature-extraction commands therefore need code maintenance
before use. Historical `genBankExtract` examples are not current working
commands. The main `GenoKit hgvs --genbank` option is a separate implementation.

## Project information

- Source: [SitaoZ/GenoKit](https://github.com/SitaoZ/GenoKit)
- Author: Sitao Zhu
- Contact: zhusitao1990@163.com
- License declared in package metadata: MIT
