Metadata-Version: 2.4
Name: dslint
Version: 0.1.0
Summary: Pre-training linter for speech datasets: catches broken training data before you rent a GPU.
License: MIT
License-File: LICENSE
Keywords: asr,data-quality,dataset,lint,speech,whisper
Requires-Python: >=3.9
Requires-Dist: huggingface-hub<2.0,>=0.23
Requires-Dist: pyarrow<22.0,>=14.0
Requires-Dist: tokenizers<0.30,>=0.19
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == 'dev'
Description-Content-Type: text/markdown

# dslint

**Catches broken training data before you rent a GPU.**

## The finding that motivated this tool

`tarteel-ai/everyayah` is a popular Quranic recitation dataset for ASR
fine-tuning. Lint it against a Whisper target and, in a few minutes and
without downloading a single audio file, you get:

- **leakage: 100.0%** — all 5,910 unique test texts also appear in train.
  A model can score perfectly on this test set by memorising, not
  transcribing. Every WER computed on this split is meaningless.
- **audio-length: 10.6%** of 187,785 train clips exceed Whisper's 30-second
  window (longest: 338s). Whisper truncates the audio but trains on the full
  transcript — those 19,000+ rows explicitly teach the model to emit words
  that were never spoken.

Reproduce it:

```
dslint tarteel-ai/everyayah --model whisper
```

## What it is

A pre-training linter for speech datasets. It inspects the dataset *before*
anyone rents a GPU and reports data problems that would silently corrupt a
fine-tune, measured against a specific model's constraints. It does not
train and it does not tune hyperparameters.

Three promises:

1. **It never downloads audio.** Hub datasets are read from their parquet
   conversion with column projection — only the text/duration/speaker
   columns are fetched. A scan costs minutes and zero dollars.
2. **It never modifies your data unless explicitly asked.** Fixing happens
   only under `--fix`, and always writes a new copy.
3. **Every FAIL ships with the fix.** Unambiguous fixes (drop malformed
   rows, drop over-window clips, drop exact duplicates) are applied under
   `--fix`. Judgement calls (how to re-split leaked data) are never guessed:
   you get both options explained and a `group_split` helper you call
   yourself.

## Install

```
pip install dslint
```

## Usage

```
# a HuggingFace dataset
dslint tarteel-ai/everyayah --model whisper

# local files (train.csv / test.csv / validation.csv, or jsonl/parquet/tsv)
dslint ./my-dataset/ --model whisper-small

# name columns explicitly when inference would guess wrong
dslint ./my-dataset/ --text transcript --duration secs --group speaker

# CI: exit code is non-zero on any FAIL, --json for machines
dslint owner/dataset --json

# apply the unambiguous fixes, writing a NEW copy (originals untouched)
dslint ./my-dataset/ --fix --out ./my-dataset-fixed/
```

In a notebook:

```python
from dslint import lint, group_split

report = lint("tarteel-ai/everyayah", model="whisper")
print(report.render())
report.to_dict()          # machine-readable
report.ok                 # False if any check failed

# re-split leaked data yourself - this is a judgement call dslint won't make:
# by content -> eval measures transcription of unseen sentences
# by speaker -> eval measures transcription of unseen voices
train_idx, test_idx = group_split(texts, test_size=0.1, seed=0)
```

## The checks

| check | severity | auto-fix |
|---|---|---|
| leakage — same text on both sides of the train/test boundary | FAIL on any overlap | no (judgement call; `group_split` provided) |
| audio-length — clips longer than the model's encoder window | FAIL above 1% (configurable), WARN below | yes: drop |
| label-length — labels longer than the model's decoder token limit | WARN | no (drop or segment; explained) |
| duplicates — exactly duplicated train rows | WARN | yes: keep first |
| malformed — empty text, null/non-positive durations | FAIL | yes: drop |

If a dataset has no duration column, audio-length SKIPs and says so — it
will not silently pass, and it will not download audio to compute durations.

## Model constraints are data, not code

The 30-second limit is not a property of the dataset and not a property of
the model — it is a property of the two meeting. Model constraints live in
a registry (`dslint/models.json`): audio window, decoder token limit,
tokenizer. The Whisper family ships with v1; adding a model is a data
change, not a code change.

```
dslint --list-models
```

## Scope

v1 is speech/ASR only. Out of scope, deliberately: images, LLM chat
templates, PII detection, near-duplicate detection, forced alignment, any
check that loads audio or runs a model.

## Development

```
pip install -e ".[dev]"
pytest                          # offline suite, synthetic fixtures
DSLINT_NETWORK_TESTS=1 pytest tests/test_e2e_everyayah.py   # the acceptance test
```
