Metadata-Version: 2.5
Name: pytest-subset
Version: 0.2.0
Summary: pytest subset plugin for subsetting parametrizes tests.
Author: cyber-paraphernalia
License-File: LICENSE
Requires-Python: >=3.10
Requires-Dist: pytest>=7
Provides-Extra: dev
Requires-Dist: coverage[toml]>=7.0; extra == 'dev'
Requires-Dist: pytest-cov>=4.0; extra == 'dev'
Requires-Dist: pytest-html>=3.2; extra == 'dev'
Requires-Dist: pytest-xdist>=3.0; extra == 'dev'
Requires-Dist: ruff>=0.6; extra == 'dev'
Description-Content-Type: text/markdown

# pytest-subset

A pytest plugin for deterministically sampling a subset of a heavily
parametrized test's variants.

## Installation

```bash
uv pip install pytest-subset
```

## Usage

```python
import pytest

@pytest.mark.subset(fraction=0.2)
@pytest.mark.parametrize("value", range(1000))
def test_something(value):
    ...
```

Running `pytest` normally will run ~20% of `test_something`'s variants,
chosen deterministically. Pass `--full` (or `-F`) to run every variant, 
e.g. in CI before a release.

## Marker reference

`@pytest.mark.subset(fraction=..., seed=None)`

- `fraction`: float in `(0, 1]`. Fraction of parametrized variants to keep.
  Optional if `subset_default_fraction` is set in your `pyproject.toml`.
- `seed`: optional int. A seed value used to generate a pseudo-random slice of 
  the parametric test space. Pins the selection for this test. If not provided,
  a default seed is generated based on the test module and Node ID. In either case,
  a given seed value will always generate the same subset of tests (i.e. deterministic)

## CLI options

- `-F`, `--full` — ignore every `subset` marker and run all variants.
- `--salt=SEED` — controls the slice for tests that don't pin their own
  `seed=`. Accepts:
  - an integer, so different values give different, still-reproducible
    slices;
  - `random`, to pick a fresh seed each run. It's printed after the run so
    you can reuse it later.

```python
import pytest

@pytest.mark.subset(fraction=0.2, seed=42)
@pytest.mark.parametrize("value", range(1000))
def test_something(value):
    ...

@pytest.mark.subset(fraction=0.2)
@pytest.mark.parametrize("value", range(1000))
def test_something_else(value):
    ...
```

`test_something` always uses seed `42` — an explicit `seed=` superceeds
`--salt` every time. `test_something_else` has no seed of its own, so
`--salt` decides its slice:

```
pytest --salt=22       # test_something_else samples using salt 22
pytest --salt=random   # a fresh seed is picked and printed after the run
```

Note that `--salt=random` picks each run's slice independently, so
slices across runs are not guaranteed to be disjoint.

## Distributed runs (pytest-xdist)

`pytest-subset` is compatible with `pytest-xdist`; `-n` needs no extra flags.

xdist has [known limitations](https://pytest-xdist.readthedocs.io/en/stable/known-limitations.html), requiring every worker to collect the same tests in the same order. To account for this, the
plugin resolves the salt once on the controller and hands it to each worker.
That includes `--salt=random`: all workers share the one freshly picked seed,
and the value printed after the run reproduces the distributed selection exactly.

> Note: `xdist` support was only added in `v0.2.0`.

```bash
pytest -n auto --salt=random
```

## Ini options

Set this under `[tool.pytest.ini_options]` in your `pyproject.toml` (or the
equivalent section of `pytest.ini`/`setup.cfg`):

- `subset_default_fraction` — fraction used by `@pytest.mark.subset` when it's
  applied bare, with no `fraction=` argument. This value is applied globally
  across the test suite.
  ```toml
  [tool.pytest.ini_options]
  subset_default_fraction = "0.3"
  ```

Then, in your test modules,

```python
import pytest

@pytest.mark.subset  # 0.3 applied here by default.
@pytest.mark.parametrize("value", range(1000))
def test_something(value):
    ...
```