Metadata-Version: 2.4
Name: mutation-testing
Version: 0.1.0
Summary: Runtime mutation injection for Python tests
Project-URL: Homepage, https://github.com/YeahWick/mutation-testing
Project-URL: Repository, https://github.com/YeahWick/mutation-testing
Project-URL: Issues, https://github.com/YeahWick/mutation-testing/issues
Author: YeahWick
License-Expression: MIT
License-File: LICENSE
Keywords: ast,mutation-testing,quality,testing
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Testing
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: pyyaml>=6.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == 'dev'
Description-Content-Type: text/markdown

# mutation-testing

A Python mutation testing framework that validates test suite quality by injecting runtime mutations and checking whether your tests catch them.

Mutations are applied at runtime using AST pattern matching — source files are never modified.

## How it works

1. You define **mutations**: intentional code changes like replacing `+` with `-` or `>` with `>=`
2. The framework **injects** each mutation into the running code by swapping the function's `__code__` object
3. Your **test suite runs** against the mutated code
4. If a test fails, the mutation was **killed** (good — your tests caught the bug)
5. If all tests pass, the mutation **survived** (bad — your tests have a gap)

The **mutation score** is the percentage of mutations killed. A higher score means stronger tests.

## Installation

Requires Python 3.10+.

```bash
pip install mutation-testing

# Or with uv
uv add mutation-testing
```

For development:

```bash
git clone https://github.com/YeahWick/mutation-testing.git
cd mutation-testing
pip install -e ".[dev]"
```

## Quick start

### 1. Define mutations in YAML

Create a `mutations.yaml` file:

```yaml
version: "1.0"

settings:
  timeout: 30

targets:
  - module: "calculator"
    file: "src/calculator.py"
    mutations:
      - id: "add-001"
        function: "add"
        description: "Replace + with -"
        original: "return a + b"
        mutant: "return a - b"

      - id: "pos-001"
        function: "is_positive"
        description: "Replace > with >="
        original: "return x > 0"
        mutant: "return x >= 0"
```

### 2. Write a test runner

```python
from mutation_testing import MutationRunner

def run_tests() -> bool:
    """Return True if all tests pass."""
    try:
        assert calculator.add(2, 3) == 5
        assert calculator.add(-1, 1) == 0
        assert calculator.is_positive(5) is True
        assert calculator.is_positive(-1) is False
        return True
    except AssertionError:
        return False

runner = MutationRunner(run_tests)
report = runner.run_from_config("mutations.yaml")
```

### 3. Run it

```bash
cd example
python run_mutations.py
```

Output:

```
============================================================
MUTATION TESTING
============================================================

[✓] [add-001] Replace + with -: KILLED
[✓] [add-002] Replace + with *: KILLED
[✓] [sub-001] Replace - with +: KILLED
[✗] [pos-001] Replace > with >=: SURVIVED
[✓] [pos-002] Replace > with <: KILLED
[✓] [clamp-001] Off-by-one in lower bound: KILLED
[✗] [clamp-002] Off-by-one in upper bound: SURVIVED

============================================================
SUMMARY
============================================================
Total mutations:  7
Killed:           5
Survived:         2
Mutation Score:   71.4%
============================================================

SURVIVING MUTATIONS (improve your tests!):
  - [pos-001] Replace > with >=
  - [clamp-002] Off-by-one in upper bound
```

The surviving mutations tell you exactly where your tests are weak — in this case, missing boundary checks for `is_positive(0)` and `clamp` at its upper bound.

## API

### Defining mutations explicitly

Instead of YAML, you can define mutations in code:

```python
from mutation_testing import MutationRunner, Mutation

runner = MutationRunner(run_tests)
report = runner.run(
    mutations=[
        Mutation(
            id="add-001",
            function="add",
            original="return a + b",
            mutant="return a - b",
            description="Replace + with -",
        ),
    ],
    module_name="calculator",
)

print(f"Score: {report.score:.1%}")
print(f"All killed: {report.all_killed}")
```

### Core classes

| Class | Purpose |
|---|---|
| `Mutation` | Defines a single mutation (id, function, original pattern, mutant pattern) |
| `MutationRunner` | High-level runner — accepts a test function, runs mutations, prints results |
| `MutationReport` | Results summary with `total`, `killed`, `survived`, `score`, `all_killed` |
| `MutationConfig` | Loads mutation definitions from a YAML file |
| `MutationInjector` | Low-level engine that injects/restores mutations at runtime |
| `MutationResult` | Result of a single mutation (mutation + killed boolean) |

### Low-level API

For direct control over injection:

```python
from mutation_testing import MutationInjector

injector = MutationInjector()

# Inject a mutation
injector.inject("calculator", "add", "return a + b", "return a - b")

# Run your tests here...

# Restore the original
injector.restore("calculator", "add")

# Or restore everything at once
injector.restore_all()
```

## Mutation coverage report

The coverage report tracks which of your test functions have corresponding mutations defined. This helps you ensure every test is validated by at least one mutation, and can fail CI when coverage drops below a threshold.

### Configure in YAML

Add a `coverage` section to your `mutations.yaml`:

```yaml
coverage:
  threshold: 100.0        # minimum percentage of tests that must have mutations
  fail_under: true         # exit non-zero if threshold not met
  test_paths:
    - "tests"
  test_mappings:           # optional: override the default naming convention
    test_boundary_check:
      - "clamp"
      - "is_positive"
```

By default, tests are mapped to source functions by naming convention — `test_add` maps to `add`, `test_is_positive` maps to `is_positive`. Use `test_mappings` when a test name doesn't match its source function.

### Generate the report

```python
from mutation_testing import MutationRunner

runner = MutationRunner(run_tests)
coverage = runner.coverage_from_config("mutations.yaml")

# Check results programmatically
if not coverage.meets_threshold:
    sys.exit(1)
```

Or use the standalone function:

```python
from mutation_testing import generate_coverage_report, print_coverage_report

report = generate_coverage_report(
    test_paths=["tests"],
    mutation_config_path="mutations.yaml",
    threshold=100.0,
)
print_coverage_report(report)

# JSON output for CI integration
print(report.to_json())
```

### Output

```
============================================================
MUTATION COVERAGE REPORT
============================================================

[✓] test_add
      functions: add
      mutations: 2 (add-001, add-002)
[✓] test_subtract
      functions: subtract
      mutations: 1 (sub-001)
[✗] test_multiply
      functions: multiply
      mutations: 0

============================================================
COVERAGE SUMMARY
============================================================
Total tests:     3
Covered:         2
Uncovered:       1
Coverage:        66.7%
Threshold:       100.0%
Status:          FAIL
============================================================

UNCOVERED TESTS (add mutations for these!):
  - test_multiply -> multiply
```

### CI usage

Use the exit code to fail CI pipelines:

```python
coverage = runner.coverage_from_config("mutations.yaml")
sys.exit(0 if coverage.meets_threshold else 1)
```

### Coverage classes

| Class | Purpose |
|---|---|
| `CoverageReport` | Full report with `total_tests`, `covered_tests`, `coverage_percent`, `meets_threshold`, `all_covered` |
| `TestCoverage` | Per-test status with `test_name`, `mapped_functions`, `mutations`, `covered` |
| `CoverageConfig` | YAML settings: `threshold`, `fail_under`, `test_paths`, `test_mappings` |

## Supported mutation patterns

The AST pattern matcher supports any valid Python expression or statement:

- **Arithmetic operators**: `+` ↔ `-`, `*`, `/`, `//`
- **Comparison operators**: `>` ↔ `>=`, `<` ↔ `<=`, `==` ↔ `!=`
- **Boolean operators**: `and` ↔ `or`
- **Return values**: `return x` → `return None`, `return -x`
- **Any expression** parseable as Python

Patterns are matched structurally via AST, so whitespace and formatting differences are ignored.

## Project structure

```
mutation_testing/
├── __init__.py     # Public API exports
├── core.py         # AST pattern matching, injection engine, Mutation/MutationResult
├── config.py       # YAML configuration loader
├── coverage.py     # Mutation coverage reporting
└── runner.py       # MutationRunner and MutationReport

example/
├── src/calculator.py       # Sample module under test
├── tests/test_calculator.py  # Sample test suite
├── mutations.yaml          # Sample mutation config
└── run_mutations.py        # Example runner script
```

## Key concepts

| Term | Meaning |
|---|---|
| **Mutation** | An intentional code change (e.g., `+` → `-`) |
| **Killed** | Tests detected the mutation (test failed) |
| **Survived** | Tests passed despite the mutation — tests need improvement |
| **Mutation score** | Killed / Total (higher is better) |
| **Mutation coverage** | Percentage of tests that have at least one mutation defined |

## License

MIT
