Metadata-Version: 2.5
Name: stepdoc
Version: 0.2.0
Summary: Turn test code into test documentation: procedures with symbolic values and executed reports.
Project-URL: Homepage, https://github.com/guillegil/stepdoc
Project-URL: Source, https://github.com/guillegil/stepdoc
Project-URL: Changelog, https://github.com/guillegil/stepdoc/blob/main/CHANGELOG.md
Author: Guillermo Gil
License-Expression: MIT
License-File: LICENSE
Keywords: documentation,pytest,test procedure,test report,testing
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: Pytest
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Documentation
Classifier: Topic :: Software Development :: Testing
Classifier: Typing :: Typed
Requires-Python: >=3.11
Requires-Dist: executing>=2.0
Provides-Extra: pytest
Requires-Dist: pytest>=8; extra == 'pytest'
Description-Content-Type: text/markdown

# stepdoc

stepdoc turns test code into test documentation. From the same pytest tests it
writes two documents:

- a **procedure**: what the test always does, with symbolic values
  (`dev.map.dac.level = <level>`, `GET /users/<user_id>`), and
- an **executed report**: what one run did, with concrete values, pass/fail per
  step and timing.

Both come from a JSON **run record**, so they can never drift apart from the code.

Status: 0.2 (alpha). See [CHANGELOG.md](CHANGELOG.md). [SPIKE.md](SPIKE.md) has the design
measurements behind it.

## Quick look

```python
@pytest.mark.parametrize("name, age", [("Ana", 0), ("Bo", 120)])
def test_user_lifecycle(api, name, age):
    with step("Create user"):
        r = api.post("/users", json={"name": name, "age": age})
        assert r.status_code == 201
        user_id = r.json()["id"]

    with step("Read user back"):
        r = api.get(f"/users/{user_id}")
        assert r.json()["age"] == age
```

```
uv run pytest --stepdoc-procedure=out/procedure.md --stepdoc-report=out/report.md
```

Procedure:

```
1. Create user
   1.1. POST /users  {"name": <name>, "age": <age>}
   1.2. Verify r.status_code == 201
2. Read user back
   2.1. GET /users/<user_id>
   2.2. Verify r.json()["age"] == age
```

Report for the `Ana, 0` case:

```
1. Create user   ✓
   1.1. POST /users  {'name': 'Ana', 'age': 0} -> 201
   1.2. Verify r.status_code == 201   ✓
2. Read user back   ✗ AssertionError: assert None == 0
   2.1. GET /users/1 -> 200
   2.2. Verify r.json()["age"] == age   ✗
```

(Shown as plain text; the Markdown files carry the same content with a parameter
table and a summary.)

Run the full example yourself (one case fails on purpose, it is the bug the
directed case finds):

```bash
uv run pytest examples/quickstart --stepdoc-procedure=out/procedure.md --stepdoc-report=out/report.md
```

## How it works

1. **Steps** come from `with step("…")` blocks (or `@step("…")` on helpers). Nesting
   decides the numbering; actions, checks and nested steps share it (1.1, 1.2, 1.2.1).
2. **Actions** come from your system under test's own events through a small
   **bridge** in `conftest.py`, for example an httpx event hook or a device's
   register callbacks. stepdoc ships no drivers.
3. **Symbolic values** come from the source: when an action is recorded, stepdoc
   finds the line of your test that caused it and later reads that expression
   with [`executing`](https://github.com/alexmojaki/executing). Literals stay
   literal, names become `<name>`.
4. **Checks** come from plain `assert` (passing ones need
   `enable_assertion_pass_hook = true`), `step("…", check=...)` and
   `stepdoc.attach_check(...)`. stepdoc never compares values itself.

### Keeping documents readable

- **Setup and teardown.** Steps run by fixtures go into a Setup section (S1, S2…)
  and a Teardown section (T1…), apart from the test's own steps.
- **Polling.** The same read (or the same request) repeated from one source line,
  such as `while not dev.map.status.pll_locked: ...`, becomes one line:
  `Poll dev.map.status.pll_locked`, and in the report `-> True (37 reads, 18 ms)`.
  Turn it off with `stepdoc_collapse_repeats = false`.
- **Reads inside checks.** A read on the same line as an `assert` is shown on the
  check's line: `Verify dev.map.adc.value == 1024 -> got 1024`.
- **Quiet helpers.** `@step("Reset device", record=False)` keeps the step and its
  checks but drops the actions inside it.
- **Naming a value.** `stepdoc.value("code", level * 2)` returns the value and shows
  it as `<code>` in the procedure, also when it travels through a helper first.

### Bridges

```python
# conftest.py
def http_bridge(r):
    stepdoc.record_action(
        "{method} {url}", "request", json.loads(r.request.content or "null"),
        value_from=("json",),        # which call argument carries the value
        target_from=("url",),        # template fields resolved from call arguments
        method=r.request.method, url=r.request.url.path, status=r.status_code,
    )

@pytest.fixture
def api():
    with httpx.Client(base_url=URL, event_hooks={"response": [http_bridge]}) as client:
        yield client

@pytest.fixture
def dev(stepdoc_dry_run):
    d = Device.simulated() if stepdoc_dry_run else Device("TCPIP::10.0.0.5")
    d.on_write(lambda reg, val: stepdoc.record_action(reg.path, "write", val, value_from=("value",)))
    d.on_read(lambda reg, val: stepdoc.record_action(reg.path, "read", val))
    return d
```

`record_action(target, op, value, *, direction=None, value_from=None, target_from=(), bridge_frames=1, **meta)`

- `op` is free text (`write`, `read`, `request`, `query`, `measure`, …).
- `direction` says which way data flows: `in` (the test sends `value`), `out`
  (`value` is a result, such as a read or a measurement) or `exchange` (a request;
  put the response in `meta`, for example `status=` or `result=`). Common op names
  default sensibly; anything else is `in`.
- Values, `meta` and targets are **redacted** before they are stored: keys such as
  `password`, `token`, `authorization` and bearer tokens become `***`. Extend with
  `stepdoc.configure_redaction(keys=[...], patterns=[...])`.

Tell stepdoc which modules are library code, so it looks past them for your line:

```toml
[tool.pytest.ini_options]
stepdoc_skip_modules = ["httpx", "mydevice"]
enable_assertion_pass_hook = true
```

## Options

| Option | Purpose |
|---|---|
| `--stepdoc-record=PATH` | Write the JSON run record (schema: `src/stepdoc/schema/run-record.schema.json`) |
| `--stepdoc-procedure=PATH` | Write the procedure. The extension picks the format: `.md`, `.html` (one self-contained file) or `.txt` |
| `--stepdoc-report=PATH` | Write the executed report, same formats |
| `--stepdoc-dry-run` | Collect the procedure without judging: assertion failures inside steps are recorded and skipped so later steps are still documented |
| `stepdoc_skip_modules` (ini) | Module prefixes skipped when looking for your source line |
| `stepdoc_collapse_repeats` (ini) | Collapse polling into one line (default `true`) |
| `stepdoc_adapters` (ini) | Adapters to run. Default: every adapter whose tool is installed. `-verify` leaves one out, `none` runs none |

Without any of these options the plugin records nothing and prints nothing.

All options work with pytest-xdist (`-n 4`): each worker records the tests it
runs, and the main process writes one record and one set of documents, with
tests in collection order. Tests on a worker that crashes are missing from the
documents.

The session-scoped `stepdoc_dry_run` fixture (or `stepdoc.is_dry_run(config)`) tells your fixtures
to use a simulated backend. Python evaluates arguments eagerly and dry-run only
documents the branches the simulation takes.

## Adapters

stepdoc works with other pytest plugins without depending on them, and they
don't depend on stepdoc. When one of them is installed, its adapter reads what
the plugin already publishes:

- **pytest-verifier**: each `verify.*` check becomes a "Verify …" line in the
  step it was made in, taken from the `pytest_verify_results` hook. An expected
  value given by name is symbolic in the procedure (`Verify 'Stored age' ==
  <age>`) and concrete in the report (`== 30 -> got 31 ✗`).
- **pytest-strategies**: the report shows each case's seed and the row it ran
  (`directed: youngest`), and the procedure's parameter table shows what each
  argument draws from (`integer 0 … 120`) instead of the sampled values.

If a plugin that judges checks runs with its adapter off, stepdoc can't see
those checks, so it shows the steps of the tests that use it as `?`, never ✓,
next to the test's own verdict. In dry-run, verify's checks are documented as
not judged, but verify still judges them, so a failing one still fails the test.

Other packages can ship adapters through the `stepdoc.adapters` entry-point
group (a subclass of `stepdoc.adapters.Adapter`). Plugins can also follow steps
without importing stepdoc, through the `pytest_stepdoc_step_started`,
`pytest_stepdoc_step_finished` and `pytest_stepdoc_record_ready` hooks
(implement them with `@pytest.hookimpl(optionalhook=True)`).

## Rendering a saved record

`stepdoc render` turns a run record into documents again, without running the
tests or needing pytest installed:

```bash
stepdoc render out/run.json --procedure procedure.html --report report.md
stepdoc render out/run.json                          # procedure as Markdown on stdout
stepdoc render out/run.json --doc report --format text
```

## Agent skill

stepdoc ships a skill that teaches AI coding agents (Claude Code, Codex, GitHub
Copilot and others) to write tests with stepdoc and review the procedure:

```bash
stepdoc skill install            # .claude/skills/ and .agents/skills/ in this repository
stepdoc skill install --claude   # only .claude/skills/ (--agents for only .agents/skills/)
stepdoc skill install --global   # in your home folder instead
stepdoc skill status             # where it is installed, and whether it is outdated
stepdoc skill uninstall
```

Reinstalling an unchanged copy does nothing; a copy you edited is never
overwritten or removed without `--force`. `--print` shows the skill instead of
writing it.

## Without pytest

`stepdoc.core` never imports pytest, so the same API works in scripts and test stations:

```python
from stepdoc import Recorder, step, dumps

with Recorder("station-42") as rec:
    with step("Configure"):
        ...
open("run.json", "w").write(dumps(rec))
```

## Development

```bash
uv sync
uv run pytest
uv run mypy
uv run python bench/bench_events.py      # cost per recorded event
uv run --python 3.11 pytest              # any supported Python (3.11+)
```
