Metadata-Version: 2.4
Name: errgrade
Version: 0.2.0
Summary: Grade the error messages your code raises: what broke, where, and what to do next.
Author: mgkvibez
License: MIT
Keywords: linter,errors,dx,code-review,developer-tools,static-analysis
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Dynamic: license-file

# errgrade

Grade the error messages your code raises: does it say **what** broke, **where**, and **what to do next**.

```
$ errgrade examples/before.py
errgrade: 4 finding(s) across 6 message(s) in 1 file(s), worst high
1 of 6 graded message(s) fall below C

[HIGH] examples/before.py:6 (vague-error)
  message: 'failed'
  why:     the message does not say what went wrong
  missing: what, where, next
  grade:   D (0/100)

[HIGH] examples/before.py:8 (literal-placeholder)
  message: 'missing {path}'
  why:     braces are shown to the reader verbatim: this string is never formatted
  missing: next
  grade:   B (75/100)

[LOW] examples/before.py:10 (no-next-step)
  message: 'config file is empty'
  why:     nothing tells the reader what to do next
  missing: next
  grade:   B (75/100)
```

Note what it does **not** flag: `amount must be positive, got {amount}` and
`currency must be one of NGN, USD` pass, because they answer all three questions.

## Install

```bash
pipx install errgrade
```

Python 3.9+, **zero runtime dependencies**. The analysis is a standard-library AST pass.

## Usage

```bash
# grade the current directory (tests excluded: their messages are not user-facing)
errgrade

# one file or directory, and see the worst offenders first
errgrade src --top 5

# machine-readable, for a bot or a CI annotation step
errgrade --json

# fail CI only on the messages a user would actually see going wrong
errgrade src --fail-on high
```

Exit codes: `0` nothing at or above the threshold, `1` findings at or above it,
`2` usage error. A linter that exits 1 for its own mistakes makes CI lie, which is why
those are separate.

`--fail-on` defaults to `medium`. `low` findings never fail a run unless you ask.

## Why it exists

Error messages are the only part of your code that runs at the worst moment of someone's
day. They are also the one thing nobody reviews: a pull request diff shows a new
`raise`, and reviewers look at the condition, not the sentence.

The cost is invisible and real. `ValueError("failed")` reaches a user as a dead end, and
reaches you as a support ticket that says "it doesn't work". The difference between that
and `amount must be positive, got -5` is four words and one value.

## How the grading works

Every message is scored on three axes, because those are the three questions a stuck
person asks:

| Axis | Weight | Satisfied by |
|---|---|---|
| what | 40 | not a known filler phrase, and at least two words |
| where | 35 | an interpolated value, a named subject ("tenant is archived"), a location noun, or a list of valid values |
| next | 25 | an imperative ("set", "pass", "check", "give"), a constraint ("must be", "expected"), or the allowed values |

100 becomes `A`, 85+ `A`, 70+ `B`, 50+ `C`, below that `D`. A message with no
interpolated value and no named subject cannot tell one failure from another, which is
what pushes it below `C`.

The values come from the syntax tree, not the text: an f-string, `.format()` or `%`
counts as interpolation because a value genuinely flows in. `"{name}"` written as a plain
string counts as a bug, because the reader sees the braces.

## Grading the bodies you return

`check` grades what you raise. `responses` grades what the caller receives, which is a
different failure: a handler can raise a good message and still answer with `{"error":
"failed"}`, or with the exception's text pasted in.

```
$ errgrade responses examples/api_errors.py
errgrade: 3 finding(s) across 5 body(ies) in 1 file(s), worst high
3 of 5 graded body(ies) fall below C

[HIGH] examples/api_errors.py:15 (leaked-internals)
  body: '<computed>'
  why:     the body embeds the exception's own text, which discloses internals and changes shape between failures
  missing: what, where, next
  grade:   D (0/100)

[HIGH] examples/api_errors.py:22 (leaked-internals)
  body: 'failed'
  why:     the body carries a traceback field, which is for the developer, not the caller
  missing: what, where, next
  grade:   D (0/100)

[MEDIUM] examples/api_errors.py:26 (internal-path)
  body: 'could not write /srv/app/exports/%s.csv'
  why:     an absolute server path in a response tells an attacker the layout and is useless to a client
  missing: where, next
  grade:   D (40/100)
```

Those are the four shapes worth failing a build over: an exception's own text leaking out,
an absolute server path disclosing the deployment layout, a message too vague to act on,
and a body with no machine-readable code for a client to branch on.

## Rules

| Severity | Rule | What it means |
|---|---|---|
| high | `vague-error` | the message does not say what went wrong (`"failed"`, `"invalid input"`) |
| high | `literal-placeholder` | braces reach the reader verbatim because the string is never formatted |
| medium | `no-context` | no value and no subject: the reader cannot tell which case this is |
| medium | `bare-exception` | `raise ValueError()` with no message at all |
| low | `no-next-step` | nothing tells the reader what to do next |

One primary finding per message: a vague message is not also reported for context and
next steps, because stacking three findings on one word is noise, not help.

The response rules, used by `errgrade responses`:

| Severity | Rule | What it means |
|---|---|---|
| high | `leaked-internals` | `str(exc)`, `repr(exc)`, `traceback.format_exc()`, `exc.args`, or a `traceback`/`stack`/`debug` field |
| medium | `internal-path` | an absolute server path inside a response body |
| medium | `vague-error-body` | the returned message is too vague to act on |
| medium | `no-machine-code` | no code a client can branch on, so consumers must match your prose |

A nested `{"error": {"code": ..., "message": ...}}` body is read as a whole, so the shape
callers actually want is not flagged for the code hidden one level down.

## Suppressing a message

Some messages are deliberately terse. Say so on the line:

```python
raise ValueError("failed")  # errgrade: ignore
```

Only that line is suppressed. There is no config file to fall out of sync.

## What this deliberately does not do

- **Python only, for now.** The grading itself takes a string and does not care where it
  came from; only the AST extraction is Python-specific.
- **It grades literal messages.** `raise ValueError(msg)` is skipped, because grading a
  variable would mean inventing its contents.
- **It is a heuristic, not a review.** A high grade means the message answers the three
  questions as far as a static tool can tell, not that it is good writing. "No issues
  found" means no known weak pattern matched, and is not a guarantee.
- **It does not touch logging.** Log lines are for people with the code open; `raise`
  strings are what users and callers read.
- **It does not rewrite your messages.** It tells you which ones are a dead end and why.

## Tests

```bash
python -m pytest -q      # 68 tests
bash scripts/verify.sh   # the release gate, 6 checks
```

Contributions: the rules live in `errgrade/analyze.py`, the scoring in
`errgrade/grade.py`, and every rule has a test in `tests/test_analyze.py`. A new
heuristic needs a case it must flag and a case it must not.

## License

MIT.
