Metadata-Version: 2.4
Name: shurrup
Version: 1.1.1
Summary: A linter for the comments, not the code
Project-URL: Homepage, https://github.com/pimoroni/shurrup
Project-URL: Repository, https://github.com/pimoroni/shurrup
Project-URL: Issues, https://github.com/pimoroni/shurrup/issues
Author-email: Philip Howard <phil@pimoroni.com>
License-Expression: MIT
License-File: LICENSE
Requires-Python: >=3.11
Requires-Dist: markdown-it-py>=3
Requires-Dist: pathspec>=0.12
Requires-Dist: rich>=13
Requires-Dist: tree-sitter-language-pack>=1.14
Requires-Dist: tree-sitter<0.26,>=0.25
Provides-Extra: nlp
Requires-Dist: spacy<4,>=3.8; extra == 'nlp'
Provides-Extra: test
Requires-Dist: pytest>=8; extra == 'test'
Description-Content-Type: text/markdown

# shurrup

A linter for the comments, not the code.

Every other linter in a project reads the code and skips the comments. This one does the opposite. It pulls the comments out of eleven languages and checks them. A comment can outweigh the code beneath it. One file can end up with two comment styles in it. A comment can describe a change no reader can check, or fill a line with words that carry no information.

shurrup runs on itself in CI. A rule that fires on prose a reader would leave alone is a rule the project turns off, and CI is where that shows up first.

Written in glorious Python, if I wanted Rust I'd drive a Toyota Tacoma.

## Install

```bash
uv tool install shurrup
```

## Use

```bash
shurrup .                              # everything it can read below here
shurrup src --format full              # with the offending line quoted
shurrup src --statistics               # counts by code
shurrup --explain SHC001               # what one code means
shurrup --list-rules

shurrup . --write-baseline .shurrup-baseline.json   # record what is already there
shurrup . --baseline .shurrup-baseline.json         # report only what is new

shurrup --commits                      # commit messages not yet pushed
shurrup --commits origin/main..HEAD    # or any range git understands
```

A commit message is prose about code, so the phrasing and length rules read it the same way. The subject is one block and each paragraph of the body is another. Lines wrap at 72 rather than 96. SHC001 stands aside: describing a change is what a message is for.

Files `.gitignore` covers are skipped, so a generated tree is not linted as source. Turn that off with `respect-gitignore = false`.

Exit code is 1 for an error, 0 for warnings alone. Switch a rule down to a warning to see what it reports without failing a build. `--strict` fails on warnings too.

Formats are `concise`, `full`, `github` (annotations, so a run in Actions marks up the diff) and `json`.

## How it fits together

Extraction and linting are separate. Neither depends on the details of the other.

An **extractor** is per language. It takes source text and returns `Comment`s: where the comment was, which delimiter opened it, and the text with the delimiters removed. What a language can hide a comment inside is the hard part, so each extractor is built on a parser for that language rather than a scan over characters:

| Language | Parser | What it has to get right |
| -------- | ------ | ------------------------ |
| Python | `tokenize`, `ast` | A string is a docstring by where it sits, and `ast` is what settles that |
| C, C++ | tree-sitter | String, character and raw literals, digit separators, `//` continued over a backslash-newline |
| JavaScript, JSX | tree-sitter | Regex literals against division, template holes that nest, markup in the middle of an expression |
| HTML | tree-sitter | `<!-- -->`, plus `<script>` and `<style>` handed to the JavaScript and CSS extractors with their positions shifted |
| CSS | tree-sitter | `/* */` only, since a stylesheet reads `//` as a value |
| YAML | tree-sitter | Block scalars, plain scalars holding an apostrophe, anchors |
| TOML | tree-sitter | `#` with no space in front of it, and the three string forms |
| INI | tree-sitter | `;` as well as `#`, and a trailing `;` that belongs to the value |
| shell | tree-sitter | Heredocs, `${#array[@]}`, and a hashbang that is a directive and not a comment |
| commit messages | `git log` | The subject, then each paragraph of the body |
| Markdown | `markdown-it-py` | Fences of any length, raw HTML blocks, link targets holding a bracket; each paragraph, heading, list item and table row is one block. A row is a list of cells, so the sentence limits leave it alone |

The **core** then merges runs of line comments into one block. A paragraph written as six `#` lines has to be measured as a paragraph. An empty comment line inside a run ends a paragraph, so a file header carrying a summary and two short notes is three units and not one long one. It also attaches the code each comment appears to document.

A **rule** is per family, and sees the language only as a field on the comment. Rules come in three scopes. One comment at a time. One file, for a question such as whether that file mixes two styles. Or every file at once, for the same block pasted into six of them.

## Severity, and why most phrasing rules are warnings

Codes are grouped by family, so `--ignore SHP` drops all the phrasing rules at once. A `?` marks a warning.

An **error** is a shape with no good reading in a code comment: a hedge, a comment describing a change nobody can check, commented-out code. A **warning** is a tic, defensible once and a problem only in quantity.

### Budgets

A single `rather than` is fine and forty in one file is not, so the tic rules carry a **per-file budget**. Under it they report nothing. Over it they report once:

```
src/thing.py:1:1: error SHP013 9 blocks in this file use rather-than (budget 3);
                  the count is the finding, search for the word to find them
```

Forty findings naming a word you can grep for is a wall. One finding naming the count is the work. Set `[rules.SHP013] budget = 0` or `budget = false` to turn the budget off and get every site instead.

No phrase is counted at two levels. A tic with a per-instance rule reports through that rule's budget; **SHM** measures only the habits that have no rule of their own.

### SHL, length

Measured against the code a comment documents where there is code to measure against, and against a flat limit elsewhere.

<!-- shurrup: ignore -->
| Code | | What it catches |
| ---- | - | --------------- |
| SHL001 | | More words of comment than the code beneath it accounts for |
| SHL002 | | A comment line past the configured width |
| SHL003 | | A trailing comment long enough to belong on its own line above |
| SHL004 | ? | A docstring of fewer words than the configured floor |
| SHL005 | | A comment with no text in it |
| SHL006 | | A paragraph past 60 words (40 warns), or a docstring paragraph past 80 |
| SHL007 | | A paragraph past 6 lines |
| SHL008 | ? | Timings and byte counts in a code comment, with no test named |
| SHL009 | | A sentence past the word limit |
| SHL010 | | A sentence carried by more commas than the limit allows |
| SHL011 | | A document longer than anyone reads in one sitting |

SHL009 and SHL010 are separate rules because a single score mixing words and commas names a number below the limit as the reason a sentence exceeded it. Each reports the count it is about.

Limits differ by register. Measured over a 4000-sentence corpus, code comments reach 37 words at the 95th percentile and markdown reaches 51, so markdown gets its own defaults for SHL006, SHL007, SHL009 and SHL010. Set them per language with `[rules.SHL009.markdown]`.

One comment is measured one way. Where there is code beneath to measure against, SHL001 takes the ratio; where there is not, SHL006 takes the flat limit. Running both produced two findings quoting the same number against the same threshold. SHL007 is independent of both, since depth is about what fits on a screen.

A docstring is measured by its longest paragraph and not by its total, since the size of the thing it documents sets how much there is to say. Measured over 1490 docstrings the longest paragraph reaches 81 words at the 99th percentile, so 80 is the limit: a laid-out docstring of any length passes, and one paragraph past that is a wall whatever it documents. SHL007 does not read docstrings at all.

Tables and worked examples inside a comment are excluded from all of these. Measured as a sentence, a `key description` block becomes one enormous run-on with a word count that describes nothing.

### SHY, style

Rules that could go either way take the majority within a file as the standard and object to the minority. A codebase that has settled on `//` gets told about the stray `/* */`; one that has settled on `/* */` is left alone.

<!-- shurrup: ignore -->
| Code | | What it catches |
| ---- | - | --------------- |
| SHY001 | | Two spellings of the same single-line comment in one file |
| SHY002 | | A docstring not written with triple double quotes |
| SHY003 | | A row of decoration in place of a sentence |
| SHY004 | | No space between the delimiter and the words |
| SHY005 | ? | One short line in a block comment |
| SHY006 | | Trailing whitespace |

### SHC, content

<!-- shurrup: ignore -->
| Code | | What it catches |
| ---- | - | --------------- |
| SHC001 | | Describes what the code was, which the reader has no way to check |
| SHC002 | | Code kept as a comment |
| SHC003 | | A marker with no name and no ticket on it |
| SHC004 | ? | Restates in words what the line below expresses in code |
| SHC005 | | A disabled line parked with a note about why |
| SHC006 | ? | Reports how an approach that is not in the file behaved |

SHC001 needed the most care. A comment documenting a change describes the code's past as well as its present. Three signals mark it:

<!-- shurrup: ignore -->
- **tense**: "previously", "used to", "formerly", "no longer". A past the reader cannot see, because the code in front of them is only the present.
- **verbs**: "renamed", "moved", "replaced", "reverted" applied to the code itself. Those describe an edit to the file, where a comment should describe what the file does.
- **reference**: "since v2.1", "as of 2024-03", "fixes #412". A point in the history.

Each of the three turns up in good comments too. "Now" reads plainly in "now that the buffer is full"; "removed" is fine when the code does the removing. So the signals are scored, not matched, and a comment is flagged at one strong signal or two weak ones. That keeps it quiet enough to leave switched on.

SHC006 catches the near neighbour SHC001 misses. A design note about a road not taken carries no temporal marker. The signal is a past-tense verb of outcome attached to an alternative absent from the file.

### SHP, phrasing

Patterns over English, so approximations. Each catches a shape that is nearly always a mistake in a code comment, even where it would pass in prose.

<!-- shurrup: ignore[SHP032] -->
SHP005 and SHP032 are tested from the verb, not the subject. The set of things a comment might personify never closes; the set of things that legitimately have a mind does. So the verb is matched first and the subject checked against that set. A scanner genuinely scans and a thread genuinely sleeps, so those verbs are absent; a file cannot spell, because whoever wrote it did the spelling.

<!-- shurrup: ignore -->
| Code | | What it catches |
| ---- | - | --------------- |
| SHP001 | | `its own`, `the theme's own` used as emphasis |
| SHP002 | | `simply`, `obviously`, `of course` |
| SHP003 | | `note that`, `it should be noted`, a sentence opening on `Actually,` |
| SHP004 | | `basically`, `essentially`, `literally` |
| SHP005 | | The parser `knows`, the page `wants`, the epicentre `breathes` |
| SHP006 | | `magic`, `clever`, `hacky`, `nasty` |
| SHP007 | | `knobs`, `levers` |
| SHP008 | | `we should`, `you need to` |
| SHP009 | | `!!`, `?!` |
| SHP010 | | `for now`, `temporarily`, `placeholder` |
| SHP011 | | `as you can see`, `needless to say` |
| SHP012 | | `this function is`, `this class does` |
| SHP013 | | `rather than` |
| SHP014 | ? | `in order to`, `due to the fact that` |
| SHP015 | | `instead of` |
| SHP016 | | `is what the`, `which is where` |
| SHP017 | | `that being`, `being what` |
| SHP018 | | `reads as`, `read as` |
| SHP019 | | `on purpose`, `deliberately`, `by design` |
| SHP020 | | Em dashes, curly quotes, ellipsis characters, non-breaking spaces |
| SHP021 | | Two spaces after a sentence |
| SHP022 | | Emoji |
| SHP023 | ? | `worth a`, `only worth saying` |
| SHP024 | | ` - ` standing in for an em dash |
| SHP025 | ? | `nothing`, `never`: defining a thing by exclusion |
| SHP026 | ? | A block opening `How`/`What`/`Where` with no main clause |
| SHP027 | | A block opening `And`/`But`, continuing the comment above |
| SHP028 | ? | `somebody chose`, `someone switches`: a decision credited to nobody in particular |
| SHP029 | ? | `dear` for expensive, `furniture` for chrome, `whilst` |
| SHP030 | | `the whole <thing>` |
| SHP031 | | `more prose than the code can carry.`: a transitive verb with its object dropped |
| SHP032 | | `a file spells`, `a phrase says`: a written thing given its author's agency |
| SHP033 | | `honest`, `story`, `narrative` |
| SHP034 | | `That is all it does`, `and nothing more` |
| SHP035 | | A block opening on a bare `It`, `This` or `They` |
| SHP036 | | `a .mpy`, `a HTTP`: the article against the sound that follows |
| SHP037 | ? | `which is what`, where `which` alone says it |
| SHP038 | ? | `handled apart from these`, where `apart from` means `except for` |
| SHP039 | ? | `taken off`, `come off`, `based off`: a phrasal verb one word replaces |
| SHP040 | ? | Two `and`s in one sentence, where the first should be a comma |
| SHP900 | | A phrase this project configured |

SHP001 declines the separative senses, where `own` marks a thing held apart: `on its own line`, `in its own thread`, `to its own directory`. Everywhere else it reads as emphasis.

That test is blunt. Measured over a 439-instance corpus it declines about 10%, so the rule is a warning and its per-file budget carries the density. Extend the set with `[rules.SHP001] allow = [...]`.

### SHM, per-file measures

Some phrasing is fine once and a tic by the twentieth time, which one comment can never show. These are measured as a proportion of the file's blocks and reported once, at the top, with the count.

<!-- shurrup: ignore -->
| Code | Default limit | What it measures |
| ---- | ------------- | ---------------- |
| SHM002 | 30% | Blocks ending on "so the ..." |
| SHM003 | 40% | Blocks pivoting on a colon |

### SHD, duplication

<!-- shurrup: ignore -->
| Code | What it catches |
| ---- | --------------- |
| SHD001 | The same block of prose in more than one file |

The one rule that needs every file at once. A block that appears verbatim in six files was pasted, and pasted prose goes stale in five places at once. Matching is on the words, with case, punctuation and wrapping taken out, so a block rewrapped to a different width still counts. Blocks under 12 words are skipped, and so is licence boilerplate.

## Settings

`shurrup.toml`, `.shurrup.toml`, or a `[tool.shurrup]` table in `pyproject.toml`, whichever turns up first at or above the path being checked.

```toml
select = ["SH"]
ignore = ["SHP014", "SHM"]
exclude = ["build", "vendor"]

# Any rule's severity and options.
[rules.SHL002]
max_length = 100

[rules.SHL009]
max_words = 30

[rules.SHL006]
max_words = 60
warn_words = 40

[rules.SHP013]
severity = "warning"

[rules.SHP001]
allow = ["line", "thread", "namespace"]

[rules.SHM003]
max_ratio = 0.4
min_blocks = 20

# Prefixes match, so this drops every phrasing rule in one file.
[per-file-ignores]
"tests/fixtures/*" = ["SH"]
"src/legacy/*.c" = ["SHP", "SHL006"]

# House style, the short way.
[prefer]
knob = "setting"
utilise = "use"

# House style, the long way.
[[phrases]]
code = "SHP901"
pattern = "\\b(kick off|fire off|spin up)\\b"
message = "say what it starts"
severity = "warning"
languages = ["python", "javascript"]
```

An option can be set for some files only, which keeps a weakened check where an ignore would give up the rule entirely:

```toml
# A generated API surface writes one member per line, and a signature alone reaches 225
# characters. The other files keep the 96 the rest of the project uses.
[per-file-rules."api/*.py".SHL002]
max_length = 240
```

Every phrase rule takes an `allow`, for vocabulary a project cannot avoid: `trivially copyable` is the C++ standard's own adjective, and `magic` is what a file's magic number is called.

```toml
[rules.SHP002]
allow = ["trivially"]
```

Options for each rule are listed by `shurrup --explain <code>`. `--select` on the command line replaces the file's list; `--ignore` adds to it.

### Starting on an existing codebase

Turning the phrasing rules on across a repository that has never seen them produces hundreds of findings, none of which is a regression. Record them and work from there:

```bash
shurrup . --write-baseline .shurrup-baseline.json
shurrup . --baseline .shurrup-baseline.json
```

A finding is fingerprinted by file, code and matched phrase, never by line, so editing the top of a file leaves the rest of the baseline intact.

### Suppressing one comment

A directive in the comment itself silences its own lines and the first line of code below it. The directive is stripped before any rule runs, and is never itself reported.

```python
# Kept for the API, which we do not control. shurrup: ignore[SHC001]
```

Bare `shurrup: ignore` silences everything on those lines. Prefixes work: `shurrup: ignore[SHP]`.

## Extending it

Both halves attach through entry points, so a project can add a language or a house style without touching this package.

```toml
[project.entry-points."shurrup.languages"]
rust = "my_pack.rust:extract"

[project.entry-points."shurrup.rules"]
house = "my_pack.rules:register"
```

An extractor is a callable taking source and returning `Comment`s, with a `SHURRUP_EXTENSIONS` attribute listing the extensions it handles. Plugins take precedence over builtins, so a project can replace one.

A rule pack's `register` takes no arguments, and applies the `@rule` decorator to what it brings:

```python
from shurrup.rules import Finding, rule

def register():
    @rule("XYZ001", "no-latin", severity="warning")
    def no_latin(comment, context, settings):
        """Latin abbreviations a reader has to expand."""
        if re.search(r"\b(e\.g\.|i\.e\.|etc\.|viz\.)", comment.text):
            yield Finding("spell it out", excerpt="e.g.")
```

Rules yield `Finding`s. Position defaults to the start of the comment, so a rule without a more precise one need not work one out.

## As a library

```python
from shurrup import lint

for problem in lint(["src"]):
    print(problem.code, problem.path, problem.line, problem.message)
```

`shurrup.core.read(path)` returns a `Context` if extraction is all you want.

## Notes

The phrasing rules are opinionated. The defaults follow one house style, and `ignore` and `severity` are there because yours will differ. Start with `--statistics` to see which rules fire before turning any of them into errors.

SHP020 bans the em dash and SHP024 bans the spaced hyphen that usually replaces it. Together they leave no dash at all, which is a policy a project has to choose rather than inherit, so **SHP024 ships off**. Turn it on with `[rules.SHP024] severity = "error"` once the dash question is settled. Any rule can be switched off the same way.

Words are matched without any notion of what they refer to. SHP007 asks for "settings" in place of `knob` and `lever`; `dial` is absent from that list because a clock has one.

`actually` is absent from SHP004 for the same kind of reason. A hedge weakens a claim and `actually` corrects one, which is the opposite job. Measured over 93 instances in four codebases, 96% of them marked observed behaviour against assumed: "the bytes actually read", "whatever the host actually reports", "these functions are actually ASCII-only". Opening a sentence it carries nothing, and SHP003 has that case.

SHP005, SHP031 and SHP032 turn on grammar and not on words: whether a verb has a subject, whether that subject is animate, whether a transitive verb has an object. A pattern answers those by listing every word that may sit between a subject and its verb. That list never closes. Installing the optional `nlp` extra puts a parse behind them instead:

```bash
pip install "shurrup[nlp]"
python -m spacy download en_core_web_sm
```

Measured over 45,000 comment blocks from the standard library, the parse reports about 30% fewer than the pattern and drops a whole class of nonsense subject: a pattern reads `Don't think` as the word `t` thinking, and `0 tells` as the number telling. Neither path is exact, so the extra is opt-in and every rule falls back to its pattern when the model is absent.

The JavaScript regex-against-division heuristic is the standard one (what the previous significant token was) and it can be fooled. A mis-scan costs a missed comment or a spurious one. JSX expressions inside markup are left unparsed.
