Metadata-Version: 2.4
Name: selectorprobe
Version: 0.1.0
Summary: Suggest the CSS selector that holds the body of an article, so a feed reader can be told where the full text is. Standard library only.
Author: Younes Z.
License: MIT
Project-URL: Homepage, https://github.com/Rezarys/selectorprobe
Project-URL: Issues, https://github.com/Rezarys/selectorprobe/issues
Keywords: css selector,rss,feed,full text,article extraction,scraping,freshrss,self hosted
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: End Users/Desktop
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Internet :: WWW/HTTP :: Indexing/Search
Classifier: Topic :: Text Processing :: Markup :: HTML
Classifier: Topic :: Utilities
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# selectorprobe

```
pip install selectorprobe
```

Point it at one article and it tells you which CSS selector holds the body of that article, so you can paste it into the full text field of a feed reader instead of hunting for it in a browser inspector.

```
selectorprobe https://example.com/2026/09/harbour-wall-reopens
```

The report looks like this. The block below is the exact output of the page fixture used in the test suite, printed by the same function the command calls.

```
Candidate 1
  content selector : div.entry-content
  strip inside it  : div.sharedaddy
  div, 2081 characters, 3 paragraphs, 1 link, score 4519.8
  starts with      : The harbour wall was rebuilt over the winter and reopened in March. Fishermen say the new slipway is easier to use at low tide. The works cost less than the fir

Check the first candidate against a second article of the same site before trusting it: one page cannot tell a stable selector from a lucky one.
```

`content selector` is the block to keep. `strip inside it` lists blocks that sit inside that block and look like page furniture, so you can drop them.

## Why the selector is short

A browser inspector hands you the full path of the node you clicked, which matches that one page and breaks on the next one. selectorprobe only ever writes selectors made of ids, `tag.class` steps, and descendant chains of those, and it refuses any name that looks generated: a hash, a framework prefix, or a name carrying a record number such as `post-4821`. It also walks down from the block it picked while a single child still holds nearly all of the text, so the title bar, the tag list and the comment area stay out of the selector.

It never prints a selector it has not counted: a candidate is only offered once the tool has checked that it matches exactly one node of the page.

## Options

- `--file` reads the argument as a local HTML file instead of downloading it.
- `--top N` prints up to N candidates, best first, default 3.
- `--json` prints the result as JSON instead of text.
- `--timeout S` sets the network timeout in seconds, default 20.
- `--user-agent S` sets the User-Agent sent with the request.

Exit code 0 when at least one candidate was found, 1 when the page yields none, 2 when the page could not be read.

## What it does not do

It does not run JavaScript. A page that ships an empty document and fills it from a script yields no candidate, and the tool says so rather than guessing.

It downloads the one URL you give it, following redirects the way the standard library does by default, and touches the network nowhere else. It stores no cookie, writes nothing to disk, sends no telemetry, and reads at most 8 MB of a page.

It does not talk to your feed reader. It prints selectors and you paste them.

## Requirements

Python 3.9 or newer. No dependency, standard library only.

## Honest limits

This has been run on HTML fixtures written for its own test suite, and never against a live website. The scoring was tuned on those fixtures. If it picks the wrong block on a real page, that is a defect worth an issue, and the page URL is the only thing needed to reproduce it.

The tool reads the markup a server sends. Sites that assemble their article in the browser are outside what it can see, and so are pages behind a login or a challenge.

A score is a comparison between blocks of the same page and nothing more. It does not compare across sites and it does not measure quality.

## Where this came from

The demand thread is [FreshRSS/FreshRSS#9340](https://github.com/FreshRSS/FreshRSS/issues/9340), where someone asked whether a tool exists that generates these values instead of copying them out of an inspector by hand. selectorprobe is not affiliated with that project and is not an extension for it: it is a standalone command that prints selectors, usable with any reader that takes a CSS selector for full text extraction.

## Licence

MIT, Younes Z.

Built with AI assistance, reviewed and tested by me.
