Metadata-Version: 2.4
Name: html-reader-llm
Version: 0.0.3
Summary: HTML simplification and intelligent extraction for LLM
Author-email: Clericpy <clericpy@gmail.com>
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: fastmcp-slim[server]>=3.4.7
Requires-Dist: minify-html>=0.18.1
Requires-Dist: python-dotenv>=1.2.2
Requires-Dist: selectolax>=0.4.7
Requires-Dist: trafilatura>=2.2.0

# html-reader-llm

HTML simplification and intelligent extraction for LLM.

## Features

- **HTML Simplification** — 7-step pipeline to clean HTML for LLM consumption
- **Render Detection** — Compare HTTP vs Chrome to decide if browser rendering is needed
- **LLM Analysis** — Page classification (list/detail) + CSS selector rule generation
- **CSS Selector Validation** — Verify selectors against actual HTML
- **MCP Server** — Expose tools to LLM agents via stdio protocol

## Installation

```bash
pip install html-reader-llm
# or
uv tool install html-reader-llm
```

## CLI Usage

```bash
# Simplify HTML
html-reader-llm simplify < page.html
html-reader-llm simplify --file page.html --stats

# Detect if browser rendering is needed
html-reader-llm detect https://example.com
html-reader-llm detect https://example.com --proxy http://127.0.0.1:7890

# Full analysis with LLM
html-reader-llm analyze https://example.com
html-reader-llm analyze https://example.com --dry-run
html-reader-llm analyze https://example.com --base-url https://api.openai.com/v1 --api-key sk-xxx

# Fetch HTML
html-reader-llm fetch https://example.com
html-reader-llm fetch https://example.com --chrome
html-reader-llm fetch https://example.com --json --detect-only

# Validate CSS selectors
html-reader-llm validate https://example.com --rules-file selectors.json
html-reader-llm validate --html-file page.html --rules-file selectors.json --json
```

## MCP Mode

Expose tools to LLM agents via stdio protocol:

```bash
# Start MCP server
uvx html-reader-llm --mcp
```

### MCP Client Configuration

**Claude Desktop** (`claude_desktop_config.json`):
```json
{
  "mcpServers": {
    "html-reader-llm": {
      "command": "uvx",
      "args": ["html-reader-llm", "--mcp"]
    }
  }
}
```

**Cursor** (`.cursor/mcp.json`):
```json
{
  "mcpServers": {
    "html-reader-llm": {
      "command": "uvx",
      "args": ["html-reader-llm", "--mcp"]
    }
  }
}
```

### Available MCP Tools

| Tool | Input | Output |
|---|---|---|
| `simplify_html` | `html` | Simplified HTML + per-step stats |
| `prepare_analysis` | `html, max_tokens?` | System prompt + user prompt for LLM analysis |
| `validate_selectors` | `html, rules` | Per-selector validation results |
| `extract_data` | `html, rules` | Extracted data (detail) or list of items (list) |

All tools accept HTML strings directly — no network calls. The caller is responsible for fetching HTML.

### Agent Workflow

```
Agent fetches HTML (browser tool, curl, etc.)
  │
  ├─ simplify_html(html)
  │   → Cleaned HTML for analysis
  │
  ├─ prepare_analysis(html)
  │   → System prompt + user prompt
  │   → Agent analyzes page, generates extraction rules
  │
  ├─ validate_selectors(html, rules)
  │   → Verify rules work against actual HTML
  │
  └─ extract_data(html, rules)
      → Get extracted content (detail values or list items)
```

## Proxy Support

All network-dependent commands support `--proxy`:

```bash
html-reader-llm fetch https://example.com --proxy http://127.0.0.1:7890
html-reader-llm detect https://example.com --proxy http://127.0.0.1:7890
html-reader-llm analyze https://example.com --proxy http://127.0.0.1:7890
html-reader-llm validate https://example.com --rules-file rules.json --proxy http://127.0.0.1:7890
```

## License

MIT

