Metadata-Version: 2.4
Name: threadsense
Version: 0.2.1
Summary: Evidence-first discussion intelligence tooling.
License: MIT License
        
        Copyright (c) 2026 Tom Mathews
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Requires-Python: >=3.11
Requires-Dist: httpx>=0.28.1
Requires-Dist: pydantic-settings>=2.13.1
Requires-Dist: pydantic>=2.12.5
Requires-Dist: rich>=14.3.3
Description-Content-Type: text/markdown

# ThreadSense

**Discussion intelligence, not scraping.** ThreadSense is a reproducible pipeline that turns public discussion threads into structured, evidence-backed product and research intelligence.

## Why ThreadSense

Scrapers give you raw payloads. AI summarizers give you prose. Neither gives you a defensible basis for decisions.

ThreadSense keeps the evidence chain intact at every stage:

1. **Acquire** — source connectors fetch threads from Reddit, Hacker News, and GitHub Discussions
2. **Normalize** — source-specific payloads are mapped into a canonical thread model with provenance metadata
3. **Analyze** — deterministic extraction identifies issues, feature requests, themes, and sentiment — each linked to the comment that produced it
4. **Synthesize** — optional local-model inference adds summaries on top of the deterministic evidence layer
5. **Report** — structured outputs in Markdown, HTML, or JSON, with full traceability from finding back to source comment

Every stage produces a persisted, inspectable artifact. Rerun any stage independently. Diff results across runs. Audit exactly where a finding came from.

## What This Enables

### Single-thread analysis

Feed a discussion URL. Get structured findings — issues, requests, themes, severity — with every claim linked to the comment that produced it.

```bash
uv run threadsense run reddit \
  "https://www.reddit.com/r/ClaudeCode/comments/1ro0qbl/anyone_actually_built_a_second_brain_that_isnt/" \
  --format markdown \
  --with-summary \
  --summary-required
```

### Cross-thread research

Search a topic across multiple subreddits. ThreadSense deterministically selects, ranks, and analyzes matching threads, then synthesizes a corpus-level report.

```bash
uv run threadsense --output-format human research reddit \
  --query "second brain OR agentic PKM" \
  --subreddit ClaudeCode \
  --subreddit LocalLLaMA \
  --subreddit AI_Agents \
  --limit 5 \
  --per-subreddit-limit 3 \
  --with-summary
```

### Domain-aware analysis

The analysis layer uses a contract system with domain-specific vocabularies (developer tools, product feedback, hiring, research, financial markets, gaming). Each domain defines its own theme keywords, issue markers, and severity calibration — so analysis adapts to context rather than applying one-size-fits-all heuristics.

## Architecture

```text
fetch → normalize → analyze → [optional inference] → report
         ↓              ↓              ↓                ↓
     raw artifact   canonical     analysis         report artifact
                    artifact      artifact
```

- **Deterministic core** — parsing, normalization, scoring, and selection are reproducible across runs
- **Inference on top** — LLM synthesis is optional and layered over deterministic evidence, never a substitute for it
- **Stable artifacts** — each stage persists a separate JSON artifact with `schema_version` and SHA256 provenance
- **Fail fast** — invalid URLs, malformed payloads, and schema inconsistencies surface immediately

## Sources and Discovery

| Capability      | Reddit | Hacker News | GitHub Discussions |
| --------------- | :----: | :---------: | :----------------: |
| Thread analysis |  yes   |     yes     |        yes         |
| Topic research  |  yes   |      —      |         —          |

## Output Modes

| Mode    | Purpose                                          |
| ------- | ------------------------------------------------ |
| `json`  | Machine-readable payloads for downstream tooling |
| `human` | Rich terminal panels and summaries for operators |
| `quiet` | Status-only output for scripts and CI            |

```bash
uv run threadsense --output-format human research reddit ...
```

See [docs/output-modes.md](docs/output-modes.md) for details.

## Who This Is For

- **Product teams** validating pain points and feature demand from community discussions
- **Founders** doing market and competitor research across technical communities
- **DevRel teams** tracking developer workflow friction and tooling sentiment
- **Researchers** studying technical communities with reproducible methodology

## Quickstart

```bash
# 1. Install
uv sync

# 2. Validate local setup
uv run threadsense preflight

# 3. Analyze a single thread
uv run threadsense run reddit \
  "https://www.reddit.com/r/ClaudeCode/comments/1ro0qbl/anyone_actually_built_a_second_brain_that_isnt/"

# 4. Research a topic across subreddits
uv run threadsense research reddit \
  --query "second brain OR agentic PKM" \
  --subreddit ClaudeCode \
  --subreddit LocalLLaMA \
  --subreddit AI_Agents
```

## CLI Commands

| Command           | Purpose                                             |
| ----------------- | --------------------------------------------------- |
| `run`             | End-to-end single-thread pipeline                   |
| `research reddit` | Cross-subreddit topic research and corpus synthesis |
| `fetch`           | Acquire raw thread data                             |
| `normalize`       | Map raw data to canonical model                     |
| `analyze`         | Deterministic evidence extraction                   |
| `infer`           | LLM-assisted synthesis                              |
| `report`          | Generate output reports                             |
| `corpus`          | Build and analyze cross-thread corpora              |
| `inspect`         | Examine persisted artifacts                         |
| `batch run`       | Process multiple threads                            |
| `preflight`       | Validate local environment                          |
| `serve`           | Local API server                                    |

Full command reference: [docs/usage.md](docs/usage.md)

## Artifact Storage

Every pipeline run produces inspectable artifacts under `.threadsense/`:

```text
.threadsense/
├── raw/<source>/          # Source payloads as fetched
├── normalized/<source>/   # Canonical thread model
├── analysis/<source>/     # Evidence-linked findings
├── reports/<source>/      # Rendered reports
├── corpora/<corpus-id>/   # Manifest, analysis, and report
└── batches/               # Batch run metadata
```

Details: [docs/artifacts.md](docs/artifacts.md)

## Local Runtime

ThreadSense runs without a local model for deterministic analysis. Summaries and synthesis require a local OpenAI-compatible endpoint (default: `http://127.0.0.1:8080/v1/chat/completions`).

Details: [docs/local-runtime-contract.md](docs/local-runtime-contract.md)

## Documentation

| Document                                                    | Content                             |
| ----------------------------------------------------------- | ----------------------------------- |
| [usage.md](docs/usage.md)                                   | Command reference                   |
| [research-reddit.md](docs/research-reddit.md)               | Reddit topic research workflow      |
| [output-modes.md](docs/output-modes.md)                     | JSON, human, and quiet output modes |
| [artifacts.md](docs/artifacts.md)                           | Artifact types and storage layout   |
| [overview.md](docs/overview.md)                             | Product and workflow overview       |
| [system-design.md](docs/system-design.md)                   | Architecture and system boundaries  |
| [local-runtime-contract.md](docs/local-runtime-contract.md) | Local inference contract            |
| [pitch.md](docs/pitch.md)                                   | Product positioning                 |

## Validation

```bash
uv run ruff check
uv run ruff format --check .
uv run mypy --strict src tests
uv run pytest
```

## Current Limits

- Topic research is implemented for Reddit; other source discovery workflows are planned
- Reddit research queries support `OR`/`|` clause unions only (intentionally narrow for deterministic alignment)
- Corpus reports are Markdown only
- The local API is a trusted local surface, not a hardened public service

## Direction

- Richer corpus presentation and operator workflows
- Discovery workflows beyond Reddit
- Evaluation and replay benchmarking
- Source-distribution and research-quality reporting
