Metadata-Version: 2.4
Name: omniseek
Version: 0.2.0
Summary: Your agent seeks what search can't find. A self-hosted perception MCP server that transcribes speech, reads behind logins, sees images and video frames, crosses languages, and remembers.
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/Battam1111/omniseek
Project-URL: Repository, https://github.com/Battam1111/omniseek
Project-URL: Issues, https://github.com/Battam1111/omniseek/issues
Project-URL: Changelog, https://github.com/Battam1111/omniseek/blob/main/CHANGELOG.md
Project-URL: Documentation, https://github.com/Battam1111/omniseek#readme
Keywords: mcp,deep-retrieval,ai-agent,llm,rag,citation-graph,transcription,web-scraping,self-hosted
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Internet :: WWW/HTTP :: Indexing/Search
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: mcp[cli]<2,>=1.0.0
Requires-Dist: semanticscholar>=0.10.0
Requires-Dist: praw>=7.7.0
Requires-Dist: atproto>=0.0.55
Requires-Dist: playwright>=1.40
Requires-Dist: httpx>=0.27
Requires-Dist: beautifulsoup4>=4.12
Requires-Dist: lxml>=5.0
Requires-Dist: feedparser>=6.0
Requires-Dist: markdownify>=0.13
Requires-Dist: pyyaml>=6.0
Requires-Dist: curl_cffi>=0.7
Requires-Dist: pydantic>=2.5
Requires-Dist: platformdirs>=4.0
Requires-Dist: python-dateutil>=2.8
Requires-Dist: yt-dlp>=2026.6.9
Requires-Dist: youtube-transcript-api>=0.6.2
Requires-Dist: python-pptx>=1.0
Requires-Dist: python-docx>=1.1
Requires-Dist: openpyxl>=3.1
Requires-Dist: pillow>=10.0
Requires-Dist: numpy>=1.26
Provides-Extra: pdf
Requires-Dist: pymupdf>=1.24; extra == "pdf"
Provides-Extra: asr
Requires-Dist: funasr>=1.0; extra == "asr"
Requires-Dist: imageio-ffmpeg>=0.5; extra == "asr"
Provides-Extra: recall
Requires-Dist: sentence-transformers>=3.0; extra == "recall"
Provides-Extra: ocr
Requires-Dist: rapidocr-onnxruntime>=1.2; extra == "ocr"
Provides-Extra: walled
Requires-Dist: patchright>=1.40; extra == "walled"
Requires-Dist: xhshow>=0.1.0; extra == "walled"
Provides-Extra: all
Requires-Dist: pymupdf>=1.24; extra == "all"
Requires-Dist: funasr>=1.0; extra == "all"
Requires-Dist: imageio-ffmpeg>=0.5; extra == "all"
Requires-Dist: patchright>=1.40; extra == "all"
Requires-Dist: xhshow>=0.1.0; extra == "all"
Requires-Dist: sentence-transformers>=3.0; extra == "all"
Requires-Dist: rapidocr-onnxruntime>=1.2; extra == "all"
Dynamic: license-file

<!-- mcp-name: io.github.Battam1111/omniseek -->

<div align="center">

<picture>
  <source media="(prefers-color-scheme: light)" srcset="assets/logo-icon.png">
  <img src="assets/logo-hero-dark.png" width="320" alt="OmniSeek">
</picture>

# OmniSeek

**Your agent seeks what search can't find.**

The answer is sitting in minute 47 of a podcast, three replies deep in a comment thread, behind a login, in another language. Your agent gets it anyway.

<sub>Self-hosted perception MCP server · one connection</sub>

[![CI](https://github.com/Battam1111/omniseek/actions/workflows/ci.yml/badge.svg)](https://github.com/Battam1111/omniseek/actions/workflows/ci.yml)
&nbsp;[![License](https://img.shields.io/badge/License-Apache_2.0-3B82F6?style=flat-square)](./LICENSE)
&nbsp;![Python](https://img.shields.io/badge/Python_3.11+-3B82F6?style=flat-square)
&nbsp;![Built for MCP](https://img.shields.io/badge/built_for-MCP-3B82F6?style=flat-square)
&nbsp;![Self-hosted](https://img.shields.io/badge/self--hosted-3B82F6?style=flat-square)

[Quick start](#quick-start) · [Tools](#tools) · [Configure](#configure) · [Contributing](#contributing)

**Languages:** English · [中文](docs/i18n/README_zh.md) · [日本語](docs/i18n/README_ja.md)

</div>

---

Search gives your agent indexed pages, in one language, in text, and stops there.

OmniSeek keeps seeking: through languages, logins, comment threads, audio, and pixels, all on your machine.

<div align="center">
  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="assets/demo-en-dark.png">
    <img src="assets/demo-en-light.png" alt="One real investigation: an English question about a changed rule. Search quotes the rule and stops; OmniSeek keeps seeking and comes back with a Chinese explainer transcribed, login-walled first-person timelines, the workaround buried in a comment thread, and the dissenting video note, each line attributed.">
  </picture>
</div>

It hears (local bilingual ASR, no cloud), sees (images and video frames, in-band), crosses languages (a Chinese query finds English results and vice versa), reads behind login walls (your credentials, your machine, off by default), and remembers (persistent retrieval memory plus a typed, source-traced evidence graph).

Every source in [the catalog](docs/sources.md), the curated roster of everything OmniSeek can reach, earned its place by beating plain search at one of five jobs (structure, unwalling, transcription, recall, monitoring): citation graphs, regulatory filings, login-walled forums, Chinese-language video. And the catalog is built to grow: a curator pipeline probes, judges, and admits new sources, and retires the ones that decay.

**[Worked examples, real outputs](docs/examples.md)** · **[A full case study](docs/case-study.md)**

---

## Quick start

### Docker (recommended)

```bash
git clone https://github.com/Battam1111/omniseek.git && cd omniseek
docker compose up -d
docker compose logs omniseek        # bearer token printed on first start
curl -s http://127.0.0.1:8765/healthz
```

Point your MCP client at `http://127.0.0.1:8765/mcp` with `Authorization: Bearer <token>`. The token is generated on first start and stored in `~/.omniseek/credentials/omniseek_http.json` (with the compose file, that's `./.omniseek/credentials/omniseek_http.json` on the host).

Two paths from here. The prebuilt core image (`docker pull ghcr.io/battam1111/omniseek`, amd64 + arm64) needs no build and carries every core sense; it is Apache-clean and ships without PDF reading, hearing (ASR + video frames), and login-walled sources. Wanting those extras is what triggers a local build: set `EXTRAS="[pdf,asr,walled]"` and run `docker compose build`, then `up -d` (the first build also fetches headless Chromium; later starts are instant). Optional but recommended: set `OMNISEEK_CONTACT_EMAIL` for a faster lane with Crossref, SEC, and Unpaywall.

### Without Docker

```bash
python -m venv .venv && . .venv/bin/activate
scripts/bootstrap.sh
python -m omniseek.serve_http
```

The bare install is the **Core** tier: every keyless API and static source, document reading minus PDF, and the lexical memory index. `pip install "omniseek[pdf,asr,recall,ocr]"` wakes the **Research** tier (PDF, hearing, cross-lingual vectors, OCR); `omniseek[walled]` adds the login-walled tier, which stays off until you bring your own accounts; `omniseek[all]` takes everything. The server prints which senses are online, and which are dormant, at every boot.

On Windows, run `bootstrap.sh` under Git Bash or WSL; Docker is the simplest path. For an always-on Linux service, see [`deploy/omniseek.service`](deploy/omniseek.service).

Prefer stdio? The install also ships an `omniseek` command that speaks MCP over stdio, for clients that launch servers themselves; [`Dockerfile.stdio`](Dockerfile.stdio) wraps the same thing in a container.

OmniSeek binds `127.0.0.1` and requires the bearer token on every request. Do not expose without a reverse proxy ([SECURITY.md](.github/SECURITY.md)).

---

## Tools

One MCP connection; no model, no agent loop inside. Your model thinks, your harness drives the loop, OmniSeek reaches. Start with `omniseek_search`; explore what's available with `omniseek_sources`.

| Tool | What it does |
|------|-------------|
| `omniseek_search` | Fan out across the whole catalog, deduplicate, rank. Cross-lingual (semantic + lexical). |
| `omniseek_read` | Normalize any URL or document (web page, PDF, arXiv) into clean text. |
| `omniseek_view` | Read images, document figures, video frames with vision. |
| `omniseek_transcribe` | Transcribe audio/video locally. Bilingual ASR, sliceable by timestamp. |
| `omniseek_field_skeleton` | Map a research field's citation neighborhood: foundational core vs. frontier. |
| `omniseek_resolve_identity` | Resolve a person's name to candidate author IDs across databases. |
| `omniseek_coauthors` | Map a researcher's collaboration network by joint-paper count. |
| `omniseek_institution_cohort` | List who actively publishes at a lab, scoped to a field. |
| `omniseek_paper_enrich` | Open-access PDF, retraction/integrity status, citation count for a paper. |
| `omniseek_paper_recommend` | Semantically similar papers (SPECTER embeddings) that keyword search misses. |
| `omniseek_graph` | Query the accumulated evidence graph: find, neighborhood, between, since, similar. |
| `omniseek_sensor` | Standing queries with novelty detection. Only tells you what is new. |
| `omniseek_ruling` | Record identity judgments (same/not-same) the graph applies at read time. |
| `omniseek_statement` | Record directed relations the graph carries forward. |
| `omniseek_curator_act` | Source lifecycle: submit, probe, judge, admit, retire. |
| `omniseek_curator_view` | Read the source-admission queue or a per-source audit dossier. |
| `omniseek_gather` | Run multiple tools in parallel, one response. |
| `omniseek_sources` | List and route: domains, regions, capabilities, health. |

The login-walled tier has no tool of its own: once you opt in per source, the same `omniseek_search(..., sources=["xiaohongshu"], raw=True)` runs through your own logged-in browser. See [walled sources](docs/walled-sources.md).

Full reference in **[tools.md](docs/tools.md)** · **[FAQ](docs/faq.md)**

Using Claude Code? [`skills/omniseek-investigate`](skills/omniseek-investigate/SKILL.md) ships the investigation methodology (sweep, zoom, structure) as a ready-made skill.

---

## Configure

OmniSeek is **catalog-first**: with no config, every benign source is on and login-walled sources are off. Tune in one file, `~/.omniseek/profile.json` ([example](deploy/profile.example.json)):

| Tier | Default |
|------|---------|
| **free** (public, no key) | **on** |
| **keyed** (a free or paid API key you supply) | on once the key is set |
| **walled** (a login you hold) | **off**; you bring your own browser |
| **circumvention** | **off**; none in the default pack |

Full reference: **[configuration](docs/configuration.md)** · **[walled sources](docs/walled-sources.md)** · **[legal posture](docs/LEGAL-POSTURE.md)**

---

## Why self-hosted

There is no OmniSeek cloud. No telemetry, no accounts, no relay: a query leaves your machine only as direct requests to the sources you enabled, and OmniSeek adds no other party to that path. Walled-source credentials stay in your own browser, presented only to the site they belong to; OmniSeek never stores, uploads, or even sees your passwords. The retrieval memory and evidence graph it accumulates over months are local files you own: stop running OmniSeek and you keep everything. Not a feature toggle. The architecture.

---

## Contributing

See [CONTRIBUTING.md](.github/CONTRIBUTING.md). The bar for a new source: it must beat plain web search via a mode (structure / unwall / transcribe / recall / monitor). The bar for fixing a decayed source: low, please do. `python tests/smoke.py` before you push.

By participating you agree to the [Code of Conduct](.github/CODE_OF_CONDUCT.md).

<div align="center">

---

**Your agent seeks what search can't find.**

[Apache-2.0](./LICENSE) · [NOTICE](./NOTICE) · [Security](.github/SECURITY.md) · [Cite](./CITATION.cff)

</div>
