Metadata-Version: 2.4
Name: retrieval-mcp
Version: 0.4.6
Summary: Installable MCP server for the Retrieval API: academic-paper search + journal memory + ACE playbook + semantic code search
Project-URL: Homepage, https://retrieval.rnarket.com
License: MIT
Keywords: ace,code-search,llm,mcp,model-context-protocol,papers,retrieval,search
Requires-Python: >=3.10
Requires-Dist: chonkie[code]>=1.6
Requires-Dist: httpx[socks]>=0.27
Requires-Dist: mcp<2,>=1.2.0
Requires-Dist: watchfiles>=1.1
Description-Content-Type: text/markdown

# retrieval-mcp

Current release: **0.4.6**, 31 tools. Platform architecture: [../overview.md](../overview.md).

The package currently declares `mcp>=1.2.0,<2`: the 31-tool server still uses the
official SDK v1 `FastMCP` API, while SDK v2 removed that import path. The upper
bound makes a normal install resolve a compatible SDK without client-side
`--with mcp<2` workarounds.

An **MCP server** for the Retrieval academic-paper API - semantic paper search,
document matching, ACE journal memory, and index inventory. Self-contained: it talks
to the backend over HTTP only (just `mcp` + `httpx`), so it installs anywhere with
`uvx` / `pip` - no repo checkout, no GPU, no models.

By default it targets the compute box on the lab LAN (`http://10.100.100.111:8000`),
which trusts LAN callers so **no key is needed**. Off-LAN, point `RETRIEVAL_API_URL` at
the public gateway (`https://retrieval.rnarket.com`) and set `RETRIEVAL_API_KEY` (`sk-...`).

## Tools

Every tool's full docstring (purpose + each argument with its default + an example)
is what your LLM sees - call them by name. Summary:

**Paper retrieval**

| Tool | What it does |
|------|--------------|
| `search_papers` | Semantic hybrid search over the live top-venue corpus (filters: `venue`, `year`, `title_only`) |
| `search_within_paper` | Every matching passage inside one paper |
| `get_paper_markdown` | Return one paper's complete Markdown, or Range-stream it to a caller-local file/directory |
| `download_paper_pdf` | Range-stream the original PDF to a caller-local path with validated resume and atomic publication |
| `download_papers` | Download `pdf`, `markdown`, or `both` for 1-10 paper IDs with two bounded concurrent transfers |
| `match_document` / `match_paper` | Content-nearest papers to a passage / to a paper |
| `list_conferences` / `corpus_stats` | Venue registry / corpus size |

**Journal work-memory** (scoped to the current project by default)

| Tool | What it does |
|------|--------------|
| `journal_record` | Record a work note (`memory`) or a file's current content (`doc`, latest-wins) |
| `journal_search` | Search memory - `keyword` (FTS5, no embedding) or `hybrid`/`dense`/`sparse` |
| `journal_recent` | List recent entries |
| `journal_index_dir` | Batch-index a local dir's files into the journal (latest-wins per file) |

**Code KB** (source stays local - only chunks are uploaded)

| Tool | What it does |
|------|--------------|
| `index_code` | AST-chunk a repo locally (40+ languages) and index it, scoped to you |
| `search_code` | Semantic code search with `path:line` citations |
| `index_inventory` | Your indexed-file tree: user -> host -> project -> dir -> file |

**Public Code** (admin-managed shared source corpus)

| Tool | What it does |
|------|--------------|
| `list_public_code_sources` | List configured sources, active revisions, coverage, profiles, and update state |
| `add_public_code_source` | Register an HTTPS Git upstream using server-side proxy/credential profile references |
| `check_public_code_updates` | Queue remote revision checks for selected sources or domains |
| `index_public_code_sources` | Queue bare mirror refresh, AST chunking, shared GPU embedding, and atomic promotion |
| `set_public_code_update_policy` | Change manual/scheduled update policy without exposing credentials |
| `public_code_job_status` | Poll durable source/file/chunk progress for checks and indexing |
| `search_public_code` | Flat domain or exact-repository hybrid/dense/keyword search with full code citations |

**ACE playbook** (accumulated, curated lessons per project)

| Tool | What it does |
|------|--------------|
| `ace_context_aware` / `ace_playbook` | Retrieve relevant / list all curated bullets |
| `ace_enhance_prompt` / `ace_smart_generate` | Attach playbook lessons to a prompt (no LLM call) |
| `ace_smart_reflect` | Curate a transferable lesson into the playbook (grow-and-refine dedup) |

### Code KB language coverage

`index_code` chunks 40+ languages structurally via tree-sitter (chonkie CodeChunker): Python,
TypeScript/TSX/JS/JSX (React), Java, Kotlin (incl. Jetpack Compose `.kt`/`.kts`),
Swift, Go, Rust, C/C++, C#, Ruby, PHP, Lua, Scala, Dart, R, Julia, Elixir, Erlang,
Haskell, OCaml, SQL, GraphQL, Protobuf, HTML, CSS/SCSS (Tailwind = CSS classes),
Vue, Svelte, shell, PowerShell, Dockerfile, Terraform/HCL, CMake, YAML/JSON/TOML/XML,
and more. Grammarless config/text files fall back to line-window chunks; docs (`.md`)
and binaries are skipped (docs belong in the journal via `journal_index_dir`).

## Install

### Claude Code

```bash
# LAN (no key):
claude mcp add retrieval -- uvx --from retrieval-mcp==0.4.6 retrieval-mcp
# Off-LAN (public gateway + key):
claude mcp add retrieval \
  --env RETRIEVAL_API_URL=https://retrieval.rnarket.com \
  --env RETRIEVAL_API_KEY=sk-... \
  -- uvx --from retrieval-mcp==0.4.6 retrieval-mcp
```

### Claude Desktop / any MCP client

`claude_desktop_config.json` (macOS: `~/Library/Application Support/Claude/`,
Windows: `%APPDATA%\Claude\`):

```json
{
  "mcpServers": {
    "retrieval": {
      "command": "uvx",
      "args": ["--from", "retrieval-mcp==0.4.6", "retrieval-mcp"],
      "env": {
        "RETRIEVAL_API_URL": "https://retrieval.rnarket.com",
        "RETRIEVAL_API_KEY": "sk-..."
      }
    }
  }
}
```

No `uv`? `pip install retrieval-mcp` then use `"command": "retrieval-mcp"`.

## Automatic code-index refresh

`index_code(...)` returns a local job ID before repository walking, hashing, AST chunking,
upload, GPU embedding, or Qdrant upsert completes. Poll that same ID with
`index_code_status()` through the `preparing`, backend queue, and terminal phases. A second
index request for the same scope reuses the active job instead of starting another scan.

With `auto_refresh=True`, the client then keeps one filesystem watcher for that absolute
repository path. Ordinary file events hash and chunk only the touched paths; ignore-rule
changes trigger a full reconcile. `search_code()` never waits for the watcher or indexing:
it returns the last completed snapshot and reports freshness separately.

Git repositories continue to honor Git's ignore rules by default. Any directory, including
non-Git projects, can add scope-relative patterns to `.retrievalignore`; callers can add
temporary patterns with `exclude_globs=["generated/**", "private.py"]`. Explicit
`delete_code()` cancels/fences stale refresh work and removes vectors, inventory, and the
saved refresh policy.

Windows drive and UNC scopes use `pathlib`'s Windows path semantics and one
case-insensitive canonical identity, while POSIX scope casing remains significant.
Historical mixed-case Windows scopes are included through bounded catalog aliases.
`path_glob` is always scope-relative; both slash styles are accepted, and `pathspec`
supplies basename, directory, `**`, and exact-file matching semantics.
Windows extended-length/device spellings (`\\?\`, `\\.\`) are rejected with guidance
to use the equivalent ordinary drive or UNC path, preventing duplicate scope identities.

## Config (env)

| Var | Default | Notes |
|-----|---------|-------|
| `RETRIEVAL_API_URL` | `http://10.100.100.111:8000` | LAN compute box (no key). Off-LAN, set to `https://retrieval.rnarket.com`. |
| `RETRIEVAL_API_KEY` | - | `sk-...` key for the gateway (create under `/auth/keys`). Required off-LAN. |
| Journal/code scope | current absolute directory path | The client derives scope from the directory you run or pass to each tool, so projects do not leak into each other. |

## Paper artifact transport

- `get_paper_markdown(..., save_path=None)` returns the complete JSON-backed Markdown
  to the MCP caller. With `save_path`, it instead uses the raw
  `GET /api/papers/{paper_id}/markdown` file route, so large Markdown is not buffered
  as JSON by the gateway or MCP process.
- Markdown files and PDFs use HTTP Range with validated `Content-Range`. If the
  connection closes after writing a valid prefix, the next request resumes from the
  exact local byte offset. Completed files are published atomically and are not
  overwritten unless `overwrite=True`.
- `download_papers` accepts at most 10 paper IDs, preflights all destination names,
  disambiguates sanitized filename collisions with a stable digest, and reuses the
  single-artifact Range paths with at most two concurrent transfers. Errors are
  reported per paper/artifact.
- The separate authenticated HTTP `POST /api/papers/download-batch` endpoint serves a
  one-shot ZIP for browser/API clients. It is capped at 10 papers and 1 GiB of source
  artifacts, allows at most two concurrent archive builds, and intentionally rejects
  Range because each generated archive is a new file.

## License

MIT
