Metadata-Version: 2.4
Name: wiki-forge
Version: 1.2.0
Summary: LLM Wiki — Auto-generate knowledge base from code & docs. CGC code intelligence, Q&A chat, knowledge graph, drift detection.
Project-URL: Homepage, https://github.com/tinhthanh/llm-wiki
Project-URL: Documentation, https://github.com/tinhthanh/llm-wiki/blob/main/docs/USER_GUIDE.md
Project-URL: Issues, https://github.com/tinhthanh/llm-wiki/issues
Author-email: FINOS Team <dev@finos.asia>
License: MIT
Keywords: code-to-doc,knowledge-base,knowledge-graph,llm,mcp,rag,wiki
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Documentation
Classifier: Topic :: Software Development :: Documentation
Requires-Python: >=3.11
Requires-Dist: codegraphcontext-rust>=0.9.10
Requires-Dist: duckdb>=1.0
Requires-Dist: httpx>=0.27
Requires-Dist: pydantic-settings>=2.0
Requires-Dist: pydantic>=2.0
Requires-Dist: python-frontmatter>=1.1
Requires-Dist: pyyaml>=6.0
Requires-Dist: rank-bm25>=0.2.2
Requires-Dist: rich>=13.0
Requires-Dist: typer>=0.9
Provides-Extra: all
Requires-Dist: fastapi>=0.100; extra == 'all'
Requires-Dist: markitdown[docx,pptx,xlsx]>=0.1; extra == 'all'
Requires-Dist: mcp>=1.0; extra == 'all'
Requires-Dist: networkx>=3.2; extra == 'all'
Requires-Dist: pdf-oxide>=0.3; extra == 'all'
Requires-Dist: pdfplumber>=0.10; extra == 'all'
Requires-Dist: pyarrow>=14.0; extra == 'all'
Requires-Dist: python-multipart>=0.0.6; extra == 'all'
Requires-Dist: uvicorn>=0.24; extra == 'all'
Provides-Extra: converters
Requires-Dist: markitdown[docx,pptx,xlsx]>=0.1; extra == 'converters'
Requires-Dist: pdf-oxide>=0.3; extra == 'converters'
Requires-Dist: pdfplumber>=0.10; extra == 'converters'
Requires-Dist: pyarrow>=14.0; extra == 'converters'
Provides-Extra: dev
Requires-Dist: mypy>=1.8; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest-mock>=3.12; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: respx>=0.21; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Provides-Extra: graph
Requires-Dist: networkx>=3.2; extra == 'graph'
Provides-Extra: mcp
Requires-Dist: mcp>=1.0; extra == 'mcp'
Provides-Extra: server
Requires-Dist: fastapi>=0.100; extra == 'server'
Requires-Dist: python-multipart>=0.0.6; extra == 'server'
Requires-Dist: uvicorn>=0.24; extra == 'server'
Description-Content-Type: text/markdown

# Wiki-Forge

**Auto-generate knowledge base from code & documents.**

[![PyPI](https://img.shields.io/pypi/v/wiki-forge.svg)](https://pypi.org/project/wiki-forge/)
[![Python 3.11+](https://img.shields.io/badge/Python-3.11+-blue.svg)](https://www.python.org)
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
[![MCP Compatible](https://img.shields.io/badge/MCP-Compatible-purple.svg)](https://modelcontextprotocol.io/)

Wiki-Forge indexes your codebase and documents into a structured, interlinked wiki — then lets you query it via Web UI, CLI, or AI agents (Claude Desktop, Cursor).

## Features

| Feature | Description |
|---------|-------------|
| **Code-to-Doc** | Auto-generate wiki from source code (19 languages) |
| **Document Ingestion** | PDF, DOCX, PPTX, HTML, Markdown → wiki pages |
| **Q&A Chat** | Ask questions in natural language, get answers with citations |
| **Knowledge Graph** | Interactive graph visualization (Sigma.js) |
| **Drift Detection** | Auto-detect code ↔ wiki contradictions (7 types) |
| **MCP Server** | Claude Desktop / Cursor integration (5 tools) |
| **Multi-Project** | Unified wiki across multiple repos |
| **CGC Engine** | Rust-accelerated code intelligence (Tree-sitter + KuzuDB) |

---

## Quick Start

### Install

```bash
# Full install (recommended)
pip install "wiki-forge[all]"

# Or minimal (CLI only, no server/MCP)
pip install wiki-forge
```

### 1. Index a codebase

```bash
cd my-project

# Free index (CGC only, no LLM cost)
wiki-forge init --cgc --no-llm

# Full wiki generation (uses LLM ~$0.04)
export GEMINI_API_KEY=your-key
wiki-forge init --cgc
```

### 2. Query

```bash
wiki-forge query "how does authentication work?"
```

### 3. Web UI

```bash
wiki-forge serve --port 5757
# Open http://localhost:5757/wiki/
```

### 4. AI Agent (Claude Desktop / Cursor)

```json
{
  "mcpServers": {
    "wiki-forge": {
      "url": "http://localhost:5757/mcp/sse"
    }
  }
}
```

---

## Install Options

```bash
pip install "wiki-forge[all]"          # Everything
pip install "wiki-forge[server]"       # Web UI + REST API
pip install "wiki-forge[cgc]"          # CGC Rust code intelligence
pip install "wiki-forge[mcp]"          # MCP server (Claude/Cursor)
pip install "wiki-forge[converters]"   # PDF, DOCX, PPTX converters
```

---

## CLI Commands

| Command | Description | LLM? |
|---------|-------------|------|
| `wiki-forge init [--cgc]` | Index repo + generate wiki | Yes |
| `wiki-forge init --cgc --no-llm` | CGC index only (free) | No |
| `wiki-forge update [--cgc]` | Incremental update (changed files only) | Yes |
| `wiki-forge query "question"` | Q&A against wiki | Yes |
| `wiki-forge serve` | Start Web UI + API server | No |
| `wiki-forge mcp` | Start MCP server (stdio) | No |
| `wiki-forge list` | List registered vaults | No |
| `wiki-forge register` | Register vault in global registry | No |
| `wiki-forge ingest doc.md` | Ingest a document | Yes |
| `wiki-forge lint` | Check wiki health (broken links, orphans) | No |
| `wiki-forge drift ./src` | Detect code ↔ wiki contradictions | Partial |
| `wiki-forge unify -p a:./a -p b:./b` | Merge multi-repo wikis | Yes |

---

## Code Intelligence (CGC)

Wiki-Forge uses [CodeGraphContext](https://github.com/tinhthanh/CodeGraphContext) (MIT) for code analysis:

- **19 languages**: Python, TypeScript, JavaScript, Rust, Go, Java, C/C++, C#, Ruby, PHP, Kotlin, Scala, Swift, Dart, Haskell, Elixir, Perl, and more
- **Rust-accelerated**: Tree-sitter parsing via Rust native extension
- **Call graph**: Function → function call edges (1,000-20,000+ per project)
- **Operational params**: Auto-extracts cron schedules, timeouts, config values

### Performance

| Repo | Files | Parse | Write | Resolve | Total | Edges |
|------|-------|-------|-------|---------|-------|-------|
| Python backend (300 files) | 300 | 1.0s | 7.3s | 3.4s | **11.6s** | 11,927 |
| Next.js frontend (300 files) | 300 | 3.0s | 4.6s | 1.5s | **9.1s** | 5,731 |
| Large TS monorepo (1220 files) | 1220 | 16s | 28s | 27s | **71s** | 49,958 |

---

## Docker

```bash
# Build
docker build -t wiki-forge \
  --build-context cgc=/path/to/CodeGraphContext .

# Run
docker run -d -p 5757:5757 \
  -v ./workspaces:/workspaces \
  -e GEMINI_API_KEY=your-key \
  wiki-forge
```

Or with docker-compose:

```bash
docker-compose up -d
open http://localhost:5757/wiki/
```

---

## MCP Tools (for AI Agents)

| Tool | Description | LLM Cost |
|------|-------------|----------|
| `list_vaults` | List all available projects | Free |
| `search_pages` | BM25 keyword search | Free |
| `read_page` | Read raw wiki page content | Free |
| `get_context` | Get ranked pages for a question (AI synthesizes locally) | **Free** |
| `query_wiki` | Full Q&A with LLM synthesis | ~$0.001 |

### Claude Desktop Config

```json
{
  "mcpServers": {
    "wiki-forge": {
      "url": "http://your-server:5757/mcp/sse"
    }
  }
}
```

### Claude Code Skill

```bash
# Install /wiki slash command globally
curl -sL https://raw.githubusercontent.com/tinhthanh/llm-wiki/main/scripts/install-skills.sh | bash

# Usage in Claude Code
/wiki "how does the auth module work?"
```

---

## Document Converters

| Format | Engine | License |
|--------|--------|---------|
| **PDF** | pdf_oxide | MIT/Apache-2.0 |
| **DOCX** | markitdown | MIT |
| **PPTX** | markitdown | MIT |
| **XLSX** | markitdown | MIT |
| **HTML** | markitdown | MIT |
| PDF (alternative) | pdfplumber | MIT |

All converters are MIT-licensed — safe for commercial SaaS.

---

## Architecture

```
wiki-forge/
├── src/llm_wiki/           ← Python backend
│   ├── cli.py              ← 17 CLI commands
│   ├── server.py            ← FastAPI (REST + SSE + MCP)
│   ├── mcp_server.py        ← MCP tools (5 tools)
│   ├── code/cgc_bridge.py   ← CGC Rust engine bridge
│   └── pipeline/
│       ├── ingest.py        ← Document ingestion (2-step LLM)
│       ├── ingest_code.py   ← Code-to-Doc (5 phases)
│       ├── query.py         ← Q&A (BM25 + graph re-ranking)
│       ├── drift.py         ← Drift detection (7 types)
│       └── lint.py          ← Wiki health check
├── web/                     ← React frontend (Vite + Tailwind)
├── workspaces/              ← Wiki data (markdown flat files)
├── docs/
│   ├── USER_GUIDE.md        ← End-user guide (Dev + BO)
│   ├── DEVELOPER_GUIDE.md   ← Developer guide
│   └── PROPOSAL.md          ← Architecture proposal
├── Dockerfile               ← Multi-stage (Rust + Node + Python)
└── docker-compose.yml       ← One-command deployment
```

---

## Cost Estimate

| Operation | LLM Calls | Cost (Gemini Flash Lite) |
|-----------|-----------|-------------------------|
| Index 300 files (code-to-doc) | ~25 | ~$0.04 |
| 1 Q&A query | 1 | ~$0.001 |
| Ingest 1 document | 2 | ~$0.002 |
| CGC code index | 0 | **$0 (free)** |
| Search / Browse / Graph | 0 | **$0 (free)** |

---

## Documentation

- [USER_GUIDE.md](docs/USER_GUIDE.md) — End-user guide for Dev + BO teams
- [DEVELOPER_GUIDE.md](docs/DEVELOPER_GUIDE.md) — Internal developer guide
- [PROPOSAL.md](docs/PROPOSAL.md) — Architecture & solution proposal

---

## License

MIT — free for commercial use.

**Dependencies**: All production dependencies are MIT/BSD/Apache-2.0 licensed.
