Metadata-Version: 2.5
Name: dlightrag
Version: 2.0.2
Summary: Multimodal RAG with knowledge graph and contextual intelligence
Project-URL: Homepage, https://github.com/hanlianlu/dlightrag
Project-URL: Repository, https://github.com/hanlianlu/dlightrag
Project-URL: Issues, https://github.com/hanlianlu/dlightrag/issues
Maintainer: HanlianLyu, hllyu
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Requires-Python: <3.15,>=3.14
Requires-Dist: aiobotocore>=3.9.0
Requires-Dist: aiofiles>=24.1.0
Requires-Dist: anthropic>=1.3.0
Requires-Dist: asyncpg>=0.31.0
Requires-Dist: azure-storage-blob>=12.30.1
Requires-Dist: bm25s>=0.3.11
Requires-Dist: botocore>=1.43.3
Requires-Dist: charset-normalizer>=3.4.0
Requires-Dist: defusedxml>=0.7.1
Requires-Dist: dlightrag-memory==2.0.2
Requires-Dist: fastapi>=0.141.1
Requires-Dist: google-genai>=2.21.0
Requires-Dist: httpx>=0.28.1
Requires-Dist: json-repair>=0.63.4
Requires-Dist: langchain-text-splitters
Requires-Dist: langfuse>=4.15.1
Requires-Dist: lightrag-hku>=1.5.7
Requires-Dist: lingua-language-detector>=2.2.0
Requires-Dist: markdown-it-py[plugins]>=3.0
Requires-Dist: markitdown[docx,pdf,pptx,xlsx]>=0.1.7
Requires-Dist: mcp>=2.1.1
Requires-Dist: nh3>=0.3.7
Requires-Dist: numpy>=2.5.0
Requires-Dist: openai>=2.54.0
Requires-Dist: openpyxl>=3.1.5
Requires-Dist: pathspec>=1.0.4
Requires-Dist: pgvector>=0.5.0
Requires-Dist: pillow>=12.3.0
Requires-Dist: pydantic-settings>=2.15.0
Requires-Dist: pydantic>=2.13.5
Requires-Dist: pygments>=2.17
Requires-Dist: pyjwt>=2.8.0
Requires-Dist: pypdfium2>=5.13.0
Requires-Dist: python-docx>=1.2.0
Requires-Dist: python-dotenv>=1.2.1
Requires-Dist: python-multipart>=0.0.22
Requires-Dist: pyyaml>=6.0
Requires-Dist: uvicorn[standard]>=0.52.1
Description-Content-Type: text/markdown

# DlightRAG

[![PyPI](https://img.shields.io/pypi/v/dlightrag)](https://pypi.org/project/dlightrag/)
[![CI](https://github.com/hanlianlu/dlightrag/actions/workflows/ci.yml/badge.svg)](https://github.com/hanlianlu/dlightrag/actions/workflows/ci.yml)
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/hanlianlu/DlightRAG)

DlightRAG is a production multimodal RAG service built on LightRAG. It combines
knowledge-graph and vector retrieval with metadata filtering, BM25, visual
retrieval, reranking, citations, highlights, and durable agentic answers. The
same runtime is available through Web, REST, MCP, and an in-process Python API.

**Runtime:** Python ≥3.14.7,<3.15 · PostgreSQL 18 ecosystem · Apache-2.0

## Architecture

<p align="center">
  <img src="docs/architecture.svg" alt="DlightRAG system context showing browser, REST, MCP, and embedded callers; optional enterprise identity and Web edge boundaries; external integrations; PostgreSQL; and corpus artifacts" width="1180" />
</p>

LightRAG supplies graph and vector retrieval. DlightRAG owns product policy,
multimodal alignment, durable ingestion and answers, security, storage adapters,
and public interfaces. Fast and Research answers share one durable conversation
tree; Research adds a per-run workspace, tools, memory, and child agents.
See [Architecture](docs/architecture.md) for module and storage ownership.

## Deployment Paths

| Path | PostgreSQL | Parser | Security |
|---|---|---|---|
| Local Docker | Compose PG18 | Self-hosted MinerU by default | Loopback, `auth_mode: none` |
| Native API | Compose or external PG18 | Any reachable MinerU or Docling | Local or explicit auth |
| Shared service | Managed or self-hosted PG18 | Independently operated parser | `simple` or `jwt` |
| Enterprise | Managed PG18 | Independently operated parser | JWKS plus claim access control |

The parser runs outside the DlightRAG app container. The checked-in Docker
configuration uses self-hosted MinerU at
`http://host.docker.internal:8210`. Docling and MinerU cloud remain supported.

## Quick Start

Install [Docker + Compose](https://docs.docker.com/get-docker/),
[`uv`](https://docs.astral.sh/uv/), `git`, and `make`.

### Interactive setup

```bash
git clone https://github.com/hanlianlu/dlightrag.git
cd dlightrag
uv run prerequisite_setup.py
```

The wizard configures models, parser, secrets, and the local stack. It defaults
to self-hosted MinerU and is safe to rerun.

### Manual setup

```bash
git clone https://github.com/hanlianlu/dlightrag.git
cd dlightrag
cp .env.example .env
```

Add the keys required by your `config.yaml` model blocks:

```bash
DLIGHTRAG_MODELS__CHAT__DEFAULT__API_KEY=...
DLIGHTRAG_MODELS__EMBEDDING__API_KEY=...
DLIGHTRAG_MODELS__CHAT__ROLES__EXTRACT__API_KEY=...
DLIGHTRAG_MODELS__CHAT__ROLES__KEYWORD__API_KEY=...
DLIGHTRAG_MODELS__CHAT__ROLES__QUERY__API_KEY=...
DLIGHTRAG_MODELS__CHAT__ROLES__VLM__API_KEY=...
DLIGHTRAG_MODELS__RERANK__API_KEY=...
```

Install and start MinerU, then start DlightRAG:

```bash
make mineru-install
make mineru-service-install  # installs and starts the background service
curl http://127.0.0.1:8210/health

docker compose up -d
docker compose ps
```

Open <http://localhost:8100/web/>. The stack publishes:

| Service | Address |
|---|---|
| REST API and Web | `http://127.0.0.1:8100` |
| MCP streamable HTTP | `http://127.0.0.1:8101` |
| PostgreSQL | `127.0.0.1:5432` |

Use `make mineru-api` when the platform cannot install a background user
service. To use Docling, replace the MinerU block in `config.yaml`; a commented
example is included there. Parser changes affect only new parses.

Configuration fields and parser operations are documented in
[Configuration](docs/configuration.md) and [Operations](docs/operations.md).

### Native API

Run PostgreSQL in Docker and the API on the host:

```bash
docker compose up -d postgres
uv sync
DLIGHTRAG_CORPUS__SIDECARS__MINERU__LOCAL_ENDPOINT=http://127.0.0.1:8210 \
  uv run dlightrag-api
```

The checked-in config is Docker-first, so a native process overrides the parser
host alias with loopback. Native managed inputs live under
`./dlightrag_storage/inputs/<workspace>`.

## Use DlightRAG

### Web

The Web UI supports workspace and file management, durable Fast and Research
conversations, answer attachments, citations, source highlights, child-agent
status, and typed Answer Artifacts. English, Chinese, and automatic browser
language modes are available under Settings.

### REST

Ingestion creates a background job; answers create durable runs and return
`202 Accepted`.

```bash
JOB=$(curl -sS -X POST http://localhost:8100/ingest \
  -H "Content-Type: application/json" \
  -d '{"source_type":"local","path":"report.pdf"}' | jq -r .job_id)
curl "http://localhost:8100/ingest/jobs/$JOB"

RUN=$(curl -sS -X POST http://localhost:8100/answer \
  -H "Content-Type: application/json" \
  -d '{"query":"What are the key findings?"}' | jq -r .run_id)
curl -N "http://localhost:8100/answer/$RUN/events"
curl "http://localhost:8100/answer/$RUN"
```

See [Interfaces](docs/interfaces.md) for requests, responses, pagination, SSE,
attachments, citations, and all transport contracts.

### MCP

For a local stdio client:

```json
{
  "mcpServers": {
    "dlightrag": {
      "command": "uvx",
      "args": ["dlightrag-mcp", "--env-file", "/absolute/path/to/.env"]
    }
  }
}
```

The Compose stack also exposes streamable HTTP on port 8101. MCP supports
retrieval, durable answers, steering, follow-up/fork/resume, child status,
corpus administration, and capability discovery. The authoritative tool list
is in [Interfaces](docs/interfaces.md#mcp-server).

### Python

```bash
uv add dlightrag
```

Create an application with `create_application(config)`, use
`application.corpora` for ingestion and `application.answers` for durable
answers, then call `application.aclose()`. Complete typed examples are in
[Interfaces](docs/interfaces.md#in-process-application).

## Core Concepts

| Concept | Meaning | Reference |
|---|---|---|
| Workspace | Isolation unit for indexed data, metadata, jobs, files, and queries | [Domain language](docs/domain-language.md) |
| Ingestion | One durable contract for local files, uploads, object storage, URLs, and SDK sources | [Interfaces](docs/interfaces.md#ingestion) |
| Retrieval | LightRAG mix retrieval plus metadata, BM25, visual fusion, rerank, and packing | [Retrieval and Answer](docs/retrieval-answer.md) |
| Answer run | One durable lifecycle shared by REST, MCP, Web, Python, and evaluation | [Durable Answer Runs](docs/durable-answer-runs.md) |
| Resource | Request-local attachment read deterministically or inspected visually on demand | [Retrieval and Answer](docs/retrieval-answer.md#answer-attachments-and-resources) |
| Source | Durable provenance and download contract for an ingested document | [Interfaces](docs/interfaces.md#sources) |

## Security

Loopback development can use `access.auth_mode: none`. Shared deployments should
use a bearer token or externally issued JWT; JWKS and claim-based workspace/action
rules are supported. DlightRAG does not issue tokens or replace an ingress WAF,
rate limiter, TLS terminator, or identity provider. See
[Security](docs/security.md).

## Development

```bash
uv sync
npm --prefix frontend ci
make hooks
make ci          # lint, security, format, types, architecture, frontend, unit
make ci-full     # plus integration tests
make ci-e2e      # plus E2E smoke
```

Use [Operations](docs/operations.md) for reset, rebuild, parser, Langfuse, and
maintenance runbooks. RAGAS evaluation is documented in
[Evaluation](docs/evaluation.md).

## Documentation

| Document | Owns |
|---|---|
| [Architecture](docs/architecture.md) | Runtime ownership, flows, storage topology, layering |
| [Domain Language](docs/domain-language.md) | Canonical product vocabulary |
| [Configuration](docs/configuration.md) | Configuration precedence, fields, defaults, examples |
| [Interfaces](docs/interfaces.md) | Python, REST, MCP, and Web contracts |
| [Retrieval and Answer](docs/retrieval-answer.md) | Retrieval, fusion, rerank, packing, citations, highlights |
| [Durable Answer Runs](docs/durable-answer-runs.md) | State machine, leases, events, recovery, retention |
| [Security](docs/security.md) | Authentication, authorization, ingress and content boundaries |
| [PostgreSQL](docs/postgresql.md) | PostgreSQL requirements, schema ownership, tuning |
| [Operations](docs/operations.md) | Executable runbooks and recovery workflows |
| [Evaluation](docs/evaluation.md) | RAGAS workflow |
| [Web Theme Design](docs/web-theme-design.md) | Web appearance and interaction decisions |

Plans, ADRs, and research notes under `docs/` are historical design evidence,
not required reading for operating DlightRAG.

## License

Apache License 2.0. See [LICENSE](LICENSE).

Built by HanlianLyu. Contributions welcome.
