Metadata-Version: 2.4
Name: docureason-framework
Version: 1.1.3
Summary: DocuReason RAG: A Grounded Framework for Multimodal Document Retrieval and Reasoning. An Enterprise-Grade Tri-Path Multimodal RAG Framework for Text, Tables, and Visual Documents
Author-email: Arpit Kumar <kumararpit17773@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/arpitkumar2004/DocuReason
Project-URL: Documentation, https://github.com/arpitkumar2004/DocuReason#readme
Project-URL: Repository, https://github.com/arpitkumar2004/DocuReason.git
Project-URL: Changelog, https://github.com/arpitkumar2004/DocuReason/blob/main/CHANGELOG.md
Project-URL: Tracker, https://github.com/arpitkumar2004/DocuReason/issues
Project-URL: Evaluation Spec, https://github.com/arpitkumar2004/DocuReason/blob/main/Documentations/eval_method.md
Project-URL: Dataset Benchmark Spec, https://github.com/arpitkumar2004/DocuReason/blob/main/Documentations/datasets_eval.md
Project-URL: Architecture Deep Dive, https://github.com/arpitkumar2004/DocuReason/blob/main/Documentations/all_about_project.md
Project-URL: System Robustness Report, https://github.com/arpitkumar2004/DocuReason/blob/main/Documentations/robust_sys.md
Keywords: docureason,rag,multimodal,retrieval,duckdb,eval,nlp,llm,colpali,vector-search
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.0.0
Requires-Dist: transformers>=4.35.0
Requires-Dist: colpali-engine>=0.3.0
Requires-Dist: peft>=0.10.0
Requires-Dist: accelerate>=0.28.0
Requires-Dist: qdrant-client>=1.8.0
Requires-Dist: duckdb>=1.0.1
Requires-Dist: fastapi>=0.100.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: sentence-transformers>=2.2.0
Requires-Dist: bm25s>=0.1.0
Requires-Dist: docling>=2.0.0
Requires-Dist: easyocr>=1.7.0
Requires-Dist: uvicorn>=0.20.0
Provides-Extra: test
Requires-Dist: pytest>=7.0.0; extra == "test"
Requires-Dist: pytest-cov>=4.0.0; extra == "test"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "test"
Requires-Dist: httpx>=0.24.0; extra == "test"
Provides-Extra: lint
Requires-Dist: ruff>=0.1.0; extra == "lint"
Requires-Dist: mypy>=1.5.0; extra == "lint"
Requires-Dist: pip-audit>=2.6.0; extra == "lint"
Provides-Extra: build
Requires-Dist: build>=1.0.0; extra == "build"
Requires-Dist: twine>=4.0.2; extra == "build"
Requires-Dist: wheel>=0.41.0; extra == "build"
Provides-Extra: dev
Requires-Dist: docureason-framework[build,lint,test]; extra == "dev"
Dynamic: license-file

# DocuReason v1.1.3 — Enterprise-Grade Tri-Path Multimodal RAG Framework

[![CI Pipeline](https://github.com/arpitkumar2004/DocuReason/actions/workflows/ci.yml/badge.svg)](https://github.com/arpitkumar2004/DocuReason/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/docureason-framework.svg)](https://pypi.org/project/docureason-framework/)
[![Python](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12-blue.svg)](https://www.python.org/)
[![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)

**DocuReason** (`docureason-framework`) is an enterprise-grade, multimodal Retrieval-Augmented Generation (RAG) framework for Python. Built for multi-format enterprise document processing, DocuReason ingests, parses, segments, indexes, routes, retrieves, synthesizes grounded answers, and evaluates document corpora across text, tabular, and visual modalities.

---

## Table of Contents

1. [Overview](#overview)
2. [Key Features](#key-features)
3. [Architecture](#architecture)
4. [Pipeline Walkthrough & Segment Breakdown](#pipeline-walkthrough--segment-breakdown)
   - [Step 1: Ingestion & Layout Parsing](#step-1-multi-format-document-ingestion--layout-parsing)
   - [Step 2: Tri-Path Routing & Indexing](#step-2-tri-path-dynamic-query-routing--multimodal-indexing)
   - [Step 3: RRF Fusion & Reranking](#step-3-reciprocal-rank-fusion-rrf--cross-encoder-reranking)
   - [Step 4: Generation & NLI Attribution](#step-4-multimodal-generation--nli-faithfulness-attribution)
5. [Benchmark Results & Performance](#benchmark-results--performance)
6. [Installation](#installation)
   - [PyPI Installation](#pypi-installation)
   - [Install from Source](#install-from-source)
   - [Kaggle & Offline Notebook Installation](#kaggle--offline-notebook-installation)
7. [Quick Start](#quick-start)
   - [Python API](#1-python-api)
   - [CLI Commands](#2-cli-commands)
   - [FastAPI REST Server](#3-fastapi-rest-server)
   - [Interactive Web Dashboard](#4-interactive-web-dashboard)
8. [Underlying Open-Source Libraries & Documentation Links](#underlying-open-source-libraries--documentation-links)
9. [Standard Library API Reference](#standard-library-api-reference)
   - [`docureason.pipeline`](#docureasonpipeline)
   - [`docureason.ingestion`](#docureasoningestion)
   - [`docureason.serving`](#docureasonserving)
   - [`src.tripath.retrieval`](#srctripathretrieval)
   - [`src.tripath.attribution`](#srctripathattribution)
   - [`src.tripath.evaluation`](#srctripathevaluation)
10. [Fine-Tuning Dataset Exporter](#fine-tuning-dataset-exporter)
11. [REST API Endpoint Reference](#rest-api-endpoint-reference)
12. [Configuration Guide](#configuration-guide)
13. [Running Tests & Validation](#running-tests--validation)
14. [CI/CD & PyPI Release Engineering](#cicd--pypi-release-engineering)
15. [License](#license)

---

## Overview

Enterprise document collections contain a mix of prose, multi-row financial tables, and embedded diagrams or charts. Standard RAG systems treat all content as plain text, leading to severe accuracy degradation on tabular data and visual figures.

**DocuReason 1.1.3** addresses this via a **Tri-Path Multimodal RAG Architecture**:
1. **Text Path**: Combines dense vector embeddings ([SentenceTransformers](https://www.sbert.net/) / [Qdrant](https://qdrant.tech/)) with sparse keyword retrieval ([BM25S](https://github.com/xhluca/bm25s)).
2. **Table / Text-to-SQL Path**: Extracts tabular regions, serializes to Markdown/HTML/JSON schemas, and executes SQL aggregations using [DuckDB](https://duckdb.org/).
3. **Vision / Chart Path**: Uses visual feature extractors ([ColPali](https://github.com/illuin-tech/colpali) / [CLIP](https://huggingface.co/docs/transformers/model_doc/clip)) and [BLIP-2](https://huggingface.co/docs/transformers/model_doc/blip-2) figure captioning for visual chart understanding.

Incoming queries are dynamically routed using soft probability scoring, retrieved hits are merged via Reciprocal Rank Fusion (RRF) and Cross-Encoder reranking, and outputs undergo NLI-based attribution to guarantee zero hallucinations.

---

## Key Features

* **Multi-Format Document Parsing**: Native support for `.pdf`, `.docx`, `.pptx`, `.xlsx`, `.html`, `.csv`, `.md`, and `.txt`.
* **Deep Layout Segmentation**: Uses TableFormer + DocLayNet via [Docling](https://ds4sd.github.io/docling/) to separate text blocks, data tables, and figures.
* **EasyOCR Fallback**: Automatic scan detection and optical character recognition for scanned PDFs or image-only document pages using [EasyOCR](https://github.com/JaidedAI/EasyOCR).
* **FastAPI Serving & Visualization Dashboard**: Production REST API endpoints and an interactive local HTML pipeline dashboard.

---

## Architecture

```mermaid
flowchart TD
    A[Raw Enterprise Documents] --> B[FormatAwareLoader & DoclingLayoutParser]
    B --> C1[Text Regions]
    B --> C2[Table Regions]
    B --> C3[Figure / Image Regions]
    
    C1 --> D1[Dense & BM25S Index]
    C2 --> D2[DuckDB SQL Engine]
    C3 --> D3[BLIP-2 / CLIP Index]
    
    E[User Query] --> F[ConfigurableRouter]
    F -->|Text Intent| G1[Text Retrieval Path]
    F -->|Table Intent| G2[Table & Text-to-SQL Path]
    F -->|Vision Intent| G3[Vision & Chart Path]
    
    G1 & G2 & G3 --> H[Reciprocal Rank Fusion - RRF]
    H --> I[Cross-Encoder Reranking]
    I --> J[Multimodal Generation Engine]
    J --> K[NLI Faithfulness Attributor]
    K --> L[Grounded Response + Citations]
```

---

## Pipeline Walkthrough & Segment Breakdown

DocuReason breaks complex multimodal document reasoning into **4 clear, modular pipeline segments**:

### Step 1: Multi-Format Document Ingestion & Layout Parsing

![Step 1: Multi-Format Document Ingestion and Layout Parsing](assets/step1_ingestion.jpg)

* **Document Loading**: Ingests unstructured enterprise files (`.pdf`, `.docx`, `.xlsx`, `.pptx`, `.html`, scanned images).
* **Layout Parsing**: Uses **Docling** (TableFormer + DocLayNet) to segment documents into distinct structural regions:
  1. **Text Blocks**: Formatted text passages annotated with section hierarchy and breadcrumbs.
  2. **Data Tables**: Extracted grids serialized into GitHub Flavored Markdown and JSON schemas.
  3. **Figure Images**: Embedded visual charts, graphs, and diagrams paired with captions.
* **OCR Fallback**: Automatically triggers **EasyOCR** when scanned or non-searchable document pages are detected.

---

### Step 2: Tri-Path Dynamic Query Routing & Multimodal Indexing

![Step 2: Tri-Path Dynamic Query Routing and Multimodal Indexing](assets/step2_routing.jpg)

* **Modality Router**: A soft-probability classifier analyzes incoming user queries to determine the search intent (Text, Table/SQL, or Vision/Chart).
* **Specialized Tri-Path Processing**:
  - **Text Path**: Combines dense vector search via **Qdrant** / SentenceTransformers with sparse BM25 keyword search via **BM25S**.
  - **Table Path**: Translates natural language table queries into SQL aggregations (`SUM`, `AVG`, `GROUP BY`) executed in-memory with **DuckDB**.
  - **Vision Path**: Embeds graphic charts and figures using **ColPali** visual representations and **BLIP-2** image captioning.

---

### Step 3: Reciprocal Rank Fusion (RRF) & Cross-Encoder Reranking

![Step 3: Reciprocal Rank Fusion (RRF) & Cross-Encoder Reranking](assets/step3_fusion_reranking.jpg)

* **Rank Fusion**: Combines candidate hits retrieved across Text, Table, and Vision paths using **Reciprocal Rank Fusion (RRF)**:
  $$RRF_{score}(d) = \sum_{k} \frac{1}{60 + rank_k(d)}$$
* **Cross-Encoder Reranking**: Passes fused candidates through a Cross-Encoder Transformer to score query-context pairs and extract the top-$K$ most relevant context chunks.

---

### Step 4: Multimodal Generation & NLI Faithfulness Attribution

![Step 4: Multimodal Generation Engine and NLI Faithfulness Attribution](assets/step4_attribution.jpg)

* **Multimodal Answer Synthesis**: Feeds top-$K$ grounded context passages to the LLM generation engine.
* **NLI Entailment Guardrails**: The `NLIFaithfulnessAttributor` decomposes generated responses into individual claims and checks NLI premise-hypothesis entailment against retrieved document sources.
* **Grounded Output**: Returns zero-hallucination answers accompanied by page-level document citations.

---

## Benchmark Results & Performance

Evaluated against industry-standard multimodal and tabular benchmarks (FinQA, TAT-QA, ChartQA, DocVQA), **DocuReason** demonstrates significant performance improvements over generic single-vector RAG baselines:

| Evaluation Metric | Industry Standard Baseline (Single-Vector RAG) | Target SLA | DocuReason Achieved (Tri-Path RAG) | Relative Improvement |
| :--- | :---: | :---: | :---: | :---: |
| **`Recall@5`** | 0.450 (45.0%) | 0.850 | **0.612 (61.2%)** | **+16.2%** vs Baseline |
| **`nDCG@5`** | 0.400 (40.0%) | 0.800 | **0.552 (55.2%)** | **+15.2%** vs Baseline |
| **`Attribution Precision`** *(Faithfulness)* | 0.520 (52.0%) | 0.900 | **0.657 (65.7%)** | **+13.7%** vs Baseline |
| **`SQL Execution Success Rate`** | 0.380 (38.0%) | 0.850 | **0.578 (57.8%)** | **+19.8%** vs Baseline |
| **`Latency (P90)`** | 4500.0 ms | 1500.0 ms | **2100.0 ms** | **2400.0 ms Faster** |

> [!NOTE]
> *For detailed dataset formulations, mathematical metrics, and evaluation harness specs, refer to [Documentations/datasets_eval.md](Documentations/datasets_eval.md) and [Documentations/eval_method.md](Documentations/eval_method.md).*

---

## Installation

### PyPI Installation

Install the official published package from PyPI:

```bash
pip install docureason-framework
```

### Install from Source

Clone the repository and install in editable mode:

```bash
git clone https://github.com/arpitkumar2004/DocuReason.git
cd DocuReason
pip install -e .
```

Verify installation:

```python
import docureason
print(docureason.__version__)  # Output: 1.1.3
```

### Kaggle & Offline Notebook Installation

To install in Kaggle or offline environments without internet access, upload the `.whl` package file as a Kaggle Dataset and install:

```python
!pip install /kaggle/input/your-dataset-name/docureason_framework-1.1.3-py3-none-any.whl
```

Or install directly from GitHub:

```python
!pip install git+https://github.com/arpitkumar2004/DocuReason.git
```

---

## Quick Start

### 1. Python API

#### High-Level Ingestion and Indexing Pipeline
```python
from docureason import DocuReasonPipeline

# Initialize the offline ingestion pipeline
pipeline = DocuReasonPipeline(
    input_dir="samples",
    output_dir="artifacts/my_index"
)

# Run document parsing, layout segmentation, table serialization, and index generation
report = pipeline.run()
print(f"Processed {report['document_count']} documents and {report['chunk_count']} chunks.")
```

#### Online Query Execution & Answer Serving
```python
from docureason.serving import QueryService

# Initialize the end-to-end serving query engine
service = QueryService(
    input_dir="samples",
    output_dir="artifacts/my_index"
)

# Execute a multimodal query
response = service.query("What was the Q3 revenue growth shown in the comparison table?")

print("Answer:", response["answer"])
print("Routing:", response["route"])
print("Top Document:", response["results"][0]["document_id"])
```

### 2. CLI Commands

DocuReason provides built-in command-line interfaces:

```bash
# Execute the full end-to-end processing pipeline
python -m docureason --input-dir samples --output-dir artifacts/test_run

# Or run via script
python scripts/run_pipeline.py
```

### 3. FastAPI REST Server

Launch the production REST API server:

```bash
uvicorn src.tripath.serving.main:app --host 0.0.0.0 --port 8000 --reload
```

### 4. Interactive Web Dashboard

Launch the local HTML dashboard to inspect pipeline metrics and indices visually:

```bash
python scripts/serve_dashboard.py
```
Open browser at: `http://127.0.0.1:8001`

---

## Underlying Open-Source Libraries & Documentation Links

DocuReason builds upon industry-standard machine learning and data processing libraries. Below is the mapping of components to their official documentation:

| Component / Engine | Purpose in DocuReason | Official Library Documentation | Primary Function / Class Used |
| :--- | :--- | :--- | :--- |
| **Docling** | Deep document layout parsing & TableFormer | [Docling Documentation](https://ds4sd.github.io/docling/) | [`DocumentConverter`](https://ds4sd.github.io/docling/concepts/architecture/) |
| **DuckDB** | In-memory Text-to-SQL tabular execution | [DuckDB Python API](https://duckdb.org/docs/api/python/overview.html) | [`duckdb.connect()`](https://duckdb.org/docs/api/python/overview.html#querying) |
| **Qdrant** | High-performance vector index storage | [Qdrant Documentation](https://qdrant.tech/documentation/) | [`QdrantClient`](https://qdrant.tech/documentation/concepts/collections/) |
| **BM25S** | Fast sparse lexical search engine | [BM25S GitHub](https://github.com/xhluca/bm25s) | [`bm25s.BM25`](https://github.com/xhluca/bm25s#quick-start) |
| **SentenceTransformers** | Dense vector text embeddings | [SentenceTransformers Docs](https://www.sbert.net/) | [`SentenceTransformer.encode()`](https://www.sbert.net/docs/package_reference/SentenceTransformer.html) |
| **Hugging Face Transformers** | Cross-Encoder reranking & NLI entailment | [Transformers Documentation](https://huggingface.co/docs/transformers/) | [`AutoModelForSequenceClassification`](https://huggingface.co/docs/transformers/main_classes/model) |
| **BLIP-2** | Image & chart visual captioning | [BLIP-2 Model Docs](https://huggingface.co/docs/transformers/model_doc/blip-2) | [`Blip2ForConditionalGeneration`](https://huggingface.co/docs/transformers/model_doc/blip-2) |
| **ColPali & CLIP** | Multi-modal visual feature extraction | [ColPali Repository](https://github.com/illuin-tech/colpali) | [`ColPaliForRetrieval`](https://github.com/illuin-tech/colpali) |
| **EasyOCR** | Scanned document OCR fallback engine | [EasyOCR Documentation](https://github.com/JaidedAI/EasyOCR) | [`easyocr.Reader`](https://github.com/JaidedAI/EasyOCR#usage) |
| **FastAPI** | Asynchronous HTTP REST microservice | [FastAPI Documentation](https://fastapi.tiangolo.com/) | [`FastAPI()`](https://fastapi.tiangolo.com/tutorial/first-steps/) |
| **MLflow** | Metrics logging & experiment tracking | [MLflow Documentation](https://mlflow.org/docs/latest/index.html) | [`mlflow.log_metrics()`](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.log_metrics) |

---

## Standard Library API Reference

### `docureason.pipeline`

#### `class docureason.pipeline.DocuReasonPipeline(input_dir: str | Path, output_dir: str | Path)`
High-level offline ingestion pipeline orchestrator. Manages layout parsing, table serialization, OCR fallback, figure captioning, and vector index construction.

* **Parameters:**
  * `input_dir` (*str | Path*): Directory path containing raw enterprise documents.
  * `output_dir` (*str | Path*): Directory path where index artifacts are stored.

##### `run() -> Dict[str, Any]`
Executes end-to-end layout segmentation, table processing, vector indexing, and artifact generation.

---

### `docureason.ingestion`

Multi-format document loaders, vision layout parsers, OCR fallback engines, and table serializers.

#### `class docureason.ingestion.DoclingLayoutParser(page_batch_size: int = 1, do_ocr: bool = False)`
Deep layout parsing wrapper utilizing Docling (TableFormer + DocLayNet) to segment text, tables, and figures.

##### `parse(document_path: str | Path) -> List[Region]`
Parses `document_path` and returns typed region bounding boxes and layouts.

#### `class docureason.ingestion.TableSerializer()`
Serializes tabular document regions into GFM Markdown tables, HTML representations, and DuckDB JSON schemas.

##### `serialize(table_region: Region) -> Dict[str, Any]`
Converts `table_region` into linearized Markdown, HTML, and structured schema dictionary `{"columns": [...], "rows": [[...]]}`.

---

### `docureason.serving`

Synchronous and asynchronous query services for production serving.

#### `class docureason.serving.QueryService(input_dir: str | Path, output_dir: str | Path)`
Production query service providing dynamic query routing, multi-path retrieval, RRF fusion, reranking, and generation.

##### `query(text: str) -> Dict[str, Any]`
Executes search, fusion, reranking, and generation for input query `text`.

---

### `src.tripath.retrieval`

Tri-path retrieval engines (Text, Table/SQL, Vision), chart understanding, and cross-encoder rankers.

#### `class src.tripath.retrieval.hybrid_retriever.HybridRetriever()`
Full multi-path retriever integrating routing, sub-path retrieval, Reciprocal Rank Fusion (RRF), parent-child chunk expansion, and cross-encoder reranking.

#### `class src.tripath.retrieval.table_sql.TableSQLRetriever()`
Text-to-SQL retriever executing dynamic queries over DuckDB in-memory database tables.

#### `class src.tripath.retrieval.ranker.Ranker()`
Cross-encoder relevance scoring module.

##### `rank(query: str, candidates: List[Dict[str, Any]]) -> List[Dict[str, Any]]`
Re-scores candidate chunks against `query` using cross-encoder attention and returns sorted top hits.

---

### `src.tripath.attribution`

Claim attribution and NLI faithfulness engine.

#### `class src.tripath.attribution.nli_attributor.NLIFaithfulnessAttributor()`

##### `attribute(answer: str, evidence: List[Dict[str, Any]]) -> Dict[str, Any]`
Deconstructs `answer` into discrete sentence claims and computes entailment precision against `evidence`.

---

### `src.tripath.evaluation`

Evaluation harness, benchmark runners, and ablation studies.

#### `class src.tripath.evaluation.eval_harness.EvaluationHarness(output_dir: str | Path)`

##### `evaluate_single(query: str, results: List[dict], relevant_ids: Optional[List[str]] = None) -> Dict[str, float]`
Computes retrieval performance metrics including Recall@K, nDCG@K, MRR, TEDS, NLI Faithfulness, and SLA target verification.

---

## Fine-Tuning Dataset Exporter

DocuReason provides a built-in `DatasetExporter` module to export processed multi-modal corpora and query logs into SFT (Supervised Fine-Tuning) and DPO (Direct Preference Optimization) dataset formats compatible with HuggingFace `datasets`:

```python
from src.tripath.evaluation.dataset_exporter import DatasetExporter

exporter = DatasetExporter(output_dir="artifacts/my_index")

# Export fine-tuning dataset for SLM training
dataset_path = exporter.export_fine_tuning_dataset(
    output_format="jsonl",
    split="train"
)
print("Exported dataset to:", dataset_path)
```

---

## REST API Endpoint Reference

When running `uvicorn src.tripath.serving.main:app --port 8000`, the server exposes the following OpenAPI endpoints:

| Method | Endpoint | Description | Request Body / Parameters |
| :--- | :--- | :--- | :--- |
| `GET` | `/health` | Server readiness check | None |
| `GET` | `/api/report` | Returns last pipeline execution report | None |
| `POST` | `/query` | Executes multimodal query and returns answer | `{"query": "string", "input_dir": "samples"}` |
| `POST` | `/api/ingest` | Triggers document ingestion pipeline | `{"input_dir": "samples", "output_dir": "artifacts/run"}` |
| `POST` | `/api/evaluate` | Evaluates retrieval metrics for query | `{"query": "string", "relevant_ids": ["doc_1"]}` |
| `GET` | `/api/benchmarks`| Returns loaded benchmark dataset spec | None |

---

## Configuration Guide & Developer Transparency

DocuReason provides a PyTorch-like configuration experience that prioritizes **developer transparency** and **fail-fast validation**.

### Minimum Required Developer Inputs

To execute any pipeline or query service, developers only need to specify **two minimum required inputs**:

1. **`input_dir`** (Data Corpus Path): Path to local document directory containing `.pdf`, `.docx`, `.pptx`, `.xlsx`, `.html`, `.csv`, or `.txt` files.
2. **`output_dir`** (Artifact Target Path): Writable path for generated index artifacts (`corpus.json`, `index.json`, vector stores).

If required inputs are omitted or point to non-existent/empty directories, DocuReason raises developer-friendly exceptions (`MissingRequiredConfigError`, `InvalidCorpusError`) before starting any heavy computation.

```python
from docureason import DocuReasonPipeline, DocuReasonConfig

# Minimum required developer inputs with balanced default configuration
pipeline = DocuReasonPipeline(
    input_dir="data/my_corpus",
    output_dir="artifacts/my_index",
    config="balanced",  # Or 'quality_max', 'latency_optimized', 'low_resource_cpu'
    verbose=True,       # Displays the Configuration Transparency Summary on startup
)
```

### PyTorch-Style Configuration Transparency

When initializing `DocuReasonPipeline` or `QueryService`, DocuReason automatically logs/prints an explicit **Configuration Transparency Summary**. Developers never need to look into internal framework code to verify active baseline defaults:

```text
================================================================================
[DocuReason Framework v1.1.3] Configuration & Pipeline Transparency
================================================================================
[Required Developer Inputs]:
  - input_dir (Data Corpus Path) : data/my_corpus [VERIFIED - 12 supported file(s)]
  - output_dir (Artifact Path)  : artifacts/my_index [VERIFIED - target ready]
--------------------------------------------------------------------------------
[Framework Active Layer Configurations & Defaults]:
  - Config Preset Profile       : 'balanced' (Framework Default Preset)
  [1. Ingestion Layer Defaults]:
      • OCR Fallback Enabled     : True (char_threshold=50, dpi=150)
      • Chunking Tokens          : child_chunk=256, parent_region=1024, overlap=32
  [2. Indexing Layer Defaults]:
      • Domain / Vector Model    : domain='general', model='Default (sentence-transformers/all-MiniLM-L6-v2)'
      • FAISS Index Configuration: index_type='hnsw', hnsw_m=32, ef_construction=200, ef_search=64
      • BM25S Parameters         : k1=1.5, b=0.75
  [3. Intent Router Layer Defaults]:
      • Activation Threshold     : 0.35 (sigmoid_lambda=1.2)
  [4. Hybrid Retrieval Layer Defaults]:
      • Top-K Per Modality       : text=20, table=20, vision=20 (RRF k=60)
  [5. Cross-Encoder Reranker Defaults]:
      • Model & Target Top-K     : model='cross-encoder/ms-marco-MiniLM-L-6-v2', final_top_k=5, parent_expansion=True
  [6. Multimodal Generation Defaults]:
      • Model Backend & Path     : backend='auto', model='deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B'
      • Context Token Budget     : max_context_tokens=4096, temp=0.1, max_new_tokens=512
  [7. Faithfulness Attribution Defaults]:
      • NLI Model & Threshold    : enable_nli=True, model='cross-encoder/nli-deberta-v3-small', threshold=0.5
================================================================================
```

### On-Demand Inspection & YAML Overrides

Developers can inspect active configuration summaries programmatically or override settings via YAML (`configs/config.yaml`) or environment variables (`DOCUREASON_SECTION_KEY`):

```python
config = DocuReasonConfig.load_from_yaml("configs/config.yaml")
config.print_summary(input_dir="samples", output_dir="artifacts/run")
```

---

## Running Tests & Validation

DocuReason maintains a comprehensive test suite covering all modules:

```bash
# 1. Install development & testing extras
pip install -e ".[dev]"

# 2. Run pytest across all test modules
python -m pytest -v

# 3. Run Ruff code quality check
ruff check .

# 4. Verify local PyPI package build and metadata
python scripts/verify_pypi_package.py
```

---

## CI/CD & PyPI Release Engineering

DocuReason incorporates an enterprise-grade CI/CD pipeline powered by **GitHub Actions** and PyPI **OIDC Trusted Publishing**:

- **Continuous Integration (`.github/workflows/ci.yml`)**:
  - Triggers on all pushes and pull requests targeting `main`.
  - Runs syntax linting (`ruff`), static type checking (`mypy`), and vulnerability audits (`pip-audit`).
  - Executes unit and integration test matrix across **Python 3.10, 3.11, and 3.12**.
  - Validates package metadata using PyPA `build` and `twine check --strict`.
- **PyPI Release Pipeline (`.github/workflows/release-pypi.yml`)**:
  - Automatically triggered upon creating a published release on GitHub.
  - Deploys `docureason-framework` directly to PyPI using secure OIDC token authentication.
  - Automatically attaches `.tar.gz` and `.whl` distribution binaries to the GitHub Release.
- **Automated Maintenance (`.github/dependabot.yml`)**:
  - Checks weekly for dependency upgrades across Python packages and GitHub Actions.

For a full technical architectural deep dive into the CI/CD pipeline, see the [CI/CD Specification Document](Documentations/cicd_pipeline.md).

---

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
