Metadata-Version: 2.4
Name: vector-unforget
Version: 4.2.0
Summary: Enterprise-grade GDPR/CCPA Vector Unlearning & PII Oblivion Middleware.
Author: Toskurim
License: AGPL-3.0
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.24.0
Requires-Dist: scipy>=1.10.0
Requires-Dist: torch>=2.0.0
Requires-Dist: prometheus-client>=0.16.0
Requires-Dist: fastapi>=0.100.0
Requires-Dist: uvicorn>=0.22.0
Requires-Dist: pydantic>=2.0.0
Provides-Extra: dashboard
Requires-Dist: streamlit>=1.28.0; extra == "dashboard"
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: black; extra == "dev"
Requires-Dist: flake8; extra == "dev"
Requires-Dist: mypy; extra == "dev"
Dynamic: license-file

# VectorUnforget

[![version](https://img.shields.io/badge/version-4.2.0-blue.svg)](https://github.com/Toskurim/vector-unforget)
[![PyPI](https://img.shields.io/pypi/v/vector-unforget.svg)](https://pypi.org/project/vector-unforget/)
[![License](https://img.shields.io/badge/License-AGPL%20v3-blue.svg)](LICENSE)
[![python](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12%20%7C%203.14-blue.svg)](pyproject.toml)
[![tests](https://img.shields.io/badge/tests-38%2F38%20passing-brightgreen.svg)](tests/)
[![observability](https://img.shields.io/badge/observability-Prometheus%20ready-orange.svg)](vector_unforget/metrics.py)

**VectorUnforget** is an enterprise-grade AI middleware engine designed for verifiable PII erasure, concept unlearning, and GDPR (Art. 17) / CCPA compliance across vector databases and RAG pipelines.

---

### Operational Overview & Value Proposition

When a user requests their data be forgotten under GDPR/CCPA, traditional vector databases force you to delete records and re-index millions of embeddings from scratch—a process taking minutes or hours and burning substantial compute.

**VectorUnforget solves this in milliseconds without re-indexing:**

1. **Understands Natural Language Intent**: Accepts zero-config commands (e.g., *Forget all financial records for John Doe*) and automatically extracts entities and sensitive concept vectors via lightweight NLP and heuristics.
2. **Instant In-Place Unlearning**: Projects the embedding space into an orthogonal subspace ($O(N \cdot D)$), mathematically neutralizing the target concept to zero residual leakage while preserving the integrity of all unrelated vector relationships.
3. **Universal Vector DB Support**: Connects natively via unified adapters to **Milvus**, **Elasticsearch**, **Pinecone**, **Weaviate**, **LanceDB**, **Qdrant**, and **ChromaDB**, as well as **LangChain** and **LlamaIndex** RAG pipelines.
4. **Cryptographic Proof of Erasure**: Automatically generates tamper-proof **SHA-256 cryptographic audit certificates** for compliance officers and Data Protection Officers (DPO), tracked in real-time via Prometheus metrics and a built-in Streamlit dashboard.

---

## Core Capabilities

* **Subspace Orthogonal Projection**: Deterministic concept unlearning using SVD and PyTorch CUDA/CPU acceleration without full index re-indexing.
* **Cascading PII Entity Graph**: Multi-hop entity resolution with exponential confidence decay to erase connected identities.
* **Hybrid Search Scrubbing**: Unified dense vector orthogonalization combined with sparse lexical BM25 token redaction.
* **Production Vector DB Adapters**: Native drivers for **Milvus**, **Elasticsearch / OpenSearch**, **Pinecone**, **Weaviate**, and **LanceDB**.
* **RAG Framework Integrations**: Native middleware for **LangChain** (`VectorUnforgetRetriever`) and **LlamaIndex** (`VectorUnforgetNodePostprocessor`).
* **Cryptographic Audit Receipts**: Tamper-evident **SHA-256** erasure certificates verifying *Zero Residual Leakage* for Data Protection Officers.
* **MLOps Telemetry**: Built-in Prometheus exposition endpoint (`GET /metrics`) tracking unlearning throughput, SVD latency, and leakage distribution.
* **Interactive DPO Console**: Streamlit-based web interface for live PII remediation, graph traversal preview, and instant compliance certificate export.

---

## Quickstart

### 1. Installation

```bash
pip install vector-unforget
```

### 2. Python SDK Example

```python
from vector_unforget.subspace_projection import SubspaceProjector
from vector_unforget.compliance import ComplianceCertificateGenerator
import numpy as np

projector = SubspaceProjector(device="auto")
embeddings = np.random.randn(100, 768).astype(np.float32)
sensitive_concept = np.random.randn(768).astype(np.float32)

unlearned = projector.project_matrix_orthogonal(embeddings, sensitive_concept, normalize=True)

cert_gen = ComplianceCertificateGenerator()
cert = cert_gen.generate_certificate(
    request_id="REQ-GDPR-001",
    entity_identifier="user_9941",
    unlearned_vector_count=100,
    pre_unlearning_leakage=0.85,
    post_unlearning_leakage=0.002,
    regulation="GDPR_Art_17"
)
print("Receipt SHA-256:", cert["cryptographic_hash_sha256"])
```

### 3. Running the REST Gateway

```bash
uvicorn vector_unforget.api.server:create_app --factory --host 0.0.0.0 --port 8000
```

### 4. Running the Interactive DPO Console

```bash
streamlit run vector_unforget/dashboard.py
```

---

## REST Endpoints Overview

| Method | Endpoint | Description |
| :--- | :--- | :--- |
| `GET` | `/health` | Service status and compute device info |
| `GET` | `/metrics` | Prometheus metrics for MLOps observability |
| `POST` | `/v1/unlearn/batch` | High-throughput batch vector orthogonal projection |
| `POST` | `/v1/graph/resolve` | Cascading multi-hop PII entity discovery |
| `POST` | `/v1/audit/verify` | Adversarial probe vector leakage verification |
| `POST` | `/v1/audit/certificate` | Generate signed SHA-256 GDPR/CCPA erasure receipts |

---

## ⚡ Empirical Performance & Scalability Benchmarks

VectorUnforget replaces expensive vector database re-indexing with **O(N × D) Orthogonal Subspace Projection**, drastically cutting GDPR Art. 17 right-to-erasure latency while ensuring complete mathematical mitigation of target concept leakage.

### Measured Results (Host CPU vs Native Faiss C++ HNSW)

All values are *physically measured wall-clock times* across dense 768-dimensional float32 vector collections.

| Vectors (N) | Latency (VU Projection) | Baseline (Faiss C++ HNSWFlat) | Empirical Speedup | Memory Peak (MB) | Residual Concept Leakage | Cryptographic Audit Proof |
|---|---|---|---|---|---|---|
| **10,000** | **31.12 ms** | 336.14 ms | **10.8x** | 58.74 MB | `-0.0` (100% Scrubbed) | SHA-256 Verified |
| **50,000** | **146.73 ms** | 3.41 s | **23.2x** | 293.57 MB | `-0.0` (100% Scrubbed) | SHA-256 Verified |
| **100,000** | **304.28 ms** | 12.52 s | **41.1x** | 587.11 MB | `-0.0` (100% Scrubbed) | SHA-256 Verified |
| **250,000** | **773.19 ms** | 45.91 s | **59.4x** | 1,467.74 MB | `-0.0` (100% Scrubbed) | SHA-256 Verified |
| **500,000** | **1.52 s** | 109.16 s (~1.8 min) | **72.0x** | 2,935.44 MB| `-0.0` (100% Scrubbed) | SHA-256 Verified |

> 📊 **Full Methodology & Reproduction**: See [BENCHMARKS.md](BENCHMARKS.md) for execution parameters, memory profiling, and test scripts (`benchmarks/benchmark_unlearning.py`).

---

## 🏲 Enterprise Grade: Homonym Disambiguation & Lossless Rollback

In mission-critical enterprise environments, blind semantic unlearning risks homonym collisions (scrubbing unrelated individuals who share a name) and accidental data loss. VectorUnforget provides native modules to eliminate both failure modes:

### 1. Metadata-Scoped Scrubber (Homonym Safety)
Enforces multi-predicate metadata gating before isolating the sensitive concept subspace. Even if two records share identical names in text/embeddings, only the targeted subject (`user_id`, `ssn`, `tenant_id`) is scrubbed:

```python
from vector_unforget.enterprise import MetadataScopedScrubber

scrubber = MetadataScopedScrubber(embedding_dim=1536)

# Extracts centroid strictly from vectors matching user_id == 'user_101'
centroid, target_indices = scrubber.filter_and_extract_centroid(
    embeddings=active_embeddings,
    metadata_list=metadata_records,
    target_entity='Mario Rossi',
    match_predicates={'user_id': 'user_101', 'tenant_id': 'prod_eu'}
)
```

### 2. Transactional Delta Ledger & Lossless Rollback
Every unlearning operation calculates and logs the orthogonal deviation vector $\delta_i = (\hat{c}^T x_i)\hat{c}$ in a cryptographically signed buffer with a configurable TTL window. If an erasure was requested in error, vectors are restored instantly without rebuilding indices:

```python
from vector_unforget.enterprise import UnlearningRollbackManager

rollback_mgr = UnlearningRollbackManager(ttl_seconds=604800)  # 7-day safe window

# Stage unlearning with automatic cryptographic delta capture
scrubbed_vectors, tx_id = rollback_mgr.stage_unlearning(
    embeddings=active_embeddings,
    target_indices=target_indices,
    centroid=centroid
)

# Accidental deletion? Revert in O(1) time complexity
restored_vectors = rollback_mgr.rollback_transaction(
    current_embeddings=scrubbed_vectors,
    tx_id=tx_id
)
```

---

## License & Commercial Use

VectorUnforget is distributed under the **GNU Affero General Public License v3.0 (AGPL-3.0)**. See [LICENSE](LICENSE) for details.

* **Open-Source / Community**: Free to use, evaluate, and modify under AGPLv3 copyleft terms.
* **Commercial / Enterprise Licensing**: For proprietary software embedding, closed-source SaaS integrations, or custom enterprise terms without AGPLv3 copyleft obligations, please open a GitHub Discussion or reach out directly via repository contact channels.
