Metadata-Version: 2.4
Name: devlacruz_hashlib
Version: 0.2.0
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Rust
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Topic :: Security :: Cryptography
License-File: LICENSE
Summary: AI Privacy Shield & TOON Compressor - Zero-Trust middleware for Python AI Agents
Author-email: Alejandro De La Cruz <devlarood@gmail.com>
License-Expression: MIT
Requires-Python: >=3.8, <4.0
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM

# devlacruz_hashlib 🛡️

> **devlacruz_hashlib (v0.2.0)** is an enterprise-grade **AI Privacy Shield & TOON Compressor**, engineered in Rust with PyO3. It serves as a Zero-Trust middleware for Python AI Agents, encrypting sensitive data locally before reaching commercial LLMs (OpenAI, Anthropic, DeepSeek) while slashing prompt token costs with tabular compression.

---

## Table of Contents

1. [The Problem It Solves](#the-problem-it-solves)
2. [What's New in v0.2.0](#whats-new-in-v020)
3. [Features](#features)
4. [Installation](#installation)
5. [Quick Start](#quick-start)
6. [Streaming LLM Responses (FSM Decoder)](#streaming-llm-responses-fsm-decoder)
7. [Deterministic Encryption & Referential Integrity](#deterministic-encryption--referential-integrity)
8. [Real-World AI Agent Integration](#real-world-ai-agent-integration)
9. [API Reference](#api-reference)
10. [Performance](#performance)
11. [Backwards Compatibility](#backwards-compatibility)
12. [Contributing](#contributing)
13. [License & Author](#license--author)

---

## The Problem It Solves

When developing AI Agents (AutoGen, LangChain, CrewAI, or custom tool-calling agents), sending **PII (Personally Identifiable Information)** or **PHI (Protected Health Information)** to external model APIs introduces critical regulatory risks (GDPR, HIPAA, SOC-2). Additionally, sending verbose JSON structures wastes tokens on repetitive keys and punctuation.

**devlacruz_hashlib** resolves both challenges seamlessly at native Rust speed:

1. **Locally Encrypts** sensitive fields using ChaCha20-Poly1305 DAEAD.
2. **Compresses** data into the compact **TOON format** (or standard JSON if needed).
3. **Preserves Referential Integrity**: Identical entities produce identical tokens, enabling the LLM to group, sort, and reason consistently.
4. **Resilient Streaming**: Decrypts responses in real time chunk-by-chunk with zero latency overhead.
5. **Anti-BPE Fuzzy Matcher**: Automatically heals tokens affected by tokenizer splits, Markdown escapes (`\<...\>`), spaces, or Unicode zero-width characters.

---

## What's New in v0.2.0

* 🧠 **DAEAD Deterministic Encryption (SIV-like):** Preserves entity equivalence across prompts using HMAC-derived nonces and Authenticated Additional Data (AAD).
* 🏷️ **Semantic Typed Tokens:** Tokens specify category tags (e.g. `<VAULT:NAME:...>`, `<VAULT:CONDITION:...>`), preventing context confusion.
* ⚡ **Streaming `VaultStreamDecoder`:** Finite State Machine (FSM) for zero-latency, chunk-by-chunk decryption over WebSockets / Server-Sent Events (SSE).
* 🛡️ **BPE & Markdown Resilience:** Fuzzy scanner transparently recovers corrupted tokens without failing decryption.
* 🧹 **Memory Protection with `zeroize`:** Key material wrapped in `SessionKey` with automatic zeroization upon cleanup or context exit.
* 🔄 **Hybrid Output:** Support for both tabular `toon` and native `json` format outputs.
* 🔒 **100% Backwards Compatible:** Retains flawless compatibility with legacy v0.1.x code signatures and string keys.

---

## Features

* 🔒 **Zero-Trust Local Cryptography:** ChaCha20-Poly1305 DAEAD. Master keys reside strictly in local memory and are never transmitted.
* 📉 **Token Optimization:** Slashes JSON token consumption by up to 60% with TOON compression.
* ⚡ **Rust Native Execution:** Drops the Python GIL (`py.allow_threads`) during heavy operations.
* 📦 **Universal ABI3 Wheels:** Works out of the box on Python 3.8 through 3.14+ across Linux, macOS, and Windows.

---

## Installation

```bash
pip install devlacruz_hashlib
```

---

## Quick Start

### Basic Usage (v0.1.x Compatible)

```python
import devlacruz_hashlib as shield
import json

data = [
    {"id": 1, "name": "Alejandro De La Cruz", "diagnosis": "Anxiety", "score": 85},
    {"id": 2, "name": "Sam Altman", "diagnosis": "Flu", "score": 92}
]

# Encrypt sensitive keys into TOON format
toon_payload, key = shield.prepare_for_ai(json.dumps(data), ["name", "diagnosis"])

print(toon_payload)
# data[2]{diagnosis,id,name,score}
# <VAULT_...>,1,<VAULT_...>,85
# <VAULT_...>,2,<VAULT_...>,92

# Decrypt LLM response
llm_reply = "Patient <VAULT_...> requires follow-up care."
restored = shield.restore_from_ai(llm_reply, key)
print(restored)
# Patient Alejandro De La Cruz requires follow-up care.
```

---

## Streaming LLM Responses (FSM Decoder)

When consuming tokens in real time from OpenAI, Anthropic, or Ollama streams, tokens may arrive sliced across chunk boundaries. Use `VaultStreamDecoder`:

```python
from devlacruz_hashlib import SessionKey, VaultStreamDecoder, prepare_for_ai
import json

data = [{"id": 1, "patient": "John Doe", "condition": "Migraine"}]
payload, session_key = prepare_for_ai(
    json.dumps(data),
    ["patient"],
    deterministic=True,
    semantic_types={"patient": "NAME"}
)

# Initialize stream decoder
decoder = VaultStreamDecoder(session_key)

# Simulated stream fragments
stream_chunks = [
    "Patient ",
    "<VAULT:NAME:ab",
    "cd12345",
    "...>",
    " reported mild pain."
]

for chunk in stream_chunks:
    output = decoder.decode_chunk(chunk)
    print(output, end="", flush=True)

# Flush any trailing buffer
print(decoder.flush(), end="")
# Output: "Patient John Doe reported mild pain."
```

---

## Deterministic Encryption & Referential Integrity

In agentic loops, LLMs often need to compare entities across rows. Standard randomized nonces turn the same name into two different tokens, causing hallucinations. With `deterministic=True`:

```python
import devlacruz_hashlib as shield
import json

records = [
    {"record_id": 101, "customer": "Alice Smith"},
    {"record_id": 102, "customer": "Alice Smith"},  # Same identity
    {"record_id": 103, "customer": "Bob Jones"}
]

json_str, key = shield.prepare_for_ai(
    json.dumps(records),
    ["customer"],
    format="json",
    deterministic=True,
    semantic_types={"customer": "NAME"}
)

data = json.loads(json_str)
# data[0]['customer'] == data[1]['customer'] -> True (<VAULT:NAME:same_hash>)
```

---

## Secure Memory with SessionKey

Keys can be explicitly cleared from memory or managed via Python context managers:

```python
from devlacruz_hashlib import SessionKey, prepare_for_ai
import json

with SessionKey() as key:
    payload, _ = prepare_for_ai(
        json.dumps([{"secret": "API_KEY_123"}]),
        ["secret"],
        deterministic=True
    )
    # Perform operations...

# 'key' is automatically zeroized from RAM after exiting the block
assert key.is_zeroized()
```

---

## Real-World AI Agent Integration

```python
import devlacruz_hashlib as shield
import json

INTERNAL_DB = {
    "Alejandro De La Cruz": {"balance": 5000, "status": "VIP"},
    "Sam Altman": {"balance": 1000, "status": "Standard"}
}

def secure_get_customer_data(encrypted_name: str, key: str) -> str:
    # 1. Decrypt locally
    real_name = shield.restore_from_ai(encrypted_name, key).strip()
    if real_name in INTERNAL_DB:
        return f"User {encrypted_name}: {json.dumps(INTERNAL_DB[real_name])}"
    return "User not found."

# Protect user prompt
masked_input, session_key = shield.prepare_for_ai(
    json.dumps([{"target": "Alejandro De La Cruz"}]),
    ["target"],
    deterministic=True
)

# Agent receives token and retrieves data securely
agent_response = "Summary for <VAULT:...>: balance 5000."
final_response = shield.restore_from_ai(agent_response, session_key)
```

---

## API Reference

### Functions

| Function | Signature | Description |
| :--- | :--- | :--- |
| `prepare_for_ai` | `(json_str: str, sensitive_keys: list[str], *, format: str = "toon", deterministic: bool = False, semantic_types: dict[str, str] = None) -> tuple[str, SessionKey \| str]` | Encrypts sensitive keys and formats output as TOON or JSON. Returns payload and session key. Drops GIL. |
| `restore_from_ai` | `(text: str, master_key: SessionKey \| str) -> str` | Scans text for `<VAULT...>` tokens, applies fuzzy healing, and returns plaintext. Compatible with both legacy and typed tokens. Drops GIL. |

### Classes

* `SessionKey()`: Wrapper for ChaCha20 master key with zeroize capabilities and context manager support (`__enter__`, `__exit__`).
* `VaultStreamDecoder(key: SessionKey | str)`: FSM decoder for processing fragmented token streams in real time.

---

## Performance

Compiled natively in Rust with SIMD and ABI3 optimizations.

**Benchmarking 10,000 JSON records (Parsing + Encryption + Compression):**
* Pure Python (`cryptography` + `json`): ~2.5 - 4.0s
* **devlacruz_hashlib (Rust + PyO3)**: **~0.08 - 0.15s** (~25x faster)

---

## Backwards Compatibility

`devlacruz_hashlib` v0.2.0 is fully backwards-compatible with earlier v0.1.x versions:
1. Calls to `prepare_for_ai(json_str, keys)` default to `format="toon"` and return `tuple[str, str]`.
2. `restore_from_ai` seamlessly accepts legacy `str` keys and `<VAULT_...>` tokens.
3. Passing raw `str` keys issues a gentle `DeprecationWarning` while executing successfully without crashing.

---

## Contributing

Contributions are welcome!

1. Fork the repository
2. Create your feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m "feat: enhance streaming decoder"`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request

---

## License

This project is licensed under the **MIT License**. See the [LICENSE](./LICENSE) file for details.

---

## Author

**Alejandro De La Cruz**  
✉️ [devlarood@gmail.com](mailto:devlarood@gmail.com)  
GitHub: [@DevLaCruz](https://github.com/DevLaCruz)

