Metadata-Version: 2.4
Name: rednblue
Version: 3.3.1
Summary: AI Adversarial Security Testing for LLM, RAG, Agent, and Vision Models
Author-email: RednBlue <support@rednblue.io>
License: Proprietary
Project-URL: Homepage, https://rednblue.io
Project-URL: Dashboard, https://dashboard.rednblue.io
Project-URL: Repository, https://github.com/mahdidrm/RednBlue_CLI
Keywords: adversarial,ml,security,ai-testing,machine-learning
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Security
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click>=8.1.0
Requires-Dist: requests>=2.28.0
Requires-Dist: cryptography>=41.0.0
Requires-Dist: pillow>=10.0.0
Requires-Dist: numpy>=1.24.0
Requires-Dist: torch>=2.0.0
Requires-Dist: torchvision>=0.15.0
Requires-Dist: tqdm>=4.65.0
Requires-Dist: colorama>=0.4.6
Requires-Dist: psutil>=5.9.0
Requires-Dist: huggingface_hub>=0.20.0
Requires-Dist: safetensors>=0.4.0
Requires-Dist: timm>=0.9.0
Requires-Dist: ultralytics>=8.0.0
Requires-Dist: eel>=0.16.0
Provides-Extra: onnx
Requires-Dist: onnx>=1.14.0; extra == "onnx"
Requires-Dist: onnx2torch>=1.5.0; extra == "onnx"
Provides-Extra: gpu
Requires-Dist: GPUtil>=1.4.0; extra == "gpu"
Dynamic: license-file


# RednBlue CLI v3.3.1

**Zero-Knowledge Adversarial Security Testing for AI Models**

RednBlue CLI is a command-line tool for testing the adversarial robustness of machine learning models and the security posture of LLM/RAG/Agent systems. Run security assessments locally — your model never leaves your infrastructure.

```
███████████               ███████████
▒▒███▒▒▒▒▒███             ▒▒███▒▒▒▒▒███
 ▒███    ▒███  ████████    ▒███    ▒███
 ▒██████████  ▒▒███ ▒▒███  ▒██████████
 ▒███▒▒▒▒▒███  ▒███  ▒███  ▒███▒▒▒▒▒███
 ▒███    ▒███  ▒███  ▒███  ▒███    ▒███
 █████   █████ ████  █████ ███████████
▒▒▒▒▒   ▒▒▒▒▒ ▒▒▒▒  ▒▒▒▒▒ ▒▒▒▒▒▒▒▒▒▒▒

Zero-Knowledge Adversarial Security Testing
```

---

## Features

- **Zero-Knowledge Protocol** — Model weights and data never leave your infrastructure
- **Image Classifiers** — Test ResNet, VGG, EfficientNet, and custom architectures
- **YOLO Detection** — Full support for YOLOv5, YOLOv8, YOLOv10, YOLOv11
- **LLM / RAG / Agent Red-Team Suite** — Jailbreak resistance, system prompt extraction, indirect prompt injection, token-level obfuscation, RAG pipeline security, agentic tool-hijacking/goal-drift/privilege-escalation, backdoor auditing, and model extraction resistance — against hosted providers (OpenAI/Anthropic/Gemini), a local Python target file, or any HTTP endpoint
- **Tier-Based Testing** — Freelancer (quick scan) and Enterprise (comprehensive)
- **Visual Explanations (XAI)** — See exactly which pixels confused the model, including the real perturbed image, not a placeholder
- **Blue Team Hardening** — `rnb defend` applies a local defensive layer to a vulnerable model
- **Encrypted Submission** — AES-256 encrypted results with HMAC-SHA256 signing
- **Signed Compliance Reports** — Map test results onto EU AI Act, NIST AI RMF, and ISO/IEC 42001 articles, with a tamper-evident trust seal
- **Multi-Jurisdiction Compliance** — EU AI Act, NIST AI RMF, ISO/IEC 42001, UK DSIT, Canada AIDA, Singapore MAIGF

---

## Installation

```bash
# Install from PyPI
pip install rednblue

# Verify installation
rnb
```

### Requirements

- Python 3.8+
- PyTorch 2.0+ (for classifier/YOLO testing)
- CUDA (optional, for GPU acceleration)
- For LLM testing against a hosted provider: `pip install openai` / `pip install anthropic` / `pip install google-generativeai` as needed — not bundled by default, since most users only test one provider

---

## Quick Start

### 1. Choose your interface

**Desktop UI (recommended for non-CLI users):**
```bash
rnb ui
```

**Command line:** see below.

### 2. Set your token (always required for `rnb llm`; required for certified/submitted runs on `rnb preview`)

```bash
# Linux/Mac
export RNB_TOKEN=RB-XXXXXX-YYYYYY

# Windows
set RNB_TOKEN=RB-XXXXXX-YYYYYY
```

Get a token at https://dashboard.rednblue.io/dashboard/credits.

### 3. Run a free preview (Classifier / YOLO)

**Image Classifier:**
```bash
rnb preview --model resnet50.pth --input ./test_images --model-type classifier
```

**YOLO Detection Model:**
```bash
rnb preview --model yolov10n.pt --input ./test_images --model-type yolo
```

### 4. Submit for certification (Classifier / YOLO)

```bash
rnb preview --model yolov10n.pt --input ./images --model-type yolo --submit
```

### 5. Run an LLM / RAG / Agent security assessment

Against a hosted provider:
```bash
rnb llm --provider openai --model gpt-4o-mini --api-key sk-xxxxx --submit
```

Against a local Python file exposing a chat function (auto-discovered, or specify `--function`):
```bash
rnb llm --file my_target.py --submit
```

Against an HTTP endpoint:
```bash
rnb llm --endpoint http://localhost:8000/chat --input-field prompt --output-field response --submit
```

Restrict to specific attack classes (see Assessment Dimensions below for codes):
```bash
rnb llm --file my_target.py --attacks JB,SPE,TM --submit
```

### 6. View your report

Go to: **https://dashboard.rednblue.io/dashboard/tests** to download your PDF report and certificate, for both Classifier/YOLO and LLM/RAG/Agent sessions.

---

## Commands

| Command | Description |
|---------|-------------|
| `rnb` | Show welcome banner and quick start |
| `rnb-welcome` | Same as `rnb` — convenience alias usable outside the main group |
| `rnb ui` | Launch the desktop graphical interface |
| `rnb preview --help` | Run adversarial attacks (classifier or YOLO) |
| `rnb llm --help` | Run the full LLM/RAG/Agent adversarial security assessment |
| `rnb status` | Check CLI version and token validity/tier |
| `rnb explain --help` | Generate a standalone visual-explanation panel for one image (Robust Integrated Gradients) |
| `rnb defend --help` | Apply local defensive hardening to a model (Blue Team) |
| `rnb report --help` | Generate a signed compliance report + trust seal from saved classifier/YOLO results |
| `rnb optimize-epsilon` | Optimize epsilon values (Enterprise) |

---

## Supported Model Formats

| Type | Extensions |
|------|------------|
| PyTorch | `.pt`, `.pth`, `.safetensors` |
| ONNX | `.onnx` |
| Ultralytics YOLO | `.pt` (auto-detected) |

**Architectures:** ResNet, VGG, DenseNet, EfficientNet, MobileNet, Inception, SqueezeNet, ShuffleNet, GoogLeNet, AlexNet, and common `timm` backbones.

> TensorFlow (`.h5`, `.pb`) is **not yet supported**. The ingestion layer
> (`rnb/core/ingestion/wrapper.py`) is architected to accept a TensorFlow
> adapter, but `tensorflow` is intentionally not bundled as a dependency
> today — adding real support is tracked as a follow-up, not claimed here.

**LLM/RAG/Agent targets:** hosted providers (OpenAI, Anthropic, Gemini), a local Python file exposing a `chat(user_message: str) -> str`-style function, or any HTTP endpoint accepting a JSON prompt field and returning a JSON response field.

---

## Assessment Dimensions

### Classifier Models

| Dimension | Description |
|-----------|-------------|
| Noise Resilience | Stability under sensor noise and interference |
| Spatial Consistency | Robustness to spatial feature shifts |
| Universal Pattern Defense | Resistance to universal perturbation patterns |
| Feature Stability | Internal representation integrity |
| Confidence Calibration | Prediction reliability accuracy |
| Iterative Stress Tolerance | Defense against sustained pressure |
| Optimization Attack Defense | Resistance to optimized adversarial inputs |
| Deep Perturbation Resistance | Resilience against deep layer perturbations |

### YOLO Detection Models

| Dimension | Description |
|-----------|-------------|
| Noise Resilience | Stability under sensor noise |
| Input Perturbation Defense | Resistance to subtle input modifications |
| Iterative Stress Tolerance | Defense against multi-step attacks |
| Detection Consistency | Reliable detection under varying conditions |
| Targeted Evasion Defense | Resistance to deliberate misclassification |
| Object Persistence | Maintains detections under perturbations |
| Multi-Object Stability | Accuracy in crowded scenes |
| Black-Box Resilience | Defense without model access |
| Query-Limited Defense | Resistance to low-query probing |

### LLM / RAG / Agent

Each class runs against the target's actual responses — no attack ever counts as a success unless the response shows positive evidence of engaging with the malicious request (see v3.3.0 Changelog entry below); a target that's unreachable or merely non-committal is never scored as compromised.

| Code | Class | Sub-attacks |
|------|-------|--------------|
| `JB` | Jailbreak Resistance | Template Battery, Many-Shot Jailbreaking, Crescendo Multi-Turn, Multilingual Jailbreak, Persona Transfer, Competing Objectives |
| `SPE` | System Prompt Protection | Direct Extraction, Differential Probing, Token Fishing |
| `LPI` | Indirect Injection Defense | Document Body Injection |
| `TM` | Token Manipulation Robustness | Unicode Homoglyph Substitution, Zero-Width Character Injection, Leetspeak Substitution, Context Dilution, Bidirectional Text Override |
| `RAG` | RAG Pipeline Security | Knowledge Boundary Probe, Corpus Poisoning (requires `inject_fn`), Context Window Stuffing, RAG Prompt Injection |
| `AGT` | Agentic Safety | Tool Hijacking via Injected Content, Goal Override / Objective Drift, Privilege Escalation Chain |
| `BD` | Backdoor Integrity | STRIP Consistency Test, Trigger Enumeration, Input Sensitivity Probe |
| `EXT` | Model Extraction Resistance | Extraction Feasibility Assessment, Response Consistency Test, Model Fingerprinting |

Use `--attacks` with a comma-separated list of codes to restrict a run (e.g. `--attacks JB,SPE,TM`). Omit `--attacks` to run all classes.

---

## Tier Comparison

| Feature | Freelancer | Enterprise |
|---------|------------|------------|
| Classifier Attacks | 5 | 8 |
| YOLO Attacks | 4 | 9 |
| Epsilon Values | 2 | 4 |
| Total Scenarios | ~10-20 | ~30-70 |
| LLM / RAG / Agent Testing | ❌ | ✅ (all 8 classes, 27 sub-attacks) |
| Epsilon Optimization | ❌ | ✅ |
| Continuous Integrity Monitoring | ❌ | 🔜 Coming soon |

---

## Output Example

```
============================================================
  RednBlue Security Preview — YOLO Detection
============================================================
  Attacks run    : 21
  Successful hits: 0/21 (0%)
  Robustness rate: 100%
  Estimated Grade: GOLD

  ⚠️  This is a preview only
  → Visit: https://rednblue.io/checkout
  → Re-run with: rnb preview --model-type yolo --submit
```

---

## Certification Grades

| Grade | Score | Meaning |
|-------|-------|---------|
| 🥇 GOLD | ≥90% | Excellent robustness, deployment ready |
| 🥈 SILVER | ≥75% | Good robustness, minor improvements recommended |
| 🥉 BRONZE | ≥50% | Moderate robustness, improvements needed |

---

## Architecture

![Architecture](docs/RnB-Diagram.png)

For the complete, AI-readable source dump of this repository, see
[`rnb.md`](rnb.md).

---

## Links

- **Platform:** https://dashboard.rednblue.io
- **Documentation:** https://docs.rednblue.ai
- **Website:** https://rednblue.io

---

## Authors

- **Dr. Mahdi Deramgozin** — Chief AI Officer
- **Dr. Saeid Samizade** — Chief Technology Officer

---

## License

Proprietary — RednBlue SAS © 2026

Made in France 🇫🇷

---

## Changelog

### [3.3.1] — 2026-07-06

#### Fixed
- **`TM-BIDI` (Bidirectional Text Override) was written but never wired into `token_manipulation.py`'s `run_attack()`**, so it never actually ran despite being listed in the module's own docstring. Now wired in as a fifth Token Manipulation sub-attack.
- **The Issue #37 false-positive fix (v3.3.0) is now extended to the remaining attack modules that shared the same vulnerable pattern**: `rag_attacker.py`'s Knowledge Boundary Probe and `extraction_attacker.py`'s Model Fingerprinting both used a raw "not refused and long enough" check, which — like the `jailbreak.py` case fixed in v3.3.0 — misclassified transport errors and generic, on-topic-unrelated deflections as genuine findings. Both now require positive evidence (topical overlap with the probe, not just absence of refusal) via the same shared `is_compliant_response()` / `is_generic_deflection()` / `has_topical_overlap()` helpers.
- `token_manipulation.py`'s shared `_run_variant()` (used by all five TM sub-attacks) had the same vulnerable pattern and is now fixed identically.
- Restored a working-copy fix to `agent_attacker.py` that had been accidentally reverted to a syntactically-broken committed version (unescaped newlines inside string literals) during a repository cleanup pass — no logic change, this is a syntax-only restoration.

**Not changed in this release** (same class of gap, lower priority — see the modules' own logic for why): `agent_attacker.py`'s three sub-attacks and `backdoor_attacker.py`'s three sub-attacks already use more specific keyword-match or baseline-comparison logic and are less prone to this exact failure mode.

---

### [3.3.0] — 2026-07-02

#### Fixed
- **Critical**: `--tier` manual override on `rnb llm`/`rnb preview` bypassed
  `RNB_TOKEN` validation entirely, allowing unlimited unpaid runs. Token is
  now always required and validated.
- **Critical**: API/transport errors (e.g. invalid API key) were scored as
  successful jailbreaks. Added target health pre-check and error-aware
  refusal detection; runs now abort cleanly on unreachable targets instead
  of producing fabricated results.
- **High**: Generic, harmless, non-refusal responses were scored as
  successful jailbreaks purely for not containing refusal language. Added
  positive-evidence (topical relevance) classification to `jailbreak.py`
  and `prompt_injection.py`'s Token Fishing.
- Removed the stale `RNB_BETA_LLM_SUBMIT` gate — `rnb llm --submit` no
  longer requires a beta flag now that Dashboard v3.1 supports LLM
  sessions.
- Desktop UI (`rnb ui`) was reading stale `dimensions`/`avg_reliability`
  fields that don't exist in `LLMTestRunner`'s actual `attacks`/`success`
  output shape, causing LLM test results to always display as fully safe
  regardless of actual findings.

#### Changed
- `--tier` on `rnb llm`/`rnb preview` now only warns on mismatch instead of
  silently overriding the token's actual tier.

---
### [3.1.0] — 2026-06

#### Fixed
- **"After Attack" panel no longer mirrors the original image.** The real
  adversarial tensor produced by the attack engine now flows through to
  the XAI panel for both classifier and YOLO models, instead of the
  attention map being computed against an unmodified copy of the input.

#### Added
- **Unified model-ingestion layer** (`rnb/core/ingestion/wrapper.py`) —
  a `BaseModelWrapper` abstraction with a PyTorch adapter (covering
  `.pt`/`.pth`/`.safetensors`) and an ONNX adapter, replacing ad-hoc
  format detection with a single, extensible interface. TensorFlow is an
  architected-but-not-yet-implemented extension point (see note above).
- **`rnb defend`** — Local Blue Team hardening. Reads a vulnerability
  profile and registers forward hooks (spatial smoothing + gradient
  clipping) on the target model's convolutional layers to dampen
  adversarial perturbations at inference time, with zero retraining.
- **`rnb explain`** — Standalone XAI playground for a single
  (model, image) pair. Adds a `RobustIntegratedGradientsExplainer`,
  which averages gradients over many noised copies of the input so the
  attention map itself can't be blinded by an adversarial perturbation
  the way single-pass saliency methods can.
- **`rnb report`** — Generates a signed compliance report from saved
  `rnb preview --save-results` output: maps Noise Resilience, Jailbreak
  Bypass Rate, and Bias Score-style metrics onto EU AI Act and NIST AI
  RMF articles, then emits a Markdown report plus a signed
  `trust_seal.svg`, both protected with the same AES-256-CBC +
  HMAC-SHA256 scheme already used for result submission.
- **`--save-results PATH`** flag on `rnb preview`, so a preview run's
  JSON output can be fed into `rnb report` later without re-running the
  attacks.
- **`rnb-welcome`** entry point now actually works (previously pointed
  at a function that did not exist in `cli.py`).

#### Changed
- `rnb/banner.py` no longer hard-codes its own version string — it now
  imports from `rnb/_version.py`, the single source of truth.
- Repo cleanup: removed `BANNER_INTEGRATION.md`, `patch_banner.py`, the
  stray root-level `banner.py`, `rnb_init.py`, `rnb/architecture.md`
  (an old AI-context dump that was accidentally being shipped inside the
  installable package), `rnb/.installed`, and `test.py` (a manual
  integration-test script that contained a hardcoded API token).
- Documentation consolidated: `QUICKSTART.md` and `CHANGELOG.md` have
  been folded into this README. The only other Markdown file in this
  repository is `rnb.md`, the AI-readable full-source dump.

#### Deferred
- Continuous Integrity Monitoring Daemon (`rnb daemon`) is **not**
  included in this release. It is reserved for a future Enterprise-only
  release.

---

### [3.0.0] — 2026-05-08

Major release. Added the `rnb/xai/` visual-explanation package
(`GradCAMExplainer`, `EigenCAMExplainer`, worst-case selection, 3-panel
composite rendering), the desktop UI (`rnb ui`, Eel-based), GPU
auto-detection, the `--device` and `--tier` flags, and multi-architecture
classifier support (ResNet, VGG, DenseNet, EfficientNet, MobileNet,
Inception, SqueezeNet, ShuffleNet, GoogLeNet, AlexNet). Fixed tier
propagation to classifier attacks, the `run_preview()` `input_path` /
`Fore` import bugs, and token-validation credit lookups.

### [2.5.1] — 2026-04-14

Added HuggingFace whitelist filtering and `timm` backbone support.
Migrated the build system from `setup.py` to `pyproject.toml`.

### [2.5.0] — 2026-04 (initial public release)

Welcome banner, `rnb status`, `rnb preview`, `rnb optimize-epsilon`,
`rnb test-llm`. Eight image-classifier attacks (`GNI, SHFP, UAP, FSP,
CCM, PGD, CW, DEEP`). AES-256-CBC + HMAC-SHA256 encrypted result
submission to the dashboard.

> Full per-commit history is available via `git log`. This section
> retains only release-level highlights.
