# Testing & Quality Roadmap

This document outlines the planned expansion of the integration and regression test suite for `rclib`. These tests aim to ensure mathematical correctness, architectural robustness, and numerical stability as the library evolves.

## 1. NVAR Pipeline Integration
While unit tests exist for the `NvarReservoir` class, a full model-level integration test is needed.
*   **Target:** Next-Generation Reservoir Computing (NVAR) architecture.
*   **Test Scenario:** Train an NVAR-based model on a chaotic system (e.g., Lorenz Attractor).
*   **Verification:** Ensure the model correctly constructs the non-linear feature vector and achieves a low MSE without random weight matrices.

## 2. Generative Stability
Generative prediction (closed-loop) is sensitive to accumulated errors.
*   **Target:** `predict_generative` method.
*   **Test Scenario:** Train on a stable periodic signal, then generate 1000+ steps autonomously.
*   **Verification:** Assert that the generated signal remains within a valid range (e.g., $[-2, 2]$) and maintains the target frequency without diverging or decaying to zero.

## 3. Deep ESN Architecture (Serial)
Verifying the stacking logic for deep networks.
*   **Target:** `serial` connection type in the `Model` class.
*   **Test Scenario:** Create a stack of 3+ reservoirs where the state of $Res_n$ serves as the input to $Res_{n+1}$.
*   **Verification:** Confirm that state propagation and concatenation are handled correctly and that the model achieves better performance on complex tasks compared to a shallow reservoir of the same total size.

## 4. Multidimensional I/O Robustness
Ensuring consistent handling of high-dimensional data across the C++/Python boundary.
*   **Target:** Linear algebra and data-passing logic.
*   **Test Scenario:** Implement a "Multiple-Input Multiple-Output" (MIMO) task with 5D input and 5D target vectors.
*   **Verification:** Validate that matrix dimensions, transpositions, and indexing are correctly handled for non-scalar time series.

## 5. Numerical Edge Cases & Stress Testing
Testing the library's resilience to extreme parameters.
*   **Target:** Error handling and stability.
*   **Test Scenario:**
    *   Large spectral radius ($> 1.0$) leading to saturation.
    *   Small RLS forgetting factors causing potential matrix ill-conditioning.
    *   Zero-length or empty input data.
*   **Verification:** Ensure the library throws meaningful exceptions (via `std::runtime_error`) or remains stable through regularization, rather than crashing or returning `NaN`.

## 6. Performance Benchmarking Regression
*   **Target:** Computational efficiency.
*   **Test Scenario:** Measure execution time for fitting a large-scale reservoir.
*   **Verification:** Establish a baseline and compare new commits against it.
*   **Status:** Implemented as a **manual, non-gating** workflow (`.github/workflows/benchmark.yml`, `workflow_dispatch`) that runs `benchmark_ridge_all_solvers` and uploads the results as an artifact. It is intentionally *not* a blocking PR check: timing on shared GitHub runners varies too much run-to-run for a hard "<10% deviation" assertion to be anything but flaky. Run it before and after a perf-sensitive change and compare the artifacts; for rigorous numbers, run the local `benchmarks/` scripts on fixed hardware.
