Metadata-Version: 2.4
Name: notebooktester
Version: 0.3.7
Summary: A simple linting tool to check if all notebooks execute
Author-email: raoul grouls <Raoul.Grouls@han.nl>
Requires-Python: >=3.11
Requires-Dist: click>=8.1.8
Requires-Dist: jupyter>=1.1.1
Requires-Dist: loguru>=0.7.3
Requires-Dist: tqdm>=4.67.1
Provides-Extra: test
Requires-Dist: pytest-asyncio>=0.25.3; extra == 'test'
Requires-Dist: pytest>=8.3.4; extra == 'test'
Requires-Dist: ray[tune]>=2.42.0; extra == 'test'
Description-Content-Type: text/markdown

# 📓 NotebookTester
[![uv](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json)](https://github.com/astral-sh/uv)
[![image](https://shields.io/pypi/v/notebooktester.svg)](https://pypi.org/project/notebooktester/)

A fast, reliable, and parallel Jupyter notebook testing tool with caching support! Test your notebooks with confidence.

## 🌟 Features

- 🚀 **Parallel Execution**: Test multiple notebooks simultaneously
- ⏱️ **Configurable Timeouts**: Set custom timeouts for notebook execution
- 💾 **Smart Caching**: Cache test results to avoid unnecessary re-runs
- 🎯 **Selective Testing**: Only test modified notebooks
- 📊 **Detailed Logging**: Comprehensive logs with test results and errors
- 🛠️ **CLI Support**: Easy to use command-line interface

## 🔧 Installation

Choose your preferred package manager:

### Using pip (slow legacy)

```bash
pip install notebooktester
```

### Using uv (10-100x faster)
see [uv docs](https://docs.astral.sh/uv/) for more info.

```bash
uv add notebooktester
```

## 🚀 Quick Start

Test a single notebook:

```bash
notebooktester path/to/your/notebook.ipynb
```

Test all notebooks in a directory:

```bash
notebooktester path/to/notebooks/directory
```

Test a specific list of notebooks:

```bash
notebooktester nb1.ipynb nb2.ipynb nb3.ipynb
```

You can mix files and directories, and combine them with any of the options below:

```bash
notebooktester nb1.ipynb nb2.ipynb -t 120 -w 4
```

## 🎮 Command Line Options

```bash
notebooktester [OPTIONS] PATHS...
```

Options:
- `-t, --timeout SECONDS`: Timeout in seconds for each notebook (default: 60). With `--report`, the minimum duration to show (default: 30)
- `-w, --workers NUMBER`: Number of parallel workers (default: CPU count)
- `-c, --cache-dir PATH`: Cache directory for test results (default: .notebookcache)
- `-v, --verbose`: Enable verbose output (full log stream, no progress bar)
- `-pb, --progressbar`: Show a live, time-weighted progress bar based on cached notebook timings (mutually exclusive with `-v`)
- `-f, --force`: Ignore cache and force test execution
- `-r, --report`: Show long-running notebooks from the cache instead of running tests
- `--version`: Show the installed version and exit

Every run prints a summary line first, e.g. `Testing 42 notebooks (30 cached/skipped, 12 to run) - estimated time ~3m10s`, using timings from previous cached runs.

## 📋 Example Usage

Basic usage:
```bash
notebooktester notebooks/
```

With custom timeout and workers:
```bash
notebooktester notebooks/ -t 120 -w 4
```

Force re-run all tests:
```bash
notebooktester notebooks/ --force
```

Find long-running notebooks from the cache (no re-run needed):
```bash
notebooktester --report
```

Same, but with a lower threshold:
```bash
notebooktester --report -t 20
```

## 🔍 Cache Behavior

NotebookTester maintains a cache of test results to optimize performance:
- Only notebooks modified since their last test run are re-tested
- If the notebook timed out, and the current timeout has not been increased, notebook is skipped
- Cached results include success/failure status and error messages
- Force flag (`-f`) bypasses the cache

## 🐢 Long-Running Notebook Report

Once you've run the tests at least once, `--report` reads straight from the cache to show which
notebooks are the slowest — no re-execution required. This pairs well with `param()` (see below):
use the report to find your slow notebooks, then wrap their expensive parameters in `param()` so
they run fast under `notebooktester` while keeping their real values everywhere else.

```bash
notebooktester --report
notebooktester --report -t 20        # lower the threshold to 20s
notebooktester --report -c .my-cache # point at a different cache dir
```

Example output:
```
>>notebooktester --report -t 20

🐢 Long-running notebooks (> 20s)

Notebook                        Time   Timeout  Status
------------------------------------------------------
notebooks/slow_train.ipynb     55.2s       60s  ✅ PASSED
notebooks/medium.ipynb         33.7s       60s  ✅ PASSED
------------------------------------------------------
2 notebook(s), 88.9s combined
```

## ⚡ Reducing Notebook Runtime with `param()`

Wrap expensive values in `param(full, test=...)` so notebooks run with a cheap stand-in under
`notebooktester`, while keeping the real value everywhere else (e.g. when run interactively):

```python
from notebooktester import param

EPOCHS = param(100, test=2)
SEARCH_SIZE = param(10_000, test=50)
```

## 📊 Output

The tool provides:
- Progress bar for test execution
- Colored console output for test results
- Detailed logs in the `logs/` directory
- Summary of passed and failed tests

Example output:
```
>>notebooktester notebooks/ -t 120 -w 4
Running tests with 4 workers
Starting notebook tests - found 10 notebooks
✅ PASSED - notebook1.ipynb: Success
❌ FAILED - notebook2.ipynb: Cell execution error
⏰ TIMEOUT - notebook3.ipynb: A cell timed out

Test Summary: 8 passed, 2 failed
```

## 🤝 Contributing

Contributions are welcome! Feel free to:
- Report bugs
- Suggest features
- Submit pull requests

## 👤 Author

Raoul Grouls
