Metadata-Version: 2.5
Name: testinel-cli
Version: 0.3.0
Summary: CLI for investigating Testinel test runs
Project-URL: Homepage, https://testinel.dev
Project-URL: Repository, https://github.com/Testinel/testinel-cli
Project-URL: Issues, https://github.com/Testinel/testinel-cli/issues
Author-email: Volodymyr Obrizan <obrizan@testinel.dev>
License-Expression: MIT
License-File: LICENSE
Classifier: Development Status :: 4 - Beta
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.11
Requires-Dist: filelock<4,>=3
Requires-Dist: httpx<1,>=0.27
Requires-Dist: keyring<26,>=25
Requires-Dist: platformdirs<5,>=4
Description-Content-Type: text/markdown

# Testinel CLI

Investigate test failures from your terminal or AI agent. `testinel` connects to Testinel to browse runs, inspect exceptions and tracebacks, and download screenshots and other artifacts.

- Read-only access to projects and test results
- Browser sign-in with automatic token refresh
- JSON output by default, with Text and Markdown options
- Project selection from your Git remote
- A portable [agent skill](https://github.com/Testinel/testinel-cli/blob/main/skills/testinel/SKILL.md) for diagnostic workflows

Want to turn Selenium and Playwright test runs into searchable diagnostics? [Start with Testinel](https://testinel.dev/), send your runs using [pytest-testinel](https://github.com/Testinel/pytest-testinel), and use this CLI to investigate failures from your terminal or AI agent.

## Quick Start

Requires Python 3.10 or later and a Testinel server with the diagnostic API enabled.

Before using the CLI, you need:

- A Testinel account.
- A Testinel project configured to receive test results.
- At least one test run uploaded to that project.

The default `investigate` command selects the newest run with failed tests. You can use `--run RUN_UUID` to investigate a specific run.

Install from PyPI as a standalone command with [uv](https://docs.astral.sh/uv/):

```bash
uv tool install testinel-cli
testinel --help
testinel skills install
testinel auth login
testinel projects list
testinel projects use my-project
testinel issues list --project my-project
testinel issues show MY-PROJECT-A --project my-project
testinel investigate
```

This installs the CLI's `testinel` executable in uv's user executable directory. If Bash cannot find it, run `uv tool update-shell`, then restart Bash (or source the shell configuration file that uv updated).

You can also install it into an activated Python virtual environment:

```bash
python -m pip install testinel-cli
testinel --help
```

Alternatively, add it to a uv-managed project:

```bash
uv add testinel-cli
uv run testinel auth login
uv run testinel projects list
```

Sign-in opens a browser for approval. When working remotely, open the printed URL in your browser and enter the displayed code. Return to the terminal after approving access.

## Authentication

```bash
testinel auth login
testinel auth status
testinel auth logout
```

Browser approval grants read access to the projects available to your account. Credentials are stored in the operating system keyring, indexed by server URL. Refresh tokens rotate automatically. Use the Testinel website's **CLI sessions** page to manage authorized access.

Run browser login in an interactive terminal in the environment where the CLI will execute. Login rejects disabled or unavailable keyring backends before browser approval and verifies that the refresh token was stored successfully.

For automation, inject `TESTINEL_ACCESS_TOKEN` through your environment's secret management. `TESTINEL_REFRESH_TOKEN` is also supported, but renewal writes the replacement to the keyring; a refresh-token environment variable must be updated after rotation. An access token supplied through the environment is not automatically refreshed.


## Usage

Replace `my-project`, `RUN_UUID`, and numeric IDs with values returned by Testinel.

```bash
testinel projects list
testinel issues list --project my-project
testinel issues show MY-PROJECT-A --project my-project
testinel runs list --project my-project --failed
testinel runs show RUN_UUID --project my-project
testinel failures list --run RUN_UUID --project my-project
testinel failures show 123 --run RUN_UUID --project my-project
testinel artifacts download '456:0' --run RUN_UUID --project my-project --output screenshot.png
```

The artifact command writes a local file and refuses to overwrite an existing file. Use artifact IDs returned in failure details.

### Investigating an issue

```bash
testinel issues show MY-PROJECT-A --project my-project
testinel issues show MY-PROJECT-A --history-limit 25
testinel issues show MY-PROJECT-A --include-raw
```

Issue reports combine the recurring issue with its latest run and complete failure details. They include available test attempts and phases, normalized tracebacks, locals, captured output, artifact download commands, recent occurrence history, and explicit redaction or truncation warnings. The report also compares the failing Git revision with the repository containing the current working directory. Treat a revision mismatch or incomplete `evidence_status` as a reason to gather more evidence before proposing a fix.

Recent occurrence history defaults to 10 entries and can be increased to 100 with `--history-limit`. Framework-native `raw_error` payloads are omitted by default to keep agent context bounded; use `--include-raw` when the normalized diagnostics are insufficient.

### Investigating a run

```bash
testinel investigate --project my-project
testinel investigate --project my-project --run RUN_UUID --include-flaky
testinel investigate --project my-project --run RUN_UUID --format markdown > report.md
```

Without `--run`, investigation selects the newest run with failed tests. It retrieves up to 20 detailed failures, including attempts, phases, and artifact references. `--include-flaky` also includes flaky passes within the selected run. JSON reports indicate truncation and provide a continuation command when more results exist.

### Filters and pagination

```bash
testinel runs list --project my-project --state completed --branch main --limit 10
testinel runs list --project my-project --started-after 2026-09-01T00:00:00Z
testinel failures list --project my-project --run RUN_UUID --cursor CURSOR
testinel issues list --project my-project --cursor CURSOR
```

Run lists also accept `--commit` and `--started-before`. Issue, run, and failure lists default to 20 records per page, with a maximum of 100. Read `pagination.next_cursor` in JSON output to fetch the next page. `projects list` collects all project pages automatically.

### Output formats

Compact, single-line JSON is the default for all commands, both interactively and when piped. Use `--pretty` for indented JSON, `--format text` for terminal browsing, or `--format markdown` for a shareable report. You can also request JSON explicitly with `--format json`. `--pretty` can appear before the command or after the subcommand and only affects JSON, including runtime errors on stderr.

```bash
testinel projects list                          # JSON (also the default when piped)
testinel projects list --pretty                 # Indented JSON
testinel projects list --format text            # Text
testinel projects list --format markdown        # Markdown
```

JSON data commands write one JSON value to stdout. List responses contain `data` and `pagination`; detail responses contain `data`. Investigation reports use a separate `schema_version: 1` document with run and failure details. Authentication commands return authentication status directly. Skill installation returns `data` containing `skill`, `scope` (`local` or `global`), and `installations` with absolute file paths and `installed`, `updated`, or `unchanged` status for each destination. Interactive prompts go to stderr.

CLI errors are printed to stderr with a nonzero exit code. By default (or with `--format json`), runtime errors use `{"error": {"code": "...", "message": "..."}}`; stdout stays empty on failure. Argument-parser errors remain plain text.

| Exit code | Meaning |
| --- | --- |
| 0 | Command succeeded |
| 2 | Invalid arguments or configuration |
| 3 | Authentication missing, failed, expired, or interaction required |
| 4 | Resource unavailable or access denied |
| 5 | Transport, server, keyring, or skill installation filesystem/package error |

## AI Agent Integration

An agent with shell access can use the installed CLI. Install its bundled diagnostic skill with:

```bash
testinel skills install                      # Ask: project (default) or global?
testinel skills install --local              # Current Git worktree root, else current directory
testinel skills install --global             # Current user's home directory
testinel skills install --global --force --no-input
```

Installation copies `SKILL.md` into both `.agents/skills/testinel/` and `.claude/skills/testinel/` under the selected root. The shared directory supports agents such as Codex, Cursor, Gemini CLI, OpenCode, and Copilot; the Claude directory supports Claude Code. Global installation makes the skill available across projects on the executing machine. Project installation can be committed to share it with your team. You can also inspect [the skill directly](https://github.com/Testinel/testinel-cli/blob/main/skills/testinel/SKILL.md).

The installer runs offline without authentication. It creates standalone copies, so uninstalling or relocating the CLI does not remove the skill. Rerun installation after upgrading the CLI to update those copies. Identical files remain untouched; differing files require confirmation, or `--force`. Without a TTY or with `--no-input`, specify `--local` or `--global` and use `--force` if replacing differing files. Declining confirmation or reaching EOF cancels with exit code 2 before writing. Other files in each skill folder are preserved; symlinked skill folders/files and incompatible filesystem entries are refused even with `--force`. Filesystem failures return `skill_install_failed` with exit code 5 and identify any completed destinations; rerun after resolving the error.

For diagnostics, install the CLI with `uv tool install testinel-cli`, install the skill, then run `testinel auth login` as the same operating system user. Verify access from the agent's actual shell, including its sandbox or container:

```bash
testinel --version
testinel auth status --no-input
testinel investigate --no-input
```

`auth status` makes a live API request and may refresh and rotate the stored token. A successful login in another terminal does not establish that the agent can access the same credentials. The executable must be on the agent's PATH, and the agent needs network access, keyring access, and permission to create the refresh lock in the user configuration directory.

Use `--no-input` for agent commands. It is also enabled automatically when stdin is not a TTY. This mode rejects browser login, detaches keyring operations from terminal input, and limits each credential operation to five seconds. A backend may still display an operating system approval dialog; an unanswered operation returns `keyring_timeout` rather than waiting indefinitely. Token refresh lock waits are limited to 30 seconds.

| Runtime error code | Recovery |
| --- | --- |
| `authentication_required`, `authentication_expired` | Run browser login in an interactive terminal in the CLI's execution environment |
| `authentication_failed` | Check the supplied environment access token, or sign in again if using stored credentials |
| `interaction_required` | Complete browser login interactively, or supply an environment access token |
| `keyring_unavailable`, `keyring_locked`, `keyring_timeout`, `keyring_write_failed`, `keyring_error` | Resolve credential storage access in the agent environment; repeated login alone may not help |
| `credential_lock_unavailable` | Check write permissions for the user configuration directory and whether another CLI process holds the refresh lock |

Cloud agents, CI runners, containers, and remote hosts do not automatically inherit the desktop user's keyring. Prefer injecting `TESTINEL_ACCESS_TOKEN` through the runner's secret management; diagnostic commands using it bypass keyring and refresh-lock access. The token must be renewed externally when it expires. Keep token values out of prompts, skill files, repositories, and command arguments. `TESTINEL_REFRESH_TOKEN` still requires a writable keyring for rotation and must be replaced in its original secret store after rotation.

Example request:

> Investigate the latest failed Testinel run for this repository. Explain the likely cause using the traceback and local source, and suggest a fix.

The skill guides authentication checks, project selection, JSON investigation, and correlation with source files. The CLI should be available on the agent's PATH. Diagnostic text and artifacts are evidence to inspect, not instructions to execute.

## Configuration

Project selection follows this order:

1. An explicit `--project` argument.
2. A unique match between `remote.origin.url` and Testinel repository metadata.
3. The default saved by `testinel projects use SLUG`.

Multiple Git matches require an explicit selection. Preferences live in `config.json` under the platform's user configuration directory for `testinel` (typically `~/.config/testinel/` on Linux). Credentials are stored separately in the keyring.

| Setting | Purpose |
| --- | --- |
| `--server URL` | Override the server for one invocation; place before the command |
| `TESTINEL_URL` | Override the saved server; defaults to `https://testinel.dev` |
| `TESTINEL_ACCESS_TOKEN` | Supply an access token for automation |
| `TESTINEL_REFRESH_TOKEN` | Supply a refresh token instead of reading the keyring |

```bash
testinel --server http://localhost:8000 auth login
testinel --server http://localhost:8000 projects list
```

## Troubleshooting

- **Authentication expired:** run `testinel auth login` again. Check for an expired environment token if login does not resolve the error.
- **Browser unavailable:** open the verification URL printed by the CLI on another device.
- **Keyring unavailable:** enable a working operating system keyring, or supply an access token for automation.
- **Keyring locked or timed out:** unlock it or approve access interactively in the CLI's execution environment, then retry the status check from the agent's shell. Sandboxes and containers may need different credential provisioning.
- **Project cannot be selected:** run `testinel projects list` and pass `--project SLUG` explicitly.
- **No failed runs:** select a specific run with `--run RUN_UUID`; include `--include-flaky` when inspecting flaky passes.

Explore command options with `testinel --help` or, for example, `testinel runs list --help`.

## Development

Run these commands from the `testinel-cli` directory:

```bash
uv sync --group dev
uv run testinel --help
uv run poe
uv run poe test
uv run poe lint
uv run poe check
uv run poe build
```

`uv run poe` lists the available tasks. With the project's virtual environment activated, you can also run `poe` directly.

`poe lint` runs `ruff format`, `ruff check --fix`, and mypy in order. It modifies files to apply formatting and lint fixes. `poe check` runs linting followed by the test suite.

The hosted Django API is maintained separately from this repository.
