Metadata-Version: 2.4
Name: smokeops
Version: 0.1.3
Summary: Turn a prompt into a production smoke check you own — local-first synthetic monitoring.
Project-URL: Homepage, https://github.com/gate3/SmokeOps
Project-URL: Repository, https://github.com/gate3/SmokeOps
Project-URL: Issues, https://github.com/gate3/SmokeOps/issues
Author: Doyin Olarewaju
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Keywords: cli,playwright,qa,smoke-test,synthetic-monitoring
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.10
Requires-Dist: litellm>=1.93.0
Requires-Dist: playwright>=1.61.0
Requires-Dist: pydantic-settings>=2.14.2
Requires-Dist: pydantic>=2.13.4
Requires-Dist: python-dotenv>=1.2.2
Requires-Dist: pyyaml>=6.0.3
Requires-Dist: rich>=15.0.0
Requires-Dist: typer>=0.16.0
Description-Content-Type: text/markdown

<p align="center">
  <img src="https://raw.githubusercontent.com/gate3/SmokeOps/main/docs/smokeops-banner.png" alt="SmokeOps" width="420" height="420" />
</p>

<h1 align="center">SmokeOps</h1>

<p align="center">Turn a plain-language prompt into a smoke check you own — local-first synthetic monitoring with Playwright.</p>

## What is SmokeOps?

SmokeOps helps you verify that critical user flows work correctly by turning natural language descriptions into automated smoke checks. You write what matters to test, SmokeOps proposes steps, you review and approve them, and you get back a JSON file you keep in your repository. Approved checks can be run or scheduled without ever calling an LLM again.

**Why this matters:** Most people either skip synthetic monitoring entirely, write fragile Playwright scripts by hand, or rely on hosted services they don't control. SmokeOps gives you AI-powered creation paired with a durable artifact (JSON in your repo) that you can review, edit, and schedule.

SmokeOps is **alpha** (actively developed, real-world friendly, some rough edges). You provide an LLM API key for the `create` step only; scheduled runs need no external service.

<p align="center">
  <img src="https://raw.githubusercontent.com/gate3/SmokeOps/main/docs/smokeops-demo.gif" alt="SmokeOps demo: create a multi-page check from a prompt, approve it, then run it" width="800" />
</p>

## Installation

Requires **Python 3.10 or later**. Install the CLI into your environment:

```bash
pip install smokeops
# or
uv tool install smokeops
```

Confirm the installation:

```bash
smokeops version
```

On your first `create` or `run`, SmokeOps installs Chromium automatically if it's missing. In CI environments, you need to keep an explicit `playwright install` step in your workflows (examples are provided below).

## Five-Minute Quickstart

This walkthrough shows the entire workflow in about 5 minutes using the included PeakAir demo site (a sample react app).

### 1. Start the demo site (optional)

The quickstart below uses **PeakAir**, a sample app included in this repository. Clone the repo and start it locally:

```bash
git clone https://github.com/gate3/SmokeOps.git
cd SmokeOps/examples/peakair-hvac
npm install
npm run dev
```

PeakAir runs at `http://localhost:3000`. Leave this terminal running.

If you installed SmokeOps from PyPI and do not want to clone the repo, skip this step. In step 3, replace the URL and prompt with a site you can reach (for example your own app or a public staging URL).

### 2. Set up LLM credentials

In a **new terminal**, work from the directory where you want `monitors/` to live (for this walkthrough, the SmokeOps repo root: `cd SmokeOps`).

Create a `.env` file or export environment variables. SmokeOps routes LLM calls through [LiteLLM](https://docs.litellm.ai/) and supports Gemini (default), OpenAI, and other providers:

```bash
export LLM_PROVIDER=gemini
export DEFAULT_MODEL=gemini-3.6-flash
export GEMINI_API_KEY=your_key_here
```

### 3. Create your first check

Tell SmokeOps what to test:

```bash
smokeops create --url "http://localhost:3000/" \
  "Verify the homepage loads and the product name PeakAir is visible"
```

By default, `create` runs **headless**: Playwright drives Chromium in the background with no visible browser window. You follow progress in the terminal (navigation, DOM pruning, step results). When all steps pass, SmokeOps shows a summary and waits for your approval.

To watch the browser while steps run:

```bash
smokeops create --headed --url "http://localhost:3000/" \
  "Verify the homepage loads and the product name PeakAir is visible"
```

`--headed` opens a real browser window, prints each step as it runs, and pauses briefly between asserts so you can follow along. When the run finishes, press Enter to close the window, then approve or reject as usual.

### 4. Approve the check

After SmokeOps runs the steps, you'll see a summary. Choose one of:

- **accept** — saves to `monitors/<name>.json` in your current directory (the check name is a slug from your prompt; edit the draft to rename before accept)
- **reject** — discards the draft
- **edit** — opens the JSON in your editor to adjust the name or summary

Accept this one. Note the path printed after accept (for example `monitors/verify-the-homepage-loads-and-the-product-name-peakair-is-visible.json`).

### 5. Run the approved check

Now run the check without the LLM, using the path from the accept step:

```bash
smokeops run monitors/<name>.json
```

Or omit the path to pick from `monitors/*.json` interactively:

```bash
smokeops run
```

No LLM is needed from this point. The check runs offline, using only the Playwright steps you approved. If it passes, you see `PASS`. If it fails, SmokeOps saves a screenshot and trace under `.smokeops/artifacts/` for debugging.

**Try a bundled journey check (clone only):** this repo also includes an approved multipage PeakAir check you can run without creating one:

```bash
smokeops run monitors/complete-the-signup-form-with-sample-contact-details-choose-the-seasonal-tune-up.json
```

---

That's the core loop: **create** (LLM proposes) → **approve** (you decide) → **run** (offline verification, on schedule).

## Why Choose SmokeOps?

| Scenario                                                | Alternative                                        | SmokeOps                                                              |
| ------------------------------------------------------- | -------------------------------------------------- | --------------------------------------------------------------------- |
| **You write Playwright by hand**                        | Time-consuming, brittle selectors, easy to drift   | LLM proposes steps, you review once, JSON stays in your repo          |
| **You use a hosted monitor (e.g., DataDog, New Relic)** | Opaque, vendor lock-in, separate UX from your code | Transparent JSON in your repo, portable, schedule with GitHub Actions |
| **You skip synthetic monitoring**                       | No visibility into critical flows                  | 5-min setup, offline execution, owned artifact                        |
| **You need custom selectors or logic**                  | Hosted services limit customization                | Edit the JSON directly; it's human-readable Playwright                |

### What SmokeOps is NOT

- **Not a full E2E test framework** — for comprehensive test suites with hundreds of scenarios, use Playwright Test
- **Not a visual regression tool** — focuses on flow verification and assertions, not pixel-perfect comparisons
- **Not designed for localhost in CI** — use a public staging URL or tunnel service for GitHub Actions
- **Not a monitoring platform** — no dashboards, alerts, or historical trend analysis; use tools like DataDog or Grafana for that

## Usage Guides

The quickstart runs through that loop once. These sections go deeper on each step.

### Create

Describe what to test in plain language:

```
On the signup page, fill in an email, submit the form, and verify the account-created message appears.
```

When you run `smokeops create`, SmokeOps:

1. Navigates to the URL you provide
2. Finds interactive elements on the page (buttons, inputs, links, form controls)
3. Asks the LLM to propose Playwright steps that match your prompt
4. Runs those steps in a real browser to verify they work
5. Shows you a summary and waits for your decision

By default, `create` runs headless (no visible browser window). Add `--headed` to watch the browser and see each step in the terminal.

#### Single check

```bash
smokeops create --url "https://example.com/signup" \
  "Complete a signup with email and password, then verify the dashboard loads"
```

```bash
smokeops create --headed --url "https://example.com/signup" \
  "Complete a signup with email and password, then verify the dashboard loads"
```

With `--headed`, SmokeOps opens a browser window, prints each step as it runs, and prompts you to press Enter to close the window before approval.

#### Suite file (batch)

For multiple checks, use a YAML suite file:

```yaml
# suite.yml
defaults:
  base_url: https://example.com

checks:
  - name: homepage-loads
    url: /
    prompt: |
      Verify the homepage loads and the company name is visible

  - name: user-signup
    url: /signup
    prompt: |
      Complete the signup form with sample contact details,
      accept the terms, and verify the account-created confirmation

  - name: user-login
    url: /login
    prompt: |
      Log in with a valid email and password, and verify the dashboard appears
```

Create all checks at once:

```bash
smokeops create --from suite.yml
```

SmokeOps walks through each check in order, proposing and waiting for your approval (accept/reject/edit/skip) for each. Approved checks are saved to `monitors/`. If a check file already exists, it is skipped.

```bash
smokeops create --from suite.yml --headed
```

With `--headed`, press Enter after each check completes to close the browser, then approve or reject.

### Approve & edit

After `create` proposes steps, you see a plain-English summary and are prompted:

```
Choose a, r, e, or s.
```

- **a** (accept) — saves `monitors/<name>.json`
- **r** (reject) — discards the draft
- **e** (edit) — opens the draft JSON in your editor (`$EDITOR` or `$VISUAL`, fallback `vi`)
- **s** (skip) — available only with `--from` (suite file)

An accepted check is saved as `monitors/<name>.json` with:

- A plain-English summary (what you asked for and what the check does)
- The Playwright steps (action, element locator, assertions)
- The URL and your original prompt for reference

If you choose **edit**, SmokeOps opens your editor. Make changes (rename the check, adjust the summary, tweak a step), save and quit. SmokeOps re-validates the JSON and re-runs the steps to confirm they still work. Then you can accept or reject.

Set your preferred editor:

```bash
export EDITOR=nano
# or
export EDITOR="code --wait"  # VS Code
```

### Run

The approved JSON file is your source of truth. `smokeops run` executes the stored steps with no LLM call and no API key:

```bash
smokeops run monitors/homepage.json
```

Omit the path in an interactive terminal to pick from `monitors/*.json`. Pass a path in CI or scripts.

```bash
smokeops run --headed monitors/homepage.json
```

With `--headed`, the browser window stays open until you press Enter.

On failure, SmokeOps writes a screenshot and Playwright trace under `.smokeops/artifacts/` and prints those paths. Passing runs do not leave artifact files.

Exported `*.spec.py` files can also be run with `smokeops run`, but `--headed` is ignored for them.

### Scheduled runs with GitHub Actions

Smoke checks run on a timer via GitHub Actions. The workflow handles setup, browser installation, and check execution. Scheduled `run` does not need an LLM API key.

#### Setup (one-time)

1. Copy [`examples/workflows/smokeops-cron.yml`](examples/workflows/smokeops-cron.yml) from this repository into **your** app repo as `.github/workflows/smokeops-cron.yml`:

```bash
cp examples/workflows/smokeops-cron.yml .github/workflows/smokeops-cron.yml
```

If you do not have a SmokeOps checkout, download the file from GitHub and save it at that path in your project.

2. Commit and push your approved check files under `monitors/`:

```bash
git add monitors/*.json
git commit -m "Add approved smoke checks"
git push
```

3. Adjust the cron schedule if needed. The template default runs every 15 minutes:

```yaml
on:
  schedule:
    - cron: "*/15 * * * *" # Every 15 minutes. Adjust for your needs.
```

#### What runs on each schedule

Each time the schedule triggers:

1. GitHub Actions checks out your repo
2. Installs SmokeOps via PyPI
3. Installs Chromium (cached across runs)
4. Runs `smokeops run` for each `monitors/*.json` file
5. If any check fails, the job fails and uploads failure artifacts

#### Debugging failed runs

When a check fails on schedule:

1. Open the failed workflow run in GitHub Actions
2. The log shows which checks passed and which failed
3. SmokeOps writes a screenshot (PNG) and Playwright trace (`.zip`) to `.smokeops/artifacts/`
4. The workflow uploads these as a GitHub Actions artifact named `smokeops-failure-artifacts`
5. Download the zip, open the PNG in an image viewer, and open the trace with `playwright show-trace`

#### Running only some checks

If you have many checks but want to run a subset on schedule, edit the workflow to use a matrix:

```yaml
strategy:
  fail-fast: false
  matrix:
    check:
      - monitors/homepage.json
      - monitors/login.json
      - monitors/checkout.json

steps:
  # ... setup steps ...
  - name: Run approved smoke check
    run: smokeops run ${{ matrix.check }}
```

#### Reaching external URLs

GitHub Actions runners are temporary cloud machines maintained by GitHub. They can reach any public URL on the internet, but they cannot reach:

- `localhost` or `127.0.0.1` (your local machine)
- Private IP addresses (internal networks)
- VPC-only resources

If your staging environment is not publicly reachable, you have two options:

1. **Public staging URL** — If you have a public staging or preview URL (for example `staging.example.com`), point your checks there.
2. **Tunnel service** — Services like [Cloudflare Tunnel](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/), [ngrok](https://ngrok.com/), or [localtunnel](https://localtunnel.me/) expose your local machine to the internet. After setting up a tunnel, you get a public URL (e.g., `https://abc123.ngrok.io`) that routes to your local machine. Use that URL in your checks.

---

## Configuration

### Environment Variables

Set these for `create` (LLM calls) and optionally `run` (for logging):

| Variable                                    | Purpose                                                                                                                                                   | Default                      |
| ------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------- |
| `LLM_PROVIDER`                              | LiteLLM provider ID (e.g., `gemini`, `openai`, `anthropic`)                                                                                               | `gemini`                     |
| `DEFAULT_MODEL`                             | Model ID without provider prefix (e.g., `gemini-3.6-flash`)                                                                                               | _(required)_                 |
| `GEMINI_API_KEY`                            | Required if `LLM_PROVIDER=gemini`                                                                                                                         | _(required if using Gemini)_ |
| `OPENAI_API_KEY`                            | Required if `LLM_PROVIDER=openai`                                                                                                                         | _(optional)_                 |
| `LOG_LEVEL` or `SMOKEOPS_LOG_LEVEL`         | Logging verbosity: `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`, or `OFF`                                                                              | `INFO`                       |
| `SMOKEOPS_PRUNE_WAIT_MS` or `PRUNE_WAIT_MS` | Max milliseconds to wait for the page to become idle during DOM pruning (soft timeout—pruning continues after). Useful for SPAs that take time to settle. | `10000`                      |

SmokeOps reads from a `.env` file in the current directory first, then falls back to environment variables.

### Example `.env`

```bash
LLM_PROVIDER=gemini
DEFAULT_MODEL=gemini-3.6-flash
GEMINI_API_KEY=your_api_key_here
LOG_LEVEL=INFO

# Optional: increase wait time for SPAs
# SMOKEOPS_PRUNE_WAIT_MS=20000
```

### Using a different LLM provider

To use OpenAI, Anthropic, or any [LiteLLM](https://docs.litellm.ai/)-supported provider, set `LLM_PROVIDER` and `DEFAULT_MODEL`:

```bash
export LLM_PROVIDER=openai
export DEFAULT_MODEL=gpt-4o-mini
export OPENAI_API_KEY=sk-...
```

No code changes needed—SmokeOps routes all LLM calls through LiteLLM.

### Logging

Control verbosity:

```bash
LOG_LEVEL=DEBUG smokeops create --url "..." "..."  # Verbose
LOG_LEVEL=OFF smokeops run monitors/check.json     # Silent
```

---

## Other Commands

### Show a check summary

Print the plain-English summary and metadata for an approved check without running it:

```bash
smokeops show                          # Pick interactively
smokeops show monitors/homepage.json   # Specific file
```

### Export to standalone Playwright

If you want to run a check as a plain Playwright script (without SmokeOps), export it:

```bash
smokeops export monitors/homepage.json
# Writes: monitors/homepage.spec.py
```

Or choose a custom path:

```bash
smokeops export monitors/homepage.json --out exports/smoke.py
```

The exported file is a runnable Playwright script with no SmokeOps dependencies.

---

## Contributing

To work on SmokeOps itself (development setup, tests, publishing), see [CONTRIBUTING.md](CONTRIBUTING.md).

## License

Licensed under the [Apache License, Version 2.0](LICENSE).  
See [NOTICE](NOTICE) for copyright.
