Metadata-Version: 2.4
Name: PyClickerRPA
Version: 0.6.0
Summary: Visual UI automation & testing tool: Windows + Android(ADB) automation, robust/self-healing image matching, OCR, structured conditions and loops with array iteration, node flow editor, smart recorder, task health checks and failure attribution, a visual test framework (assertions, visual regression, HTML/JUnit reports) and a web console.
Author: Equinox
License-Expression: MIT
Project-URL: Homepage, https://github.com/Equinox/PyClickerRPA
Keywords: autoclicker,rpa,automation,opencv,template-matching,ocr,gui,windows,android,adb,testing,visual-regression,ui-testing
Classifier: Environment :: Win32 (MS Windows)
Classifier: Operating System :: Microsoft :: Windows
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Desktop Environment
Classifier: Topic :: Software Development :: Testing
Classifier: Topic :: Multimedia :: Graphics :: Capture :: Screen Capture
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: End Users/Desktop
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.23
Requires-Dist: opencv-python>=4.8
Requires-Dist: pywin32>=305; sys_platform == "win32"
Requires-Dist: pynput>=1.7
Requires-Dist: PySide6>=6.5
Provides-Extra: ocr
Requires-Dist: paddleocr>=2.7; extra == "ocr"
Requires-Dist: paddlepaddle>=2.5; extra == "ocr"
Provides-Extra: web
Requires-Dist: fastapi>=0.110; extra == "web"
Requires-Dist: uvicorn[standard]>=0.27; extra == "web"
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Requires-Dist: httpx>=0.24; extra == "dev"
Dynamic: license-file

# PyClickerRPA

A visual UI **automation & testing** tool. Automate Windows apps *and* Android
devices/emulators by what's on screen — image/text matching + mouse/keyboard (or
ADB) input — then turn those flows into a **visual test framework** with
assertions, visual-regression baselines, HTML/JUnit reports, and a **web console**
for remote runs.

Python/PySide6, started as a port of a Qt/C++ AutoClicker. Legacy `.json` tasks
remain supported; structured control-flow blocks are a Python-only extension.

## Install

```
pip install PyClickerRPA
```

Optional extras:

```
pip install "PyClickerRPA[ocr]"   # PaddleOCR — find-text / read-text steps
pip install "PyClickerRPA[web]"   # FastAPI + uvicorn — web console
```

## Entry points

```
autoclicker                          # desktop GUI (flow editor / list editor / test tab)
autoclicker-run task.json --window "Notepad"   # headless run against a window
autoclicker-run task.json --adb                # run against an Android device (adb)
autoclicker-lint task.json                     # health-check a task (exit code = error count)
autoclicker-testkit suite.json --report out.html --junit out.xml   # run a test suite
autoclicker-web --suites-dir examples          # web console at http://127.0.0.1:8000
```

## Automation engine

- **Targets**: any visible window, the whole screen (multi-monitor + high-DPI
  aware), or an **Android device/emulator via ADB** (`--adb [serial]`).
- **Image matching**: OpenCV template match with **self-healing** by default —
  template → multi-scale → **ORB feature match** fallback (survives scaling /
  minor UI changes); **transparent-mask** templates (PNG alpha) ignore
  don't-care regions. Per-step `matchMode` = `robust` / `template` / `feature`.
- **Also**: pixel-color checks, optional PaddleOCR text find / read-to-variable.
- **Smart wait**: `wait_stable` waits until the screen (or an ROI) stops
  changing — replaces guessed `sleep`s (fast when ready, patient when slow).
- **~30+ step types**: click image/text/position, click-all, relative/offset
  clicks, wait / random / wait-stable, drag / **drag-from-image** / long-press / pinch / scroll,
  key / type / hotkey, loop / goto / while / do-until, if image/text/color/
  count/var with **true/false branching** (if-else), set-var, read-text→variable,
  activate-window, HTTP request, run-Python-script, call sub-task, groups, and a
  **stop** step to end a run — with retry / repeat / until policies.

## Authoring (desktop GUI)

Drag-and-drop **node flow graph** and an interoperable **step list**, with
nested **If/Else**, **While**, fixed-count loops, **ForEach** arrays, and
**Break/Continue** — no step numbers required. Double-click a block to edit its
body or branches. Legacy condition nodes retain their true/false ports.

Select consecutive steps and choose **包进… → 遍历数组** to reuse them for each
item. Enter a custom JSON array or generate a numeric range, then reference the
current item in text actions, assignments, and conditions. For example, iterate
`[1,2,3,4,5,6,7,8,9,10]` with variable `server`, click **`{server}服`**, perform
the actions, and return to the server-selection screen inside the same body.
Enable exact text matching so `1服` does not also match `11服`. A failed action
inside a block stops execution unless its failure policy explicitly skips it.

Also includes a snippet library, task packages (`.zip` with nested templates),
on-preview coordinate/color picking, template crop, single-step test run;
global hotkeys (F6 run/stop, ESC stop), scheduler, and an image/color
**monitor/trigger** with tray icon; a **Test** tab to load and run visual test
suites.

### Smart recorder

The action **recorder** (global hooks) doesn't just replay coordinates and
sleeps — with **智能录制 / smart mode** on (default) it records what things *look
like* instead of where they *were*:

- **Clicks become image steps.** A patch is cropped around the click and grown
  (48 → 192 px) until it is both textured enough and **unique on screen**, then
  emitted as a `click_image` step — so the flow still works after the window
  moves or the resolution changes. Identical patches are de-duplicated by
  content hash.
- **Drags become `drag_image`.** The drag's *start* is anchored to an image and
  the *displacement* is stored, so start and end move together.
- **Pauses become `wait_stable`.** A long pause is only converted if the
  recorder actually **observed the screen settle** during it; on a permanently
  animating screen it keeps the fixed wait (a `wait_stable` that never settles
  would fail the run).

Anything that can't be made robust falls back to the plain coordinate/fixed-wait
step and says why in the log.

## Diagnostics

**Task health check** (`autoclicker-lint task.json`, or 诊断 → 任务体检 in the GUI)
statically scans a task and separates *will definitely break* from *works but
fragile*:

- **errors** — missing template / sub-task file, image step with no template,
  loop / goto / fail-jump target out of range, unbalanced groups. The GUI asks
  for confirmation before running a task with errors; the CLI exit code is the
  error count.
- **warnings** — steps that depend on absolute coordinates, long fixed waits,
  a match threshold low enough to hit the wrong thing, templates too small to be
  unique. These are exactly the things the smart recorder now avoids, so the
  check doubles as a "what should I re-record?" list.

**Failure attribution.** A run captures a screenshot per step; the first
successful run becomes the baseline timeline (`.timelines/<task>/`). When a later
run fails, the frames are compared step by step and the report names the step
where the screen **first diverged** from the successful run — which is usually
*earlier* than the step that failed (step 7 can't find its button because a popup
appeared at step 4). You get the diverging step, the per-step changed-pixel
ratios, and a baseline|actual|heatmap image. If the screens match, it says so —
that points at timing/flakiness rather than a UI change.

## Visual test framework (`autoclicker.testkit`)

Turn automation into tests:

- **Assertions**: image / text / color present-or-absent, variable comparison.
- **Visual regression**: baseline diff (first run captures the baseline; later
  runs flag changed-pixel ratio and emit a baseline|actual|heatmap image).
- **Cases & suites**: declarative JSON suites *or* pytest fixtures (`screen`,
  `visual`); **data-driven** cases (a data table / CSV runs the case per row).
- **Reports**: self-contained **HTML** (embedded screenshots + per-step
  timeline, optional GIF) and **JUnit XML** for CI; retries mark **flaky**.

```python
# pytest style
def test_login(screen, visual):
    screen.assert_image("assets/login_btn.png")
    screen.assert_text("Username")
    visual.check("login.png", tolerance=0.02)
```

## Web console (`autoclicker-web`)

Browser UI to trigger suites (single or batched), watch **live per-line logs**
and per-case results, schedule recurring runs, list ADB devices and grab
**screenshots**, browse **persisted run history**, and open the HTML report — all
served by FastAPI with a zero-build vanilla-JS front end.

## Notes

- Host is **Windows** (Win32 capture + `SendInput`); ADB targets are
  cross-platform via the `adb` CLI (must be on `PATH`).
- OCR's first run downloads PaddleOCR models (~100 MB).
- Some target windows may require running as administrator.

## License

MIT
