Metadata-Version: 2.4
Name: pyguitest-recorder
Version: 0.8.2
Summary: Record desktop GUI activity on X11, XWayland, macOS and Microsoft Windows, and generate pyguitest test scripts
Author: Dennis K. Paulsen
License-Expression: GPL-2.0-or-later
Project-URL: homepage, https://github.com/ctrondlp/pyguitest-recorder
Project-URL: repository, https://github.com/ctrondlp/pyguitest-recorder
Project-URL: issues, https://github.com/ctrondlp/pyguitest-recorder/issues
Project-URL: documentation, https://github.com/ctrondlp/pyguitest-recorder/tree/main/docs
Project-URL: changelog, https://github.com/ctrondlp/pyguitest-recorder/blob/main/CHANGELOG.md
Project-URL: pyguitest, https://github.com/ctrondlp/pyguitest
Keywords: gui-recorder,record-and-replay,gui-testing,test-automation,desktop-automation,ui-testing,code-generation,test-generation,pyguitest,linux,bsd,freebsd,windows,win32,macos,x11,xrecord,cgeventtap,xwayland,accessibility,atspi,at-spi
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Environment :: X11 Applications
Classifier: Environment :: Win32 (MS Windows)
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: POSIX :: BSD :: FreeBSD
Classifier: Operating System :: POSIX
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Environment :: MacOS X
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Desktop Environment
Classifier: Topic :: Software Development :: Testing
Classifier: Topic :: Software Development :: Testing :: Acceptance
Classifier: Topic :: Software Development :: User Interfaces
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Code Generators
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pyguitest>=0.16.1
Requires-Dist: tomli>=2.0; python_version < "3.11"
Provides-Extra: x11
Requires-Dist: python-xlib>=0.33; extra == "x11"
Provides-Extra: macos
Requires-Dist: pyguitest[macos]; extra == "macos"
Provides-Extra: atspi
Requires-Dist: pyguitest[atspi]; extra == "atspi"
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: ruff>=0.6; extra == "dev"
Requires-Dist: mypy>=1.11; extra == "dev"
Dynamic: license-file

# pyguitest-recorder

[![CI](https://github.com/ctrondlp/pyguitest-recorder/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/ctrondlp/pyguitest-recorder/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/pyguitest-recorder)](https://pypi.org/project/pyguitest-recorder/)
[![License](https://img.shields.io/pypi/l/pyguitest-recorder)](https://github.com/ctrondlp/pyguitest-recorder/blob/main/LICENSE)

Record desktop GUI activity and generate [pyguitest](https://github.com/ctrondlp/pyguitest)
scripts from it.

The point is not to replay a macro. It is to turn what you did into test code
you would have been willing to write by hand — so a recorded click on a button
becomes

```python
gui.button("Save").click()
```

rather than `gui.move_mouse(180, 90); gui.click()`, which stops working the
moment the window moves, the theme changes, or someone adds a toolbar item.

## Quick start

```sh
pip install 'pyguitest-recorder[x11,atspi]'          # Linux, XWayland
pip install pyguitest-recorder 'pyguitest[windows]'  # Windows
pip install 'pyguitest-recorder[macos]'              # macOS

pyguitest-recorder --doctor              # can this machine record? why not?
pyguitest-recorder -o login_test.py      # record until Escape, Escape
python3 login_test.py                    # replay it (python on Windows)
```

> ⚠️ **Keyboard capture sees every application's keystrokes**, not only the one
> you are recording — including your password manager. That is what X11's
> RECORD extension, Windows' low-level input hooks and macOS's event tap all
> do, and it is why this tool exists at all. Close what you would not want in a
> file, and read [Privacy](#privacy) before recording anything that touches a
> login.

Recording needs X11 or XWayland on a Linux desktop, native Windows, or macOS.
On Linux that means no Wayland session will do: the recorder reaches XWayland
clients and nothing else, and says so rather than producing a file with silent
gaps.
[Why that is permanent](https://github.com/ctrondlp/pyguitest-recorder/blob/main/docs/developers/architecture.md#why-wayland-has-no-capture-backend).

Three flags carry most of the value:

```sh
pyguitest-recorder -o login_test.py --save-session rec.json   # keep both
pyguitest-recorder --regenerate rec.json -o out.py            # re-render, no recording
pyguitest-recorder --record-motion                            # capture hovers too
```

Recording stops on **Escape pressed twice**, not only Ctrl-C — a recorder you
can only stop from its own terminal is one you cannot stop while driving a
full-screen application. **Ctrl+1** records a check on whatever the pointer is
over. Both are rebindable; see
[docs/recipes.md](https://github.com/ctrondlp/pyguitest-recorder/blob/main/docs/recipes.md#rebinding-the-stop-and-check-keys).

[docs/getting-started.md](https://github.com/ctrondlp/pyguitest-recorder/blob/main/docs/getting-started.md) walks through all of this
properly.

## What comes out

```python
"""Generated by pyguitest-recorder. Edit freely.

Profile:     pyguitest-0.16
Recorded on: x11 (mutter)
Timeouts:    seconds; the wait the recording observed, rounded up (floor 10s, cap 300s)
"""

import pyguitest
from pyguitest import Capability, Role


def main() -> None:
    """Replay the recorded interaction."""
    with pyguitest.connect() as gui:
        gui.require(
            Capability.ELEMENT_ACTION,
            Capability.ELEMENT_TREE,
            Capability.WINDOW_ACTIVATE,
        )

        editor = gui.wait_for_window("Example", timeout=10)
        gui.activate_window(editor)
        gui.text_field("Name").set_text("Ada")
        gui.button("Save").click()


if __name__ == "__main__":
    main()
```

Plain pyguitest source, depending on nothing from this package. Three things
about it are deliberate.

### Elements lead, coordinates follow

Each click is resolved at record time against the accessibility tree, and
falls down a ladder only as far as it has to:

```
    gui.button("Save").click()      a named element — survives redesigns,
                                    themes, resizes and added toolbar items
        ↓  nothing accessible under the pointer?
    window-relative coordinates     survives the window moving
        ↓  no window accounts for the point?
    gui.move_mouse(842, 612)        absolute — breaks when anything moves
```

`--absolute-coordinates` forces the bottom rung; `--relative-coordinates`
forces the middle one.

The recorder **refuses to name an element it cannot corroborate** — if the
element's process does not own the window under that point, or its own extents
do not contain the point it was looked up at, the click becomes a coordinate
instead. A coordinate that works beats a named element that does not. Every
such refusal is written into the generated file's docstring, because "why is
this script all coordinates?" is the first thing its reader asks. The rules,
and the GTK4 measurements behind them, are in
[docs/developers/architecture.md](https://github.com/ctrondlp/pyguitest-recorder/blob/main/docs/developers/architecture.md#when-it-refuses-to-name-an-element);
the fix when the answer is your own application is
[docs/testable-guis.md](https://github.com/ctrondlp/pyguitest-recorder/blob/main/docs/testable-guis.md).

**Typed text is the exception, and gets named anyway.** A run of typing asks
the toolkit what has *focus* rather than what is under the pointer — focus
involves no geometry, so it still works where hit-testing has failed. A GTK4
application whose clicks all degrade to coordinates still produces
`gui.text_field("Name").set_text("Ada")`, including for a field reached by
Tab or focused by the application itself.

### Scripts declare what they need

Every file opens with `gui.require(...)` naming the capabilities it uses, so a
recording made on X11 and replayed somewhere weaker fails on the first line
with a typed exception instead of halfway through with a click that went
nowhere.

**And the file is checked before it is offered.** Generation compiles the
script, confirms every `gui.<method>` call exists on the installed
`pyguitest.Session`, and checks each `Capability` and `Role` constant against
the same — because a recorder that emits a plausible script naming a function
the library does not have is worse than no recorder.

### Waits, not sleeps

The difference between a recorder and a macro player is what happens to the
three seconds you spent waiting for a dialog. A macro player sleeps for three
seconds: slow when the machine is fast, broken when it is slow.

Each pause is instead asked what it was waiting for, and answered from what
the recorded events themselves saw:

| What the recording shows | What comes out |
|--------------------------|----------------|
| The next action is in a window nothing had seen before | `wait_for_window` |
| The next action is on a new element, in a window already open | `wait_for_element` |
| Nothing observable changed | `wait_for_idle(win.pid)` |
| None of the above | `gui.wait(...)`, and a comment saying why |

So a 12.4-second gap becomes

```python
# the recording waited 12.4s here for 'Save As' to open
saveas = gui.wait_for_window("Save As", timeout=13)
```

`timeout` is in seconds, and it is the wait that comment reports, rounded up to
the next whole second — 12.4 becomes 13 — floored at ten seconds and capped at
five minutes. Nothing is multiplied on the way out: read the number and you are
reading how long the recording itself waited, which is what lets the comment and
the call be checked against each other instead of looking like a bug.

The trade is deliberate. Headroom for a slower machine used to be baked into
every number, which also hid how long each wait was; now a replay machine much
slower than the one that recorded surfaces as a timeout rather than being
absorbed. Every generated file says the same in its header, and the number is a
plain literal when it needs raising — see
[troubleshooting.md](https://github.com/ctrondlp/pyguitest-recorder/blob/main/docs/troubleshooting.md#the-script-waits-too-long-or-not-long-enough).

Inference runs when a script is generated, not when a recording is made — so
`--regenerate` re-analyzes an old recording under whatever rules exist now,
and rendering the same file twice cannot compound.

## Checks: what makes it a test

A recording of actions alone is not a test. It passes as long as nothing
raises, whatever the application actually did — click Save, and a script that
never looks at the result passes just as happily against a build where saving
silently fails.

Point at what should have changed and press **Ctrl+1**:

```python
gui.button("Save").click()
saveas = gui.wait_for_window("Save As", timeout=10)

# Check: 'Status' reads 'Saved'
gui.expect_text(role=Role.LABEL, name="Status", equals="Saved")
```

What comes out depends on what was under the pointer — a checkbox gives
`expect_checked`, a label with something to say gives `expect_text`, anything
else named gives `expect_showing`. These are pyguitest `Session` methods, so a
generated script depends on nothing but pyguitest — and it needs 0.16.1 or
newer, as the install section explains. They name what was wrong instead of
raising a bare
`AssertionError`, and each retries until its timeout so a check cannot race a
redraw. Full table in
[docs/recipes.md](https://github.com/ctrondlp/pyguitest-recorder/blob/main/docs/recipes.md#checks-what-makes-it-a-test).

## Install

### From PyPI

```sh
pip install 'pyguitest-recorder[x11,atspi]'          # Linux, XWayland
pip install pyguitest-recorder 'pyguitest[windows]'  # Windows
pip install 'pyguitest-recorder[macos]'              # macOS
```

`x11` brings `python-xlib`, which capture needs on Linux. `atspi` is what
lets a click be recorded as a name instead of a coordinate there, through
pyguitest's own AT-SPI backend.

On native Windows, capture needs no extra at all — it's pure `ctypes` — so
`pyguitest-recorder` alone gives you coordinate-only recording; `pyguitest
[windows]` adds named elements, through UI Automation rather than AT-SPI.

On macOS, `pyguitest-recorder[macos]` is self-contained: it re-exports
pyguitest's own `macos` extra, so the two packages' PyObjC/Quartz dependency
can't drift apart.

### From a clone

To work on the recorder itself, or track `main` ahead of a release:

```sh
git clone https://github.com/ctrondlp/pyguitest-recorder
cd pyguitest-recorder
pip install -e '.[x11,atspi,dev]'

pyguitest-recorder --doctor
```

To run without installing anything, put `src/` on the import path —
[every flag works identically](https://github.com/ctrondlp/pyguitest-recorder/blob/main/docs/recipes.md#running-without-installing):

```sh
PYTHONPATH=src python3 -m pyguitest_recorder --doctor
```

### The part pip cannot do for you

`dogtail`, which element resolution goes through, **declares no dependencies
of its own**: PyGObject and pyatspi have to come from your distribution. Miss
them and nothing errors — `--doctor` reports element resolution off and every
click in every recording comes out as a coordinate, which looks like the
recorder being bad at its job rather than a missing package.

On Fedora `python3-gobject python3-pyatspi at-spi2-core`; on Debian and Ubuntu
`python3-gi python3-pyatspi gir1.2-atspi-2.0`. pyguitest's
[install guide](https://github.com/ctrondlp/pyguitest/blob/main/docs/install.md)
carries the full table, including Arch, openSUSE and FreeBSD.

**pyguitest 0.16.1 or newer is required outright** — the floor the generated
code is verified against. Generated scripts call the `expect_` family as
pyguitest `Session` methods, which do not exist before 0.9.0; double-click a
named element with `Element.double_click`, which 0.10.0 added; once `motion`
is set to `"natural"` or `"recorded"`, move the pointer with
`Session.move_mouse_naturally`, which 0.10.1 added; and 0.11.0 is the first
release that imports on Microsoft Windows at all. `Element.expand()`/
`.collapse()` and reading `.selectable` directly, which 0.12.0 added, are what
the floor required until 0.14.0 raised it again: that is the release whose
`macos` backend a macOS recording resolves windows and elements through, and
whose `macquartz` vocabulary its key names are translated through, so a
recording made on a Mac has nothing older to replay or regenerate against.
0.15.0 was the step nothing here strictly needed -- the pyguitest this output
was last verified against, and no more. 0.15.1 was the first floor step in a
while an older install can genuinely get wrong rather than merely lack: a
generated script under `locators = "element"` (the default) scopes an element
search with `gui.window_element(title)`, and an older pyguitest there could
resolve that call to a shell-owned decoration proxy instead of the real window
on a real GNOME/Mutter desktop, found live. 0.16.0 is the release this output
is verified against now, and it is not a quiet step either -- five calls in a
generated script behave differently there. `gui.tap_key(...)` on Windows now
carries the scan code and extended bit the layout gives the key, where `Home`,
an arrow or Right Ctrl went out as a key no keyboard sends; `expect_checked`
reads a selected radio button as checked, not as None; macOS `double_click()`
counts its clicks, so a Cocoa application sees a double click; `gui.drag`
posts drag events instead of moves; and `find_window`/`wait_for_window` name
the topmost of two windows sharing a title rather than the one behind.
0.16.1 is a patch step with the same character: on Linux, `Element.click()` on a
GTK switch or toggle button did nothing, and the `input` backend's pointer
landed one pixel short of an absolute position, so a recorded coordinate click
missed by one.
`validate()` checks
`gui.*` calls against the installed `Session` and what is called on an element
against the installed `Element`, so a script naming a method this pyguitest does
not have is reported INVALID when it is generated. `--doctor` prints the
installed pyguitest next to the generator profile, which is what the emitted
calls were checked against.

The older floors still matter for what the scripts *do*. 0.5.0 is where role
lookups learned to accept both spellings of a renamed role, and where
`Window.app_id` starts being populated on X11, which is what lets a window whose
title drifts still be found; earlier versions lack `element_at`/`extents` and
`double_click` as well.

## Privacy

Keyboard capture — XRecord on X11 and XWayland, a low-level input hook on
Windows, an event tap on macOS — sees **every application's keystrokes**, not
only the one you are recording, including your password manager.

- Text typed into an AT-SPI password field is detected and never written into
  the generated script; it gets `os.environ["SECRET_1"]` instead. A check
  recorded against a password field is redacted the same way.
- `--sensitive` treats *all* text that way.
- Raw event logs are off unless `--record-raw` is passed.
- A saved recording is a credential-bearing artifact. Treat it like one —
  redaction happens when a script is generated, not when events are saved.

## Configuration

`$XDG_CONFIG_HOME/pyguitest-recorder/config.toml` wherever that is set. With
it unset the default is per-platform — `~/.config/pyguitest-recorder/config.toml`
on Linux and the BSDs, `%APPDATA%\pyguitest-recorder\config.toml` on Windows,
since `~/.config` is neither conventional nor discoverable there. Either way
`~/.pyguitest-recorder.toml` is read if the first is absent. Precedence is
defaults → file → command line. See
[config.example.toml](https://github.com/ctrondlp/pyguitest-recorder/blob/main/config.example.toml).

## Status

**Early, but the engine is complete and every X11 path has been run live** —
including capture of a real application, AT-SPI element resolution against a
real accessibility bus, and focus-based targeting for typed text.
`scripts/live-capture-check.py` runs the whole pipeline against a private Xvfb
on every push, and it has found two bugs that no unit test could have.

**Windows recording is newer, but has now captured a real keystroke.** A
`SetWindowsHookExW` hook, driven against a purpose-built native window and
against real Windows 11 Notepad, recorded and correctly replayed typed text,
clicks, menus, tabs, a dropdown, a window drag mid-recording, and a
maximize/restore cycle. Its `SysListView32` selection carries the one
asterisk: the mechanism passed on its own, and the single most complex full
run did not reproduce it — left open rather than claimed fixed.
`scripts/win32-live-capture-check.py` is the Windows counterpart of
`live-capture-check.py`. See
[docs/developers/status.md](https://github.com/ctrondlp/pyguitest-recorder/blob/main/docs/developers/status.md) for what those runs
found and fixed, and what is still outstanding.

**macOS is newer still, and has been through the same round trip.** A
listen-only `CGEventTap` captured real events from a granted macOS 26.7 Mac,
and a record → generate → replay → re-record cycle on that machine compared two
recordings event by event: pointer, buttons, scroll, chords and typed text all
came back, with windows and elements resolving for both halves of a click.
What no live run has shown yet is a person's own keyboard — the tap skips the
installing process's own events, so every check so far posted them from a
separate process, and no real hardware key has been through it.
`scripts/` has no macOS counterpart of the two live-check scripts above.

The per-part verification table, and the known gaps — no UI yet, two
recordings of a real desktop application (one on GhostBSD, one of Windows 11
Notepad), and what GTK4 hit-testing costs — are in
[docs/developers/status.md](https://github.com/ctrondlp/pyguitest-recorder/blob/main/docs/developers/status.md).

## Documentation

- [docs/getting-started.md](https://github.com/ctrondlp/pyguitest-recorder/blob/main/docs/getting-started.md) — from nothing to a test
  you can run
- [docs/recipes.md](https://github.com/ctrondlp/pyguitest-recorder/blob/main/docs/recipes.md) — every flag that matters, by the task it
  serves
- [docs/troubleshooting.md](https://github.com/ctrondlp/pyguitest-recorder/blob/main/docs/troubleshooting.md) — "why is my script all
  coordinates?", and the rest
- [docs/testable-guis.md](https://github.com/ctrondlp/pyguitest-recorder/blob/main/docs/testable-guis.md) — how to build a GUI that can
  be tested at all; written to be handed to application developers
- [docs/developers/architecture.md](https://github.com/ctrondlp/pyguitest-recorder/blob/main/docs/developers/architecture.md) and
  [docs/developers/status.md](https://github.com/ctrondlp/pyguitest-recorder/blob/main/docs/developers/status.md) — why Wayland has no
  capture backend, the element-resolution rules, and what has actually been run

## License

GPL-2.0-or-later, the same as [pyguitest](https://github.com/ctrondlp/pyguitest)
— which this imports at run time and generates source for. See [LICENSE](https://github.com/ctrondlp/pyguitest-recorder/blob/main/LICENSE).
