Metadata-Version: 2.5
Name: mua-driver
Version: 0.3.0
Summary: Mobile-use driver for Android and iOS devices, inspired by CUA Driver.
Author: MUA Driver contributors
License-Expression: MIT
License-File: LICENSE
Keywords: android,computer-use,ios,mcp,mobile-automation
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Python: >=3.11
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == 'dev'
Provides-Extra: image
Requires-Dist: pillow>=10.0; extra == 'image'
Description-Content-Type: text/markdown

# MUA Driver

A mobile-use driver for agents. It controls Android and iOS devices and exposes
them over MCP, a JSON CLI, and a small Python SDK. It runs no model of its own:
an agent brings the decisions, MUA carries them out and reports what actually
happened on the device.

- **Android** devices and emulators through `adb`, with an optional
  Accessibility Helper for the things `adb` cannot do
- **iOS** Simulators through `xcrun simctl` and `idb`, physical devices through
  `xcrun devicectl` and WebDriverAgent
- **MCP** over stdio or authenticated HTTP, with a capability-filtered tool list

## Install

```bash
python3 -m pip install -e '.[dev]'
```

Host tools: Android platform-tools (MUA resolves `MUA_ADB_PATH`, then
`ANDROID_SDK_ROOT`/`ANDROID_HOME`, then `PATH`); Xcode command line tools with
`xcrun` for iOS Simulators; Facebook `idb` for iOS touch and text input; Xcode
15+ with Developer Mode and a signed, running WDA for physical iOS devices.

## Quick start

```bash
mua-driver call list_devices
mua-driver call screenshot --device emulator-5554 --output /tmp/screen.png
mua-driver call tap_element --device emulator-5554 --platform android \
  --using text --value "Continue"
mua-driver mcp                      # MCP server over stdio
```

```python
from mua_driver import MuaDriver

driver = MuaDriver.create()
driver.tap_element(device_id="emulator-5554", platform="android",
                   using="text", value="Continue")
```

`mua-driver call --help` lists every tool. [docs/](docs/) has the details.

## Tool coverage

| Capability | Tools | Android | iOS Simulator | iOS device |
| --- | --- | --- | --- | --- |
| Discovery | `list_devices`, `list_apps`, `is_app_installed` | yes | yes | yes |
| Visual/UI | `observe`, hierarchy reads, `resolve_target`/`tap_target`, semantic element actions, waits, assertions | yes | yes | yes |
| Action safety | preflight target validation, semantic-first targeting, postcondition verification, bounded retry | yes | yes | yes |
| Sequencing | `batch_commands` with per-step trace and optional trailing observation | yes | yes | yes |
| Input | tap, drag/fling (velocity-aware)/pinch, element drag, W3C actions, text, keyboard | yes | yes | yes |
| Platform input | `key`, `back`, `shell` / `pinch`, `shake` | yes | yes | partial |
| App lifecycle | `launch_app`, `stop_app`, `install_app`, `uninstall_app` | yes | yes | yes |
| App state | foreground state, restart, metadata, explicit reset | yes | yes | yes |
| Navigation | `open_url`, custom deep links | yes | yes | yes |
| Device state | permissions, location, orientation, appearance | yes | partial | partial |
| Files and diagnostics | container files, logs, crash logs, bugreport, video, battery | yes | yes | yes |
| Recording | start/stop/cancel screen recording | yes | yes | yes |
| Sync | `wait_for_idle`, screen-change and activity waits | yes | yes | yes |
| Alerts | `get_alert`, `handle_alert` | yes | yes | yes |
| Environment | IME selection, locale/timezone, wifi/airplane mode, volume | yes | partial | unsupported |
| Pasteboard | `get_clipboard`, `set_clipboard` with restore, paste with restore | helper | yes | yes |
| Foldables | `get_posture`, `set_posture`, `get_display_segments`, `locate_point` | yes | unsupported | unsupported |
| Audit and trace | `get_audit_log`, `start_trace`…`export_trace`, `replay_trace` | yes | yes | yes |
| Automation context | `list_contexts`, `get_context` — which context is active | no | native only | setup |

## What MUA does differently

**Actions are verified by their effect.** Every semantic action re-resolves its
target on a live hierarchy before dispatch, declares what it expects
(`screen_change`, `element_present`, `text_value`, `activity`, …), waits for
that outcome, and reports `verified` or a structured failure with a bounded
retry. A tool that reports success has either dispatched a real action or told
you it degraded. See [docs/action-safety.md](docs/action-safety.md).

**Gestures are motion, not two events.** `adb shell input swipe` delivers a
press and a release with no movement between them on many devices. MUA ships its
own Accessibility Helper and prefers it for drags, flings and pinches, so a
ScrollView actually scrolls and a pinch actually produces a scale event. See
[docs/android-helper.md](docs/android-helper.md).

**Platform limits are named, not hidden.** A capability the platform does not
expose is reported `unsupported_by_platform` with a reason. The verification
suite records the same way: every tool ends as `verified`, `requires_setup`, or
`unsupported` — never a silent skip.

## Compared to other mobile drivers

Read from each project's own README at the time of writing. *Not stated* means
the project does not document the capability; it is not a claim that it is
missing.

| | **MUA Driver** | [mobile-use](https://github.com/minitap-ai/mobile-use) | [ARTEMIS](https://github.com/google/artemis) | [mobile-mcp](https://github.com/mobile-next/mobile-mcp) |
| --- | --- | --- | --- | --- |
| Ships | driver only, no model | full agent | agent + test framework | MCP server |
| Android device + emulator | yes | yes | yes | yes |
| iOS simulator | yes | yes | no, on roadmap | yes |
| iOS physical device | yes (WDA) | not stated | no, on roadmap | yes |
| Interfaces | MCP (stdio, HTTP), CLI, Python SDK | agent library, MCP | MCP, CLI, Python SDK, web console | MCP, TS CLI |
| Own on-device helper | yes (`android/self_apk`) | no | yes (helper APK) | no |
| Verifies the effect of an action | every semantic action | not stated | checker, per-profile verification levels | not stated |
| Re-validates target before dispatch | every semantic action | not stated | safety net (XML, then pixel) | not stated |
| Finding an element | semantic, then coordinate | accessibility tree; weak on games | index, coordinate, visual (OCR + vision model) | accessibility tree first, coordinates second |
| Multi-touch | pinch, fling, element drag | not stated | not stated | swipe |
| Unicode text input | yes (helper) | not stated | not stated | text entry |
| Clipboard | read, replace, restore | not stated | not stated | read, replace |
| Crash logs | list, get, clear | not stated | logcat collection | list, get |
| Foldable posture | model and actuate | not stated | not stated | fold/unfold |

The comparison is about shape, not depth: all four expose an MCP tool surface,
and all four can find an element semantically before falling back to
coordinates. What differs is in this repository's column — it ships an on-device
helper, and it verifies the effect of an action rather than reporting that a
command was accepted.

Where MUA is behind: ARTEMIS runs a vision model as a third fallback for Canvas
and Compose interfaces. MUA runs no model of any kind, so a target it cannot
find semantically is reached by coordinate or not at all — see
[docs/targeting.md](docs/targeting.md).

## Verification

Both device surfaces are measured, not asserted. A suite drives the real driver
against a live device and records an explicit verdict per tool, asserting against
what the test app recorded receiving — event counts, pointer contact time, peak
velocity, pinch span, selected range — rather than against a returned `ok`.

```bash
MUA_ANDROID_TEST_DEVICE=emulator-5554 python -m pytest tests/verification -q
MUA_IOS_SIMULATOR_UDID=6F7241FF-... python -m pytest tests/verification_ios -q
```

Android: 123 tests in about 2¾ minutes, 163 verdicts across 129 tools — 161
verified, 1 `requires_setup` (clipboard read, which Android 10+ grants only to the
focused app), 1 `unsupported` (`get_screen_state`, iOS-only in this surface).

iOS: the same seven modules against a booted Simulator — 141 tests, about eleven
minutes, 159 verdicts across 122 tools, with every tool the schema lets a caller
reach on iOS covered. 125 verified; the other 34 are platform limits the Simulator
cannot lift, each recorded with the measurement behind it and a typed refusal from
the driver rather than an `ok`.

Neither run has a failing or unrun tool, and a coverage gate fails either one if a
tool lacks a verdict or fails one.

[DRIVER_VERIFICATION.md](DRIVER_VERIFICATION.md) has the method, both results and
the defects they found — among them an iOS `tap` path that passed floats to a
command that parses integers, which had broken every element-addressed tool on
that platform; a selector table in the iOS backend that resolved `content-desc`
nowhere, which made `scroll_element_into_view` scroll a screen looking for an
element the hierarchy already held; an Android `set_permission` that returned `ok`
for a grant the platform had silently declined to make; and an Android
`handle_alert("accept")` that returned `ok` after pressing *Deny*, because the two
dialog buttons were held in a set and sets do not keep an order.

## Boundaries

- Android defaults to platform tools (`adb shell input`, `adb exec-out`, package
  manager); the accessibility helper is opt-in and never installed or enabled
  without being asked.
- iOS Simulator discovery, screenshots, launch and termination use `simctl`;
  gesture, text and accessibility operations use `idb`. Physical iOS uses
  `devicectl` for discovery and diagnostics, WDA for interaction.
- The MCP server is stdio JSON-RPC with `initialize`, `tools/list`, and
  `tools/call`.

## Documentation

| Document | Covers |
| --- | --- |
| [docs/cli.md](docs/cli.md) | CLI usage, action sequences with `batch_commands` |
| [docs/mcp.md](docs/mcp.md) | MCP registration, capability-filtered tool list, HTTP transport |
| [docs/action-safety.md](docs/action-safety.md) | preflight, postconditions, bounded retry |
| [docs/targeting.md](docs/targeting.md) | semantic-first targeting and coordinate fallback |
| [docs/python-sdk.md](docs/python-sdk.md) | embedding the driver |
| [docs/android-helper.md](docs/android-helper.md) | the accessibility helper, backend selection, shell policy |
| [docs/ios.md](docs/ios.md) | physical iOS devices, WDA, per-target capabilities |
| [docs/device-primitives.md](docs/device-primitives.md) | screen, orientation, buttons, clipboard |
| [docs/crashes.md](docs/crashes.md) | crash resource model |
| [docs/foldables.md](docs/foldables.md) | foldable posture model |
| [docs/traces.md](docs/traces.md) | action traces and replay |