Metadata-Version: 2.5
Name: gothos
Version: 0.2.2
Summary: Provider-agnostic Android AI agent — automates any device using any LLM with safety gates, memory, planning and visual grounding
Project-URL: Homepage, https://github.com/GouthamRajam/GothOS
Project-URL: Repository, https://github.com/GouthamRajam/GothOS
Project-URL: Issues, https://github.com/GouthamRajam/GothOS/issues
Project-URL: Changelog, https://github.com/GouthamRajam/GothOS/releases
Author: GouthamRajam
License-Expression: MIT
License-File: LICENSE
Keywords: accessibility,adb,ai-agent,android,automation,grounding,langgraph,llm,mcp,mobile,safety,testing
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: Pydantic :: 2
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
Classifier: Typing :: Typed
Requires-Python: >=3.12
Requires-Dist: aiosqlite>=0.20
Requires-Dist: langchain-core>=0.3
Requires-Dist: langgraph>=0.2
Requires-Dist: pydantic>=2.0
Provides-Extra: console
Requires-Dist: fastapi>=0.115; extra == 'console'
Requires-Dist: uvicorn[standard]>=0.32; extra == 'console'
Provides-Extra: dev
Requires-Dist: mypy>=1.13; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.24; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.8; extra == 'dev'
Provides-Extra: ios
Requires-Dist: pymobiledevice3>=4.0; extra == 'ios'
Description-Content-Type: text/markdown

# GothOS

[![CI](https://github.com/GouthamRajam/GothOS/actions/workflows/gothos-ci.yml/badge.svg)](https://github.com/GouthamRajam/GothOS/actions)
[![Python 3.12+](https://img.shields.io/badge/python-3.12%2B-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
[![Code style: ruff](https://img.shields.io/badge/code%20style-ruff-orange.svg)](https://github.com/astral-sh/ruff)
[![Typed: strict mypy](https://img.shields.io/badge/typed-strict%20mypy-blue.svg)](https://mypy-lang.org/)

Provider-agnostic Android AI agent — automates any Android device using any LLM (Claude, GPT-4o, Gemini, Ollama). Combines safety gates, persistent memory, structured extraction, visual grounding, and LangGraph orchestration into a single agent system.

## Architecture

```mermaid
graph TB
    subgraph GothOS["🤖 GothOS Agent"]
        direction TB
        CORE["core<br/>LangGraph state machine"]
        
        subgraph Perception["👁️ Perception"]
            SCREEN["screen<br/>UI hierarchy parser"]
            GROUND["grounding<br/>4-stage cascade"]
            EXTRACT["extract<br/>structured data"]
        end
        
        subgraph Intelligence["🧠 Intelligence"]
            PLANNER["planner<br/>goal decomposition"]
            LLM["llm<br/>provider factory"]
            MEMORY["memory<br/>SQLite + nav graphs"]
        end
        
        subgraph Control["🛡️ Control"]
            SAFETY["safety<br/>action classifier"]
            DEVICE["device<br/>async ADB bridge"]
        end
    end
    
    CORE --> PLANNER
    CORE --> SAFETY
    CORE --> DEVICE
    CORE --> SCREEN
    CORE --> MEMORY
    PLANNER --> LLM
    GROUND --> LLM
    EXTRACT --> LLM
    GROUND --> SCREEN
    DEVICE --> ANDROID["📱 Android Device"]
    
    style GothOS fill:#1a1a2e,stroke:#e94560,color:#fff
    style Perception fill:#16213e,stroke:#0f3460,color:#fff
    style Intelligence fill:#16213e,stroke:#0f3460,color:#fff
    style Control fill:#16213e,stroke:#0f3460,color:#fff
    style ANDROID fill:#3c6e47,stroke:#2d5a3a,color:#fff
```

### Agent loop

```mermaid
stateDiagram-v2
    [*] --> Plan
    Plan --> SafetyCheck
    SafetyCheck --> Execute: approved
    SafetyCheck --> Replan: blocked
    Execute --> Observe
    Observe --> Record: success
    Observe --> Replan: failed
    Record --> Plan: more steps
    Record --> [*]: goal met
    Replan --> Plan
```

### Grounding cascade (zero LLM calls for ~80% of actions)

| Stage | Method | LLM calls | Latency |
|-------|--------|-----------|---------|
| 1 | Accessibility tree lookup | 0 | ~0 ms |
| 2 | OCR text-box lookup | 0 | ~0 ms |
| 3 | LLM semantic lookup | 1 text call | ~500 ms |
| 4 | LLM visual coordinate | 1 multimodal call | ~2 s |

## Quick Start

```bash
# One-click setup (auto-installs ADB, creates venv, installs GothOS)
./start.sh                # macOS / Linux
.\start.bat               # Windows

# Or manual install
pip install -e .          # runtime only
pip install -e ".[dev]"   # + pytest, ruff, mypy
```

### Docker

```bash
docker build -t gothos .
docker run --rm gothos serve              # MCP server
docker run --rm gothos run "open settings" --provider anthropic
```

## CLI

```bash
gothos run "open settings and enable dark mode" --provider anthropic
gothos run "send hello to Mom" --mode flash   # fast reactive mode
gothos run "complex multi-step task" --mode pro  # thorough planning
gothos devices                          # list connected Android devices
gothos screenshot -o screen.png         # capture device screenshot
gothos hierarchy --format text          # dump UI accessibility tree
gothos plan "send hello to Mom"         # generate plan without executing
gothos benchmark                        # run core benchmark suite
gothos benchmark cross-app -o results.json  # cross-app suite with JSON output
gothos console                          # launch web dashboard
gothos console --port 9000 --no-open    # custom port, no auto-open
gothos serve                            # start MCP server
gothos serve --install claude-code      # configure for Claude Code
```

## MCP Server — IDE integration

GothOS exposes 5 tools over the [Model Context Protocol](https://modelcontextprotocol.io/) so AI coding assistants can drive a connected device:

| Tool | What it does |
|------|-------------|
| `execute_goal` | Run a natural-language task on the device |
| `control_session` | Pause, resume, or cancel a running session |
| `capture_state` | Snapshot: screenshot + UI elements + device info |
| `replay_actions` | Step-by-step action log for debugging |
| `health_check` | Verify ADB, device, and LLM readiness |

### Setup for your IDE

```bash
pip install -e .

# One-command install for your editor:
gothos serve --install claude-code   # Claude Code
gothos serve --install cursor        # Cursor
gothos serve --install windsurf      # Windsurf
gothos serve --install vscode        # VS Code
```

Then ask your AI assistant: *"Use GothOS to open Settings on the connected phone and take a screenshot."*

## Dual mode — Flash vs Pro

GothOS supports two execution strategies, selectable per-run:

| | Flash | Pro |
|---|---|---|
| **Strategy** | Reactive, one-step-at-a-time | Full multi-step plan upfront |
| **LLM calls** | 1 per step (minimal) | N for planning + 1 per step |
| **Replanning** | Never (reacts to what it sees) | Automatic on failure |
| **Best for** | Routine tasks, known UI flows | Complex multi-app sequences |
| **Latency** | Lower | Higher (but more reliable) |

```bash
gothos run "open settings" --mode flash          # fast reactive
gothos run "send hello to Mom on WhatsApp" --mode pro  # thorough planning
gothos run "enable dark mode" --mode auto         # let GothOS decide (default)
```

**Auto mode** inspects the goal's complexity (word count, action keywords) and picks Flash or Pro automatically.

## Quick start

### Full agent — automate any Android task

```python
import asyncio
from gothos.core import GothOSAgent, AgentConfig

async def main():
    agent = GothOSAgent(AgentConfig(
        llm_provider="anthropic",
        model="claude-sonnet-4-20250514",
    ))
    await agent.initialize()

    result = await agent.run(
        goal="Send 'hello' to Mom on WhatsApp",
        app_package="com.whatsapp",
    )
    print(f"{'Success' if result.success else 'Failed'}: {result.steps_taken} steps")

    await agent.shutdown()

asyncio.run(main())
```

### Device bridge — control Android directly

```python
from gothos.device import DeviceBridge, ActionExecutor

bridge = DeviceBridge()
await bridge.connect()
executor = ActionExecutor(bridge)

await executor.tap(540, 1200)
await executor.type_text("Hello world")
await executor.press_back()
await executor.scroll_down()

info = await bridge.get_device_info()
print(f"{info.model} ({info.screen_width}x{info.screen_height})")
```

### Screen analysis — understand what's on screen

```python
from gothos.screen import ScreenAnalyzer
from gothos.device import dump_ui_hierarchy

xml = await dump_ui_hierarchy(bridge)
state = ScreenAnalyzer.analyze(xml, app_package="com.whatsapp")
print(f"{len(state.interactive_elements)} interactive elements")
print(f"Keyboard visible: {state.has_keyboard}")
```

### Task planner — decompose goals into steps

```python
from gothos.planner import TaskPlanner

planner = TaskPlanner(llm_provider="anthropic", model="claude-sonnet-4-20250514")
plan = await planner.create_plan(
    goal="Open Settings and enable Dark Mode",
    app_package="com.android.settings",
    screen_description="Home screen with app grid",
)
for step in plan.steps:
    print(f"  {step.step_number}. {step.action_type} → {step.target_description}")
```

### Safety — classify actions before executing

```python
from gothos.safety import SafetyPolicy, SafetyPolicyConfig

policy = SafetyPolicy.from_config(SafetyPolicyConfig(
    confirm_on_warn=True,
    channel_type="console",
    blocked_apps=["com.venmo"],
))

result = await policy.check_action(
    action_type="tap",
    element_text="Pay Now",
    app_package="com.example.shop",
)
if not result.approved:
    print(f"Blocked: {result.reason}")
```

### Grounding — find elements without an LLM

```python
from gothos.grounding import A11yNavigator

nav = A11yNavigator()
match = nav.find_element(a11y_tree_xml, "Send")
if match:
    x, y = match.element.center
    print(f"Tap at ({x}, {y}), confidence={match.confidence}")
```

### Memory — record and recall agent experience

```python
from gothos.memory import MemoryStore, ActionRecorder, MemoryRetriever, AppNavigator

store = MemoryStore("memory.db")
await store.initialize()

recorder = ActionRecorder(store)
async with recorder.session("com.whatsapp", "send hello to Mom", "claude") as rec:
    await rec.step("tap", target="Mom")
    await rec.step("type", input_text="Hello!")
    await rec.step("tap", target="Send")

retriever = MemoryRetriever(store, AppNavigator(store))
ctx = await retriever.get_relevant_experience("com.whatsapp", "send hi to Dad")
print(retriever.format_for_prompt(ctx))

await store.close()
```

## Supported LLM providers

| Provider | Install | Provider key |
|----------|---------|-------------|
| Anthropic (Claude) | `pip install langchain-anthropic` | `"anthropic"` |
| OpenAI (GPT-4o) | `pip install langchain-openai` | `"openai"` |
| Google (Gemini) | `pip install langchain-google-genai` | `"google"` |
| Ollama (local) | `pip install langchain-ollama` | `"ollama"` |

## Modules

| Module | What it does |
|--------|-------------|
| **core** | LangGraph agent orchestrator — plan/execute/observe/replan loop with telemetry, retry, caching, and step timeouts |
| **device** | Async ADB bridge — connect to Android devices, execute actions (tap/type/swipe/scroll), capture screenshots and UI hierarchies |
| **planner** | LLM-based task decomposition — turns "send hello to Mom" into concrete tap/type/scroll steps |
| **screen** | Screen analysis — parse uiautomator XML, compute screen signatures for memory, diff screen states |
| **safety** | Rule-based action classifier (safe/warn/block/critical), human confirmation gates, system dialog handler |
| **memory** | SQLite-backed persistent memory — task runs, action sequences, screen knowledge, navigation graphs |
| **grounding** | 4-stage element location cascade — accessibility tree, OCR, LLM semantic, LLM visual |
| **extract** | Structured data extraction from screens via LLM, MCP tool interface |
| **llm** | Shared provider-agnostic LLM factory for all modules |
| **mcp** | MCP server with 5 tools — lets AI coding assistants drive the device |
| **console** | FastAPI web dashboard — live device view, session control, WebSocket stream |
| **benchmark** | Task suites and harness for measuring agent reliability |

## Benchmarks

GothOS ships with built-in task suites for measuring agent reliability:

```bash
gothos benchmark                        # run the core suite (10 tasks)
gothos benchmark cross-app              # cross-app workflows (3 tasks)
gothos benchmark cross-app --mode pro -o results.json
```

| Suite | Tasks | Categories |
|-------|-------|------------|
| `core` | 10 | navigation, settings, search, device, app_interaction |
| `cross-app` | 3 | cross-app workflows spanning multiple apps |

Custom suites can be loaded from JSON files. Results export to JSON with per-category and per-difficulty breakdowns.

## Web Console

A live dashboard for monitoring and controlling GothOS agents in the browser.

```bash
pip install -e ".[console]"   # installs FastAPI + uvicorn
gothos console                # opens http://127.0.0.1:8000
```

Features:
- **Live device screen** — auto-refreshing screenshot panel
- **Session management** — start, pause, cancel agent sessions from the UI
- **Action stream** — real-time WebSocket feed of agent steps
- **Mode selector** — pick Flash / Pro / Auto per session
- **REST API** — `/api/sessions`, `/api/device`, `/api/screenshot`, `/api/hierarchy`

## Production features

### Observability — structured telemetry

Every agent run emits a trace with per-node spans and aggregated metrics:

```python
result = await agent.run("open settings", app_package="")
print(result.telemetry["metrics"])
# {'llm_calls': 3, 'llm_avg_latency_ms': 420.5, 'actions_executed': 5,
#  'action_success_rate': 100.0, 'safety_checks': 5, 'replans': 0, ...}
```

Spans nest automatically (`plan > llm_call > grounding`) and export to dict/JSON for dashboards.

### LLM retry with exponential backoff

All LLM calls (planning, replanning, grounding) are wrapped in configurable retry:

```python
config = AgentConfig(
    llm_provider="anthropic",
    model="claude-sonnet-4-20250514",
    llm_retry_attempts=4,       # total tries
    llm_retry_base_delay=1.0,   # initial backoff
)
```

Transient 429/503/overloaded errors are retried automatically. Non-retryable errors propagate immediately.

### LLM response cache

Identical planning calls (same goal + screen signature + mode) hit an in-memory LRU cache:

```python
config = AgentConfig(
    ...,
    cache_enabled=True,         # default
    cache_max_entries=256,
    cache_ttl_seconds=600,
)
```

### Step timeout enforcement

Each action step is wrapped in `asyncio.wait_for(timeout=step_timeout_seconds)`. The config field (default 30s) is now enforced — hung ADB commands or stuck grounding calls are killed cleanly.

### Config validation

`AgentConfig` validates `llm_provider` and `execution_mode` at construction time. Typos like `"antrhopic"` or `"turbo"` fail immediately with a clear error instead of crashing deep in the call stack.

## Development

```bash
# Run tests (498 tests)
pytest -v

# Lint + format
make all                  # fmt → lint → typecheck → test

# Or individually
ruff check .
ruff format .
mypy gothos --ignore-missing-imports
```

Pre-commit hooks are configured — run `pre-commit install` after cloning.

CI runs automatically on push/PR via GitHub Actions (Python 3.12 + 3.13, ruff, mypy).

## License

MIT — see [LICENSE](LICENSE).
