Metadata-Version: 2.4
Name: app_use
Version: 0.0.1.dev5
Summary: A library for controlling mobile applications via native debugging bridges
Home-page: https://github.com/app-use/app-use
Author: Erick Torres-Moreno
Author-email: info@app-use.dev
License: MIT
Project-URL: Homepage, https://github.com/app-use/app-use
Project-URL: Bug Tracker, https://github.com/app-use/app-use/issues
Project-URL: Documentation, https://github.com/app-use/app-use#readme
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: dart-vm-client==0.0.1.dev4
Requires-Dist: pydantic>=2.0.0
Requires-Dist: python-dotenv>=0.21.0
Requires-Dist: langchain-core>=0.1.0
Provides-Extra: openai
Requires-Dist: langchain-openai>=0.0.1; extra == "openai"
Requires-Dist: openai>=1.0.0; extra == "openai"
Provides-Extra: groq
Requires-Dist: langchain-groq>=0.0.1; extra == "groq"
Requires-Dist: groq>=0.4.0; extra == "groq"
Provides-Extra: google
Requires-Dist: google-generativeai>=0.3.0; extra == "google"
Requires-Dist: langchain-google-vertexai>=0.0.1; extra == "google"
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.5.0; extra == "anthropic"
Requires-Dist: langchain-anthropic>=0.0.1; extra == "anthropic"
Provides-Extra: all
Requires-Dist: langchain-openai>=0.0.1; extra == "all"
Requires-Dist: openai>=1.0.0; extra == "all"
Requires-Dist: langchain-groq>=0.0.1; extra == "all"
Requires-Dist: groq>=0.4.0; extra == "all"
Requires-Dist: google-generativeai>=0.3.0; extra == "all"
Requires-Dist: langchain-google-vertexai>=0.0.1; extra == "all"
Requires-Dist: anthropic>=0.5.0; extra == "all"
Requires-Dist: langchain-anthropic>=0.0.1; extra == "all"
Dynamic: author-email
Dynamic: home-page
Dynamic: license-file
Dynamic: requires-python

# App Use - AI Control for Mobile Apps


![ChatGPT Image May 16, 2025 at 03_14_14 PM](https://github.com/user-attachments/assets/baccb232-4690-4b47-824b-e77166e23aa7)



> **Warning:** This project is currently under active development. APIs will change! Currently in experimental stages.

App Use is the easiest way to connect AI agents with mobile applications. It enables AI to control your mobile apps through native debugging bridges, providing a powerful yet simple interface for app automation.

Currently, App Use focuses on Flutter apps via the Dart VM Service, with plans to expand to other frameworks like React Native in the future.

## Overview

App Use allows you to automate interactions with mobile apps, such as:
- Clicking buttons and interactive widgets
- Entering text into fields
- Scrolling and navigating screens
- Building AI-powered agents that can interact with mobile app UIs

Perfect for testing, automation, and AI agent applications.

## Getting Started

### Installation

Install the package using pip:

```bash
pip install -r requirements.txt
```

### Quick Start with LLM Agents

Use AI agents to intelligently interact with your mobile app:

```python
import asyncio
import os
from dotenv import load_dotenv
from langchain_openai import ChatOpenAI

from app_use.agent.service import Agent
from app_use.app.app import App

# Load environment variables (like OPENAI_API_KEY)
load_dotenv()

async def main():
    # WebSocket URI for a running Flutter app
    VM_SERVICE_URI = "ws://127.0.0.1:50505/ws"
    
    # Set up your LLM
    llm = ChatOpenAI(
        api_key=os.environ.get("OPENAI_API_KEY"),
        model="gpt-4o",  # or any other model
        temperature=0
    )
    
    # Create an App instance to connect to the Flutter application
    app = App(vm_service_uri=VM_SERVICE_URI)
    
    # Create an agent to control the app
    agent = Agent(
        task="Explore the app and find all buttons, then click on the Login button if it exists.",
        llm=llm,
        app=app,
    )
    
    try:
        # Run the agent
        history = await agent.run()
        
        # Print results
        if history.is_successful():
            print("✅ Agent successfully completed the task!")
        else:
            print("⚠️ Agent couldn't complete the task.")
        
        # Show the final result/message
        print(f"Final result: {history.final_result()}")
        
    except Exception as e:
        print(f"❌ Error: {e}")
    finally:
        # Make sure to close resources
        await agent.close()
        app.close()

if __name__ == "__main__":
    asyncio.run(main())

### Setting Up a Flutter App for Control

> Currently, App Use supports Flutter apps, with more frameworks coming soon.

1. Makesure to enable the flutter driver extension in your mobile app

```dart
import 'package:flutter_driver/driver_extension.dart';

void main() {
  enableFlutterDriverExtension();
  runApp(const HomeScreen());
}
```

2. Run your Flutter app with the VM service enabled:

```bash
flutter run
```

3. Look for the VM service URI in the console output, which looks like:
   `ws://127.0.0.1:50505/ws`


## Features

- **Smart Widget Interaction**: Automatically attempts to interact with parent or child widgets if direct interaction fails
- **Automatic Resource Management**: Proper cleanup of connections and services
- **Context Manager Support**: Use the `with` statement for proper resource handling
- **Intelligent Port Management**: Automatically handles port conflicts
- **Extensive UI Tree Access**: Full access to the application's UI tree
- **Flexible Element Finding**: Find UI elements by key, text, type, or hierarchical position

### Extending App Use

You can extend App Use by:
- Creating custom Agent implementations
- Adding new widget interaction patterns
- Adding support for additional app frameworks
- Building domain-specific UI automation tools

## Community & Support

Contributions are welcome! Please feel free to submit a Pull Request.

App Use is actively maintained and designed to make mobile app control as simple and reliable as possible.

        
