Content
# Open-AutoGLM-GUI
> Web control platform based on [Open-AutoGLM](https://github.com/THUDM/AutoGLM) project
## Project Overview
Open-AutoGLM-GUI is a powerful mobile automation testing platform that provides a user-friendly web interface to control Android, HarmonyOS, and iOS devices.
## Core Features
### 📱 Device Management
- **Multi-platform support**: Android (ADB), HarmonyOS (HDC), iOS (WebDriverAgent)
- **WiFi connection**: Support wireless device connection
- **Real-time screen casting**: Scrcpy real-time screen casting
- **Device status monitoring**: Real-time display of device connection status
### 🤖 AI Agent
- **Intelligent task execution**: Intelligent task scheduling based on GLM large model
- **Real-time log stream**: View Agent thinking process and execution steps
- **Multi-modal support**: Support VLM (Visual Language Model)
### 📁 Project Management
- **Project creation and management**: Create, edit, and delete projects
- **Card-based display**: Intuitive project card layout
- **Project details view**: View project-associated scripts and tasks
### 📜 Script Management
- **Script editor**: Online writing and editing of test scripts
- **Version management**: Support script version control
- **Script execution**: One-click script execution
### 📊 Task Management
- **Task creation**: Create automated testing tasks
- **Task status tracking**: Real-time view of task execution status
- **Task report**: Generate detailed task execution reports
### ⚙️ Configuration Management
- **LLM configuration**: Configure large language model parameters
- **VLM configuration**: Configure visual language model parameters
- **ADB configuration**: Configure ADB connection parameters
## Technology Stack
### Backend
- **FastAPI**: Web framework
- **Uvicorn**: ASGI server
- **WebSocket**: Real-time communication
- **SQLite**: Lightweight database
- **Open-AutoGLM**: Core Agent engine
### Frontend
- **Jinja2**: HTML template engine
- **TailwindCSS**: UI style framework
- **Font Awesome**: Icon library
- **WebSocket API**: Real-time data stream
### Device Control
- **ADB**: Android device control
- **HDC**: HarmonyOS device control
- **WebDriverAgent**: iOS device control
- **Scrcpy**: Real-time screen casting
## Quick Start
### Environment Requirements
- Python 3.10+
- ADB (Android Debug Bridge) - for Android devices
- HDC (HarmonyOS Device Connector) - for HarmonyOS devices
- Xcode + WebDriverAgent - for iOS devices
### Installation Steps
```bash
# Clone repository
git clone https://github.com/joe-qai/Open-AutoGLM-GUI.git
cd Open-AutoGLM-GUI
# Create virtual environment
python -m venv venv
source venv/bin/activate # Windows: venv\Scripts\activate
# Install dependencies
pip install -r requirements.txt
# Install development dependencies (optional)
pip install -e ".[dev]"
```
### Start Service
```bash
# Default port 8001
python web_ui/main.py
# Custom port
python web_ui/main.py --port 9000
```
### Access Interface
Open browser and visit: `http://localhost:8001`
## Docker Deployment
### Method 1: Docker
```bash
# Build image
docker build -t open-autoglm-gui:latest .
# Run container (Host network mode)
docker run -d \
--name autoglm-gui \
--network host \
open-autoglm-gui:latest
```
### Method 2: Docker Compose
```bash
# Start service
docker-compose up -d
# View logs
docker-compose logs -f
# Stop service
docker-compose down
```
## Project Structure
```
Open-AutoGLM-GUI/
├── backend/ # Backend core module
│ ├── agent/ # Agent management module
│ ├── db/ # Database module
│ ├── llm/ # LLM protocol adaptation
│ ├── mcp/ # MCP tool encapsulation
│ ├── scrcpy/ # Scrcpy screen casting
│ └── utils/ # Tool functions
├── web_ui/ # Web interface module
│ ├── static/ # Static resources
│ │ ├── css/ # Style files
│ │ ├── fonts/ # Font files
│ │ └── js/ # JavaScript files
│ ├── templates/ # HTML templates
│ │ ├── partials/ # Template fragments
│ │ └── *.html # Page templates
│ ├── scrcpy/ # Scrcpy frontend support
│ ├── tests/ # Web UI tests
│ └── main.py # FastAPI entry
├── phone_agent/ # Open-AutoGLM core code
│ ├── actions/ # Operation execution module
│ ├── adb/ # ADB related functions
│ ├── hdc/ # HDC related functions
│ ├── xctest/ # iOS test support
│ ├── config/ # Configuration management
│ └── model/ # Model client
├── docs/ # Documentation
├── examples/ # Example code
├── tests/ # Integration tests
└── resources/ # Resource files
```
## Environment Variables
| Variable Name | Description | Default Value |
| --- | --- | --- |
| `PHONE_AGENT_BASE_URL` | LLM API base URL | `http://localhost:8000/v1` |
| `PHONE_AGENT_MODEL` | Model name | - |
| `PHONE_AGENT_API_KEY` | API key | - |
| `PHONE_AGENT_MAX_STEPS` | Maximum execution steps | `100` |
| `PHONE_AGENT_DEVICE_TYPE` | Device type | `adb` |
| `PHONE_AGENT_WDA_URL` | iOS WDA URL | - |
| `PHONE_AGENT_LANG` | Language | `cn` |
## Usage Instructions
### Connect Device
1. **Android**: Enable USB debugging, use `adb devices` to confirm connection
2. **HarmonyOS**: Use HDC tool to connect device
3. **iOS**: Configure WebDriverAgent and connect
### Execute Task
1. Select connected device on AI Agent page
2. Input task instructions (support natural language)
3. Click execute button to start task
4. Real-time view execution logs and screen casting
## Development Guide
### Code Specifications
- Python follows PEP8 specifications
- Use `ruff` for code checking
- Use `mypy` for type checking
- Use `black` for code formatting
### Development Commands
```bash
# Code checking
ruff check . --select I
# Code formatting
ruff format .
# Type checking
mypy phone_agent/
# Run tests
pytest web_ui/tests/
# Install pre-commit hook
pre-commit install
```
## License
MIT License
## Contribution
Welcome to submit issues and pull requests!
## Acknowledgments
- [Open-AutoGLM](https://github.com/THUDM/AutoGLM) - Core Agent engine
- [Scrcpy](https://github.com/Genymobile/scrcpy) - Screen casting tool
- [FastAPI](https://fastapi.tiangolo.com/) - Web framework
- [TailwindCSS](https://tailwindcss.com/) - CSS framework
