Content
# cua-driver MCP for OpenClaw
Wire [cua-driver](https://github.com/trycua/cua) — an open-source macOS desktop automation daemon — into OpenClaw (and Hermes) as an MCP server. This gives AI agents full desktop control: screenshots, mouse clicks, keyboard input, app launch, drag-and-drop, and accessibility tree navigation.
> **33+ tools** exposed over MCP. Zero cloud dependency. No licensing fees. Apache 2.0.
## What it unlocks
| Capability | Before | After |
|---|---|---|
| Desktop screenshots | ❌ | ✅ zoom tool |
| Click non-browser apps | ❌ | ✅ click, right_click, double_click |
| Drag files in Finder | ❌ | ✅ drag tool |
| Type into any app | ❌ | ✅ type_text |
| Keyboard shortcuts | ❌ | ✅ hotkey, press_key |
| Launch/kill apps | ❌ | ✅ launch_app, kill_app |
| Detect UI elements | ❌ | ✅ get_window_state (AX tree) |
| Scroll pages | ❌ | ✅ scroll tool |
| Agent cursor overlay | ❌ | ✅ start_session, move_cursor |
## Quick Start
### Via Hermes (recommended)
```bash
hermes computer-use install
openclaw mcp set cua-driver '{
"command": "'"$(which cua-driver)"'",
"args": ["mcp"]
}'
```
### Standalone
```bash
# 1. Download latest release from https://github.com/trycua/cua/releases
# or use the one-liner:
curl -fsSL https://github.com/trycua/cua/releases/download/cua-driver-rs-latest/cua-driver-rs-*-darwin-universal.tar.gz \
| tar xz -C /Applications
# 2. Register as OpenClaw MCP server
openclaw mcp set cua-driver '{
"command": "'"$(which cua-driver)"'",
"args": ["mcp"]
}'
# 3. Launch daemon
open -n -g -a CuaDriver --args serve
# 4. Grant permissions (one-time)
# System Settings → Privacy & Security
# → Screen Recording → tick CuaDriver
# → Accessibility → tick CuaDriver
# 5. Verify
cua-driver check_permissions --daemon
```
## For Hermes
Add to `~/.hermes/config.yaml`:
```yaml
mcp_servers:
cua-driver:
command: "$HOME/.local/bin/cua-driver"
args: ["mcp"]
```
## Prerequisites
- macOS 15.0+
- OpenClaw installed (or any MCP-compatible agent framework)
- [cua-driver (trycua/cua)](https://github.com/trycua/cua) — Apache 2.0
- No Docker, no API keys, no cloud accounts needed
## Architecture
```
┌──────────────┐ MCP/stdin ┌──────────────┐
│ OpenClaw │◄─────────────────────│ cua-driver │
│ (Ibrahim) │ 55+ agent tools │ daemon │
│ │ + 33 desktop tools │ │
│ ┌────────┐ │ │ ┌────────┐ │
│ │ cua- │──┤ Screenshots, clicks, │ │ macOS │ │
│ │ driver │ │ typing, AX tree... │ │ AX API │──┤
│ │ MCP │──┤ │ │ CGEvent│ │
│ └────────┘ │ │ └────────┘ │
└──────────────┘ └──────────────┘
```
## Permission Caveat
macOS attributes TCC grants by **bundle identity** (`com.trycua.driver`). You must:
1. Launch CuaDriver.app **as its own process** (via Finder or `open -n`)
2. Grant Screen Recording + Accessibility to CuaDriver
3. Use `--daemon` flag when checking permissions:
```bash
cua-driver check_permissions --daemon
```
Running from a terminal checks the terminal's grants, not the driver's. The --daemon flag forces the driver's own TCC identity.
## Full Tool List
| Tool | Description |
|---|---|
| `zoom` | Screenshot a window region (returns JPEG) |
| `click` / `double_click` / `right_click` | Mouse actions by element or pixel |
| `type_text` | Type into focused element |
| `press_key` / `hotkey` | Keyboard keys and shortcuts |
| `drag` | Drag from one point to another |
| `scroll` | Scroll by line or page |
| `get_window_state` | AX tree + screenshot of a window |
| `list_windows` / `list_apps` | Discover windows and running apps |
| `launch_app` / `kill_app` | Start/stop applications |
| `set_value` | Set element values (sliders, dropdowns, text fields) |
| `start_session` / `end_session` | Manage agent cursor sessions |
| `set_agent_cursor_*` | Cursor overlay visibility, style, motion |
| `move_cursor` | Move agent cursor overlay (user's mouse stays) |
| `check_permissions` | Check Accessibility + Screen Recording grants |
| `get_screen_size` | Get display dimensions |
| `start_recording` / `stop_recording` | Record agent actions for replay |
## License
This setup guide is MIT. cua-driver is Apache 2.0. OpenClaw has its own license.
