windows-computer-use

# Windows Computer-Use Windows Computer-Use provides a local Windows desktop operator behind an MCP facade: ```text token-protected Node MCP facade -> token-protected Python operator HTTP API -> Windows host desktop ``` ## Safety Warning This project can control the visible Windows session. In `host` control mode, tools can move the mouse, type/paste text, press hotkeys, capture screenshots, open URLs, launch processes, and run PowerShell commands on the machine where the operator is running. Keep the operator and MCP facade bound to `127.0.0.1` unless you fully understand the risk. Do not expose either HTTP service to an untrusted network. The MCP endpoint requires a bearer token because it exposes privileged desktop-control tools. ## Components - `src/windows_computer_use/operator.py` - Windows host operator with `/health`, `/activity`, `/task`, `/stop`, and `/actions/{action}`. - `mcp/server.mjs` - MCP facade that saves screenshots, locks active action chains, and delegates to the operator. - `plugins/windows-gate` - intent guard for Windows desktop control. - `scripts/start-local-stack.ps1` and `scripts/stop-local-stack.ps1` - local host-desktop runtime helpers. ## Canonical MCP Tools The facade keeps the Linux-compatible public surface where possible: `desktop_status`, `desktop_screenshot`, `desktop_screenshot_file`, `gui_screenshot`, `gui_snapshot`, `gui_observe`, `gui_act`, `gui_wait_for`, `gui_verify`, `gui_desktop_overview`, `gui_lock_status`, `gui_cancel_active`, `desktop_click`, `desktop_type`, `desktop_hotkey`, `desktop_scroll`, `desktop_wait`, `desktop_stop`, `ai_open`, `ai_status`, `ai_prompt`. Windows-specific aliases are also exposed for trusted local shell/process launch work: `windows_launch` and `windows_shell`. ## Quick Start Start the operator and local MCP facade in observe mode: ```powershell .\scripts\start-local-stack.ps1 -Restart ``` The script creates `generated\secrets\operator-token.txt`, reads the token from that file inside the child processes, exports it as `WINDOWS_COMPUTER_USE_MCP_TOKEN`, and starts: - Operator: `http://127.0.0.1:3038` - MCP: `http://127.0.0.1:3039/mcp` Codex can be pointed at the MCP endpoint: ```powershell codex mcp add windows-computer-use --url http://127.0.0.1:3039/mcp --bearer-token-env-var WINDOWS_COMPUTER_USE_MCP_TOKEN ``` To allow mutating host-desktop actions, start with explicit host-control acknowledgement: ```powershell .\scripts\start-local-stack.ps1 -Restart -HostControl ``` Stop the local stack: ```powershell .\scripts\stop-local-stack.ps1 ``` Useful runtime variables: - `WINDOWS_COMPUTER_USE_CONTROL_MODE` - default `observe` in the helper script; set `host` only for trusted local host-desktop control. - `WINDOWS_COMPUTER_USE_OPERATOR_HOST` - default `127.0.0.1`. - `WINDOWS_COMPUTER_USE_OPERATOR_PORT` - default `3038`. - `WINDOWS_COMPUTER_USE_OPERATOR_TOKEN` - required for all non-health endpoints. - `WINDOWS_COMPUTER_USE_MCP_TOKEN` - required for `/mcp` and `/activity`; the helper script defaults it to the generated operator token. - `WINDOWS_COMPUTER_USE_AI_URL` - default `https://chatgpt.com/`. - `WINDOWS_COMPUTER_USE_MAX_REQUEST_BYTES` - default `1048576` for operator JSON request bodies. - `WINDOWS_COMPUTER_USE_MAX_SHELL_TIMEOUT_SECONDS` - default `120` for PowerShell commands. ## Smoke Test After the operator and MCP facade are running with `-HostControl`: 1. Call `desktop_status` and confirm `controlMode: "host"` and `affectsHostDesktop: true`. 2. Call `desktop_screenshot_file`. 3. Call `gui_act` with `action: "open-url"` and `url: "https://www.youtube.com/results?search_query=music"`. 4. Call `desktop_wait` for a few seconds. 5. Call `gui_snapshot` and confirm the browser/YouTube state is visible. 6. Call `gui_verify` with browser/page expectations where useful. 7. Call `gui_cancel_active` or `desktop_stop`. ## Local Tests ```powershell python3 -m unittest discover -s tests npm --prefix mcp ci --ignore-scripts --no-audit node --test mcp/server-surface.test.mjs plugins/windows-gate/windows-gate.test.mjs npm --prefix mcp audit --omit=dev ``` ## Trusted Local Control `windows_shell`, `windows_launch`, `open-uri`, `desktop_type`, `desktop_hotkey`, and related GUI tools are intended for trusted local automation. `desktop_type` uses the system clipboard to paste text into the focused control. Known Windows boundaries still apply: UAC secure desktop, lock screen, admin/non-admin window isolation, and disconnected/minimized desktop sessions may limit GUI control.