windows-desktop-companion

<div align="center"> <img src="app/build/icon.png" alt="Windows Desktop Companion" width="120" /> # Windows Desktop Companion **A floating AI companion for Windows that can see your screen and act on your PC.** Summon it anywhere with a hotkey. It sees your screen, understands which app you're in, looks things up on the web, and takes real actions — editing configs, running PowerShell, driving other apps — approving anything risky right inside the chat window. [](https://github.com/wyofalcon/windows-desktop-companion/actions/workflows/ci.yml) [](https://github.com/wyofalcon/windows-desktop-companion/releases) [](LICENSE)  </div> --- ``` chat window ──stdin (stream-json)──▶ headless Claude Code (claude-sonnet-5) ◀──stdout (stream events)───────┘ │ ├─▶ built-in tools: PowerShell, │ file read/edit, web search └─▶ MCP server "windows-tools" (screenshot, UIA, input injection) ``` - **Brain:** `claude-sonnet-5` at **max reasoning effort** via the Claude **Agent SDK** — one persistent streaming session with crash-resume. - **Eyes & hands:** the `windows-tools` MCP server (screenshots, window enumeration, UI Automation, and — as a fallback — raw input injection). - **Command execution:** Claude Code's built-in shell tool running **PowerShell**. - **Principle:** prefer structured control (config files, CLIs, `ms-settings:` URIs, the UI Automation tree) over pixel clicking. <!-- Demo GIF goes here once recorded:  --> ## Install & run (end users) > [!IMPORTANT] > This tool lets an AI model **run commands and control your PC**. Read > [SECURITY.md](SECURITY.md) before installing — you approve every risky action, > but you are the last line of defense. **Requirements** - **Windows 10 or 11.** - **[Claude Code](https://claude.com/claude-code) installed and logged in.** The companion is a front-end for it — verify `claude --version` works in PowerShell first. Your login is reused; no API key required for a subscription account. **Steps** 1. Download the latest build from the [**Releases**](https://github.com/wyofalcon/windows-desktop-companion/releases) page: - **`Windows Desktop Companion-Setup-x.y.z.exe`** — installer (adds a Start Menu shortcut), **or** - **`Windows Desktop Companion-x.y.z-portable.exe`** — single portable exe, no install. 2. Run it. Windows SmartScreen may warn on an unsigned build — **More info → Run anyway**. 3. Press **`Ctrl+Alt+Space`** (or the fallback shown in the window header) to summon the chat. Type to talk. Everything is bundled: the Python `windows-tools` server ships as a self-contained `windows-tools.exe`, so **you do not need Python** installed. ### Using it - **Esc** (or the **—** header button) collapses the chat into a floating **chat-head bubble** that stays on top of everything. - **Click** it → the chat expands beside it. **Drag** it → snaps to the nearest screen edge and remembers its spot. It **pulses** while the brain is working and shows an amber **badge** when a reply/permission arrived while collapsed. **Right-click** → Open chat / Quit. Try: - *"What's on my screen right now?"* → it screenshots and describes it. - *"Switch VS Code to the Light+ theme."* → it edits `settings.json` (not GUI clicking) and confirms. - *"Why are my keyboard's lights flashing?"* → it identifies the device via PowerShell, web-searches the vendor docs, and **asks before changing anything**. ## Guardrails Every action falls into one of three tiers, enforced in [`app/src/main/bridge.ts`](app/src/main/bridge.ts): - **Auto-allowed (read-only):** `screenshot`, `active_window`, `list_windows`, `ui_tree`, `find_element`, web search, file reads, and a strict read-only PowerShell allowlist (`Get-*`, `Test-Path`, `Select/Where/Sort/Measure`…). - **Gated (require an Allow click):** file edits/writes, mutating PowerShell, `invoke_element`, `click`, `type_text`, `send_keys` — each shown with a one-line description of what will happen. - **Always ask, flagged as dangerous:** deletes, registry writes, disk ops, closing apps — anything that could lose work. Permission decisions happen in the chat window, never in a hidden terminal. See [SECURITY.md](SECURITY.md) for the full model. ## Build from source Requires **Node 20+**, **Python 3.11/3.12**, and Claude Code logged in. Full dev instructions are in [CONTRIBUTING.md](CONTRIBUTING.md); the short version: ```powershell # 1. Python venv for the MCP server py -m venv .venv .\.venv\Scripts\python.exe -m pip install -r requirements.txt # 2. Dev build + launch (see run-companion.ps1 for the WSL-share mirror note) ./run-companion.ps1 ``` **Package the standalone binaries yourself:** ```powershell ./build-release.ps1 # -> .\release\ (NSIS installer + portable .exe) ``` This freezes the MCP server with PyInstaller and packages the app with electron-builder. Tagged releases (`git tag v0.1.0 && git push --tags`) build the same artifacts on `windows-latest` via [GitHub Actions](.github/workflows/release.yml) and attach them to a Release. ## Does it run on Linux? **No — Windows only, by design.** The companion's core value (seeing the screen, reading the UI Automation tree, driving apps) rests on Windows-native libraries (`pywinauto`'s UIA backend, `pywin32`, `comtypes`) and PowerShell. There is no equivalent surface bundled for Linux/X11, so a Linux build would be a chat window with none of the "eyes and hands" — which defeats the purpose. If you want a Linux port with AT-SPI/`xdotool`-based tools, that's a welcome (large) contribution. ## Project layout ``` MSAI/ ├─ .mcp.json # registers windows-tools for interactive `claude` ├─ requirements.txt # Python MCP-server deps ├─ run-companion.ps1 # dev: build + launch (mirror-to-local) ├─ build-release.ps1 # release: PyInstaller + electron-builder ├─ scripts/make-icon.py # regenerates app/build/icon.{ico,png} ├─ prompts/companion-system-prompt.md ├─ mcp-server/ # windows-tools MCP server (Python, stdio) │ ├─ run.py · windows-tools.spec (PyInstaller) │ └─ windows_tools/{server,screen,win,uia,inputs,dpi}.py └─ app/ # Electron + TypeScript chat window ├─ electron.vite.config.ts · package.json (electron-builder config) ├─ build/icon.ico └─ src/{main,preload,renderer}/… ``` ## Out of scope for v1 A local Ollama "quick query" lane (a router seam is left in the bridge), OCR, a settings UI, auto-start on boot, multi-monitor polish, and voice. ## License [MIT](LICENSE) © 2026 wyofalcon. Built as a front-end for [Claude Code](https://claude.com/claude-code) using the [Claude Agent SDK](https://www.npmjs.com/package/@anthropic-ai/claude-agent-sdk).