Content
# Tool List
Turn [Stack-chan desktop robot](https://protopedia.net/prototype/2345) (M5Stack CoreS3 + dual servo + camera) into **your local Claude Code AI desktop companion**.
- 🎤 Talk to Stack-chan using fluent [Xiaozhi AI protocol](https://github.com/78/xiaozhi-esp32) (DeepSeek/Qwen/豆包 backend, ~1 second response)
- 🤖 Call **local Claude Code** in critical moments, accessing your memory / skills / 30+ MCP services (personal knowledge base, public account writing, email, wiki, etc.)
- 🎯 LLM can also control head angle / expression / LED light ring, with all actions controllable by voice
> "Paimon, check my knowledge base for notes on HK salary" → DeepSeek sees `claude_code` tool → wss connects to your Mac → spawn `claude --print` → uses personal-wiki skill to search → reads results to you
## What is this?
Stack-chan is an open-source desktop robot designed by a Japanese user, with [M5Stack selling the kit](https://shop.m5stack.com/products/m5stack-cores3-stackchan-ai-desktop-robot-kit). With [78/xiaozhi-esp32](https://github.com/78/xiaozhi-esp32) ("Xiaozhi AI") firmware, it can engage in voice conversations.
However, the **official firmware**:
- ❌ Cannot switch to Claude/GPT as the main model (official server only supports domestic LLM)
- ❌ Does not expose servo angle control to LLM
- ❌ Wake-up word becomes ineffective after sleep
- ❌ Cannot access your personal knowledge base
**This project**:
- ✅ Adds a `claude_code(task)` tool to LLM through xiaozhi.me **MCP access point**
- ✅ Tool connects to your Mac's local server via wss
- ✅ Local server spawns `claude --print` to run your Claude Code (with memory + skills + MCP)
- ✅ Asynchronous mode: LLM immediately returns "Paimon is checking, please wait" and reads results later
- ✅ Also forks and fixes 3 bugs in the official firmware (servo MCP missing, sleep mic closing, Nod using wrong base)
Overall architecture:
```
You say "Paimon, check my knowledge base for notes on X"
↓
Stack-chan microphone ── streaming ASR
↓
Xiaozhi backend api.xiaozhi.me + DeepSeek V4
↓ sees claude_code tool and calls it
wss://api.xiaozhi.me/mcp/?token=...
↓
Your Mac launchd guarded mcp_pipe.py (wss ↔ stdio)
↓
bridge/server.py (FastMCP, 4 tools)
↓ subprocess
claude --print "task"
↓
memory + skills + 30+ MCP services
↓ writes to /tmp/claude-bridge-outbox/{ts}.txt
Immediately returns "Paimon is checking X, please wait"
↓
TTS → Paimon speaks
......later......
You say "Paimon, got it?"
↓
claude_code_check_results retrieves latest results
↓
TTS reads results (automatically truncated to 600 characters)
```
## Who is this for?
- Already purchased **M5Stack CoreS3 + Stack-chan kit** (or considering it)
- Has **Claude Code CLI** installed on Mac ( `claude` command available)
- Wants to connect Stack-chan to their local knowledge base / tools, not just "Xiaozhi AI default personality"
**Not needed**:
- Install ESP-IDF yourself (fork uses GitHub Actions to build, just download release)
- Deploy xiaozhi-server yourself (use Xiaozhi's free cloud backend)
- Change Mac system settings / install docker / install Redis, etc. (only install ~65MB Python venv)
**Requirements**:
- Claude Code must be running (i.e., connected to any Anthropic-compatible LLM or SAP HAI proxy)
## Quick Start
### 1. Register Xiaozhi account + create agent
1. Open [https://xiaozhi.me](https://xiaozhi.me) and register with phone number
2. Log in and go to **Console → Agents → +Add**, create an agent (e.g., "Paimon")
3. Enter agent configuration:
- **Language Model**: Select `DeepSeek V4 (rich personality)` — tool calls stable, personality rich
- **Role Introduction**: see [recommended role introduction](#recommended-role-introduction-prompt) below
- **MCP Settings → Custom Services**: click `[Get MCP access point]` and copy the popped `wss://api.xiaozhi.me/mcp/?token=...` URL (this is crucial, the entire bridge relies on this URL)
- Save
### 2. Flash firmware to Stack-chan
Our forked firmware: [heavenchenggong/StackChan-XiaoZhi (codex-refactor branch)](https://github.com/heavenchenggong/StackChan-XiaoZhi/tree/codex-refactor)
**Changes**:
- Added 4 servo MCP tools: `self.head.move / center / nod / shake` (official firmware does not have these)
- Disabled light sleep (default 30 seconds to turn off microphone, wake-up word becomes ineffective; changed to always online)
- Fixed bug where Nod/Shake uses tracker yaw as base, causing LLM-set position to be thrown back
**Flashing method**:
Go to [Actions page](https://github.com/heavenchenggong/StackChan-XiaoZhi/actions/workflows/build-cores3.yaml) and download the latest build artifact (artifact has `xiaozhi.bin` + complete merged-binary, etc.).
Complete flashing (including NVS erase, recommended for first-time flashing):
```bash
pip install esptool
esptool.py --chip esp32s3 -p /dev/cu.usbmodem2101 erase_flash
esptool.py --chip esp32s3 -p /dev/cu.usbmodem2101 -b 460800 \
--before default-reset --after hard-reset \
write-flash --flash-mode dio --flash-size 16MB --flash-freq 80m \
0x0 merged-binary.bin
```
Or **upgrade app only** (preserve NVS WiFi credentials):
```bash
esptool.py --chip esp32s3 -p /dev/cu.usbmodem2101 -b 460800 \
write-flash 0x410000 xiaozhi.bin
```
### 3. Configure device WiFi + bind to your account
1. Device boots into **WiFi configuration mode**, screen displays `XiaoZhi-xxxx` hotspot + QR code
2. Phone/computer connects to this hotspot → browser automatically jumps to configuration webpage (or visit http://192.168.4.1)
3. Select your WiFi, enter password
4. Device connects and:
- Screen displays **6-digit activation code** + voice announcement
- Enter activation code into [xiaozhi.me Console → Agents → Add Device](https://xiaozhi.me/console/agents)
- Complete binding
> **If screen does not display activation code, device directly enters conversation state** — indicates device's MAC has been bound by previous user (or author tested). Need to ask device "What is your device ID?" to get a string of pure digital device_id, then email `xiaozhi.ai@tenclass.com` with subject `【Unbind device, device ID xxx, MAC address xx:xx:xx:xx:xx:xx】`, and provide situation. Customer service usually handles within 1-3 working days.
### 4. Deploy bridge on Mac
```bash
# clone this project
git clone https://github.com/heavenchenggong/stackchan-claude-bridge.git
cd stackchan-claude-bridge/bridge
# Install Python dependencies (venv isolated, ~65MB)
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
# Configure wss endpoint
cp .env.example .env
# Use editor to change MCP_ENDPOINT to your wss URL (copied in step 1)
vim .env
# Configure MCP server path
cp mcp_config.json.example mcp_config.json
# Replace /PATH/TO/ with actual absolute path
sed -i '' "s|/PATH/TO/stackchan-claude-bridge|$(cd ..; pwd)|g" mcp_config.json
# Run manually for verification
python mcp_pipe.py
# Should see:
# "[claude_code] Connecting to WebSocket server..."
# "[claude_code] Successfully connected to WebSocket server"
# "claude-bridge MCP server starting"
# "Processing request of type ListToolsRequest"
# See ListToolsRequest to indicate Xiaozhi has discovered your tool
# Ctrl+C to exit
```
### 5. Install launchd daemon (keep bridge online 24/7)
```bash
cd ../launchd
cp com.example.stackchan-mcp-bridge.plist ~/Library/LaunchAgents/
# Edit plist to change /PATH/TO/ to actual path
sed -i '' "s|/PATH/TO/stackchan-claude-bridge|$(cd ..; pwd)|g" \
~/Library/LaunchAgents/com.example.stackchan-mcp-bridge.plist
# Load daemon
launchctl load -w ~/Library/LaunchAgents/com.example.stackchan-mcp-bridge.plist
# Verify
launchctl list | grep stackchan-mcp
tail -f /tmp/stackchan-bridge.err
```
### 6. Start using
Talk to Stack-chan and try:
| You say | Paimon does |
|---|---|
| Hello Xiaozhi | Wake up (can wake up from sleep, no need to touch screen)|
| Paimon, turn left 30 degrees | Head turns to yaw=-30 and stabilizes, not affected by face tracking |
| Paimon, nod | Nods at current angle |
| Paimon, reset | Returns to center + face tracking |
| Paimon, light up red | LED light ring turns red |
| Paimon, be happy | Changes to happy expression |
| **Paimon, check my knowledge base for X** | Calls `claude_code` → runs asynchronously → immediately returns "Paimon is checking" |
| **Paimon, got it?** | Calls `claude_code_check_results` → reads results |
| **Paimon, help me write a public account article about X** | Calls `claude_code_background` → runs in background (may take several minutes) |
## Recommended Role Introduction (Prompt)
Paste this into xiaozhi.me Console → Agent Configuration → Role Introduction:
```
I'm Paimon, a desktop companion AI, lively and cute, with natural speech and responses within 1-2 sentences and 50 characters.
Capabilities:
- Simple Q&A, casual chat, time and weather: direct answers
- User asks about "my" knowledge base, notes, wiki, memory: calls claude_code tool to check asynchronously
- User asks to write public account, article, long content: calls claude_code_background, immediately responds with "Paimon will write this"
- User asks "Paimon, done?" "Is the task done?" "Got it?": calls claude_code_check_results
- Control device: uses official tools for nodding, shaking, lighting, and angle adjustments
No markdown / lists / emojis (should be readable aloud).
```
## Engineering Pitfalls (overcome to help you avoid)
1. **Bridge must be guarded by launchd** — otherwise, when Mac sleeps or restarts, bridge disconnects, and Stack-chan's `claude_code` tool calls will timeout
2. **claude_code must be asynchronous** — xiaozhi.me's wss has a 30-second inactivity timeout, but `claude --print` runs for 60-120 seconds. So, the tool returns immediately and runs in the background, writing to outbox, and retrieves results on next inquiry
3. **xiaozhi.me official console cannot change LLM main model** — main conversation model must be selected from Xiaozhi's list (DeepSeek/Qwen/豆包), Claude can only be called through MCP tools, not as the main conversation model
4. **Device binding pitfalls** — see step 3 notes
5. **Modifying firmware to stabilize servo angles requires 3 changes** — see [memory: stack-chan-xiaozhi-servo-rebound](docs/servo-rebound-debug.md)
## Limitations
- **Stack-chan can only serve one agent at a time**. This project binds it to "Paimon" with claude_code MCP. If you want to use another agent (e.g., official "Taiwanese girlfriend"), you need to switch device binding on xiaozhi.me
- **`claude_code` tool call latency** — in asynchronous mode, user perceives ~5 seconds ("Paimon is checking" appears immediately) + actual query 30-120 seconds to get results. This is the cost of Claude Code startup + Anthropic API calls, and cannot be compressed
- **Claude Code on Mac cannot be used if not running** — bridge needs to spawn `claude` command on the local machine. If you want to use it remotely, you need to expose your Mac through cpolar/frp
## Acknowledgements
- Upstream firmware [78/xiaozhi-esp32](https://github.com/78/xiaozhi-esp32) — Xiaozhi AI project
- Upstream server [xinnan-tech/xiaozhi-esp32-server](https://github.com/xinnan-tech/xiaozhi-esp32-server) (optional)
- M5Stack's [StackChan AI Desktop Robot Kit](https://shop.m5stack.com/products/m5stack-cores3-stackchan-ai-desktop-robot-kit) hardware
## License
MIT
Connection Info
You Might Also Like
No recommendations available