Content
# Speech MCP Echo
> **Based on**: [Kvadratni/speech-mcp](https://github.com/Kvadratni/speech-mcp) - the original Speech MCP project
Voice interface for multiple AI CLIs - Claude Code, Gemini CLI, Codex CLI, and MCP-compatible tools.
## Features
- **Multi-CLI Support**: Works with Claude Code, Gemini CLI, Codex CLI, and any MCP-compatible tool
- **Continuous Listening** *(New in v0.2.0)*: Non-blocking background listening with automatic retry on silence
- **Configurable STT**: Local (faster-whisper) or cloud (Groq Whisper, OpenAI Whisper, Google Speech)
- **Configurable TTS**: Local (pyttsx3) or cloud (Google Cloud TTS, OpenAI TTS)
- **Bilingual Support**: English and Chinese (Traditional/Simplified) text processing
- **JARVIS Summarizer**: Condenses long responses into concise, entertaining summaries
## Quick Start
### Installation
```bash
# From git (recommended)
pip install 'speech-mcp-echo[recommended] @ git+https://github.com/lukechi1219/speech-mcp-echo.git@v0.2.1'
# Or from wheel
pip install 'speech-mcp-echo[recommended] @ https://github.com/lukechi1219/speech-mcp-echo/releases/download/v0.2.1/speech_mcp_echo-0.2.1-py3-none-any.whl'
```
### CLI Integration
#### Claude Code (Primary Target)
Add to `~/.claude.json`:
```json
{
"mcpServers": {
"speech-mcp-echo": {
"command": "speech-mcp-echo"
}
}
}
```
**Optional**: Auto-approve speech tools in `~/.claude/settings.json` to avoid confirmation prompts:
```json
{
"permissions": {
"allow": [
"mcp__speech-mcp-echo__start_conversation",
"mcp__speech-mcp-echo__voice_listen",
"mcp__speech-mcp-echo__voice_speak",
"mcp__speech-mcp-echo__voice_reply",
"mcp__speech-mcp-echo__voice_config",
"mcp__speech-mcp-echo__voice_status",
"mcp__speech-mcp-echo__start_listening",
"mcp__speech-mcp-echo__check_listening",
"mcp__speech-mcp-echo__cancel_listening"
]
}
}
```
Then restart Claude Code and say: **"Let's have a voice conversation"**
#### Other CLIs
**Goose CLI**:
```bash
goose session --with-extension "speech-mcp-echo"
```
**Gemini CLI**: Add to `~/.gemini/settings.json` (same format as Claude Code)
**Codex CLI**: Add to `~/.codex/config.toml`:
```toml
[mcp_servers.speech-mcp-echo]
command = "speech-mcp-echo"
```
## Why These Technology Choices?
### STT: Groq Whisper (Recommended)
We chose **Groq Whisper API** as the recommended STT engine because:
- **Most accurate**: Uses whisper-large-v3-turbo model (~12% WER vs ~23% for local base model)
- **Blazing fast**: 216x real-time speed on Groq's LPU hardware
- **No local resources**: No model download, no GPU/CPU load
- **Generous free tier**: Free API with generous rate limits
- **Easy setup**: Just set `GROQ_API_KEY` environment variable
- **OpenAI-compatible**: Uses the same `openai` SDK with a different base URL
```bash
# Set your Groq API key (get one free at https://console.groq.com)
export GROQ_API_KEY="gsk_..."
```
### STT: faster-whisper (Local Alternative)
**faster-whisper** is a good alternative for fully offline operation:
- **Lightweight**: ~150MB vs ~1.5GB for OpenAI's original whisper
- **Fast**: 4x faster than original whisper using CTranslate2 optimization
- **Offline**: Works completely offline, no API costs
- **CPU-friendly**: Runs well on CPU with int8 quantization
### TTS: Google Cloud TTS (Recommended)
We chose **Google Cloud TTS** as the primary TTS engine because:
- **No heavy dependencies**: Unlike Kokoro which requires PyTorch (~2GB), Google Cloud TTS uses a simple REST API
- **High quality**: Neural voices with natural prosody
- **Multilingual**: Excellent support for English, Chinese (Traditional/Simplified), and 40+ languages
- **Flexible auth**: Multiple authentication methods (see below)
- **Cost-effective**: Free tier includes 4 million characters/month
- **Cross-platform**: Works on macOS, Linux, and Windows
#### Google Cloud TTS Authentication Options
Choose the method that works best for your setup:
**Option A: gcloud CLI (Recommended for developers)**
```bash
# Install gcloud CLI
brew install google-cloud-sdk # macOS
# or: https://cloud.google.com/sdk/docs/install
# Login and configure
gcloud auth login
gcloud config set project YOUR_PROJECT_ID
```
**Option B: Service Account (Recommended for servers/production)**
```bash
# 1. Create service account in Google Cloud Console
# - Go to IAM & Admin > Service Accounts
# - Create account with "Cloud Text-to-Speech API User" role
# - Download JSON key file
# 2. Set environment variable
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account-key.json"
# 3. Optionally set project ID (if not in key file)
export GOOGLE_CLOUD_PROJECT="your-project-id"
```
**Option C: Python Client Library**
```bash
# Install the library
pip install google-cloud-texttospeech
# Then use Option A or B for authentication
# The library will auto-detect credentials
```
The adapter automatically tries these methods in order and uses the first one that works.
#### Why Not Kokoro?
Kokoro is an excellent local TTS engine, but it requires:
- PyTorch (~2GB download)
- CUDA for optimal performance
- Additional language models (misaki)
For most users, Google Cloud TTS provides better quality with simpler setup. Kokoro remains available as an optional local alternative for users who prefer fully offline operation.
## Configuration
Configuration is stored at `~/.config/speech-mcp-echo/config.json`:
```json
{
"stt": {
"engine": "groq",
"model": "whisper-large-v3-turbo",
"language": "auto",
"timeout": 45,
"silence_retry_count": 10,
"retry_prompt_type": "beep"
},
"tts": {
"engine": "google",
"voice": "cmn-TW-Standard-B",
"language": "cmn-TW"
},
"summarizer": {
"enabled": true,
"personality": "jarvis",
"language": "en"
}
}
```
### Continuous Listening Settings
| Setting | Default | Description |
|---------|---------|-------------|
| `stt.timeout` | 45 | Seconds to wait for speech per attempt |
| `stt.silence_retry_count` | 10 | Retries when silence detected (~7.5 min tolerance) |
| `stt.retry_prompt_type` | "beep" | Prompt on retry: "beep", "voice", or "silent" |
```
### Supported Languages
| Language | STT (Groq/faster-whisper) | TTS (Google Cloud) |
|----------|---------------------------|-------------------|
| English | ✅ | ✅ en-US, en-GB |
| Chinese (Traditional) | ✅ | ✅ cmn-TW |
| Chinese (Simplified) | ✅ | ✅ cmn-CN |
| Japanese | ✅ | ✅ ja-JP |
### Environment Variables
API keys are read from environment variables:
- `GROQ_API_KEY` - For Groq Whisper STT (recommended)
- `OPENAI_API_KEY` - For OpenAI Whisper STT and TTS
- `GOOGLE_APPLICATION_CREDENTIALS` - For Google Cloud services (optional if using gcloud CLI)
- `ANTHROPIC_API_KEY` - For Claude-based summarization
## Architecture
```
src/speech_mcp_echo/
├── __init__.py # Main entry point
├── __main__.py # Module execution
├── audio_processor.py # Audio capture, playback, cues
├── constants.py # Centralized constants
├── server.py # Unified MCP server (all tools)
├── config/ # Configuration management
├── core/
│ └── voice_engine.py # STT, TTS, summarization
├── resources/
│ └── audio/ # Audio cue files (.wav)
├── stt_adapters/ # Speech-to-text engines
│ ├── groq_whisper_adapter.py # Cloud (recommended)
│ ├── faster_whisper_adapter.py # Local
│ ├── openai_whisper_adapter.py # Cloud
│ └── google_speech_adapter.py # Cloud
├── tts_adapters/ # Text-to-speech engines
│ ├── google_tts_adapter.py # Cloud (recommended)
│ └── openai_tts_adapter.py # Cloud
├── summarizer/ # Response summarization
│ ├── local_summarizer.py # Rule-based with JARVIS personality
│ └── llm_summarizer.py # LLM-based (placeholder)
└── utils/
└── logger.py # Centralized logging
```
## Development
```bash
# Clone and install in development mode
git clone https://github.com/lukechi1219/speech-mcp-echo.git
cd speech-mcp-echo
pip install -e ".[dev]"
# Run tests
pytest tests/
```
## License
MIT
## Credits
Adapted from [Kvadratni/speech-mcp](https://github.com/Kvadratni/speech-mcp) with multi-CLI support.
