Content
# voice-mcp
Speech-to-text **and** text-to-speech as one small service, exposed over **both**
an MCP (streamable-HTTP) interface and a plain REST interface.
- **STT** — [faster-whisper](https://github.com/SYSTRAN/faster-whisper) on CPU
(int8). Transcribe voice messages (ogg/opus, mp3, wav, m4a…) to text.
- **TTS** — Vietnamese speech via [gTTS](https://github.com/pndurette/gTTS),
with an optional Google Cloud TTS fallback. Server-side ffmpeg transcodes the
mp3 to **Ogg/Opus** (`libopus`, 48 kbit/s) so the clip renders as a native
**Telegram voice bubble**. Returns a **URL to a cached `.ogg`**, never a base64
blob. No bot needs ffmpeg — it is baked into the image.
This is the packaged evolution of the original single-file `whisper-mcp` (STT
only); the STT behaviour and API are kept 100% backward-compatible.
## About
voice-mcp gives AI agents (Claude Code, chat bots, automation) a single, self-hosted
voice endpoint: send audio, get text back; send text, get a shareable voice clip back.
It runs on a modest CPU VPS, keeps audio on your own infrastructure (nothing is sent to
a third-party STT/TTS service beyond the optional cloud-TTS fallback you opt into), and
exposes the same two capabilities over both MCP and REST so any tool can consume it.
Built for Vietnamese first (Whisper + gTTS), usable for any language Whisper/gTTS
support.
## Interfaces
| Route | Auth | Description |
| --- | --- | --- |
| `POST /mcp` (MCP streamable-HTTP) | Bearer | tools: `transcribe`, `speak` |
| `POST /transcribe` `{audio_base64, language}` | Bearer | → `{text, language}` |
| `POST /speak` `{text, lang}` | Bearer | → `{url, cached, engine, format}` |
| `GET /audio/<sha>.ogg` | none | serves a cached `.ogg`/`.mp3` (opaque hashed name) |
| `GET /healthz` | none | liveness / model info |
MCP endpoint path is `/mcp` with **no trailing slash**.
`speak` (both the MCP tool and REST) returns a short **URL string** to an
`.ogg` (Ogg/Opus) clip, never base64 audio — a base64 blob in a tool result
would blow up the model's context window. The file is served by
`GET /audio/<sha>.ogg` with `Content-Type: audio/ogg` (the same route also
serves the intermediate `.mp3` as `audio/mpeg`).
## Run
### Prebuilt image (GHCR)
```bash
docker run -d --name voice-mcp \
-p 127.0.0.1:8767:8767 \
-e MCP_TOKEN=your-secret \
-e WHISPER_MODEL=small \
-e PUBLIC_BASE_URL=https://whisper.veasy.vn \
-v voice-models:/models \
-v voice-audio:/audio \
--memory 1200m --cpus 2 \
ghcr.io/<owner>/voice-mcp:latest
```
### Build locally
```bash
docker build -t voice-mcp .
# or, without Docker:
pip install -e ".[dev]"
MCP_TOKEN=dev PUBLIC_BASE_URL=http://localhost:8767 voice-mcp
```
### Config (env)
See [`.env.example`](.env.example): `MCP_TOKEN`, `WHISPER_MODEL`, `AUDIO_DIR`,
`PUBLIC_BASE_URL`, and the optional `GOOGLE_APPLICATION_CREDENTIALS`.
## Model sizing
CPU int8, rough footprint / accuracy trade-off:
| Model | RAM (approx) | Notes |
| --- | --- | --- |
| `tiny` / `base` | ~0.3–0.5 GB | fastest, lower accuracy |
| `small` | ~0.7–1 GB | **default** — good VN accuracy / speed balance |
| `medium` | ~2–3 GB | better accuracy, heavier |
| `large-v3` | ~4–5 GB | best accuracy, needs plenty of RAM |
Cap the container (e.g. `--memory 1200m --cpus 2`) on small hosts so it can't OOM
the box; `small` fits comfortably.
## TTS engine rationale
The provider chain lives in [`src/voice_mcp/tts.py`](src/voice_mcp/tts.py):
1. **gTTS** (Google Translate TTS, `lang="vi"`) — primary, free, no credentials,
outputs mp3.
2. **Google Cloud TTS** (voice `vi-VN-Wavenet-A`, female) — fallback, only used
when gTTS raises (e.g. HTTP 429). Requires `GOOGLE_APPLICATION_CREDENTIALS`;
skipped entirely when unset.
Deliberately **not** used (do not re-add):
- **Piper** — voice quality rejected.
- **edge-tts** — Microsoft blocks datacenter IPs (403 from the VPS).
The mp3 is then transcoded to **Ogg/Opus** by ffmpeg
(`ffmpeg -y -loglevel error -i in.mp3 -c:a libopus -b:a 48k out.ogg`), baked
into the image. Both the `.mp3` (kept for the chain) and the returned `.ogg` are
cached content-addressed by `sha256(text + lang + engine)` in `AUDIO_DIR`, so
repeated requests return the `.ogg` instantly without re-running gTTS or ffmpeg.
## Serving audio
The app serves `GET /audio/<file>` itself (streams the `.ogg`/`.mp3` with the
matching Content-Type, no auth — the filename is an opaque hash), so no extra
web-server config is required. For efficiency you can instead let nginx serve
`AUDIO_DIR` directly:
```nginx
location /audio/ {
alias /path/to/audio/;
add_header Cache-Control "public, max-age=31536000, immutable";
}
```
## Speech-to-speech helper pattern
Chain the two tools for a voice-in → voice-out loop:
```python
text = transcribe(audio_base64=clip_b64, language="vi") # STT
reply = my_llm(text) # your logic
url = speak(text=reply, lang="vi") # TTS -> Ogg/Opus URL
# download `url` (an .ogg) and send it back as a Telegram voice message
```
## Development
```bash
pip install -e ".[dev]"
ruff check .
pytest # pure-logic tests (cache keying, config parsing) — no network/models
```
## Contributing
Contributions are welcome. See [CONTRIBUTING.md](./CONTRIBUTING.md) for the dev setup,
the `ruff` + `pytest` checks to run before a PR, and the design principles (two APIs /
never return bulk audio inline / engines as a fallback chain).
## License
[MIT](./LICENSE) © Edward Nguyen
