Content
# voxstudio
Self-hosted, **multilingual voice I/O studio** with support for Chinese and other languages. ASR + LLM + TTS engines sit behind **one OpenAI-compatible contract**, with a core orchestration layer and thin apps (CLI / Web / MCP / mobile client).
> Design lineage: benchmarked against [VoxWeaver Studio](https://github.com/nicekate/VOXWEAVER-STUDIO) and [Voicebox](https://github.com/jamiepine/voicebox). Focus = **multilingual speech, fully self-hosted deployment, and one swappable engine contract**.
## Architecture
```
┌─ CLI (thin client — first surface)
├─ Web Studio (browser)
core service ─┼─ MCP server (agent voice)
(orchestration)├─ desktop app (optional)
└─ mobile client
│ core = I/O loop + voice profiles + long-text chunking + persona/refine
│
└── engines (OpenAI-compatible; hosted↔local = base-URL swap)
├─ ASR parakeet.cpp (mudler/parakeet.cpp)
├─ TTS VoxCPM2 PyTorch (this repo: engines/voxcpm2-server) ← quality-first
│ VoxCPM.cpp (liuzl/VoxCPM.cpp — offline/portable fallback)
└─ LLM llama.cpp (Gemma)
```
The core never talks to a specific engine — only to the OpenAI-compatible contract (`/v1/audio/speech`, `/v1/audio/transcriptions`, `/v1/chat/completions`, plus a `/v1/voices` extension). Switching an engine between a remote GPU host and a local machine is a base-URL change.
## Layout
| Path | What |
|---|---|
| `engines/voxcpm2-server/` | Our TTS engine wrapper — FastAPI over OpenBMB VoxCPM2 |
| `packages/` | Shared TypeScript contracts, clients, configuration, text, audio, and orchestration |
| `platforms/bun/` | Filesystem, process, recording, and playback adapters for Bun apps |
| `core/` | Transitional Python parity implementation and research-facing core |
| `apps/cli/` | Compiled TypeScript `vox` CLI plus the transitional Python fallback |
| `docs/` | Product design docs |
The product workspace uses Bun 1.3.14. Shared packages use Web APIs and remain independent
of Bun; operating-system integration stays in `platforms/`. The Python parity code forms a
uv workspace with one light, cross-platform lock. `engines/` is excluded from it because
the TTS engine pins a CUDA torch build and resolves for x86_64 Linux only.
## Quick start
```bash
cp config.example.yaml voxstudio.yaml # point it at your engines
bun ci
bun run build:cli
./apps/cli/dist/vox health # probe all three engines
./apps/cli/dist/vox say -f article.txt --voice alice -o out.wav
./apps/cli/dist/vox transcribe recording.wav
./apps/cli/dist/vox chat "用三句话介绍一下你自己" --speak -o reply.wav
./apps/cli/dist/vox voices add alice --audio sample.wav --text "参考音的逐字稿"
./apps/cli/dist/vox voices add bob --audio sample.wav --language zh # transcript via ASR
./apps/cli/dist/vox voices add carol --record 15 --language zh # record, ASR, register
```
The build produces one standalone executable containing the Bun runtime and TypeScript
dependencies. Windows writes `apps/cli/dist/vox.exe`. Playback and microphone recording
remain optional external integrations: install FFmpeg for `ffplay` and `ffmpeg`, and pass
`--device` to select a non-default microphone.
The Python CLI remains available as a migration fallback and parity oracle:
```bash
uv sync --locked
uv run vox health
```
Long text is chunked at ~15 seconds of *estimated speech* — roughly 85 Chinese
characters, or 275 English ones — and the pieces are joined by trimming each one's edge
silence and inserting a single fixed pause. Both numbers are empirical: a single TTS
generation drifts away from the reference voice as it runs, and raw concatenation
produces seams of wildly uneven length. The estimate comes from a per-script speech rate
table measured against the engine, so the budget means the same thing in every language
it speaks. See `docs/chunking.md`.
## Model stack
| Layer | Engine |
|---|---|
| ASR | parakeet.cpp (`nemotron-3.5-asr-streaming-0.6b`, Mandarin-capable) |
| TTS | **VoxCPM2 PyTorch** (this repo) — 48kHz, 30 languages + 9 Chinese dialects, voice cloning + zero-shot voice design |
| LLM | Gemma (llama.cpp) |
## Status
The engine backend and compiled TypeScript CLI are verified end-to-end against live
engines. Long-text synthesis streams, and named voices support file input, microphone
recording, automatic ASR, and transcript editing. Native CI builds and executes the CLI on
macOS arm64, Linux x64, and Windows x64. Signed release artifacts, Web, MCP, desktop, persona
rewriting, and a duplex conversation loop are not built yet.
## Related
- Upstream C++ TTS engine (fallback): [liuzl/VoxCPM.cpp](https://github.com/liuzl/VoxCPM.cpp)
- Upstream C++ ASR engine: [mudler/parakeet.cpp](https://github.com/mudler/parakeet.cpp)
> Secrets (`.env`, upstream keys, tokens) and deployment topology are never committed to this public repo.
