Content
# rhythm-vibe-studio
MCP server for "vibe coding" music with LilyPond-first workflows and resilient fallbacks.
> **Full documentation (single handbook):** [docs/HANDBOOK.md](docs/HANDBOOK.md) — run/deploy, vision, architecture, tests, types, theory, UI history, prompt mapping, cloud agent workflows. **Index:** [docs/README.md](docs/README.md).
## Rhythm Vibe Studio — backend + frontend commands
| Goal | What to run |
|------|-------------|
| **Production-style (one server)** — FastAPI serves `/api/*` and the built SPA | `uv sync --extra dev --extra scrape` then `uv run python -m rhythm_vibe_studio.web.build` then `uv run rhythm-vibe-studio-webui --host 127.0.0.1 --port 7860` -> open [http://127.0.0.1:7860](http://127.0.0.1:7860) |
| **Development (hot reload)** — Vite + separate API | **Terminal A:** `uv run rhythm-vibe-studio-webui --host 127.0.0.1 --port 7860` — **Terminal B:** `cd frontend && pnpm install && pnpm dev` (or `cd frontend && npm install && npm run dev`) -> UI at [http://127.0.0.1:5173](http://127.0.0.1:5173) (proxies `/api` to port **7860**) |
Optional: point Vite at another API with `frontend/.env.development.local` -> `VITE_STUDIO_API_ORIGIN=http://host:port`.
**Remote static UI** (GitHub Pages, etc.): set **`VITE_STUDIO_API_ORIGIN`** at build time to your public FastAPI origin. **Cloudflare Worker:** build `frontend/dist`, deploy `cloudflare/` with Wrangler (`STUDIO_API_ORIGIN` secret). Details: [docs/HANDBOOK.md § Part 1](docs/HANDBOOK.md#part-1--operations-run-deploy-test).
## Release automation
- [.github/workflows/pages.yml](.github/workflows/pages.yml) deploys the docs site and the static `/studio/` bundle to GitHub Pages.
- [.github/workflows/publish-containers-and-helm.yml](.github/workflows/publish-containers-and-helm.yml) validates the root Dockerfile and Helm chart on pull requests, then publishes multi-arch images to GHCR and optional Docker Hub or LSCR targets on `master` and version tags.
- [.github/workflows/mirror-repositories.yml](.github/workflows/mirror-repositories.yml) ensures GitLab and Codeberg repos exist before pushing mirrors, and syncs a Docker-based Hugging Face Space from the GitHub source tree.
- [scripts/bootstrap_release_targets.sh](scripts/bootstrap_release_targets.sh) bootstraps the same external targets manually with `gh`, `hf`, and the GitLab or Codeberg APIs.
Container runtime defaults:
- Root image entrypoint: `rhythm-vibe-studio-webui --host 0.0.0.0 --port 7860`
- Health probe: `/api/healthcheck`
- Writable workdir volume: `/data`
Secrets, variables, and chart publishing details live in [docs/HANDBOOK.md](docs/HANDBOOK.md#112-release-automation-containers-helm-and-mirrors).
## Documentation
| Doc | Description |
|-----|-------------|
| **[docs/HANDBOOK.md](docs/HANDBOOK.md)** | **Primary reference** — how to run everything (MCP, Studio, Vite, tests, hosting), vision, architecture, web UI research, implementation map, testing, types, theory, cloud agents |
| [docs/README.md](docs/README.md) | Short TOC + links into the handbook |
| [docs/ROCKSMITH_KNOWLEDGEBASE.md](docs/ROCKSMITH_KNOWLEDGEBASE.md) | Rocksmith-style practice feature research and Studio parity map |
| Stub files (`docs/VISION_AND_SPECIFICATION.md`, `ARCHITECTURE.md`, …) | Redirect to the handbook section (old links keep working) |
## What this server supports now
- Fetch existing music assets from the web (`fetch_music_from_web`).
- Convert between common music formats (`convert_music`) with best-effort routing:
- Notation: `lilypond`, `musicxml`, `midi`, `abc`, `pdf`
- Audio containers: `wav`, `mp3`, `m4a`
- Fallback: `json_fallback`
- Audio/file to sheet workflow (`audio_or_file_to_sheet`) using transcription/conversion chain.
- Song transposition by semitones (`transpose_song`).
- Musescore API proxy (`musescore_api`) with:
- public endpoint usage without login where possible
- login token via env (`MUSESCORE_API_TOKEN`) or session tool (`set_musescore_auth_token`)
- Robust fallback behavior when strict LilyPond or converter steps fail.
## Canonical text-based social shorthand
For "Reddit/phone shorthand", this server treats **ABC notation** as the primary universal text format,
with **ChordPro** as a secondary lead-sheet shorthand.
Tool:
- `normalize_reddit_music_text`: normalizes informal text into a robust event-based fallback model.
Why this choice:
- ABC is compact and human-typable in comments/chats.
- ABC has broad tooling support and maps well to formal notation pipelines.
- ChordPro is widely used for quick chord+lyric communication.
## Error handling model
When strict conversions fail (e.g. LilyPond compile errors), server returns:
- `ok=false` plus diagnostic `message`
- `fallback` object (`RobustMusicFallback`) containing:
- ambiguous but actionable event data
- shorthand text snapshot
- warnings to continue downstream tasks
This keeps multi-step agent tasks moving even when one notation step fails.
## Verification (design phase)
There is **no automated test suite** in this repo while Studio and MCP surfaces are still in active design. Do not add pytest, Vitest, or Playwright tests without an explicit stabilization decision.
- **Studio UI:** run the app and verify affected flows in a real browser; see [AGENTS.md](AGENTS.md) and [docs/HANDBOOK.md](docs/HANDBOOK.md).
- **CLI/MCP:** `uv run rhythm-vibe-studio-cli <tool> --help` plus one real invocation with stdout/stderr shown.
- **Fixtures:** `samples/studio-fixtures/` (`minimal.abc`, `minimal.ly`, `minimal.mid`) for manual import smoke.
## Quickstart
**With uv (recommended):**
Run from the project directory so the installed package uses source (`src/`). Run `uv sync` to pick up the latest code.
```bash
cd C:\GitHub\rhythm-vibe-studio
uv sync
uv run rhythm-vibe-studio
# or: uvx --from . rhythm-vibe-studio
```
Select transport explicitly when needed:
```bash
uv run rhythm-vibe-studio --transport stdio
uv run rhythm-vibe-studio --transport streamable-http --host 127.0.0.1 --port 8000
uv run rhythm-vibe-studio --transport sse --host 127.0.0.1 --port 8000
```
Supported transports: `stdio`, `streamable-http`, `sse` (`http` is accepted as an alias for `streamable-http`).
Optional extras (audio/transcription, scraping):
```bash
uv sync --extra audio --extra scrape
```
`audio` enables ML transcription via Spotify Basic Pitch for the audio->MIDI step.
If the extra is not installed (or the runtime fails), the project uses a best-effort
ffmpeg+autocorrelation fallback so audio workflows still run.
**Rhythm Vibe Studio** (FastAPI + React): see the **Rhythm Vibe Studio — backend + frontend commands** section at the top of this file for the exact matrix (single server vs Vite dev). Minimal API-only start:
```bash
uv sync
uv run rhythm-vibe-studio-webui --host 127.0.0.1 --port 7860
```
Production bundle path: API routes from FastAPI and static assets under `src/rhythm_vibe_studio/web/static/` after:
```bash
uv run python -m rhythm_vibe_studio.web.build
```
If you still need the legacy Gradio UI module, install the optional extra:
```bash
uv sync --extra legacy_ui
```
Studio screenshot:
- `docs/screenshots/rhythm-vibe-studio-overview.png`
Refresh top Hugging Face music Spaces into `vendor/huggingface_spaces` and regenerate report:
```bash
uv run rhythm-vibe-refresh-spaces --query music --top-n 5 --limit 100
```
Optional cleanup of stale cloned Space folders:
```bash
uv run rhythm-vibe-refresh-spaces --query music --top-n 5 --prune
```
**With pip:**
```bash
python -m venv .venv
. .venv/Scripts/Activate.ps1
pip install -e .
pip install -e ".[audio,scrape]" # optional
rhythm-vibe-studio
```
To launch the web UI with pip-managed environments:
```bash
pip install -e .
rhythm-vibe-studio-webui
```
## External binaries/tools (recommended)
- `lilypond` for LilyPond -> PDF/MIDI rendering
- `ffmpeg` for audio container conversion
- `MuseScore` CLI (future route expansion for advanced engraving/export)
Without these tools, the server still returns structured fallback outputs.
## Running with uvx (no PyPI publish)
The package is not on PyPI. Run the tool from the **local project path** so uv installs from the directory:
```bash
# From the project directory (recommended)
cd C:\GitHub\rhythm-vibe-studio
uvx --from . rhythm-vibe-studio
```
Or from anywhere using an absolute path:
```bash
uvx --from "C:/GitHub/rhythm-vibe-studio" rhythm-vibe-studio
```
Do **not** use `uvx rhythm-vibe-studio` alone—that looks for the package on PyPI and will fail.
## Cursor MCP config (example)
Using uvx with local path (recommended):
```json
{
"mcpServers": {
"rhythm-vibe-studio": {
"type": "stdio",
"command": "uvx",
"args": ["--from", ".", "rhythm-vibe-studio"],
"env": { "rhythm_vibe_mcp_WORKDIR": "C:/GitHub/rhythm-vibe-studio" },
"cwd": "C:/GitHub/rhythm-vibe-studio"
}
}
}
```
Alternative (Python from venv):
```json
{
"mcpServers": {
"rhythm-vibe-studio": {
"command": "python",
"args": ["-m", "rhythm_vibe_studio.server"],
"cwd": "c:/GitHub/rhythm-vibe-studio"
}
}
}
```
## CLI (direct tool commands)
`rhythm-vibe-studio-cli` / `rhythmvibe-cli` expose one subcommand per MCP tool (generated from tool schemas).
```bash
uvx --from . --with-editable . rhythmvibe-cli --help
uvx --from . --with-editable . rhythmvibe-cli healthcheck
uvx --from . --with-editable . rhythmvibe-cli analyze-audio-performance --input-ref "cello_samples/New Recording 31_formats/New Recording 31.opus"
uvx --from . --with-editable . rhythmvibe-cli batch-convert-audio --input-ref "cello_samples"
uvx --from . --with-editable . rhythmvibe-cli audio-or-file-to-sheet --input-ref "cello_samples/New Recording 31.m4a" --prefer-output lilypond
```
For ML transcription with `basic_pitch`, install the `audio` extra:
```bash
uvx --python 3.11 --from ".[audio]" --with-editable . --with "setuptools<81" rhythmvibe-cli audio-or-file-to-sheet --input-ref "cello_samples/New Recording 31.m4a" --prefer-output lilypond
```
## Current tool surface
- `healthcheck()` — workdir, artifacts_dir, MuseScore env + session token flag, binary availability (lilypond, ffmpeg), and supported_formats list
- `fetch_music_from_web(url)`
- `plan_music_conversion(input_format, output_format)`
- `convert_music(input_ref, output_format)`
- `audio_or_file_to_sheet(input_ref, prefer_output="pdf")`
- `transpose_song(input_ref, semitones, output_format="musicxml")`
- `normalize_reddit_music_text(text, title="reddit_vibe_idea")`
- `convert_text_notation_to_lily_or_fallback(text, target_format="lilypond", title="text_notation_piece")`
- `compose_story_lily(prompt, title="Theme", tempo_bpm=56, instrument="Solo", clef=null, midi_instrument=null, output_format="lilypond")`
- `batch_convert_audio(input_ref)`
- `analyze_audio_performance(input_ref)`
- `set_musescore_auth_token(token)`
- `musescore_api(endpoint, method="GET", payload_json="{}", base_url="")`
## Notes on "any/all conversion combinations"
The architecture is built to support full matrix conversion, but some routes depend on external engines.
When a direct route does not exist yet, the server emits fallback output and diagnostics instead of failing hard.
Planned expansion areas:
- route planner with multistep intermediate formats
- richer audio-to-score transcription and quantization choices
- deeper MuseScore integration routes (CLI + public APIs + optional authenticated flows)
