Content
# Muster
A reusable Rust core for **orchestrating many LLM agents in parallel** — exposed
both as a linkable crate and as an MCP server.
> *muster* — to assemble a force and bring it under command. That's the job:
> running one agent is easy; coordinating N of them without cost, redundant work,
> race conditions, and result-merge chaos getting out of hand is the hard part.
> With 3–6 concurrent agents, token use doesn't scale linearly — it explodes.
Muster exists to **impose budget, deduplicate work, and resolve result conflicts
deterministically and auditably.** It does *not* implement the agents, the
per-subtask logic, or the LLM client — the client is injected through a trait.
Output verification is a separate layer.
🇧🇷 *Versão em português: [readme-pt.md](readme-pt.md).*
## Install
### Option A — prebuilt binary (recommended)
Grab the archive for your platform from the
[latest release](https://github.com/JoasASantos/muster-mcp/releases/latest):
| Platform | Asset |
|----------|-------|
| Linux x86_64 | `muster-mcp-<version>-x86_64-unknown-linux-gnu.tar.gz` |
| Linux ARM64 | `muster-mcp-<version>-aarch64-unknown-linux-gnu.tar.gz` |
| macOS Intel | `muster-mcp-<version>-x86_64-apple-darwin.tar.gz` |
| macOS Apple Silicon | `muster-mcp-<version>-aarch64-apple-darwin.tar.gz` |
| Windows x86_64 | `muster-mcp-<version>-x86_64-pc-windows-msvc.zip` |
```sh
# macOS / Linux — extract and put it on your PATH
tar -xzf muster-mcp-*.tar.gz
install -m755 muster-mcp ~/.local/bin/muster-mcp # or anywhere on PATH
# register with Claude Code (user scope = available in every project)
claude mcp add --scope user muster -- ~/.local/bin/muster-mcp
claude mcp get muster # should say: ✔ Connected
```
Then **open a new Claude Code session** (MCP servers load at session start) and
the `muster` tools appear. Released binaries include the optional Anthropic
provider — set `ANTHROPIC_API_KEY` to use a real model; otherwise they run the
built-in deterministic echo provider.
### Option B — build from source
```sh
cargo build --release # echo provider only (no network deps)
cargo build --release --features anthropic # + real Anthropic provider
claude mcp add --scope user muster -- "$PWD/target/release/muster-mcp"
```
## Hard premises
- **Classify before keeping.** Every piece of content is classified
re-fetchable / durable / reasoning before a retention decision. Tool output is
re-fetchable → discarded. Decisions and findings are durable → persisted.
Reasoning doesn't compress without loss → persisted verbatim.
- **No subtask runs twice.** Dedup by content hash is mandatory. A repeated hash
returns the cached (or in-flight) result instead of re-executing.
- **Budget is a hard constraint, not a suggestion.** When a cap is reached, new
work is backpressured or rejected — never a silent overflow.
## Components (built in dependency order)
| # | Module | Type | Responsibility |
|---|--------|------|----------------|
| 1 | `governor` | `TokenGovernor` | Per-agent + global hard caps, real-time tracking, reserve/commit reconciliation, backpressure. Backs `budget.query`. |
| 2 | `rate_limiter` | `RateLimiter` | One shared semaphore + windowed request/token limits across all workers. |
| 3 | `queue` | `WorkQueue` | Bounded queue, content-hash dedup with single-flight + result cache, backpressure. |
| 4 | `scheduler` | `Orchestrator` | Fan-out subtasks to workers / fan-in results. `orchestrate.spawn`, `orchestrate.status`. |
| 5 | `blackboard` | `Blackboard` | Shared state, `RwLock` + broadcast updates. `blackboard.read` / `blackboard.write`. |
| 6 | `aggregator` | `Aggregator` | Conflict resolution (vote / source-priority / recency) with provenance. |
| 7 | `replay` | `ReplayLog` | Every LLM call logged; deterministic re-execution via `ReplayClient`. |
The LLM client is injected through the `llm::LlmClient` trait. The core never
embeds one; `dummy::EchoClient` is provided for tests, examples, and running the
MCP server standalone.
## Token accounting (reserve → commit)
The global cap is a **lifetime** spend ceiling; the per-worker cap bounds any one
agent.
1. Before a call, the worker **reserves** the local estimate
(`LlmClient::estimate_tokens`); it counts against the cap immediately.
2. After the call, the reservation is **committed** with the authoritative
`TokenUsage` total — the official source reconciles the local approximation.
3. A reservation dropped without committing (failed/panicked call) releases the
estimate automatically: budget never leaks.
4. `reserve()` **waits** (backpressure) only when the shortfall is *transient*
(other reservations in flight). When the shortfall is *permanent* (lifetime
budget already spent), it **rejects** instead of blocking forever.
## Crate usage
```rust
use std::sync::Arc;
use serde_json::json;
use muster::dummy::EchoClient;
use muster::scheduler::{Orchestrator, OrchestratorConfig};
#[tokio::main]
async fn main() {
let orch = Orchestrator::new(OrchestratorConfig::default(), Arc::new(EchoClient::new()));
for _ in 0..4 { orch.spawn(); } // fan out
let r = orch.execute(json!({ "prompt": "hello" })).await.unwrap();
println!("{}", r.payload); // fan in (deduplicated)
println!("{:?}", orch.budget_query());
orch.shutdown().await;
}
```
Run the bundled examples:
```sh
cargo run --example basic # minimal end-to-end
cargo run --example stress --release # load test (see below)
```
## Load test
`examples/stress.rs` fires 500 submissions (50 distinct, each repeated 10×)
across 12 workers, with a tight lifetime budget and a 4-slot shared rate limiter.
A monitor samples the live budget to prove the cap is never breached. Sample run:
```
DEDUP
submitted ............ 500
reached queue ........ 50 (distinct work)
collapsed by dedup ... 450 (90.0%) [in-flight 450 + cached 0]
EXECUTION
real LLM calls ....... 23
succeeded ............ 230
rejected (budget) .... 270
TOKEN ECONOMY
spent (committed) .... 11615 tokens
naive w/o dedup ~..... 252500 tokens => ~95% saved by dedup
BUDGET GUARANTEE
global cap ........... 12000
peak observed used ... 11615
HARD CAP HELD ✓ (peak 11615 <= cap 12000)
BACKPRESSURE
peak workers Blocked . 9 / 12 (rate limit = 4 concurrent)
```
Dedup collapsed 90% of the work; the hard cap held under load and then *rejected*
the unaffordable tail instead of overflowing; the shared rate limiter kept ≤4
calls in flight (up to 9 workers backpressured).
## Why Muster — a worked example
Once `muster` is registered, you don't call tools by hand — you ask Claude a
normal question and it orchestrates. The point lands when the work contains
*repeats*, which real fan-outs always do.
**Prompt** (paste into a Claude Code session that has `muster`):
```
I need to review 6 backend modules in parallel, but with a cost ceiling
because I've blown my token budget before. The modules are:
auth, billing, auth, notifications, billing, search.
Use the muster server to coordinate it, then tell me the total cost and how
many calls actually ran — was parallelizing worth it?
```
**What Muster does** — `auth` and `billing` appear twice, so dedup collapses
them: 6 submissions become **4 real executions**, and the budget snapshot
reflects 4, not 6.
**Example result:**
```
Spawned 4 workers and submitted all 6 modules.
Dedup collapsed the duplicates — only 4 distinct modules actually ran
(auth and billing were each requested twice and served from cache).
budget.query → global_committed: 4 executions' worth of tokens (not 6)
global_reserved: 0 (all reservations reconciled)
never exceeded the cap
Worth it: 6 requested → 4 executed = 33% of the work avoided on this batch,
and the hard cap guarantees the fan-out can't run away with your budget.
```
That 33% is just from two repeats in a list of six. At fleet scale the
[load test](#load-test) above shows the same mechanism erasing **90%** of the
work (500 submissions → 50 executions) while the token cap holds under pressure.
## MCP server
`muster-mcp` speaks Model Context Protocol over stdio (newline-delimited
JSON-RPC 2.0). It implements `initialize`, `tools/list`, `tools/call`, `ping`
and these tools:
- `orchestrate.spawn` — create worker(s)
- `orchestrate.status` — per-worker state (idle/working/blocked/done) + budget
- `blackboard.read` / `blackboard.write` — shared state
- `budget.query` — global + per-worker token budget
- `orchestrate.submit` / `orchestrate.resolve` *(additive)* — drive work and
consolidate results
```sh
cargo build
printf '%s\n' \
'{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}' \
'{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"orchestrate.spawn","arguments":{"count":3}}}' \
'{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"orchestrate.submit","arguments":{"payload":{"prompt":"hi"}}}}' \
| ./target/debug/muster-mcp
```
To talk to a real model, replace `EchoClient` in `src/bin/mcp.rs` with your own
`LlmClient` implementation.
## Stack
Tokio runtime. Bounded work queue, `broadcast` for blackboard updates, bounded
channels for backpressure, `tokio::sync::Semaphore` for rate limiting,
`Arc<RwLock>` for the blackboard. Content hashing via SHA-256.
## Tests
```sh
cargo test # 36 unit + 7 integration tests
cargo clippy --all-targets # clean
```
Connection Info
You Might Also Like
No recommendations available