Content
# LLM Council — MCP Server
MCP Server (Model Context Protocol), which allows Claude Code / Claude
Desktop to query not one model, but **a council of several LLMs**. The question
is sent in parallel to several models, they anonymously rank each other's answers,
and the chairman model synthesizes the final answer.
At the output — one, better-verified answer. At the same time, the entire discussion
(answers of models, peer-ranking, confidence signal) is available for analysis.
![llm-council in action: question goes to the council of several LLMs (Stage 1), models anonymously rank each other's answers as Response A–F (Stage 2), in case of disagreement, a low confidence signal is raised, then the chairman from another family of models synthesizes the final result with attribution [A]/[B] (Stage 3)](docs/demo.gif)
---
## Quick Start (≈3 minutes)
**Requirements:** [uv](https://docs.astral.sh/uv/) (`brew install uv`), Python ≥ 3.10,
[OpenRouter](https://openrouter.ai/keys) key.
```bash
# 1. Get the code
git clone https://github.com/azalio/llm-council.git
cd llm-council
# 2. Install dependencies (MCP server is installed here)
uv sync
# 3. Register access
cp .env.example .env
# then edit .env — set the key:
# OPENROUTER_API_KEY=...
# 4. Check that the server starts (Ctrl-C to stop)
uv run mcp_server/server.py
```
Next, register the server in the MCP client (below) and ask:
*«Ask the council: …»*.
> The server reads `.env` from the project root, so after filling `.env`
> the token in the MCP client config does not need to be repeated.
---
## Registration in MCP Client
### Claude Code
With one command (user scope):
```bash
claude mcp add llm-council -- uv --directory /path/to/llm-council run mcp_server/server.py
```
…or manually in `~/.claude.json` / project `.mcp.json`:
```json
{
"mcpServers": {
"llm-council": {
"command": "uv",
"args": ["--directory", "/path/to/llm-council", "run", "mcp_server/server.py"]
}
}
}
```
### Claude Desktop
Add to `claude_desktop_config.json` (Settings → Developer → Edit Config):
```json
{
"mcpServers": {
"llm-council": {
"command": "uv",
"args": ["--directory", "/path/to/llm-council", "run", "mcp_server/server.py"]
}
}
}
```
If you don't want to use `.env`, pass the access directly in the config:
```json
{
"mcpServers": {
"llm-council": {
"command": "uv",
"args": ["--directory", "/path/to/llm-council", "run", "mcp_server/server.py"],
"env": { "OPENROUTER_API_KEY": "sk-or-v1-..." }
}
}
}
```
Replace `/path/to/llm-council` with your path to the cloned repository.
### Codex
Add to `~/.codex/config.toml`:
```toml
[mcp_servers.llm-council]
command = "uv"
args = ["--directory", "/path/to/llm-council", "run", "mcp_server/server.py"]
startup_timeout_sec = 60
tool_timeout_sec = 3600 # the council takes a long time — the default Codex timeout won't be enough
[mcp_servers.llm-council.env]
OPENROUTER_API_KEY = "sk-or-v1-..."
```
Without `tool_timeout_sec`, Codex will interrupt the call before the council has time to answer.
An alternative to increasing the timeout is not to block the call and poll the result via
`start_council_async` + `poll_council_task` (see "Tools").
### OpenCode
Add an entry to the `"mcp"` block of the file `~/.config/opencode/opencode.json`:
```json
{
"mcp": {
"llm-council": {
"type": "local",
"command": ["uv", "--directory", "/path/to/llm-council", "run", "mcp_server/server.py"],
"enabled": true,
"environment": { "OPENROUTER_API_KEY": "sk-or-v1-..." }
}
}
}
```
`command` in OpenCode is an array (executable + arguments), access is passed
through `environment`. If the repository has a filled `.env`, keys in `environment` can
be omitted.
---
## Tools
| Tool | Description |
|------|-------------|
| `ask_council` | Ask a question, get a synthesized answer from the chairman. Returns `conversation_id` — pass it back for a multi-turn dialogue. |
| `start_council_async` / `poll_council_task` | Run the council in the background and poll the result — for clients with a short call timeout (e.g., Codex). `start_council_async` returns `task_id`, then `poll_council_task(task_id)` every 15–30 seconds, until `status` becomes `done`/`error`. |
| `list_conversations` | List of saved council discussions |
| `get_conversation` | Full discussion flow by a specific `conversation_id` |
| `get_available_models` | List of council models and chairman in the current config |
| `get_council_metrics` | Process-local KPI (success rate, degradation, percentiles of latency by stages) in JSON |
### `ask_council` Parameters
- `mode="auto"` (by default) — the orchestrator chooses **quick**
(only the chairman for simple questions), **standard** (usual council of
3 stages) or **deep** (plus a round of criticism/edits). `thorough=true` —
an outdated alias for deep.
- `conversation_id="..."` — continue a previous dialogue (multi-turn).
- `clarify_when_unclear=true` — for an ambiguous first question, return one
clarifying question, without starting the entire council.
- `include_debug=true` — add diagnostics to the answer (stage timings,
number of fallen models).
- `bypass_cache=true` — skip the cache of first-turn answers.
Long `ask_council` calls send MCP progress + heartbeat, so the client sees
activity, not "hanging". If the client has a short tool call timeout
(like Codex by default) — either increase it in the config (`tool_timeout_sec`), or
use `start_council_async` + `poll_council_task` instead of `ask_council`.
---
## Configuration
| Variable | Required | Default | Purpose |
|----------|----------|---------|---------|
| `API_PROVIDER` | no | `openrouter` | the only registered provider |
| `OPENROUTER_API_KEY` | yes | — | OpenRouter key |
| `COUNCIL_TIMEOUT_SECONDS` | no | `3600` | time budget for the full council |
| `LLM_COUNCIL_ROOT` | no | repo root | root for SQLite storage `data/` |
The council models and chairman are specified in `backend/config.py`. The chairman must
be from a different provider family than each active council member — otherwise, the config
import fails at startup (fail-fast).
`API_PROVIDER` is resolved through a small provider registry
(`backend/providers/registry.py`), not through a hardcoded `if`: currently, only
`openrouter` is registered there. To add a new provider —
implement `backend/providers/<name>.py` according to the `Provider`
protocol (`backend/providers/base.py`: `build_request`, `parse_response`,
`resolve_auth`) and register the loader in `PROVIDER_REGISTRY`.
---
## Optional: Web UI
FastAPI backend + React frontend show the same council with a visual analysis
(model answers, raw rankings, confidence banners):
```bash
./start.sh # backend on :8001, frontend on :5173
```
---
## Based on
Fork of [karpathy/llm-council](https://github.com/karpathy/llm-council) with
added MCP server, multi-turn dialogues, quick/deep modes,
adaptive routing, and answer caching.
## License
[MIT](LICENSE)
