mcp-relay

# MCP Relay  An intelligent proxy/router for Model Context Protocol (MCP) that enables Claude Desktop and other MCP clients to dynamically discover and use tools from multiple MCP servers while maintaining a single connection point. This system prevents context bloat by intelligently selecting only the most relevant tools for each task. ## 🎯 The Problem MCP clients like Claude Desktop must load all servers at startup and cannot dynamically add servers during conversations. When using multiple MCP servers: - **Exa server**: 7 search tools (web, research papers, Twitter, companies, etc.) - **Puppeteer**: Browser automation tools - **Context7**: Documentation search tools - **Desktop Commander**: 18+ desktop automation tools Loading all servers directly leads to: - 🚨 **Context bloat**: 100+ tools consuming most of the context window - 🔒 **Static configuration**: Cannot add servers without restarting Claude - 💸 **Increased costs**: More tokens consumed per request - 🎯 **Poor tool selection**: AI struggles to choose from too many options ## 💡 The Solution MCP Relay acts as an intelligent proxy that: 1. **Single Connection**: Claude Desktop connects only to MCP Relay 2. **Backend Management**: Maintains connections to multiple MCP servers 3. **Smart Discovery**: Finds relevant tools across all servers using semantic search 4. **Dynamic Provisioning**: Loads only needed tools within token budgets 5. **Transparent Routing**: Executes tools on appropriate backend servers **Example**: Instead of configuring 10 MCP servers with 100+ tools, configure just MCP Relay. Then dynamically discover and use only the 2-3 tools you need. ## 🚀 Features ### Core Capabilities - **Proxy Architecture**: Single MCP server that routes to multiple backend servers - **Dynamic Tool Discovery**: Find tools across all servers without manual configuration - **Semantic Search**: Natural language queries to find the right tools - **Smart Provisioning**: Load only relevant tools within token budgets - **Transparent Execution**: Route tool calls to appropriate backend servers - **Native MCP Server**: Direct integration with Claude Desktop via mcp-proxy - **Cross-Server Intelligence**: Unified view of tools from Puppeteer, Exa, Context7, etc. - **Token Optimization**: 90%+ reduction in context usage vs. loading all servers - **Zero Configuration**: Claude Desktop needs only MCP Relay configuration ### Production Features (v0.3.0) - **🔒 DNS Rebinding Protection**: Secure MCP endpoint with configurable host whitelist - **🔄 Connection Pooling**: Persistent stdio sessions for 10-100x performance improvement - **📊 Structured Logging**: JSON-formatted logs with request IDs and correlation - **⏱️ Three-Tier Timeouts**: Tool → Server → Global timeout hierarchy - **📝 TOML Configuration**: Human-friendly config format with comments - **🔁 Enhanced Error Handling**: Automatic retry with exponential backoff - **🏥 Health Monitoring**: Circuit breaker pattern with automatic recovery - **📈 Progress Tracking**: Real-time feedback for long-running operations ### Advanced Features (v0.4.0) - **💬 MCP Prompts Support**: 5 built-in proxy-aware prompts for guided workflows - `discover_and_execute` - End-to-end tool workflow guidance - `server_capabilities` - Comprehensive system overview - `semantic_tool_search` - Search with ranking explanations - `execute_with_fallback` - Resilient execution with alternatives - `budget_aware_discovery` - Tools within token/count constraints - Automatic backend prompt discovery and aggregation - See [Prompts Documentation](docs/PROMPTS.md) for details ### Admin Dashboard (v0.5.0) - `/admin/dashboard` serves a read-only HTML dashboard with live stats: connected servers, tool catalog, cache hit rate, and connection pool metrics. The page refreshes automatically by calling `/api/admin/summary`, so you always have quick visibility into relay health without external tooling. ## 📋 Prerequisites - Python 3.12+ - [uv](https://docs.astral.sh/uv/) package manager ## 🔧 Installation 1. Clone the repository: ```bash git clone https://github.com/yourusername/mcp-relay.git cd mcp-relay ``` 2. Create and activate virtual environment: ```bash uv venv source .venv/bin/activate # On Unix/macOS # .venv\Scripts\activate # On Windows ``` 3. Install dependencies: ```bash uv sync ``` 4. Install package in development mode: ```bash uv pip install -e . ``` ## 🏃 Running the Server ### As HTTP API Server ```bash # Start the server mcp-relay # Or with uvicorn for development uvicorn mcp_relay.main:app --reload ``` The server will run on `http://localhost:8000` API documentation available at: - Swagger UI: `http://localhost:8000/docs` - ReDoc: `http://localhost:8000/redoc` - MCP Endpoint: `http://localhost:8000/mcp` (streamable HTTP transport) ### As MCP Server (Recommended) MCP Relay is now a native MCP server that works directly with Claude Desktop: 1. **Start the server**: ```bash mcp-relay ``` 2. **Install mcp-proxy** (if not already installed): ```bash uv tool install mcp-proxy ``` 3. **Add to Claude Desktop**: Edit `~/Library/Application Support/Claude/claude_desktop_config.json`: ```json { "mcpServers": { "mcp-relay": { "command": "/Users/YOUR_USERNAME/.local/bin/mcp-proxy", "args": ["http://localhost:8000/mcp"] } } } ``` Note: Replace `YOUR_USERNAME` with your actual username. 4. **Restart Claude Desktop** You'll see MCP Relay's tools available in Claude, including: - `discover_tools` - Find relevant tools based on queries - `provision_tools` - Select tools within token budgets - `register_tool` - Add new tools to the registry - `list_mcp_servers` - View registered MCP servers - And more! See [MCP Native Usage Guide](docs/MCP_NATIVE_USAGE.md) for detailed instructions. ## 🔍 API Endpoints ### Tool Discovery ```bash POST /api/tools/discover ``` Discover relevant tools based on semantic search. **Request:** ```json { "query": "I need to perform calculations", "tags": ["math", "calculation"], "limit": 5 } ``` **Response:** ```json { "tools": [ { "tool_id": "calculator", "name": "Calculator", "description": "Perform mathematical calculations", "score": 0.95, "matched_tags": ["math", "calculation"], "estimated_tokens": 50 } ], "query_id": "uuid", "timestamp": "2024-01-01T00:00:00" } ``` ### Tool Provisioning ```bash POST /api/tools/provision ``` Select and format tools for LLM consumption with token budget enforcement. **Request:** ```json { "tool_ids": ["calculator", "web-search"], "max_tools": 3 } ``` **Response:** ```json { "tools": [ { "name": "Calculator", "description": "Perform mathematical calculations", "parameters": { "type": "object", "properties": {...} }, "token_count": 50 } ], "metadata": { "total_tokens": 150, "gating_applied": true } } ``` ## 🔄 How It Works 1. **Claude Desktop Configuration**: Configure only MCP Relay ```json { "mcpServers": { "mcp-relay": { "command": "mcp-proxy", "args": ["http://localhost:8000/mcp"] } } } ``` 2. **Backend Server Connection**: MCP Relay connects to multiple MCP servers ``` MCP Relay → puppeteer (browser tools) → exa (search tools) → context7 (documentation) → filesystem (file operations) ``` 3. **Natural Language Discovery**: "I need to search for research papers" ``` Claude → discover_tools → Semantic Search → Returns relevant tools ``` 4. **Real-time Tool Execution**: Tools are loaded on-demand ``` execute_tool("exa_research_paper_search", {...}) → Validates → Loads → Executes ``` No provisioning needed! Tools are dynamically loaded when you use them. ## 🎯 Usage Examples ### Running the Demo ```bash # Make sure the server is running first mcp-relay # In another terminal, run the interactive demo python demo.py ``` ### Manual Testing ```bash # Test the server endpoints python test_server.py ``` ### Example: Finding Math Tools ```python import httpx import asyncio async def find_math_tools(): async with httpx.AsyncClient() as client: response = await client.post( "http://localhost:8000/api/tools/discover", json={ "query": "I need to solve equations", "tags": ["math"], "limit": 3 } ) tools = response.json()["tools"] print(f"Found {len(tools)} relevant tools") for tool in tools: print(f"- {tool['name']}: {tool['score']:.3f}") asyncio.run(find_math_tools()) ``` ## 🧪 Testing ```bash # Run all tests pytest # Run with coverage pytest --cov=mcp_relay # Run specific test files pytest tests/test_discovery_service.py -v # Run integration tests pytest tests/test_integration.py -v ``` ## 📊 Architecture Details 1. **Tool Registration**: Tools are registered with metadata, tags, and token estimates 2. **Semantic Search**: User queries are embedded using sentence transformers 3. **Relevance Scoring**: Tools are scored based on: - Cosine similarity between query and tool embeddings - Tag matches (adds 0.2 boost per matching tag) 4. **Real-time Loading**: Tools are validated and loaded on-demand during execution 5. **MCP Formatting**: Selected tools are formatted according to MCP protocol ## 🔧 Configuration The system uses sensible defaults but can be configured: - **Max Tokens**: Default 2000 tokens per request - **Max Tools**: Default 10 tools per request - **Embedding Model**: `all-MiniLM-L6-v2` (384-dimensional embeddings) ### Security Configuration **DNS Rebinding Protection** (v0.3.0+): The `/mcp` endpoint is protected from DNS rebinding attacks by validating Host and Origin headers. - **Default**: Only localhost access (`localhost`, `127.0.0.1`, `::1`) - **Custom Whitelist**: Add trusted hosts/IPs via `ALLOWED_MCP_HOSTS` environment variable ```bash # Allow access from specific LAN IPs export ALLOWED_MCP_HOSTS="localhost,127.0.0.1,192.168.1.100,192.168.1.101" # Allow access from custom hostname export ALLOWED_MCP_HOSTS="localhost,127.0.0.1,myserver.local,app.internal" # Docker example docker run -p 8000:8000 \ -e ALLOWED_MCP_HOSTS="localhost,127.0.0.1,192.168.1.100" \ mcp-relay ``` **⚠️ Security Note**: Only add hosts you trust. The whitelist allows full access to the MCP endpoint and all connected backend servers. ### MCP Server Configuration File - All backend MCP servers are defined in `config/mcp-servers.toml`. When running in Docker, set `MCP_DATA_DIR` (default `/data`) and mount a single host directory; the server file automatically resolves to `$MCP_DATA_DIR/config/mcp-servers.toml`. - Override the location with the `MCP_SERVERS_PATH` environment variable only if you need a non-standard layout. - The same path is used by the `/api/mcp/add_server` endpoints and the MCP registry service, so updates persist automatically when the file lives on a volume. - Each entry can either launch a local process (via `command`/`args`/`env`) **or** point at an existing MCP endpoint via `url`. For remote servers, set `transport` (e.g., `"streamable-http"`) and optional `headers`: ```json { "docs-http": { "url": "http://localhost:8844/mcp", "transport": "streamable-http", "headers": { "Authorization": "Bearer example" } }, "legacy-sse": { "url": "http://localhost:9000/mcp", "transport": "sse" } } ``` When `url` is provided, MCP Relay connects to that remote MCP server directly (supporting both streamable HTTP and SSE transports) instead of spawning a local process. ## 🐳 Docker Usage Use Docker to ship the proxy with all dependencies (Python stack plus Node.js/npm for stdio MCP servers, and optionally CUDA-enabled PyTorch wheels) without polluting the host environment. ```bash # Build the image docker build -t mcp-relay . # Prepare persistent data directory (config/logs/storage live here) mkdir -p data/config data/logs data/storage cp config/mcp-servers.toml data/config/mcp-servers.toml # Run on CPU (single bind mount handles config, logs, and storage) docker run --rm -p 8000:8000 \ -v $(pwd)/data:/data \ -e MCP_DATA_DIR=/data \ mcp-relay # Enable GPU embeddings (requires NVIDIA Container Toolkit) docker run --rm -p 8000:8000 --gpus all \ -v $(pwd)/data:/data \ -e MCP_DATA_DIR=/data \ -e MCP_RELAY_ENCODER_DEVICE=cuda:0 \ mcp-relay ``` The container exposes FastAPI + MCP on port `8000`. Override any runtime option with additional `-e VAR=value` flags (e.g., `MCP_RELAY_SKIP_BOOTSTRAP=1` for smoke tests). The single `-v $(pwd)/data:/data` mount keeps configs, logs, and SQLite storage together; `/data/config/mcp-servers.toml` is updated automatically when you add/remove servers. ### Docker Compose `docker-compose.yml` wraps these defaults so you can bring the proxy up with one command: ```bash # Customize environment (optional) cat <<'EOF' > .env MCP_DATA_VOLUME=./data MCP_DATA_DIR=/data MCP_RELAY_ENCODER_DEVICE=cpu # set to cuda:0 after installing NVIDIA toolkit MCP_RELAY_SKIP_BOOTSTRAP=0 EOF docker compose --env-file .env up --build ``` The compose service maps port `8000` and mounts `${MCP_DATA_VOLUME}` into `/data`, so configs, logs, and storage files stay together. To run with GPU embeddings, install the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) and start Compose with `docker compose --env-file .env --build --gpus all up`, then set `MCP_RELAY_ENCODER_DEVICE=cuda:0` in your `.env`. ## 📖 Documentation - [Tool Discovery System](docs/DISCOVERY.md) - How semantic search and tags work together - [Adding New MCP Servers](docs/ADDING_SERVERS.md) - Step-by-step guide to integrate new servers - [AI Integration Guide](docs/AI_INTEGRATION.md) - How AI assistants can automatically add MCP servers - [Architecture Overview](ARCHITECTURE.md) - System design and component interactions ## 📁 Project Structure ``` mcp-relay/ ├── src/ │ └── mcp_relay/ │ ├── __init__.py │ ├── main.py # FastAPI application │ ├── api/ │ │ ├── models.py # Pydantic models │ │ ├── tools.py # Tool management endpoints │ │ └── mcp.py # MCP server endpoints │ ├── models/ │ │ └── tool.py # Domain models │ └── services/ │ ├── discovery.py # Semantic search │ ├── gating.py # Tool selection logic │ └── repository.py # Tool storage ├── tests/ │ ├── test_*.py # Test files │ └── test_integration.py # Integration tests ├── demo.py # Interactive demo ├── test_server.py # Manual testing script └── pyproject.toml # Project configuration ``` ## 🔌 Integration with MCP Servers ### Registering Tools from MCP Servers ```python # 1. Clear existing demo tools DELETE /api/tools/clear # 2. Register tools from your MCP servers POST /api/tools/register { "id": "exa_research_paper_search", "name": "research_paper_search", "description": "Search across 100M+ research papers with full text access", "tags": ["search", "research", "academic"], "estimated_tokens": 250, "server": "exa", "parameters": { "type": "object", "properties": { "query": {"type": "string"}, "numResults": {"type": "number", "default": 5} }, "required": ["query"] } } ``` ### Using with LLM Orchestration 1. **LLM receives user query**: "Find recent papers on quantum computing" 2. **Orchestrator queries mcp relay**: ``` POST /api/tools/discover {"query": "find research papers", "limit": 3} ``` 3. **System returns relevant tools**: Only research tools, not file editors 4. **Orchestrator provisions tools**: ``` POST /api/tools/provision {"tool_ids": ["exa_research_paper_search"], "max_tokens": 500} ``` 5. **LLM executes directly with MCP server**: Using the provisioned tool definition ### AI-Assisted Server Registration AI assistants can automatically add new MCP servers: ```python # User: "Add this Slack MCP server to mcp relay" # AI: Connects to server, discovers tools, and registers everything POST /api/mcp/ai/register-server { "server_name": "slack", "config": { "command": "npx", "args": ["@slack/mcp-server"], "env": {"SLACK_TOKEN": "xoxb-..."} }, "tools": [ // AI provides all discovered tools with metadata ] } # Result: Slack server + all tools registered and ready for use ``` ## 🧑‍💻 Development ### Code Quality ```bash # Format code black . # Run linter ruff check . --fix # Type checking mypy . # Run all checks black . && ruff check . --fix && mypy . && pytest ``` ### Adding New Tools Tools can be added to the repository in `services/repository.py`: ```python Tool( id="my-tool", name="My Tool", description="Description for semantic search", tags=["category", "function"], estimated_tokens=100, parameters={ "type": "object", "properties": {...}, "required": [...] } ) ``` ## 🤝 Contributing 1. Fork the repository 2. Create a feature branch (`git checkout -b feature/amazing-feature`) 3. Make your changes and add tests 4. Run quality checks (`black`, `ruff`, `mypy`, `pytest`) 5. Commit your changes (`git commit -m 'Add amazing feature'`) 6. Push to the branch (`git push origin feature/amazing-feature`) 7. Open a Pull Request ## 📄 License This project is licensed under the Apache License 2.0 - see [LICENSE](LICENSE) for details. ## 🙏 Acknowledgments - Built with [FastAPI](https://fastapi.tiangolo.com/) - Semantic search powered by [Sentence Transformers](https://www.sbert.net/) - MCP protocol integration via the official [MCP Python SDK](https://github.com/modelcontextprotocol/python-sdk) - Demo UI using [Rich](https://github.com/Textualize/rich)