Content
# 🧪 MCP Lab
A focused testing environment for MCP (Model Context Protocol) servers with multiple LLM providers. Compare how different models handle tool calling, track metrics, and analyze performance.
## Features
- **Multi-Provider Support**: Test with Groq, OpenAI, and Anthropic models
- **MCP Server Integration**: Connect to multiple MCP servers simultaneously
- **Smart Media Rendering**: Automatically embed plots, images, and iframes from tool outputs
- **Detailed Metrics**: Track tokens, latency, tool calls, and iterations
- **Debug Mode**: Toggle detailed tool execution visibility
- **Analytics Dashboard**: Built-in database viewer with filtering and export
- **SQLite Storage**: All conversations saved with full metadata for analysis
## Quick Start
### 1. Install Dependencies
```bash
pip install -r requirements.txt
```
### 2. Configure Environment
```bash
cp .env.example .env
# Edit .env with your API keys
```
Required: At least one LLM provider API key:
- `GROQ_API_KEY` - Get from [console.groq.com](https://console.groq.com)
- `OPENAI_API_KEY` - Get from [platform.openai.com](https://platform.openai.com)
- `ANTHROPIC_API_KEY` - Get from [console.anthropic.com](https://console.anthropic.com)
### 3. Configure MCP Servers
Edit `config/mcp_config.json`:
```json
{
"mcpServers": {
"rosbag_reader": {
"command": "/path_to_venv/bin/python",
"args": [
"/path_to_ws/mcp-rosbags/src/server.py"
],
}
}
}
```
### 5. Run the Application
```bash
python run.py
```
Open your browser to:
- **Main Lab**: http://localhost:8000
## Project Structure
```
mcp-lab/
├── app.py # FastAPI backend
├── config.py # Configuration
├── run.py # Application launcher
├── requirements.txt # Python dependencies
├── .env # Your API keys (create from .env.example)
├── mcp_lab.db # SQLite database (created automatically)
│
├── config/
│ └── mcp_config.json # MCP server configurations
│
├── services/
│ ├── llm_service.py # Multi-provider LLM management
│ ├── mcp_service.py # MCP server management
│ ├── mcp_tools_bridge.py # Tool conversion bridge
│ └── db_service.py # Database operations
│
├── handlers/
│ ├── chat_handler.py # Chat message processing
│ └── websocket_manager.py # WebSocket connections
│
└── frontend/
├── index.html # Main chat interface
├── analytics.html # Analytics dashboard
└── style.css # Styling
```
## Usage Guide
### Testing MCP Tools
1. **Start a conversation** - The AI will automatically discover available tools
2. **Toggle Debug Mode** - See detailed tool execution, arguments, and results
3. **Compare Providers** - Switch between providers/models to compare behavior
### Understanding Metrics
- **Tokens**: Approximate token count for the exchange
- **Latency**: Total time from request to response
- **Tools Called**: Number of MCP tool invocations
- **Iterations**: Number of reasoning cycles (simple agent in a loop)
### Media Rendering
The system automatically detects and renders:
- `file:///path/to/plot.html` → Embedded iframe
- Image files → Inline images
- Plotly/chart URLs → Embedded visualizations
### Analytics Dashboard
- View all conversations with filtering
- Compare provider/model performance
- Export data as JSON or CSV
- Track average metrics per provider
## Adding MCP Servers
Add to `config/mcp_config.json`:
```json
{
"mcpServers": {
"your-server": {
"command": "path-to-executable",
"args": ["arg1", "arg2"],
"env": {
"ENV_VAR": "value"
}
}
}
}
```
## Troubleshooting
### MCP Servers Not Connecting
- Verify MCP server packages are installed
- Check the command paths in `mcp_config.json`
- Look at console logs for specific error messages
### No Models Available
- Verify at least one API key is set in `.env`
- Check that the API key is valid
- Ensure you have access to the models
### Database Issues
- Delete `mcp_lab.db` to start fresh
- Check write permissions in the directory
### Debug Mode
Set `DEBUG=True` in `.env` for:
- Auto-reload on code changes
- Detailed logging
- Error stack traces
## License
Apache 2
## Acknowledgments
Built for testing MCP (Model Context Protocol) servers with various LLM providers. MCP is originally developed by Anthropic.
Connection Info
You Might Also Like
MarkItDown MCP
Converting files and office documents to Markdown.
Time
Obtaining current time information and converting time between different...
Filesystem
Model Context Protocol Servers
Sequential Thinking
Offers a structured approach to dynamic and reflective problem-solving,...
Git
Model Context Protocol Servers
Context 7
Context7 MCP Server -- Up-to-date code documentation for LLMs and AI code editors