Content
# Beyond MCP
> It's time to push beyond MCP Servers... Right?
>
> Let's breakdown real engineering trade offs between MCP, CLI, File System Scripts, and Skills based approaches for building reusable toolsets for your AI Agents.
>
> Watch the full video breakdown here: [Beyond MCP](https://youtu.be/OIKTsVjTVJE)
## Purpose of this Repo
- MCP Servers are the standard way to build reusable toolsets for your AI Agents. But they are not the only way.
- MCP Servers come with a massive cost - **instant context loss**.
- When you have a single, or a few MCP Servers, this is not a big deal. But as you scale to many agents, many tools, and many contexts - this cost quickly becomes a bottleneck.
- So what are the alternatives that big players are using to build powerful, reusable, context preserving toolsets for their AI Agents?
_Here we explore 4 concrete approaches in this repo, all implementing access to Kalshi prediction market data._
## The 4 Approaches
### `apps/1_mcp_server/` - MCP Server
### `apps/2_cli/` - CLI
### `apps/3_file_system_scripts/` - File System Scripts
### `apps/4_skill/` - Skill
## Quick Start
### 1. MCP Server
```bash
cp .mcp.testing .mcp.json
claude --mcp-config .mcp.json
prompt: "kalshi: get exchange status"
```
### 2. CLI
```bash
# or by agent
claude
prompt: "/prime_kalshi_cli_tools"
prompt: "kalshi: Get exchange status"
prompt: "kalshi: List events"
prompt: "kalshi: List events in JSON"
prompt: "kalshi: List events in JSON, limit 100"
# or by hand
cd apps/2_cli
uv sync
uv run kalshi status
uv run kalshi events
uv run kalshi events --json
uv run kalshi events --json --limit 100
```
### 3. File System Scripts
```bash
# by agent
claude
prompt: "/prime_file_system_scripts"
prompt: "kalshi: Get exchange status"
prompt: "kalshi: List events"
...
# or by hand
cd apps/3_file_system_scripts/scripts
uv run status.py
uv run *.py
```
### 4. Skill
```bash
cd apps/4_skill/
claude
prompt: "kalshi markets: Get exchange status"
prompt: "kalshi markets: search for events about 'best ai'" # Note this will trigger the cache build on first run which will take several minutes
...
```
## The 4 Approaches In Detail
- `apps/1_mcp_server/` - MCP Server
- `apps/2_cli/` - CLI
- `apps/3_file_system_scripts/` - File System Scripts
- `apps/4_skill/` - Skill
### 1. MCP Server (`apps/1_mcp_server/`)
**Classic Model Context Protocol implementation**
- ✅ **Standardized integration** - Works with any MCP-compatible client
- ✅ **Tool discovery** - Auto-exposes 15 tools to LLMs
- ✅ **Clean abstractions** - MCP protocol handles complexity
- ❌ **Instant context loss** - Every tool call loses conversational context
- ❌ **Wrapper overhead** - Delegates to CLI via subprocess
**Architecture:**
```
Claude/LLM → MCP Protocol → MCP Server → subprocess → CLI → Kalshi API
```
**Key files:**
- `server.py` - FastMCP server with 15 tool definitions
- Wraps CLI commands in MCP tool interface
- Each tool call is stateless
**When to use:** Building tools for multiple LLM clients, need standardized protocol, context loss is acceptable.
---
### 2. CLI (`apps/2_cli/`)
**Direct HTTP API access via command-line interface**
- ✅ **Single source of truth** - Direct API calls, no wrappers
- ✅ **Dual output modes** - Human-readable or pure JSON
- ✅ **Smart caching** - Pandas-based search with 6-hour TTL
- ✅ **Minimal overhead** - Direct httpx calls, no SDK
- ✅ **Improved Context** - Agent reads ~half as much context as the MCP Server
**Architecture:**
```
Claude → subprocess → CLI (13 commands) → Direct HTTP → Kalshi API
```
**Key files:**
- `kalshi_cli/cli.py` - All 13 commands (552 lines)
- `kalshi_cli/modules/client.py` - HTTP client & search cache
- `kalshi_cli/modules/formatting.py` - Output formatters
**When to use:** Need direct API control, want both CLI and programmatic access, caching important, okay with subprocess overhead.
---
### 3. File System Scripts (`apps/3_file_system_scripts/`)
**Progressive disclosure via standalone scripts**
- ✅ **Progressive disclosure** - Only load scripts you need (~200-300 lines each)
- ✅ **Complete isolation** - Each script is fully self-contained
- ✅ **Zero dependencies** - HTTP client embedded in each script
- ✅ **Context efficient** - Agent only reads relevant scripts
- ⚠️ **Code duplication** - HTTP client repeated in each script
- ⚠️ **No shared state** - Cache and utilities duplicated
**Architecture:**
```
Claude → Read tool → Individual script → Embedded HTTP client → Kalshi API
```
**Available scripts (10):**
- `status.py` - Exchange operational status
- `markets.py` - Browse markets with filters
- `market.py` - Detailed market information
- `orderbook.py` - Bid/ask depth
- `trades.py` - Recent trading activity
- `search.py` - Keyword search (with caching)
- `events.py` - List event collections
- `event.py` - Event details
- `series_list.py` - Browse all ~6900 series
- `series.py` - Series information
**When to use:** Context preservation critical, want progressive disclosure, okay with code duplication, need standalone portability.
---
### 4. Skill (`apps/4_skill/.claude/skills/kalshi-markets/`)
**Claude Code Agent Skills with embedded scripts**
- ✅ **Model-invoked** - Claude autonomously decides when to use
- ✅ **Progressive disclosure** - Same scripts as approach #3
- ✅ **Team sharing** - Commit to git for team access
- ✅ **Discovery** - Description triggers automatic activation
- ✅ **Context preservation** - Agent reads only what's needed
- ⚠️ **Claude Code specific** - Only works in Claude Code
- ⚠️ **Learning curve** - Requires understanding Skill system
**Architecture:**
```
Claude (detects trigger) → Loads SKILL.md → Runs scripts → Kalshi API
```
**Structure:**
```
.claude/skills/kalshi-markets/
├── SKILL.md (concise description & instructions)
└── scripts/ (copies of all 10 file system scripts)
```
**When to use:** Using Claude Code, want automatic skill discovery, team collaboration via git, need context preservation with progressive disclosure.
---
## My Approach ([IndyDevDan](https://www.youtube.com/@indydevdan))
### External Tools
1. 80% Just use **MCP servers**. Don't overthink it.
2. 15% **CLI** - If you need modify, extend, or control tools and context.
3. 5% **Scripts or Skills** - For serious context preservation, portability or ecosystem reuse
### New Tools
1. 80% Just use **CLI** + Prime Prompt (works for you, your team and your agents).
2. 10% Wrap in **MCP Server** when I need multiple agents at scale - and don't want to add 'another' thing for my agents to focus on.
3. 10% **Scripts or Skills** - For serious context preservation, portability or ecosystem reuse.
## Key Technical Details
**API Access:**
- Base URL: `https://api.elections.kalshi.com/trade-api/v2`
- No authentication required (read-only public data)
- ~6900 market series available
**Search Caching:**
The Kalshi API doesn't provide a native search endpoint, which creates a challenge for finding markets by keyword. Our solution: intelligent local caching.
- **The Problem:** No API search endpoint means we'd need to paginate through thousands of markets on every search
- **The Solution:** Build a complete local cache once, then search instantly using pandas
- **First run:** 2-5 minutes to fetch all ~6900 markets and build cache
- **Subsequent searches:** Instant (searches cached pandas DataFrame)
- **Cache location:** `.kalshi_cache/` at project root (shared across CLI and scripts)
- **TTL:** 6 hours (auto-refresh when stale)
- **Search scope:** Searches titles, subtitles, tickers, series names, and descriptions
**Why the delay matters:**
- First search in a session will take 2-5 minutes while the cache builds
- Users will see progress messages during cache building
- After initial build, searches are instant for 6 hours
- This trade-off enables comprehensive keyword search across ALL markets instead of just the first 100-500 results from paginated API calls
**Path Resolution:**
- All scripts use absolute path resolution via `Path(__file__).resolve()`
- Works correctly when invoked from any directory
- Cache always resolves to project root
## Trade-off Comparison
| | MCP | CLI | Scripts | Skills |
| ------------------------------ | -------------------------------------------------------------------------------- | ------------------------------ | ------------------------------ | --------------------------------- |
| **Agent Invoked** | Yes | No | No | Yes |
| **Context Window Consumption** | High | Medium (Depends) | Low (w/incr) | Low (w/incr) |
| **Customizable** | No (unless you own) | Yes | Yes | Yes |
| **Portability** | Low | Medium | High | High |
| **Composability** | Yes (MCP Prompts) | Yes but requires local prompts | Yes but requires local prompts | Yes but requires local prompts |
| **Simplicity** | High | Medium | Medium | Medium |
| **Engineering Investment** | Low if external, Medium if custom | Medium | Medium | Low if external, Medium if custom |
| **Feature Set** | Tools, Resources, Prompts, Elicitation, Completion, Sampling, Logging, Auth. etc | Whatever you build | Whatever you build | Whatever you build |
### Key Insights
**Context Window Consumption:**
- **MCP & CLI** consume full context on every tool call
- **Scripts & Skills** use progressive disclosure - only load what's needed
**Agent Invoked:**
- **MCP & Skills** are automatically triggered by Claude based on context
- **CLI & Scripts** require explicit agent decision to use
**Customizable:**
- **MCP** is locked unless you own/fork the server
- **CLI, Scripts, Skills** are fully under your control
**Portability:**
- **Scripts & Skills** are most portable (just Python files)
- **CLI** requires installation but works anywhere
- **MCP** needs MCP-compatible client setup
## When to Use Each Approach
### Choose MCP Server if:
- Building for multiple LLM clients (not just Claude)
- Need standardized tool protocol
- Context loss per call is acceptable
- Want automatic tool discovery across clients
- Using external MCP servers you don't control
### Choose CLI if:
- Need both human CLI and programmatic access
- Want single source of truth for API logic
- Direct HTTP control is important
- Willing to accept subprocess overhead
- Building general-purpose tooling
### Choose File System Scripts if:
- Context preservation is critical
- Want maximum portability (just Python + httpx)
- Need progressive disclosure (minimize token usage)
- Okay with code duplication for isolation
- Building one-off integrations
### Choose Skill if:
- Using Claude Code (and the ecosystem) specifically
- Want autonomous skill discovery
- Team collaboration via git is important
- Need context preservation + progressive disclosure
- Building reusable team capabilities
## Project Structure
```
beyond-mcp/
├── apps/
│ ├── 1_mcp_server/ # MCP Server implementation
│ │ ├── server.py # 15 MCP tools wrapping CLI
│ │ └── README.md
│ ├── 2_cli/ # CLI implementation
│ │ ├── kalshi_cli/
│ │ │ ├── cli.py # 13 commands (552 lines)
│ │ │ └── modules/ # HTTP client, cache, formatters
│ │ └── README.md
│ ├── 3_file_system_scripts/ # Progressive disclosure scripts
│ │ ├── scripts/ # 10 standalone scripts
│ │ │ ├── status.py
│ │ │ ├── markets.py
│ │ │ ├── market.py
│ │ │ ├── orderbook.py
│ │ │ ├── trades.py
│ │ │ ├── search.py
│ │ │ ├── events.py
│ │ │ ├── event.py
│ │ │ ├── series_list.py
│ │ │ └── series.py
│ │ └── README.md
│ └── 4_skill/ # Claude Code Skill
│ └── .claude/skills/kalshi-markets/
│ ├── SKILL.md # Skill description & instructions
│ └── scripts/ # Same 10 scripts as #3
└── .kalshi_cache/ # Shared cache directory (CLI & scripts)
```
## Resources
- [MCP Python SDK](https://github.com/modelcontextprotocol/python-sdk)
- [Claude Code Skills Documentation](https://docs.claude.com/en/docs/agents-and-tools/agent-skills)
- [Kalshi API Documentation](https://docs.kalshi.com/)
- [FastMCP Framework](https://github.com/jlowin/fastmcp)
## Master **Agentic Coding**
> Prepare for the future of software engineering
Learn tactical agentic coding patterns with [Tactical Agentic Coding](https://agenticengineer.com/tactical-agentic-coding?y=byndmcp).
Follow the [IndyDevDan YouTube channel](https://www.youtube.com/@indydevdan) to improve your agentic coding advantage.
