Content
# Token-Slim-MCP
**A local MCP server for reducing token consumption, compatible with all LLM clients (written in Rust)**
Claude Code / Codex CLI / Cursor / Claude Desktop / Gemini CLI and other MCP (Model Context Protocol) compatible clients can be connected. Once set up, it can be used to reduce token consumption for file reading, searching, and JSON processing.
- Dependencies: `serde`, `serde_json`, and `regex` only. No external communication, completely local.
- Standard implementation of MCP over stdio (newline-separated) JSON-RPC 2.0.
- Single binary size: approximately 1.7MB.
## How it works
Most token consumption in LLMs occurs when directly inserting file contents, search results, and API responses into the context. This server compresses them before insertion.
| Tool | Alternative | Reduction Content | Reduction Estimate |
|---|---|---|---|
| `read_slim` | Normal Read/cat | Remove comments, empty lines, and token cap (keep beginning and end) | 10〜60% |
| `read_slim` (mode=outline) | Full file read | Extract only function/class/header lines | **90%+** |
| `grep_slim` | grep / Search tool | Only `path:line number:matched line`, limit number of results, exclude vendor/binaries | Large |
| `dir_map` | ls -R / find | One-line-per-entry tree with limited depth and number of entries | Large |
| `json_slim` | Full JSON paste | Minify + depth limit + array sampling + long string truncation | 50〜99% |
| `text_slim` | Full log paste | Empty line/space compression, duplicate line aggregation | Depending on content |
| `token_count` | — | Estimate token count before pasting (±20%) | — |
All tools' responses will have a reduction record in the format of `[token-slim] ~4835→~238 tok (-95%)`.
## Installation
Rust ([rustup](https://rustup.rs/)) is required.
### Method A: cargo install (recommended, path-independent)
```bash
git clone https://github.com/tacyan/token-slim-mcp.git
cd token-slim-mcp
cargo install --path .
```
The binary will be located at `~/.cargo/bin/token-slim-mcp` (subsequent configuration examples use this path).
### Method B: Manual build
```bash
git clone https://github.com/tacyan/token-slim-mcp.git
cd token-slim-mcp
cargo build --release
# → <cloned location>/target/release/token-slim-mcp
```
For Method B, replace `~/.cargo/bin/token-slim-mcp` with the **absolute path** of `<cloned location>/target/release/token-slim-mcp` in subsequent configuration examples.
> **Note**: Some MCP clients may not expand `~`. If connection fails, use the full path (e.g., `/Users/<username>/.cargo/bin/token-slim-mcp`, Linux: `/home/<username>/...`) for specification. The path can be confirmed with `which token-slim-mcp`.
## Client configuration (effective once set)
### Claude Code
```bash
# Effective for all projects (user scope)
claude mcp add --scope user token-slim \
--env TOKEN_SLIM_MAX_TOKENS=4000 \
-- ~/.cargo/bin/token-slim-mcp
```
### Codex CLI (`~/.codex/config.toml`)
```toml
[mcp_servers.token-slim]
command = "~/.cargo/bin/token-slim-mcp"
[mcp_servers.token-slim.env]
TOKEN_SLIM_MAX_TOKENS = "4000"
```
### Cursor (`~/.cursor/mcp.json` or project `.cursor/mcp.json`)
```json
{
"mcpServers": {
"token-slim": {
"command": "~/.cargo/bin/token-slim-mcp",
"env": { "TOKEN_SLIM_MAX_TOKENS": "4000" }
}
}
}
```
### Claude Desktop
Configuration file locations:
- macOS: `~/Library/Application Support/Claude/claude_desktop_config.json`
- Windows: `%APPDATA%\Claude\claude_desktop_config.json`
Same `mcpServers` format as Cursor (Claude Desktop does not expand `~`, so full path recommended).
### Gemini CLI (`~/.gemini/settings.json`)
```json
{
"mcpServers": {
"token-slim": {
"command": "~/.cargo/bin/token-slim-mcp"
}
}
}
```
## Recommended settings for model usage (important)
MCP tools may not be prioritized over standard Read/Grep by default. Add the following to your project's `CLAUDE.md` or `AGENTS.md` to ensure consistent reduction:
```
For file reading, code searching, and directory confirmation, prioritize using token-slim MCP's
read_slim / grep_slim / dir_map. For large JSON, use json_slim, and for long logs, use text_slim.
```
## Environment variables (default settings)
| Variable | Default value | Meaning |
|---|---|---|
| `TOKEN_SLIM_MAX_TOKENS` | 4000 | Output token limit for read_slim / text_slim |
| `TOKEN_SLIM_GREP_MAX_RESULTS` | 50 | Result limit for grep_slim |
| `TOKEN_SLIM_DIR_MAX_ENTRIES` | 300 | Total entry limit for dir_map |
| `TOKEN_SLIM_JSON_MAX_DEPTH` | 6 | Depth limit for json_slim |
| `TOKEN_SLIM_JSON_MAX_ARRAY` | 20 | Array size limit for json_slim |
| `TOKEN_SLIM_JSON_MAX_STRING` | 200 | String length limit for json_slim |
Tool call with arguments (e.g., `max_tokens`) take priority over environment variables.
## Tool reference
### read_slim
```jsonc
{ "path": "src/main.rs", // Required
"mode": "slim", // slim (default) | outline | raw
"max_tokens": 4000, // Output limit (exceeding part will be snipped)
"offset": 100, "limit": 50, // Line range specification
"strip_comments": true } // Comment removal in slim mode
```
Supported languages (comment removal): Rust, JS/TS, Python, Go, Java, C/C++, C#, Swift, Kotlin, Ruby, Shell, SQL, Lua, HTML/XML, YAML, TOML, etc.
### grep_slim
```jsonc
{ "pattern": "fn \\w+_slim", // Required (Rust regex)
"path": ".", // Root (single file also possible)
"ext": "rs,toml", // Extension filter
"max_results": 50,
"ignore_case": false,
"literal": false } // true for literal matching
```
### dir_map
```jsonc
{ "path": ".", "depth": 3, "max_entries": 300 }
```
### json_slim
```jsonc
{ "json": "{...}", // or "path": "package-lock.json"
"max_depth": 6, "max_array": 20, "max_string": 200 }
```
### text_slim
```jsonc
{ "text": "long log...", "level": "normal" } // normal | aggressive
```
### token_count
```jsonc
{ "text": "..." } // or "path": "file"
```
## Notes
- Reduction is **lossy** and not suitable for compilation or execution purposes (dedicated for LLM context insertion).
- Token estimation uses a heuristic independent of tokenizer (ASCII 4 chars ≈ 1 token + non-ASCII 1 char ≈ 1 token) with ±20% error margin.
- When output limit is exceeded, the first 70% and last 20% are kept, with a snip marker in between.
## Development
```bash
cargo test # 11 unit tests + 1 MCP integration test
cargo build --release
```
Manual verification:
```bash
printf '%s\n' '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"t","version":"0"}}}' | ./target/release/token-slim-mcp
```
## License
MIT
