offload-mcp

# offload-mcp [](https://opensource.org/licenses/MIT) [](https://www.npmjs.com/package/offload-mcp) **MCP server for offloading routine coding-assistant work to a cheaper model.** The default model chain uses Gemma because the models are useful, open, and fun to experiment with. Running them locally can be heavy on RAM, GPU, and setup; the Gemini API (key from Google AI Studio) makes them easy to use for small routine tasks at almost no cost. You can use any supported model ID. ## Install Get a free API key from <https://aistudio.google.com/apikey>. Choose one install method. ### Option 1: npx (recommended) `npx` downloads and runs `offload-mcp@latest` on demand. You do not need to install the package globally. Your MCP client runs this command whenever it starts the server. JSON-style MCP config: ```json { "mcpServers": { "offload-mcp": { "command": "npx", "args": ["offload-mcp@latest"], "env": { "GOOGLE_AI_API_KEY": "your_key" } } } } ``` TOML-style MCP config: ```toml [mcp_servers.offload-mcp] command = "npx" args = ["offload-mcp@latest"] env = { GOOGLE_AI_API_KEY = "your_key" } ``` To test that npm can resolve the package: ```bash npx offload-mcp@latest ``` That starts an MCP stdio server, so it will wait for an MCP client instead of printing a normal CLI screen. ### Option 2: global npm install Install once: ```bash npm install -g offload-mcp ``` Then use the binary directly in your MCP config. JSON-style MCP config: ```json { "mcpServers": { "offload-mcp": { "command": "offload-mcp", "env": { "GOOGLE_AI_API_KEY": "your_key" } } } } ``` TOML-style MCP config: ```toml [mcp_servers.offload-mcp] command = "offload-mcp" env = { GOOGLE_AI_API_KEY = "your_key" } ``` To update a global install later: ```bash npm update -g offload-mcp ``` ## Use Ask your assistant to offload routine work: ```text offload a commit message for the current diff offload this translation to Mexican Spanish: <text> use offload to summarize src/index.ts ``` For local diffs and files, `offload_source` is the important path because the MCP server reads the input directly: ```text offload_source(task="commit_message", source="git_diff") offload_source(task="pr_description", source="git_staged_diff") offload_source(task="code_summary", source="file", path="src/index.ts") ``` Footer example: ```text —— Offloaded via gemma-4-31b-it · 307 model tokens · ~1,420 primary input tokens avoided · [offload-mcp](https://github.com/peterhadorn/offload-mcp) ``` `model tokens` come from the API response. `primary input tokens avoided` is an estimate and only appears when using `offload_source`. ## Tasks ```text commit_message pr_description code_summary translate changelog_entry naming_suggestion classify extract_data code_review_single docstring subject_lines freeform ``` Use `freeform` for anything else: ```text offload(task="freeform", content="ECONNREFUSED 10.0.1.5:5432", prompt="Rewrite as a user-friendly error message. Output only the message.") ``` ## Status `offload_status` shows local usage counters: ```text Today: 47/14400 calls (0.3%), 28,500 model tokens processed Month: 312 calls over 8 days (avg 39/day), 187,400 model tokens processed Estimated primary input avoided: today ~12,800 tokens, month ~74,200 tokens Tasks today: commit_message: 18 docstring: 12 code_summary: 9 ``` Stats are stored locally at `~/.offload-mcp/usage.json` by default. Only counters are stored, not task content. ## Config | Env var | Default | Description | |---------|---------|-------------| | `GOOGLE_AI_API_KEY` | - | Required | | `OFFLOAD_MODEL` | `gemma-4-31b-it` | Preferred model | | `OFFLOAD_FALLBACK_MODELS` | `gemma-4-26b-a4b-it` | Comma-separated fallback models | | `OFFLOAD_TIMEOUT_MS` | `20000` | Per-model request timeout | | `OFFLOAD_RETRIES_PER_MODEL` | `1` | Attempts per model before falling back (1 = no retry) | | `OFFLOAD_RPD_LIMIT` | `14400` | Local daily call limit. Lower it if your Gemini API account has a stricter quota. | | `OFFLOAD_LOG_PATH` | `~/.offload-mcp/usage.json` | Local usage stats | By default, requests try `gemma-4-31b-it` first and fall back to `gemma-4-26b-a4b-it` on timeouts, rate limits, and transient server errors. Set `OFFLOAD_FALLBACK_MODELS=` to disable fallback. ## Data offload-mcp sends task content to the configured Gemini API model. Do not offload secrets, private customer data, proprietary code, or regulated data unless your policy allows it. `offload_source` with `source="file"` reads any file path the MCP server process can access. Treat the `path` and `cwd` parameters as trusted local input from your MCP client. MIT