Content
# clipboard-vision-mcp
[](https://linux.do/)
[中文](README.md) | [English](README.en.md)
Give your coding assistant or AI tool "eyes" - register this MCP service, and it can understand the screenshots in your clipboard. Automatically send images to OpenAI-compatible visual models and get text descriptions, OCR results, or analysis.
```text
LLM（without vision）──MCP/stdio──► clipboard-vision-mcp ──OpenAI-compatible API──► Visual Model ──► Text Result
```
## Quick Start
### 1. Prepare the Runtime Environment
Requires **Node.js ≥ 20**, and an **OpenAI-compatible API key** that can access visual models (can be filled in the MCP Host configuration below). The `OPENAI_*` variable names are used in accordance with OpenAI-compatible interface habits, but do not represent the official OpenAI model.
The simplest way is to directly use `npx -y clipboard-vision-mcp` in the MCP Host configuration without installing it globally in advance. If you want to install the command locally first, you can execute:
```bash
npm install -g clipboard-vision-mcp
```
To debug from the source code, clone the repository and run:
```bash
git clone <this-repo> image-recognition-mcp
cd image-recognition-mcp
npm install
npm run build
```
### 2. Prepare Clipboard Tools
The system needs to be able to read images from the clipboard:
- **macOS**: Install [pngpaste](https://github.com/jcsalterego/pngpaste) (`brew install pngpaste`)
- **Windows**: System built-in PowerShell, no additional installation required
- **Linux Wayland**: `wl-clipboard`'s `wl-paste`
- **Linux X11**: `xclip`
### 3. Configure MCP Host
In your MCP Host (ZCode, Claude Desktop, etc.) configuration file, find the `mcpServers` field and add the following content.
The following example uses Qwen / DashScope; if you use Gemini, LiteLLM, one-api, New API, or other OpenAI-compatible gateways, just replace `OPENAI_API_KEY`, `OPENAI_MODEL`, and `OPENAI_BASE_URL`.
```jsonc
{
"mcpServers": {
"clipboard-vision": {
"command": "npx",
"args": ["-y", "clipboard-vision-mcp"],
"env": {
"OPENAI_API_KEY": "your-api-key",
"OPENAI_MODEL": "qwen-vl-plus",
"OPENAI_BASE_URL": "https://dashscope.aliyuncs.com/compatible-mode/v1"
}
}
}
}
```
If you are using the official OpenAI interface, you can delete `OPENAI_BASE_URL` and change the model to a vision-supported OpenAI model, such as `gpt-4o-mini`.
The configuration path reference is: ZCode in `~/.zcode/v2/config.json`, Claude Desktop in `~/Library/Application Support/Claude/claude_desktop_config.json`.
If running from a local clone, use `"command": "node"`, and `args` points to the absolute path of `dist/index.js`.
### 4. Verification
Copy a screenshot to the clipboard and ask your AI assistant:
> Analyze the screenshot in my clipboard, what text is on it?
If the MCP Host has loaded the service correctly, the assistant will usually call `analyze_clipboard_image` or use the default clipboard input `recognize_image`, and then return the text content in the image.
## Features
- Four dedicated tools, adapted to different scenarios:
- `analyze_clipboard_image`: General screenshot analysis, suitable for "What's in this picture?"
- `extract_clipboard_text`: OCR-oriented, extracting text from screenshots, preserving line breaks and code formatting
- `diagnose_clipboard_error`: Debugging tool, analyzing error screenshots, stack traces, terminal output, or abnormal UI states
- `recognize_image`: The most flexible tool, supporting clipboard, local file, URL, data URL, and base64 images
- Four image sources: Clipboard (default), local file path, HTTP(S) URL, base64 / data URL
- Configurable OpenAI-compatible provider, model (default `gpt-4o-mini`), detail level, maximum token number, and timeout
- Send local, base64, and clipboard image format and size verification before sending
- Support closing local file input or limiting readable directories with allowlist
- stdio transport, compatible with any MCP Host
## Configuration Items
In addition to setting directly in the `env` of MCP Host, it also supports the `.env` file in the project root directory:
```bash
cp .env.example .env
```
The service will automatically load `.env` during startup, while retaining the environment variables passed by MCP Host.
| Environment Variable | Default Value | Description |
| --- | --- | --- |
| `OPENAI_API_KEY` | Required | OpenAI-compatible API key |
| `OPENAI_MODEL` | `gpt-4o-mini` | Visual model name |
| `OPENAI_BASE_URL` | OpenAI default address | OpenAI-compatible gateway address; not required when using official OpenAI |
| `OPENAI_TIMEOUT_MS` | `60000` | Request timeout (milliseconds) |
| `LOCAL_FILE_INPUT_ENABLED` | `true` | Set to `false` to disable local file path input |
| `LOCAL_FILE_ALLOWED_ROOTS` | Empty | Path allowlist, comma-separated, e.g., `/tmp,~/Pictures`; empty means allow all paths |
Provider examples:
```bash
# Qwen / DashScope OpenAI-compatible endpoint
OPENAI_API_KEY=...
OPENAI_MODEL=qwen-vl-plus
OPENAI_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1
# Gemini OpenAI-compatible endpoint
OPENAI_API_KEY=...
OPENAI_MODEL=gemini-2.5-flash
OPENAI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai
# LiteLLM or self-hosted OpenAI-compatible gateway
OPENAI_API_KEY=...
OPENAI_MODEL=your-vision-model
OPENAI_BASE_URL=http://localhost:4000/v1
# Official OpenAI interface
OPENAI_API_KEY=sk-...
OPENAI_MODEL=gpt-4o-mini
```
## Tool Description
### `analyze_clipboard_image`
| Parameter | Type | Required | Default Value | Description |
| --- | --- | --- | --- | --- |
| `prompt` | string | No | `Describe this image in detail, including any visible text (OCR).` | Question or instruction for the image |
| `detail` | `"auto"` \| `"low"` \| `"high"` | No | `auto` | Visual detail level, `low` is faster and more token-efficient |
| `maxTokens` | integer | No | `1024` | Maximum response token number |
Reads the current clipboard image.
### `extract_clipboard_text`
| Parameter | Type | Required | Default Value | Description |
| --- | --- | --- | --- | --- |
| `prompt` | string | No | `Extract all visible text from this image. Preserve line breaks and code formatting when possible.` | Question or instruction for the image |
| `detail` | `"auto"` \| `"low"` \| `"high"` | No | `auto` | Visual detail level, `low` is faster and more token-efficient |
| `maxTokens` | integer | No | `1024` | Maximum response token number |
Reads the current clipboard image, with a default prompt optimized for OCR scenarios.
### `diagnose_clipboard_error`
| Parameter | Type | Required | Default Value | Description |
| --- | --- | --- | --- | --- |
| `prompt` | string | No | `Analyze this screenshot for error messages, stack traces, terminal output, or UI failure states. Explain the likely cause and actionable next steps.` | Question or instruction for the image |
| `detail` | `"auto"` \| `"low"` \| `"high"` | No | `auto` | Visual detail level, `low` is faster and more token-efficient |
| `maxTokens` | integer | No | `1024` | Maximum response token number |
Reads the current clipboard image, with a default prompt optimized for error diagnosis.
### `recognize_image`
| Parameter | Type | Required | Default Value | Description |
| --- | --- | --- | --- | --- |
| `image` | string | No | `"clipboard"` | Path / URL / data URL / base64 / `"clipboard"` |
| `prompt` | string | No | `Describe this image in detail, including any visible text (OCR).` | Question or instruction for the image |
| `detail` | `"auto"` \| `"low"` \| `"high"` | No | `auto` | Visual detail level, `low` is faster and more token-efficient |
| `maxTokens` | integer | No | `1024` | Maximum response token number |
Returns `{ content: [{ type: "text", text: "..." }] }`, with `isError: true` and error message on failure.
Local files, data URLs, raw base64, and clipboard input must be PNG, JPEG, GIF, WebP, or BMP format, with a single file not exceeding 20 MiB. HTTP/HTTPS URLs are passed directly to the OpenAI-compatible API.
## Local Running
```bash
npm run dev # tsx runs directly without building
# or
npm run build && npm start
```
The service communicates with MCP messages through stdio - reading JSON-RPC from stdin and writing responses to stdout.
## Project Structure
```text
clipboard-vision-mcp/
├── package.json
├── tsconfig.json
├── .env.example
└── src/
├── index.ts # MCP server entry, registering tools and stdio transport
├── config.ts # Loading and validating env configuration
├── tools/
│ └── recognize.ts # Visual tool definition and handler
├── providers/
│ └── openai.ts # OpenAI-compatible visual call
└── inputs/
├── index.ts # resolveImage() dispatcher
├── types.ts
├── image.ts # Image MIME, size, and magic-byte verification
├── file.ts # Local path to base64 conversion
├── url.ts # HTTP(S) URL pass-through
├── base64.ts # base64 / data URL
└── clipboard.ts # macOS / Windows / Linux clipboard image reading
```
## License
MIT
