Content
# qwen-vision-mcp
Let any **text model** (Claude, GLM, GPT, Gemini...) "look at images" on-demand through MCP. The backend uses the qwen visual model from Aliyun DashScope, called through the Anthropic-compatible interface.
## Why do we need this?
Text models (GLM-4, Claude Haiku, GPT-4o-mini, etc.) cannot directly understand images. However, many tasks (e.g., screenshot error reporting, UI debugging, chart analysis, document OCR) require visual understanding.
`qwen-vision-mcp` packages the qwen visual model into an MCP tool `look(image, question)`, allowing any MCP-compatible agent to call it on-demand - only when needed, with the main conversation still driven by your affordable text model.
## Features
- **🧠 Stateful multi-turn persistence**: Within the same session, qwen remembers all previous images and questions, supporting "take another look at that image."
- **🚆 Automatic trimming to prevent exceeding limits**: When a single request body exceeds the 6MB limit of DashScope, it automatically replaces earlier image blocks with text pointers (preserving text questions and assistant responses), ensuring long-term multi-image conversations do not crash.
- **🔌 Process isolation**: Each client session runs an independent MCP server process, with messages isolated.
- **📦 Minimal dependencies**: Only depends on `@modelcontextprotocol/sdk`; runs directly with Bun without compilation.
- **🔁 Clean rollback on failure**: Automatically removes the last user message if a network exception or HTTP error occurs, ensuring history remains clean.
## How it works
```
Text Model (GLM/Claude/GPT) ──MCP stdio──▶ qwen-vision-mcp server
│
▼
DashScope qwen visual model
(Anthropic compatible interface)
```
The main model decides "when to look at images and what questions to ask," while qwen only "takes a look and answers specific questions." Model behavior is guided by the tool `description`, which helps to analyze, focus, and ask follow-up questions.
## Installation
Requires [Bun](https://bun.sh/) runtime + DashScope API Key (obtained from [Aliyun Bailian Console](https://bailian.console.aliyun.com/)).
```bash
git clone <your-fork-url> qwen-vision-mcp
cd qwen-vision-mcp
bun install # Install @modelcontextprotocol/sdk
cp .env.example .env # Fill in QWEN_API_KEY
```
`.env` content:
```env
QWEN_BASE_URL=https://dashscope.aliyuncs.com/apps/anthropic/v1
QWEN_API_KEY=sk-your-key
QWEN_MODEL=qwen3.7-plus
QWEN_MAX_TOKENS=1024
QWEN_MAX_REQUEST_BYTES=5000000
```
## Integrating with MCP Clients
### Claude Code
In `~/.claude.json` (or project-level `.mcp.json`), add:
```json
{
"mcpServers": {
"qwen-vision": {
"command": "bun",
"args": ["run", "/absolute/path/to/qwen-vision-mcp/index.ts"],
"env": { "QWEN_API_KEY": "sk-..." }
}
}
}
```
### Cursor / Continue / Other MCP Clients
Any client supporting stdio MCP server can integrate, with similar configuration - specify `bun` as the command, `run /path/to/index.ts` as arguments, and put `QWEN_API_KEY` in the environment.
### Proma
In Proma workspace's `mcp.json`, add:
```json
"qwen-vision": {
"type": "stdio",
"command": "bun",
"args": ["run", "/absolute/path/to/qwen-vision-mcp/index.ts"],
"env": { "QWEN_API_KEY": "sk-...", "QWEN_MODEL": "qwen3.7-plus" },
"enabled": true
}
```
## Exposed Tools
### `look(image, question)` — Stateful
Let qwen look at an image and answer a specific question.
- **image**: Local absolute path / http(s) URL / `data:image;base64,...`
- **question**: Specific, focused question about the image
**Important**: This tool is **stateful**. Within the same session, qwen remembers all previous images and questions.
- Ask follow-up questions about the same image → Call continuously, qwen maintains context
- Switch to a completely unrelated new image → Call `reset_vision_session` first to clear history
### `reset_vision_session()` — Clear History
When to call: When the current image discussion is complete, switching to an unrelated new image, or restarting from a clean state.
When not to call: During multi-turn follow-ups on the same image or related image discussions.
## Configuration
| Environment Variable | Default | Description |
|---|---|---|
| `QWEN_BASE_URL` | `https://dashscope.aliyuncs.com/apps/anthropic/v1` | DashScope Anthropic compatible entry |
| `QWEN_API_KEY` | Required | DashScope API Key |
| `QWEN_MODEL` | `qwen3.7-plus` | Visual model ID (can be changed to `qwen-vl-max-latest`, etc.) |
| `QWEN_MAX_TOKENS` | `1024` | Single response token limit |
| `QWEN_MAX_REQUEST_BYTES` | `5000000` | Single request body byte limit (DashScope hard limit 6MB) |
| `QWEN_ENABLE_THINKING` | `false` | Thinking model switch. Default off (visual QA 7-10x speedup); set `true` for complex chart analysis/inference questions |
## Known Limitations
- **Process isolation ≠ persistent across restarts**: If the process is killed (client exit, crash), in-memory messages are lost = automatic reset. Persistence during the **active session** works fine; persistence across restarts is not supported.
- **URL images require DashScope server accessibility**: Local intranet image beds / blocked URLs will report `Failed to download multimodal content`; **local paths are most stable**.
- **Historical image trimming strategy**: When exceeding `QWEN_MAX_REQUEST_BYTES`, replace earlier image blocks with text pointers, preserving the last image and all text. This means qwen loses visual memory for "very old images" (but can still "read" previous descriptions from conversation history).
## Troubleshooting
| Error | Cause | Solution |
|---|---|---|
| `404` | Incorrect BASE_URL | Confirm using `/apps/anthropic/v1` instead of `/compatible-mode/v1` |
| `model not found` | Incorrect QWEN_MODEL ID | Check [DashScope documentation](https://help.aliyun.com/zh/model-studio/) for the correct ID |
| `Failed to download multimodal content` | URL image inaccessible to DashScope server | Use local path instead |
| `Exceeded limit on max bytes to request body : 6291456` | Request body exceeds 6MB | Automatically trimmed; if still errors, adjust `QWEN_MAX_REQUEST_BYTES` or manually call `reset_vision_session` |
| Slow response (10s+) | `qwen3.7-plus` is a thinking model, defaulting to generating hundreds of tokens per turn | Thinking is disabled by default; confirm `QWEN_ENABLE_THINKING=false` (or switch to non-thinking models like `qwen-vl-max-latest`) |
## Development and Testing
```bash
# Start server (stdio mode, waiting for client connection)
bun run index.ts
# Protocol layer smoke test (send initialize + tools/list)
( \
printf '%s\n' \
'{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}' \
'{"jsonrpc":"2.0","method":"notifications/initialized"}' \
'{"jsonrpc":"2.0","id":2,"method":"tools/list","params":{}}' ; \
sleep 2 \
) | bun run index.ts
# End-to-end test (look at a local image)
( \
printf '%s\n' \
'{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}' \
'{"jsonrpc":"2.0","method":"notifications/initialized"}' \
'{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"look","arguments":{"image":"/tmp/test.png","question":"Describe this image in one sentence"}}}' ; \
sleep 15 \
) | bun run index.ts
```
## License
[MIT](./LICENSE)
