Umi-OCR-MCP

# Umi-OCR MCP Server Expose local OCR capabilities of [Umi-OCR v2](https://github.com/hiroi-sora/Umi-OCR) to AI Agents (Hermes, Claude Code, Codex, etc.) via the MCP protocol. Automatically start the Umi-OCR process without manual service startup. ## Directory Structure ``` Umi-OCR-MCP/ ├── server.py # MCP Server (core) ├── pyproject.toml # Dependency declaration (uv run automatically installs) ├── requirements.txt # pip dependency declaration (alternative) ├── config.yaml # Hermes config access template └── README.md ``` ## Prerequisites | Dependency | Description | |------|------| | Umi-OCR v2.1.5+ | Download from [umi-ocr.com](https://umi-ocr.com/en/download) **Paddle version (recommended)**, install and enable HTTP API (Settings -> Service -> Enable HTTP API, default port 1224) | | Python 3.11+ | Recommended to manage with uv | | uv | Package manager for automatic dependency installation with `uv run` | ## Quick Start ### 1. Confirm Umi-OCR Path Default path: `YOUR_UMI_OCR_PATH\Umi-OCR.exe` Specify with environment variable `UMI_OCR_EXE` if different. > Windows note: Ensure correct escaping in YAML and environment variables if the path contains Chinese characters/spaces/special characters. ### 2. Test MCP Service ```bash cd YOUR_PROJECT_PATH\Umi-OCR-MCP uv run server.py ``` The first run of `uv run` will automatically read `pyproject.toml`, create a temporary virtual environment, and install `mcp` and `requests` dependencies. ### 3. Integrate with Hermes Agent Merge the content of `config.yaml` into the `mcp_servers` section of Hermes's `config.yaml`: ```yaml mcp_servers: umi-ocr-mcp: command: uv args: - run - --directory - YOUR_PROJECT_PATH/Umi-OCR-MCP - YOUR_PROJECT_PATH/Umi-OCR-MCP/server.py env: UMI_OCR_URL: "http://127.0.0.1:1224/api/ocr" UMI_OCR_EXE: "YOUR_UMI_OCR_PATH\\Umi-OCR.exe" ``` > Key point: The `--directory` parameter tells uv where to find `pyproject.toml`, **cannot be omitted**. If omitted, uv will not find dependencies and report a ModuleNotFoundError. > Path format: Recommended to use forward slashes `D:/path/to/`. Backslashes in YAML need to be escaped as `D:\\path\\to\\`. ## Working Principle ``` AI Agent -> MCP stdio -> server.py 1. Detect port 1224 availability 2. If not available -> Automatically start Umi-OCR.exe (exponential backoff waiting, max 30s) 3. If available -> Call HTTP API for image recognition 4. OCR text -> Confidence filtering (>0.85) 5. Lightweight post-processing (common OCR typo correction) 6. Return pure text to Agent ``` ### Service Keepalive Automatically detect port 1224 on each call to `extract_text_umi_v2`. If the Umi-OCR process exits unexpectedly, it will be restarted automatically on the next call. ### Post-processing Rules Built-in regular expression replacement, correcting unambiguous OCR common errors (not affecting AI understanding): | Original | Corrected | |------|------| | packspace | backspace | | AMDV | AMD-V | | Windows102004 | Windows 10 2004 | | 打并 | 打开 | | 重新新 | 重新 | Post-processing only corrects **determinable** OCR noise, and results outside the rule scope are preserved for AI reading and understanding. ## Frequently Asked Questions ### ModuleNotFoundError: No module named 'requests' uv run uses an isolated environment, which cannot see global packages. **Solution**: The project already contains `pyproject.toml`. Ensure to start with `uv run --directory <project directory>`, and uv will automatically install dependencies. ### Umi-OCR Startup Timeout - Check if `UMI_OCR_EXE` path is correct - The first startup of Umi-OCR requires loading PaddleOCR models, which may take 15-30 seconds on slower machines - Enable "Startup automatically" or "Minimize to tray" in Umi-OCR settings to avoid waiting each time ### API Returns Error Code Umi-OCR v2 API format: ``` POST /api/ocr {"base64": "<base64 string>"} ``` Return: ```json {"code": 100, "data": [{"text":"...","score":0.99}], "msg":"success"} ``` - code=100: Success - code=300: Base64 decoding failure (passing an array instead of a string) - code=802: Missing base64 field ### Deployment on Other Computers 1. Install uv 2. Install Umi-OCR (download Paddle version from [umi-ocr.com](https://umi-ocr.com/en/download)) and enable HTTP API (port 1224) 3. Modify default paths in `config.yaml` and `server.py` 4. Confirm port 1224 is not occupied 5. The first `uv run` requires an internet connection to download dependencies automatically ## API Reference ### Tool Overview | Tool | Purpose | Category | Token Characteristics | |------|------|------|-----------| | `quick_ocr_status` | Simple service status | Check | Only ~5 character output | | `check_ocr_status` | Complete service status | Check | ~200 character output | | `extract_text_umi_v2` | Single image OCR | Core | Standard output | | `ocr_image_base64` | Base64 direct OCR | Core | No need to write file steps | | `ocr_batch` | Multi-image batch OCR | Batch | Process multiple images in one call | | `ocr_directory` | Directory scan batch OCR | Batch | No need to list and build a list | | `ocr_pdf_page` | PDF single-page direct OCR | PDF | No need to render and save file steps | ### quick_ocr_status Simple status check, suitable for high-frequency polling. ``` Parameters: None Return: "running" | "stopped" | "error: ..." ``` **Token comparison**: ~5 chars vs `check_ocr_status` ~200 chars, saving 97%. ### check_ocr_status Complete service status information. ``` Parameters: None Return: Service running status, listening address, API endpoint, executable file path ``` ### extract_text_umi_v2 OCR extracts local image text. Built-in paragraph merging and confidence filtering. ``` Parameters: file_path: str -- Absolute image path (required) is_handwritten: bool -- Whether handwritten notes, default False Return: str -- Recognized text or error message ``` ### ocr_image_base64 Directly extract text from Base64 encoded images, saving file writing steps. ``` Parameters: image_base64: str -- Base64 encoded string (including data URL prefix) is_handwritten: bool -- Whether handwritten notes, default False Return: str -- Recognized text or error message ``` ### ocr_batch Batch OCR multiple local images, returning all results in one call. ``` Parameters: file_paths: List[str] -- List of absolute image paths is_handwritten: bool -- Whether handwritten notes, default False Return: str -- Results separated by input order ``` ### ocr_directory Scan directory for all images and batch OCR. Recursive mode can handle subdirectories. ``` Parameters: directory_path: str -- Absolute directory path (required) extensions: str -- Comma-separated extensions, default "png,jpg,jpeg,bmp,webp" recursive: bool -- Whether recursive subdirectory, default False is_handwritten: bool -- Whether handwritten notes, default False confidence_threshold: float -- Confidence threshold, default 0.85 Return: str -- Compact format: [total] + file name + text ``` ### ocr_pdf_page Directly render PDF specified page as image and OCR. One-step process. Depends on PyMuPDF. ``` Parameters: pdf_path: str -- Absolute PDF file path (required) page_number: int -- Page number (1-based), default 1 is_handwritten: bool -- Whether handwritten notes, default False dpi: int -- Rendering resolution, default 200 confidence_threshold: float -- Confidence threshold, default 0.85 Return: str -- Recognized text or error message ``` ### Confidence Threshold Description All OCR tools internally use `confidence_threshold` to filter low-quality results. Use newly exposed parameters for direct control: | Scenario | Recommended Threshold | Description | |------|---------|------| | Clear print | 0.90+ | High precision, strict filtering | | Standard document | 0.85 (default) | Balanced precision and recall | | Scanned textbook | 0.70-0.80 | Paper quality varies, need higher tolerance | | Handwritten notes | 0.60-0.75 | Handwritten recognition rate naturally lower | ## MCP Fixed Prompts MCP protocol supports **Prompts** - predefined fixed instruction templates. Agents use the dedicated tool `get_prompt(name)` to retrieve and return standardized step-by-step workflow instructions. server.py has built-in 3 Prompts, covering the most common OCR scenarios. ### Calling in Hermes After restarting the MCP connection, Hermes will automatically register the `mcp__umi_ocr__get_prompt` tool. Calling method: ``` # List all available Prompts mcp__umi_ocr__list_prompts() # Retrieve specific Prompt mcp__umi_ocr__get_prompt(name="ocr-workflow-quick") ``` Prompts return **instruction text** (not execution results), and Agents read and call corresponding Tools to complete actual OCR. ### ocr-workflow-quick **Single-image quick OCR standard process.** | Step | Action | Tool | |------|------|------| | 1 | Confirm service online | `quick_ocr_status` | | 2 | Extract text | `extract_text_umi_v2(file_path)` | | 3 | Quality insufficient → Retry with lower threshold | `extract_text_umi_v2(..., confidence_threshold=0.65)` | Applicable: Screenshot, single worksheet photo, whiteboard photo. ### ocr-workflow-pdf **PDF page-by-page OCR standard process.** | Step | Action | Tool | |------|------|------| | 1 | Confirm service online | `quick_ocr_status` | | 2 | Try OCR on the first page | `ocr_pdf_page(pdf_path, page_number=1)` | | 3 | Text blurry → Increase DPI to 300 | `ocr_pdf_page(..., dpi=300)` | | 4 | Severe character loss → Lower threshold to 0.70 | `ocr_pdf_page(..., confidence_threshold=0.70)` | | 5 | Quality OK → Extract page by page | Loop `ocr_pdf_page(pdf_path, page_number=N)` | Applicable: Scanned高考真题 PDF, electronic teaching materials, thesis. ### ocr-workflow-batch **Batch OCR standard process for whole textbooks/worksheets.** | Step | Action | Tool | |------|------|------| | 1 | Confirm service online | `quick_ocr_status` | | 2 | Scan directory for all images | `ocr_directory(dir, recursive=true)` | | 3 | Check 2-3 results | Human or Agent judgment | | 4 | Individual failure → Retry separately | `extract_text_umi_v2(path, confidence_threshold=0.65)` | | 5 | Concatenate into complete document | Sort and merge by file name | Applicable: Scanned multi-page textbooks, multi-page worksheet collections.