py-mcp-full

# py-mcp-full > PY Crawler Development Full-Stack Toolset — MacCMS Video Site Analysis + Intelligent Source Code Generation + Selector Debugging + Play Link Debugging + Interface Testing + Code Specification Evaluation + Full-File Operations Based on the [MCP (Model Context Protocol)](https://modelcontextprotocol.io/) protocol, providing **18 professional tools** for AI programming assistants, covering the entire process of video crawler development. ## ✨ Features - 🔍 **In-Depth Website Structure Analysis** — Automatic identification of MacCMS v8/v10/Collection API/non-standard architecture - 🤖 **Intelligent Source Code Generation** — One-click generation of T3/T4 crawler source code based on analysis results - 🎯 **Selector Debugging** — Real-time verification of XPath/CSS selector matching results - 🎬 **Play Link Debugging** — Recursive analysis of m3u8/mp4 direct links, supporting v8/v10 JS variable extraction - 🧪 **Interface Testing** — Automated testing of five major interfaces (home/category/detail/search/play) - 📋 **Specification Evaluation** — 12 weighted checks for crawler source code specifications - 📁 **Full-File Operations** — read/write/delete/mkdir/move/copy/info/cwd ## 🚀 Quick Start ### Installation ```bash git clone https://github.com/sunniu-hjdhnx/py-mcp-full.git cd py-mcp-full npm install ``` ### Configure MCP Client Add the following configuration to MCP clients like Claude Desktop, Cursor, or Cherry Studio: ```json { "mcpServers": { "py_mcp_full": { "command": "node", "args": ["/path/to/py-mcp-full/index.js"], "env": { "ROOT": "/path/to/your/spider/project" } } } } ``` > The `ROOT` environment variable specifies the root directory of the crawler project, and file operation tools are based on this path. ### Simplified Deployment (Optional) Use esbuild to package into a single file without node_modules: ```bash npm install -g esbuild esbuild index.js --bundle --platform=node --format=cjs --target=node22 --outfile=dist/index.cjs ``` Configure after packaging: ```json { "mcpServers": { "py_mcp_full": { "command": "node", "args": ["dist/index.cjs"], "env": { "ROOT": "/path/to/your/spider/project" } } } } ``` ## 🛠 Tool List (18) ### Crawler Development Tools (10) | Tool | Function | Required Parameters | |------|----------|--------------------| | `analyze_website` | In-depth analysis of website structure, identifying CMS type | `url` | | `create_spider_source` | Intelligent generation of crawler source code | `url` | | `debug_selector` | Verification of CSS selector matching results | `url`, `selector` | | `debug_play_link` | Recursive debugging of play link extraction | `url` | | `test_interface` | Testing of five major interfaces | `source_code`, `interface` | | `evaluate_source` | Evaluation of code specification compliance | `source_code` | | `fetch_url` | Fetching page content | `url` | | `edit_file` | Replacing file content | `path`, `search_text`, `replace_text` | | `find_in_file` | Searching file content | `path`, `keyword` | | `list_directory` | Listing directory content | `path` | ### File Operation Tools (8) | Tool | Function | Required Parameters | |------|----------|--------------------| | `read_file` | Reading file (supporting segmentation) | `path` | | `write_file` | Writing file (overwrite/append) | `path`, `content` | | `delete_file` | Deleting file or directory | `path` | | `create_directory` | Creating directory (supporting recursion) | `path` | | `move_file` | Moving/renaming | `source`, `destination` | | `copy_file` | Copying file | `source`, `destination` | | `file_info` | Getting file detailed information | `path` | | `get_cwd` | Getting current working directory | — | ## 📖 Usage Process ``` ① analyze_website(url="https://example.com") → Identify CMS type, obtain HTML structure parameters ② debug_selector(url="...", selector="//div[contains(@class,'item')]") → Verify selector correctness ③ create_spider_source(url="...", mode="T4") → Generate complete crawler source code ④ test_interface(source_code="...", interface="all") → Test five major interfaces for normality ⑤ evaluate_source(source_code="...") → Check specification compliance ``` ## 🏗 Project Structure ``` py-mcp-full/ ├── index.js # Main program (1409 lines) ├── package.json # Dependency configuration ├── manifest.json # Tool Schema definition └── 使用说明.md # Chinese detailed documentation ``` ## Repository Description This README is only for explaining basic project information and repository boundaries, for technical learning and exchange only, without usage guidance, access guidance, or resource description. ## Acknowledgements [hjdhnx's Homepage](https://github.com/hjdhnx) ## 📄 License MIT