Content
# Agentic Playwright MCP
Let AI Agent write Python scripts to control the browser's MCP Server framework.
Based on Playwright, supports optional [CloakBrowser](https://github.com/CloakHQ/CloakBrowser) anti-detection engine.
## Core Concept
**AI is not just calling tools, but writing Python scripts.** The more you use it, the smarter the system becomes.
```
User Intent → Skill Router (Keyword Fast Screening + LLM Fine Sorting) → Hit → Parameterized Script Generation → Script Engine Execution → Browser Operation
↳ If not hit → Experience Library Search → Generate Temporary Script → Sandbox Execution → Save Experience
↳ If rule fails → LLM Intent Parsing (Fallback) → Structured Intent → Generate Script
↳ If execution fails → Self-healing Mechanism → Visual Fallback → Record New Knowledge
```
## Four-Layer Architecture
```
Layer 0: Panel (User Interaction) ← panel/: inject.js + panel_manager.py
Layer 3: Domains (Site Experience) ← domains/*.yaml + workspace/knowledge/
Layer 2: Skills (Muscle Memory) ← controls.py + skill_library/
Layer 1: Helpers (Primitive) ← actions.py: goto/click/fill/screenshot
```
| Layer | Responsibility | Evolution Method |
|------|------|---------|
| **Layer 0** | Browser Inner Interaction Panel | Shadow DOM Injection, Script/MCP Bidirectional Control |
| **Layer 1** | Atomic Operation | Unchanged |
| **Layer 2** | Control Function | Extend New Function |
| **Layer 3** | Site Experience | Selector Self-healing + Knowledge Accumulation |
## Current Capability
| Scenario | Degree | Description |
|------|------|------|
| **Simple Task** (Search, Navigation, Screenshot) | ✅ Available | Directly Run Through |
| **Medium Task** (Login, Fill Form, Page Turning) | ⚠️ Limited | Have Template, Need to Adapt to Site |
| **Complex Task** (Multi-step, Cross-page) | ⚠️ Limited | Agent Loop Can Run, Limited Reasoning Ability |
**Adapted Sites**: Baidu, Sogou, Dangdang, Bilibili, Toutiao, CSDN, Encyclopedia, Weather, Weibo, Jujin, IT Home, Bird Tutorial, Open China
## Interaction Panel (Layer 0)
Automatically inject an interaction panel into the browser after launching, allowing users to communicate with the automation program through input box and button.
```
┌─────────────────────────┐
│ 🤖 Agentic Panel [—] │ ← Default Minimized, Click to Expand
├─────────────────────────┤
│ Input │
│ [________________] [Submit]│ ← User Input Data, Program Read through panel_read()
│ │
│ Log │
│ ┌─────────────────────┐ │
│ │ Searching... │ │ ← Program Write through panel_log()
│ │ Found 10 results │
│ └─────────────────────┘ │
│ │
│ Agentic Playwright │
└─────────────────────────┘
```
**Technical Features**:
- **Shadow DOM Isolation**: Panel style is not affected by host page
- **Keyboard Event Isolation**: Page JS cannot intercept panel input
- **Automatic Survival Protection**: Automatically rebuild after being removed by page
- **Cross-page Persistence**: Automatically take effect through `addInitScript` injection, navigation/refresh/new tab
**Three Operation Methods**:
| Method | Scenario | Example |
|------|------|------|
| Script Function | `run_script` / agent loop | `panel_log("Progress 50%")` |
| MCP Tool | Claude and other clients | `panel_prompt(question="Continue?")` |
| Python API | Project internal code | `get_panel_manager().log(page, "msg")` |
## Quick Start
```bash
# Clone
git clone https://github.com/zceeeeee/agentic-playwright-mcp.git
cd agentic-playwright-mcp
# Install
pip install -e .
playwright install chromium
# Start GUI
browser-agent gui --port 8081
```
Open browser and visit **http://localhost:8081**
## Docker Deployment
```bash
# Clone
git clone https://github.com/zceeeeee/agentic-playwright-mcp.git
cd agentic-playwright-mcp
# Build Image
docker compose build
# Start Service (Background Running)
docker compose up -d
# View Log
docker compose logs -f
```
Service started, MCP client can connect through `http://localhost:8000`
**Environment Variables**: Create `.env` file in project root directory to configure API Key:
```env
ANTHROPIC_API_KEY=sk-ant-your-key-here
OPENAI_API_KEY=sk-your-key-here
# LLM Intent Parsing Fallback (Optional, Automatically Called when Rule Fails)
# OPENAI_BASE_URL=https://api.openai.com/v1
# OPENAI_MODEL=gpt-4o-mini
```
**MCP Configuration (Claude Desktop)**:
```json
{
"mcpServers": {
"browser": {
"url": "http://localhost:8000/mcp"
}
}
}
```
**Stop Service**:
```bash
docker compose down
```
## Usage
### Web GUI
```bash
browser-agent gui --port 8081
```
### CLI
```bash
browser-agent setup # First-time configuration (interactive)
browser-agent serve # MCP Service
browser-agent run "Help me search for Python tutorials on Baidu" # Single Execution
browser-agent doctor # Check Environment
browser-agent gui # Start Web GUI
```
### Python SDK
```python
from src.sdk import AgentLoop
with AgentLoop(headless=True) as agent:
result = agent.run("Help me search for Python tutorials on Baidu")
print(result.output)
```
### MCP (Claude Desktop)
```json
{
"mcpServers": {
"browser": {
"command": "browser-agent",
"args": ["serve"]
}
}
}
```
## MCP Tool List (18)
| Tool | Description |
|------|------|
| `run_task` | Natural Language-driven Autonomous Agent Loop |
| `browse_skills` | Find skill library by keyword or URL |
| `get_skill` | Get skill source code and documentation |
| `run_script` | Execute Python script in restricted sandbox |
| `analyze_page` | Screenshot + Multimodal LLM analyze page |
| `browser_launch` | Launch Chromium browser |
| `browser_launch_with_domain` | Launch browser with site cookie |
| `auth_list` | List login status of all sites |
| `auth_save` | Save current site cookie |
| `auth_delete` | Delete site cookie |
| `screenshot` | Take screenshot of current page |
| `ping` | Health check |
| `panel_toggle` | Show/Hide interaction panel |
| `panel_read` | Read user input data and events |
| `panel_log` | Write log to panel |
| `panel_set_title` | Set panel title |
| `panel_prompt` | Ask user for input and wait for response |
| `panel_set_fields` | Dynamically update panel form fields |
## Script Engine Available Functions
```python
# Navigation
goto("https://example.com")
go_back()
# Element Operation (support multiple alternative selectors)
click("#button", ".fallback-btn")
fill("#input", "hello")
# Domain Configuration Drive (with self-healing)
smart_click("search_button", domain="baidu")
smart_fill("search_input", "Python Tutorial", domain="baidu")
# Combination Operation
smart_login("github", "user", "pass") # Save cookie automatically after login
smart_search("baidu", "Python Tutorial")
smart_fill_form("example", {"name": "Zhang San", "email": "test@test.com"})
# Cookie Persistence
save_cookies("baidu") # Manually save current site cookie
load_cookies("baidu") # Load saved cookie (rebuild context)
# JavaScript Execution
run_js('document.querySelector("#kw").value = "Python"')
# Wait
wait_for_navigation(timeout=10)
wait_for_element("#result", timeout=10)
wait(2.0)
# Page Information
url = get_url()
title = get_title()
text = get_text()
screenshot("page.png")
# Interaction Panel (Layer 0)
panel_log("Searching...") # Write log to panel
answer = panel_prompt("Please enter keyword:") # Ask user for input (blocking wait)
data = panel_read() # Read user input data
events = panel_read_events() # Read and clear event queue
panel_show() # Show panel
panel_hide() # Hide panel
panel_set_title("Task Progress") # Set panel title
panel_set_fields([ # Dynamically update form fields
{"name": "keyword", "label": "Keyword", "type": "text", "placeholder": "Enter search term"},
{"name": "action", "label": "Operation", "type": "select", "options": ["Search", "Cancel"]},
])
```
## Experience Evolution System
Reference Browser Harness's `agent-workspace` mode:
```
workspace/
├── scripts/ # Successful scripts (automatically saved and reused)
├── selectors/ # Selector experience (success/failure records)
└── knowledge/ # Site knowledge (gotchas + patterns)
```
- **Script Reuse**: Automatically reuse saved script for same task second time
- **Selector Experience**: Record success/failure times of each selector, sorted by reliability
- **Site Knowledge**: Record special behavior and precautions of each site
## LLM Client
Unified AI model call interface, support OpenAI compatible API and Anthropic Claude.
```python
from src.core.llm_client import get_llm_client
client = get_llm_client()
# Interface 1: Free text conversation
reply = client.chat("Explain what Playwright is in one sentence.")
# Interface 2: Structured JSON output
result = client.chat_json(
"User said: Help me search for Python tutorials on Baidu",
system_prompt="Extract site and keyword",
schema={"site": "string", "keyword": "string"},
)
# → {"site": "baidu", "keyword": "Python Tutorial"}
```
**Provider Configuration** (`.env`):
```env
# Switch provider: "openai" (default) | "anthropic"
LLM_PROVIDER=openai
# OpenAI compatible API
OPENAI_API_KEY=sk-your-key
OPENAI_BASE_URL=https://api.openai.com/v1
OPENAI_MODEL=gpt-4o-mini
# Anthropic
ANTHROPIC_API_KEY=sk-ant-your-key
ANTHROPIC_MODEL=claude-haiku-4-5-20251001
# General parameters
LLM_TEMPERATURE=0
LLM_MAX_TOKENS=1024
LLM_TIMEOUT=30
```
Launch GUI will automatically pop up guide interface if no API Key is configured.
## CloakBrowser Anti-detection Engine
```bash
pip install -e ".[stealth]"
USE_CLOAKBROWSER=true browser-agent gui
```
| Detection Service | Playwright | CloakBrowser |
|---------|-----------|-------------|
| reCAPTCHA v3 | 0.1 (bot) | **0.9** (human) |
| Cloudflare Turnstile | FAIL | **PASS** |
| FingerprintJS | DETECTED | **PASS** |
## Cookie Persistence
Support save and restore login status (cookie + localStorage) by site.
```bash
# Automatically load saved cookie when launching
browser_launch_with_domain("baidu")
# Save cookie automatically after login
smart_login("github", "user", "pass")
# Manual management
auth_list # View login status of all sites
auth_save("baidu") # Manually save
auth_delete("baidu") # Delete
```
**Storage Location**: `~/.agentic-playwright/auth/{domain}.json`
**Automatic Adaptation**: Add `domains/*.yaml` site, automatically support corresponding cookie management, no extra configuration.
## Project Structure
```
agentic-playwright-mcp/
├── src/
│ ├── server.py # MCP entry (18 tools)
│ ├── cli.py # CLI (serve/run/doctor/gui)
│ ├── sdk.py # Python SDK
│ ├── core/
│ │ ├── agent_loop.py # Agent loop engine
│ │ ├── skill_router.py # Skill router (keyword fast screening + LLM fine sorting)
│ │ ├── llm_client.py # LLM client (chat / chat_json dual interface)
│ │ ├── auth_manager.py # Cookie persistence management
│ │ ├── script_engine.py # Script execution engine (inject panel function)
│ │ ├── script_generator.py # Task intent parsing (rules)
│ │ ├── intent_parser.py # LLM intent parsing (fallback)
│ │ ├── experience.py # Experience evolution system
│ │ ├── browser_manager.py # Dual-engine browser management (automatic panel injection)
│ │ ├── event_bus.py # Event hook system
│ │ ├── recovery.py # Error recovery
│ │ └── vision.py # Vision module
│ ├── panel/ # Layer 0: Interaction panel
│ │ ├── inject.js # Shadow DOM panel (injected into browser)
│ │ └── panel_manager.py # Panel manager (Python side)
│ ├── gui/app.py # Web GUI
│ ├── layer_1/actions.py # Atomic operation
│ ├── layer_2/controls.py # Advanced control function
│ ├── layer_3/ # Domain configuration + self-healing
│ └── skill_library/ # Standard script library
├── domains/ # Site selector configuration (19)
├── workspace/ # Experience storage
├── tests/ # 685 tests
├── docs/ # MkDocs documentation
├── examples/ # Example scripts
└── Makefile
```
## Development
```bash
make dev # Install dependencies
make test # Run tests (570)
make lint # Code check
make format # Automatic fix
make docs # Start documentation server
```
## License
MIT
MCP Config
Below is the configuration for this MCP Server. You can copy it directly to Cursor or other MCP clients.
mcp.json
Connection Info
You Might Also Like
No recommendations available