diffgate

**English** | [English](./README.en.md) <p align="center"> <img src="https://capsule-render.vercel.app/api?type=waving&color=gradient&customColorList=6,11,20&height=180&section=header&text=DiffGate&fontSize=58&fontColor=ffffff&fontAlignY=38&desc=Structured%20validation%20gate%20for%20Agent%20editing%20loops&descAlignY=62&descSize=14" alt="DiffGate banner" /> </p> <p align="center"> <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="MIT License" /></a> <img src="https://img.shields.io/badge/python-3.12%2B-blue.svg" alt="Python 3.12+" /> <img src="https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/SuperMarioYL/diffgate/main/bench/catch_rate.json" alt="Catch rate" /> <img src="https://img.shields.io/badge/languages-9-success" alt="9 languages" /> <img src="https://img.shields.io/badge/Agent-ready-7c3aed" alt="Agent ready" /> <img src="https://img.shields.io/badge/MCP-compatible-0ea5e9" alt="MCP compatible" /> </p> > **DiffGate is a structured validation gate for Agent editing loops — verify if the edit is correct, AST says.** ## Table of Contents - [Why DiffGate](#why-diffgate) - [Architecture](#architecture) - [Quick Start](#quick-start) - [Demo](#demo) - [How it works](#how-it-works) - [Comparison with similar projects](#comparison-with-similar-projects) - [Configuration](#configuration) - [Roadmap](#roadmap) - [Pricing](#pricing) - [License & Contribution](#license--contribution) - [Share this](#share-this) ## Why DiffGate Cursor, Claude Code, Codex, GPT-5.5, and other Agents often return `success` but with empty diffs, incorrect file modifications, or only comments promising changes. Tessl's 1281 tests show this is one of the most common failure types in large codebases — but **no Agent framework performs structured validation after the edit step**. DiffGate inserts a gate between Agent tool calls and the next loop: parsing ASTs before and after the edit, comparing them with the Agent's claimed `actions`, and returning `exit_code=1` if they don't match. This turns "silent lies" into "noisy errors". ## <img src="https://api.iconify.design/tabler:topology-star-3.svg?color=%230071E3&width=24" height="22" align="absmiddle" alt=""> Architecture <p align="center"> <picture> <source media="(prefers-color-scheme: dark)" srcset="./assets/atlas-dark.svg"> <source media="(prefers-color-scheme: light)" srcset="./assets/atlas-light.svg"> <img src="./assets/atlas-light.svg" width="880" alt="Coding Agent sends EditClaim (before/after blobs + claimed actions), DiffGate uses tree-sitter to parse both sides and align claims with AST diff, Verdict either passes (exit 0) or returns exit_code 1 as structured backpressure"> </picture> </p> Coding Agents (Cursor / Claude Code / Codex / LangGraph) send an **`EditClaim`** after each edit — before blob, after blob, and claimed actions. `cli.py` and `mcp_server.py` are thin shells around `verifier.verify(EditClaim) → Verdict`. The core uses tree-sitter to parse ASTs (Python / TypeScript / TSX / JavaScript / Go / Rust / Java / C++ / Ruby) and aligns each claim with the actual structured diff. **Verdict** either passes (exit 0) or returns `exit_code=1` as **structured backpressure**, forcing the Agent to retry. ## Quick Start ```bash pipx install diffgate # ≤30s diffgate verify --before X.py --after X.py.new --claim "rename foo→bar" diffgate mcp-server --stdio # Register with Claude Code / Cursor's mcp.json ``` Add the following to `~/.config/claude-code/mcp.json` (or Cursor's equivalent): ```json { "mcpServers": { "diffgate": { "command": "diffgate", "args": ["mcp-server", "--stdio"] } } } ``` See [`examples/claude_code_hook.md`](./examples/claude_code_hook.md) and [`examples/cursor_integration.md`](./examples/cursor_integration.md) for full hook and integration tutorials. ## <img src="https://api.iconify.design/tabler:photo.svg?color=%230071E3&width=24" height="22" align="absmiddle" alt=""> Demo <p align="center"> <img src="./assets/demo.gif" alt="diffgate verify terminal demo: Claude Code claims a rename but actually does nothing, DiffGate returns exit_code 1" width="820" /> </p> _{↑ Terminal recording (rendered by CI using <a href="https://github.com/charmbracelet/vhs">vhs</a> from <a href="./docs/demo.tape">docs/demo.tape</a>, generated automatically when tagging).} 60-second demo: Claude Code claims to rename `foo` to `bar` in `module_x.py` → DiffGate parses both ASTs → finds 0 renames → `exit_code=1` → Agent retries. ## How it works Three local processes, all offline: ``` [ coding agent ] ──tool_calls──► [ diffgate MCP server (python) ] │ ▼ [ verifier core ] ├── tree-sitter parsers (py/ts/tsx/js/go/rs/java/cpp/ruby) └── claim → ast_change matcher ``` The core data primitive is **`EditClaim`**: ```python EditClaim { before_blob: str after_blob: str claimed_actions: [ {kind: "rename"|"add"|"delete"|"move"|"signature_change", symbol: str, scope: str} ] } → Verdict { passed: bool, mismatches: [...], structural_diff: ast_summary } ``` `cli.py` and `mcp_server.py` are thin shells around `verifier.verify(edit_claim) → Verdict`. No daemons, databases, or network calls. ### Scope-aware validation — v0.2.0 The `scope` field in `claimed_actions` is now strictly validated. Claiming `add MyClass.helper` only passes if `helper` is actually added to `MyClass` — Agents can't confuse class methods with module-level functions. ```bash # Agent claims to add a method helper to class A, but actually adds a module-level function → exit_code 1 diffgate verify --before a.py --after a.py.new --claim "add helper in A" ``` ## Comparison with similar projects Honest comparison: | Dimension | DiffGate | [Aider](https://github.com/Aider-AI/aider) test-loop | [langgenius/dify](https://github.com/langgenius/dify) | [ChromeDevTools/chrome-devtools-mcp](https://github.com/ChromeDevTools/chrome-devtools-mcp) | | --------------------------------- | ------------------- | ---------------------------------------------------- | ----------------------------------------------------- | ------------------------------------------------------------------------------------------- | | Prevents "empty diff / wrong file" fake success | ✓ | — | — | — | | Behavioral correctness (test runs) | — | ✓ | partial | — | | Agent workflow orchestration | — | partial | **✓ (far surpasses DiffGate)** | — | | Browser / DevTools-side observation | — | — | — | **✓ (far surpasses DiffGate)** | | Cross-Agent / cross-IDE | ✓ (MCP protocol) | Aider-only | Dify-only | Chrome-only | | Deployment | single process / local | single process | multi-service / container | Chrome extension | DiffGate only solves one problem (structured lying), not replacing any of the above. Aider is still a better pair-programmer; Dify is still better at Agent orchestration; chrome-devtools-mcp is still unparalleled on the browser side. ## Configuration | Key | Type | Default | Meaning | | ---------------- | ------ | ------------------ | ---------------------------------------------------------- | | `languages` | list | `[py, ts, tsx, js, go, rs, java, cpp, ruby]` | Enabled tree-sitter parsers | | `strict_renames` | bool | `true` | Rename claims must affect all references; `false` only checks claimed locations | | `mcp.transport` | enum | `stdio` | `stdio` or `sse` | | `bench.traces` | path | built-in 200 traces | `diffgate bench` uses ground-truth JSONL | See `diffgate --help` for full configuration. ## Roadmap - [x] **m1 — `diffgate verify`**：CLI + Python/TS parsers + 20 silent-lie fixtures - [x] **m2 — `diffgate mcp-server`**：MCP tool `verify_edit`, works with Claude Code / Cursor out-of-the-box - [x] **m3 — `diffgate bench`**：Replay traces, output precision/recall - [x] **v0.2 — scope-aware validation**: Strict `scope` field matching, preventing class method vs. module function confusion - [x] **v0.3 — CLI/MCP alignment + multilanguage**: Structured `--claim-file` (including stdin), multi-file validation, added Java / C++ / Ruby parsers, and three silent-lie fixes - [ ] **DiffGate Cloud** (paid): Aggregated catch-rate, SSO, Prometheus exporter for teams - [ ] **Framework integration**: LangGraph / Mastra / Autogen official gate ## Pricing **Self-hosting is always free.** CLI, MCP server, tree-sitter parsers, and bench tool are all MIT-licensed, with no phone-home. **Paid product (v0.2) — DiffGate Cloud**: A hosted aggregation panel for internal Dev Platform teams at companies like Byte, Alibaba, Tencent, Meituan, and JD. Collects catch-rates across engineers and teams, with SSO, audit logs, and Prometheus exporter, prioritizing Java / C++ parsers. Pricing is approximately **¥1,200 / engineer / year** (tiered by capacity), about 1/3 of a Cursor Business seat — positioning it as a safety net for your paid Agent seats. Pilot process: 14-day free trial → Day 14 provides a team-wide catch-rate report + estimated "engineer hours saved" → annual contract (minimum ¥100k, scalable by seat). Stripe (USD) + Alipay / WeChat Pay (CNY) dual-channel settlement. If you're on the internal Dev Platform of one of these companies, email `leo.stack@outlook.com` to discuss a pilot. ## License & Contribution MIT, see [LICENSE](./LICENSE). Bugs, false positives, and false negatives are welcome as issues — just provide the `EditClaim` before/after blobs, and reproduction is cheap. PRs should open an issue to align on scope. ## Share this ``` DiffGate — Structured validation gate for Coding Agent editing loops. You've paid for Cursor / Claude Code seats, add a safety net, turning "claimed success" into exit-code 1. OSS + MCP. https://github.com/SuperMarioYL/diffgate ``` --- _{MIT © 2026 SuperMarioYL}