ShipDiff

<div align="center"> # shipdiff ### API change intelligence for coding agents **Diff what shipped. Know what broke. Fix it automatically.** [](https://www.npmjs.com/package/@shipdiff/cli) [](#roadmap) [](#requirements) [](#) [](#) [](#testing) [](#license) </div> --- Every docs and context product serves the **present tense**: *"here is how Stripe works today."* None can answer *"what is different between v8 and v12, and what will break"* — because that is a **diff**, and a diff does not live in documentation. **shipdiff derives that diff from the shipped package artifacts** — the versioned SDKs every provider publishes to npm and PyPI — so it works for *any* package, with or without the vendor's cooperation. It gives coding agents (and CI) the three things retrieval can't: 1. **Diffs, not snapshots** — what changed between two versions, and what breaks. 2. **The long tail** — every package, not just the top 20 with polished docs. 3. **Negative knowledge** — what *stopped* existing. `charges.create` lives in an LLM's weights forever; shipdiff can assert it was removed in v12 and point to the replacement. ## Contents - [Why shipdiff](#why-shipdiff) - [Features](#features) - [How it works](#how-it-works) - [Requirements](#requirements) - [Installation](#installation) - [Quick start](#quick-start) - [Use with a coding agent (MCP)](#use-with-a-coding-agent-mcp) - [Programmatic API (SDK)](#programmatic-api-sdk) - [Tool reference](#tool-reference) - [CLI reference](#cli-reference) - [Project auto-detection](#project-auto-detection) - [Coverage model](#coverage-model) - [Curating a vendor](#curating-a-vendor) - [Churn ranking](#churn-ranking) - [Configuration](#configuration) - [The change schema](#the-change-schema) - [Development](#development) - [Roadmap](#roadmap) - [Contributing](#contributing) - [License](#license) ## Why shipdiff An agent asked to *upgrade* a dependency is doing archaeology across two states of the world with tooling built for one. Documentation describes the present; changelogs are prose, often absent, and — increasingly — **marketing** (a provider's `llms.txt` is a prompt shipped at the domain root). shipdiff takes the opposite stance: > **We diff what they actually shipped.** Every fact is derived mechanically from published artifacts and carries a `source: "artifact" | "changelog"` tag, so it's verifiable and adversarially robust in a way self-published context never is. The vendor can't spin a type signature they already released. ## Features - 🔍 **Version diffing** — removals, renames, signature changes, deprecations, with breaking-vs-additive severity. - ⚰️ **Removal ledger** — ask whether a symbol is `live`, `deprecated`, or `removed` at a given version, and what replaces it. - 🧹 **Post-generation linting** — flag removed/deprecated API in generated code against the *exact installed* versions. - 🔧 **Safe codemods** — auto-rewrite removed/deprecated imports to their replacement (import site + every usage), with duplicate collapsing. - 📦 **Two ecosystems** — npm (via the TypeScript compiler API) and PyPI (via a static `ast` sidecar), one language-neutral model. - 🌍 **Works on any package** — curated vendors get semantic enrichment; everything else is diffed on demand. No backend required. - 📈 **Churn ranking** — rank packages by `churn × blast radius` to prioritize coverage. - 🔌 **MCP-native** — four tools an agent can call directly; also a CLI and a programmatic SDK. ## How it works ```mermaid flowchart TD subgraph SRC["Published artifacts — no vendor cooperation required"] NPM["npm registry<br/>tarball (.tgz)"] PYPI["PyPI<br/>wheel / sdist"] end subgraph RC["@shipdiff/registry-client · fetch → unpack → cache · no backend"] FETCH["resolve version + download + on-disk cache"] end NPM --> FETCH PYPI --> FETCH subgraph EX["Extractors"] ENPM["@shipdiff/extractor-npm<br/>TS compiler API over .d.ts"] EPYPI["@shipdiff/extractor-pypi<br/>static ast sidecar (.pyi / .py)"] end FETCH -->|package dir| ENPM FETCH -->|dist dir| EPYPI SURF["SurfaceDescriptor<br/>language-neutral · @shipdiff/core"] ENPM --> SURF EPYPI --> SURF DIFF["@shipdiff/differ<br/>ChangeRecord[] · severity · rename inference"] SURF -->|"diff v_from vs v_to"| DIFF subgraph ENR["Curated layer — optional, upgrades mechanical → semantic"] VEND["vendors/*.json<br/>replacements · codemods"] SNAP["snapshot/**<br/>prebuilt curated diffs"] end VEND -.-> DIFF SNAP -.-> DIFF subgraph TOOLS["@shipdiff/sdk · the tool surface"] T1["changes_between"] T2["symbol_status"] T3["check_snippet"] T4["apply_fixes"] end DIFF --> T1 DIFF --> T2 CHK["@shipdiff/checker"] --> T3 CHK --> T4 SURF -.-> CHK subgraph DELIVERY["Delivery"] MCP["@shipdiff/mcp<br/>MCP server (stdio)"] CLI["@shipdiff/cli<br/>diff · status · check · fix · scan · rank"] end TOOLS --> MCP TOOLS --> CLI MCP --> AGENTS["Coding agents<br/>Claude Code · Cursor · …"] CLI --> CI["CI / pre-commit"] classDef core fill:#1f6feb,stroke:#1f6feb,color:#ffffff; classDef tool fill:#238636,stroke:#238636,color:#ffffff; class SURF,DIFF core; class T1,T2,T3,T4 tool; ``` Two ideas do the heavy lifting: - **The extractor → differ core is isomorphic** (highlighted blue). The same code powers on-demand client diffs *and* the offline ingestion pipeline that builds curated snapshots. The differ **is** the ingestion pipeline; prose changelogs are only an enrichment layer. - **Graceful degradation.** A curated vendor is served instantly and semantically from a snapshot or `vendors/*.json`; any other package on npm or PyPI still works via on-demand artifact diffing — mechanical, but real. That is how the long tail is covered *without the vendor lifting a finger*. ## Requirements | | | |---|---| | **Node.js** | ≥ 20 | | **Package manager** | [pnpm](https://pnpm.io) (workspace) | | **Python** | 3.9+ — only for PyPI extraction (`SHIPDIFF_PYTHON` to override) | ## Installation ```bash # Run without installing npx @shipdiff/cli --help # Install the CLI globally (provides the `shipdiff` command) npm i -g @shipdiff/cli # Or use it as a library npm i @shipdiff/sdk ``` <details> <summary>Build from source (for development)</summary> ```bash git clone <repo-url> shipdiff cd shipdiff pnpm install pnpm build ``` Then run the CLI with `node packages/cli/dist/index.js …` (or `cd packages/cli && npm link` for a global `shipdiff`). </details> The examples below assume the global `shipdiff` command; substitute `npx @shipdiff/cli` if you prefer not to install. ## Quick start ```bash # What breaks upgrading the OpenAI SDK? shipdiff diff openai 4.20.0 latest --breaking # Did charges.create survive to the latest Stripe SDK? shipdiff status stripe charges.create latest # Lint a file against the CURRENT project's installed versions (auto-detected) shipdiff check bot.ts # Scan an entire project for removed/deprecated API calls shipdiff scan ./src # Auto-apply safe upgrade codemods shipdiff fix bot.ts --write # Rank vendors by churn × blast radius shipdiff rank --vendors # PyPI works too shipdiff status requests Session latest --pypi ``` ## Use with a coding agent (MCP) shipdiff exposes an [MCP](https://modelcontextprotocol.io) server over stdio. The fastest way to wire it into your editor is one command: ```bash shipdiff mcp install # auto-detect editors and add the shipdiff server shipdiff mcp install -c cursor,claude-code # or target specific ones shipdiff mcp status # see where it's configured shipdiff mcp uninstall # remove it ``` `mcp install` detects installed editors and merges a `shipdiff` server entry into each one's MCP config — **Claude Code, Cursor, Windsurf, Claude Desktop, VS Code, and Codex**. It's global by default (works in every project); pass `--project` to write repo-local config you can commit and share. Existing servers are preserved (the file is backed up first), and it's idempotent. Restart your editor afterward. <details> <summary>Manual configuration</summary> Add this to your client's MCP config (e.g. <code>~/.cursor/mcp.json</code>): ```json { "mcpServers": { "shipdiff": { "command": "npx", "args": ["-y", "@shipdiff/mcp"] } } } ``` </details> The agent then has the full **write → check → fix** loop without leaving its tool surface: ``` changes_between → see what breaks before upgrading symbol_status → verify a symbol is still live (negative knowledge) check_snippet → lint generated code against installed versions apply_fixes → auto-remediate what check_snippet flagged ``` ## Programmatic API (SDK) ```bash npm i @shipdiff/sdk ``` ```ts import { changesBetween, symbolStatus, checkSnippet, applyFixes, } from "@shipdiff/sdk"; // 1. What will break? const { breaking } = await changesBetween("openai", "4.20.0", "latest"); // 2. Is this symbol still alive? const status = await symbolStatus("stripe", "charges.create", "latest"); // → { status: "removed" | "deprecated" | "live", replacement?, since?, coverage } // 3. Lint generated code against exact installed versions const { violations } = await checkSnippet(code, [ { package: "openai", version: "4.28.0" }, ]); // 4. Auto-fix it const { code: patched, applied, skipped } = await applyFixes(code, [ { package: "openai", version: "4.28.0" }, ]); ``` ## Tool reference MCP tool names are `snake_case`; the SDK exports the same logic in `camelCase`. ### `changes_between(package, from_version, to_version)` Diff a package's public API between two versions. Returns aggregated `ChangeRecord[]`, a `breaking[]` convenience view, and a `coverage` tag. ```jsonc { "coverage": "derived-live", "breaking": [ { "symbol": "createChatCompletion", "change": "removed", "severity": "breaking", "source": "artifact" } ] } ``` ### `symbol_status(package, symbol, version)` The **verification gate**: is a symbol `live`, `deprecated`, or `removed` at a version — and what replaces it. Accepts a full path or a leaf (`charges.create`). Pass `baseline` to confirm removal and narrow the window. ```jsonc { "status": "removed", "since": "removed between 3.3.0 and 4.28.0", "replacement": { "symbol": "chat.completions.create" }, "coverage": "curated" } ``` ### `check_snippet(code, manifest)` Static lint pass over generated code. Resolves every symbol used from a tracked dependency against the exact installed version and returns `violations[]` (with line/column and a `suggestedFix` when a replacement is known). ### `apply_fixes(code, manifest)` Codemod execution. Rewrites removed/deprecated **named imports** to their replacement at the import site and every usage, collapsing duplicates. Returns the patched `code`, an `applied[]` list, and a `skipped[]` list. Structural (dotted-path) replacements and default/namespace imports are reported as suggestions — **never** auto-rewritten. ## CLI reference ``` shipdiff diff <package> <from> <to> Diff two versions shipdiff status <package> <symbol> <version> Symbol lifecycle at a version shipdiff check <file> Lint a file (auto-detects installed versions) shipdiff fix <file> Apply safe codemods to a file shipdiff scan [dir] Scan a whole project shipdiff rank [package ...] Rank packages by churn × blast radius shipdiff mcp install|uninstall|status Wire the MCP server into your editor(s) ``` | Flag | Applies to | Meaning | |------|-----------|---------| | `--pypi` | all | Use the PyPI ecosystem instead of npm | | `--json` | all | Machine-readable output | | `--breaking` | `diff` | Show only breaking changes | | `--baseline <ver>` | `status` | Older version to confirm removal / window | | `--pkg name@version` | `check`, `fix` | Pin versions explicitly (repeatable; overrides auto-detect) | | `--write` | `fix` | Apply edits to disk (default is a dry run) | | `--vendors` | `rank` | Rank every curated vendor | | `--no-dev` | `scan` | Ignore `devDependencies` | | `-c <editors>` | `mcp` | Target editors (comma-sep): claude-code, cursor, windsurf, claude-desktop, vscode, codex | | `--project` | `mcp` | Write repo-local config instead of global | | `--dry-run` | `mcp` | Preview changes without writing | **Exit codes** are output-format independent (so `--json` still gates CI): `diff` exits `1` on breaking changes, `status` on a removed symbol, `check`/`scan` on any violation. ## Project auto-detection `check`, `fix`, and `scan` need no manual version list. They walk up to the nearest `package.json`, read every dependency, and pin each to the version **actually installed** in `node_modules` — the truth your generated code will run against. Dependencies declared but not installed are skipped with a warning, never guessed. ```console $ shipdiff check src/app.ts resolved 1 installed deps from /path/to/project (1 declared deps not installed, skipped) ✗ src/app.ts:3:15 OldClient does not exist in acme-sdk@2.0.0 (removed or never existed) ``` Pass `--pkg name@version` to override. Auto-detection is npm-only today. ## Coverage model Every result carries a `coverage` field so you always know how much to trust it: | Coverage | Meaning | |----------|---------| | `curated` | Served from a prebuilt snapshot / vendor enrichment — instant, semantic (replacements, codemods). | | `derived-live` | Diffed on demand from the shipped artifacts — mechanical, but real, for any package. | ## Curating a vendor Any npm/PyPI package works out of the box (mechanical diff). To upgrade one to **curated** — semantic replacements and codemods — drop a file in `vendors/`: ```json { "package": "your-sdk", "ecosystem": "npm", "enrichment": { "replacements": { "oldMethod": { "symbol": "newMethod", "note": "why", "codemod": "hint" } } } } ``` No code changes. That is the entire "easy to add vendors" contract. Curated examples ship in [`vendors/`](vendors) (`openai`, `stripe`, `langchain`). ## Churn ranking `shipdiff rank` scores packages by **churn × blast radius** — the build/coverage queue *and* an "APIs that broke your code most" list in one computation. - **Churn** is proxied by major-bumps-per-year, derived from the registry's publish-time history. - **Blast radius** is proxied by download volume, **log-damped** so a popular-but-stable package (e.g. core S3) can't dominate on volume alone. ```console $ shipdiff rank --vendors churn ranking (npm) — score = breaking/yr × log(downloads) score maj/yr majors downloads package 11.71 1.5 23 62.0M stripe 8.00 1.0 7 114.3M openai 4.96 0.7 3 11.1M langchain ``` ## Configuration All configuration is via environment variables; sensible defaults require none. | Variable | Default | Purpose | |----------|---------|---------| | `SHIPDIFF_CACHE` | `~/.shipdiff/cache` | On-disk cache for artifacts and extracted surfaces | | `SHIPDIFF_SNAPSHOT_DIR` | `./snapshot` | Prebuilt curated diffs | | `SHIPDIFF_VENDORS_DIR` | `./vendors` | Vendor enrichment descriptors | | `SHIPDIFF_PYTHON` | `python3` | Interpreter for the PyPI extractor | | `SHIPDIFF_NPM_REGISTRY` | `https://registry.npmjs.org` | npm registry base URL | | `SHIPDIFF_PYPI_REGISTRY` | `https://pypi.org` | PyPI base URL | | `SHIPDIFF_NPM_DOWNLOADS` | `https://api.npmjs.org/downloads/point/last-month` | npm downloads endpoint (for `rank`) | ## The change schema The `ChangeRecord` is shipdiff's public contract, published as JSON Schema at [`schemas/change-record.schema.json`](schemas/change-record.schema.json). Providers may one day emit *into* this schema instead of us deriving it — at which point the treadmill ends and we receive instead of scrape. ```jsonc { "package": "openai", "ecosystem": "npm", "symbol": "createChatCompletion", "kind": "function", "change": "removed", // removed | added | renamed | signature-changed | deprecated | type-* "severity": "breaking", // breaking | additive | safe "from_version": "3.3.0", "to_version": "4.28.0", "replacement": { "symbol": "chat.completions.create" }, "source": "artifact" // artifact = mechanically derived; changelog = human/enrichment } ``` ## Development This is a pnpm workspace of nine packages built with TypeScript project references. ``` packages/ ├── core # language-neutral data model + published change schema ├── extractor-npm # .d.ts surface extraction via the TS compiler API ├── extractor-pypi # static Python surface extraction (ast sidecar) ├── differ # surface pair → ChangeRecord[] with severity ├── registry-client # fetch / cache / diff + enrichment + snapshots + ranking ├── checker # check_snippet, apply_fixes, project scan ├── sdk # the tools as a programmatic API ├── mcp # MCP server (stdio) └── cli # diff · status · check · fix · scan · rank vendors/ # per-vendor enrichment descriptors schemas/ # published JSON Schema ``` ```bash pnpm build # compile all packages (tsc -b) pnpm watch # incremental rebuild pnpm test # offline test suite pnpm test:watch # watch mode ``` ### Testing ```bash pnpm test # deterministic, no network — seeds the cache, uses exact versions pnpm test:network # + live npm/PyPI, the MCP server, and CLI end-to-end (opt-in) ``` The offline suite exercises the full orchestration — fetch resolution, extraction, diffing, enrichment, snapshots, the checker, codemods, and project auto-detection — with **no network**, so it's fast and CI-friendly. `test:network` (sets `SHIPDIFF_NETWORK=1`, run `pnpm build` first) adds real-registry, MCP, and CLI end-to-end coverage. ## Roadmap - [x] Publish packages to npm (`@shipdiff/cli`, `@shipdiff/mcp`, …) - [ ] Extract surfaces from local `node_modules` (skip the re-download for installed deps) - [ ] PyPI project auto-detection (requirements / poetry / site-packages) - [ ] Multi-version aggregation across intermediate releases - [ ] Container-rename propagation (suppress member removals when a parent is renamed) - [ ] Bundled Tier-1/2 snapshots + the ingestion pipeline - [ ] CI workflow and coverage reporting ## Contributing Contributions are welcome. Good first steps: - **Curate a vendor** — add a `vendors/*.json` enrichment file (see [Curating a vendor](#curating-a-vendor)). - **Improve extraction** — edge cases in the npm (`.d.ts`) or PyPI (`ast`) extractors. - **Add coverage** — new fixtures and tests (`pnpm test` must stay green offline). Please run `pnpm build && pnpm test` before opening a PR. ## License [MIT](LICENSE)