icp-scout

<p align="center"> <img src="assets/hero.svg" alt="ICP Scout — Search the public web. Find companies that fit. Keep the evidence." width="100%"> </p> <p align="center"> <strong>Evidence-first company discovery from the public web.</strong><br> Search, crawl, extract, score, and export qualified companies—with a source URL behind every useful fact. </p> <p align="center"> <a href="https://github.com/GrimmReaper0/icp-scout/actions/workflows/ci.yml"><img alt="CI" src="https://img.shields.io/github/actions/workflow/status/GrimmReaper0/icp-scout/ci.yml?branch=main&style=flat-square&label=tests"></a> <a href="https://github.com/GrimmReaper0/icp-scout/releases/latest"><img alt="GitHub release" src="https://img.shields.io/github/v/release/GrimmReaper0/icp-scout?style=flat-square&label=release"></a> <a href="https://github.com/GrimmReaper0/icp-scout/stargazers"><img alt="GitHub stars" src="https://img.shields.io/github/stars/GrimmReaper0/icp-scout?style=flat-square&color=c8ff32&labelColor=141414"></a> <a href="LICENSE"><img alt="MIT License" src="https://img.shields.io/badge/license-MIT-141414?style=flat-square"></a> <img alt="Zero runtime dependencies" src="https://img.shields.io/badge/runtime_dependencies-0-c8ff32?style=flat-square&labelColor=141414"> <img alt="Node 20.11+" src="https://img.shields.io/badge/node-%E2%89%A520.11-3157ff?style=flat-square&labelColor=141414"> </p> --- Most lead tools give you a row in a database and ask you to trust it. **ICP Scout shows its work.** Give it an ideal customer profile in plain language. It finds candidate company websites, respectfully crawls a few relevant public pages, extracts company facts and public business contacts, detects technology signals, and assigns a deterministic fit score. Every material result carries the page where it was found. ```text FIT COMPANY DOMAIN WHY ──────────────────────────────────────────────────────────────────────────────────────── 93 Sunbird Botanics sunbirdbotanics.co Required: shopify 72 Moss & Mirth mossandmirth.sg Required: shopify 48 PixelFoundry Agency pixelfoundry.agency Required: shopify 18 Northline Labs northlinelabs.co.uk Relevant terms: skincare, beauty ``` > [!NOTE] > ICP Scout is a research tool, not a proprietary contact database. It finds public company information, preserves evidence, and leaves outreach decisions to you. ## See it work in 30 seconds No account. No API key. No install step. The demo starts four miniature websites on localhost, runs the real crawler and scorer, then shuts everything down. ```bash git clone https://github.com/GrimmReaper0/icp-scout.git cd icp-scout npm run demo ``` Machine-readable output: ```bash node src/cli.mjs demo --no-save --json ``` Start the local dashboard: ```bash npm run serve # http://127.0.0.1:7331 ``` <p align="center"> <img src="assets/dashboard-preview.svg" alt="ICP Scout local dashboard preview" width="94%"> </p> ## Why people reach for it | Need | What ICP Scout does | |---|---| | **Find companies, not random pages** | Filters aggregators, deduplicates domains, and crawls company-owned sites. | | **Know why a lead matched** | Produces a 0–100 deterministic score with a point-by-point breakdown. | | **Verify the useful facts** | Keeps source URLs and short evidence excerpts for company, location, email, technology, and description signals. | | **Avoid another data silo** | Exports plain JSON and safe CSV; saved runs are ordinary local files. | | **Use it from an agent** | Includes a dependency-free MCP stdio server with discovery, scanning, and run-history tools. | | **Keep the default footprint small** | Zero runtime dependencies; built on Node.js standard-library primitives. | ## Three ways to use it ### 1. Score a domain list—no search service required ```bash node src/cli.mjs scan \ sunbirdbotanics.co mossandmirth.sg \ --query "independent skincare brands in Singapore using Shopify" \ --must-have shopify \ --exclude agency,marketplace \ --locations Singapore \ --technologies Shopify \ --output leads.csv ``` You can also pass a newline-delimited file: ```bash node src/cli.mjs scan --file examples/domains.txt --config examples/skincare-singapore.json ``` ### 2. Discover companies with self-hosted SearXNG ```bash docker compose up -d searxng node src/cli.mjs discover \ "independent Shopify skincare brands in Singapore" \ --must-have shopify \ --exclude agency,directory,marketplace \ --locations Singapore \ --limit 25 ``` Or use Brave Search: ```bash BRAVE_SEARCH_API_KEY=... node src/cli.mjs discover \ "B2B payroll startups in Europe" \ --provider brave ``` ### 3. Give the tools to an MCP client ```json { "mcpServers": { "icp-scout": { "command": "node", "args": ["/absolute/path/to/icp-scout/src/cli.mjs", "mcp"], "env": { "SEARXNG_URL": "http://localhost:8080" } } } } ``` Available tools: - `discover_leads` — search, crawl, score, and save evidence-backed companies - `scan_websites` — score a known website list against an ICP - `list_runs` — list locally saved research runs - `get_run` — retrieve one saved run See [docs/MCP.md](docs/MCP.md) for Claude Desktop, Cursor, Windsurf, and protocol examples. ## How scoring works The default scorer does **not** ask a model whether a company “feels right.” It measures public evidence across active dimensions, rebalances unused dimensions, then applies explicit exclusion penalties. ```text Query relevance up to 40 weighted points Preferred signals up to 15 weighted points Required signals up to 20 weighted points Technology fit up to 10 weighted points Location fit up to 10 weighted points Contactability up to 10 weighted points Evidence depth up to 5 weighted points Exclusions -30 each, capped at -60 ``` The exact active weights are normalized to 100. A run records the score breakdown, matched and missing criteria, reasons, and supporting pages. An optional OpenAI-compatible reranker can be layered on top, but the evidence-first deterministic result remains available. ## What gets extracted - Company and site names from page metadata and Schema.org JSON-LD - Public role and named business emails, including common human-readable obfuscations - Optional MX-domain checks—**not** mailbox verification - Phone numbers, location claims, and social profiles - Shopify, WooCommerce, WordPress, Webflow, Wix, Squarespace, Next.js, React, Vue, Svelte, HubSpot, Intercom, Klaviyo, Stripe, Segment, analytics, hosting, and other web signals - Relevant public pages such as About, Company, Team, Customers, Locations, and Contact - Evidence snippets and source URLs for auditability ## Safety and respectful crawling Public-web research still deserves boundaries. ICP Scout: - honors `robots.txt` by default, including specific user-agent groups and crawl delays; - blocks localhost, private, link-local, metadata, multicast, and reserved network destinations by default; - resolves DNS before requests and revalidates every redirect to reduce SSRF and DNS-rebinding risk; - limits redirects, request time, response size, pages per site, and global concurrency; - skips binary and low-value paths; - neutralizes spreadsheet-formula injection in CSV exports; - never guesses private email addresses or bypasses authentication. Read [docs/RESPONSIBLE_USE.md](docs/RESPONSIBLE_USE.md) before using it at scale. ## Architecture ```mermaid flowchart LR A[Plain-language ICP] --> B{Input mode} B -->|discover| C[SearXNG or Brave] B -->|scan| D[Domain list] C --> E[Candidate normalization] D --> E E --> F[Safe respectful crawler] F --> G[HTML + JSON-LD extraction] G --> H[Technology + contact signals] H --> I[Deterministic scorer] I --> J[Evidence-backed leads] J --> K[Terminal] J --> L[JSON / safe CSV] J --> M[Local dashboard] J --> N[MCP tools] ``` The dependency graph is intentionally boring. Network, DNS, streams, files, HTTP, crypto, and test tooling come from Node.js itself. That makes the core easy to inspect, embed, and fork. ## Configuration Create a documented starter file: ```bash node src/cli.mjs init ``` ```json { "$schema": "./icp-scout.schema.json", "name": "Shopify skincare brands in Singapore", "query": "independent skincare and beauty brands based in Singapore using Shopify", "include": ["skincare", "beauty", "direct-to-consumer"], "mustHave": ["shopify"], "exclude": ["agency", "marketplace", "directory"], "locations": ["Singapore"], "technologies": ["Shopify"], "minScore": 45, "search": { "provider": "searxng", "limit": 25, "searxngUrl": "http://localhost:8080" }, "crawl": { "maxPages": 4, "concurrency": 3, "respectRobots": true, "verifyMx": false, "allowPrivate": false } } ``` The repository includes a JSON Schema for editor completion and validation. See [docs/CONFIGURATION.md](docs/CONFIGURATION.md). ## Commands ```text icp-scout demo Offline zero-key demo icp-scout discover <ICP query> Search, crawl, score, and save companies icp-scout scan <url...> Score known websites against an ICP icp-scout serve Start the local dashboard icp-scout mcp Start the MCP stdio server icp-scout init [path] Create a starter configuration icp-scout runs List saved runs icp-scout show <run-id> Print a saved run ``` Run `node src/cli.mjs help` for every option. ## Development Requirements: Node.js 20.11 or newer. There are no runtime or development package dependencies. ```bash npm test # 28 unit, integration, CLI, server, and protocol tests npm run lint # syntax, JSON, secret, size, and merge-marker checks npm run check # all repository checks npm run smoke # complete offline pipeline smoke test ``` Before publishing your fork, replace badge and repository placeholders in one command: ```bash npm run prepare:repo -- your-github-username ``` Then read [docs/LAUNCH.md](docs/LAUNCH.md). It contains a concrete launch checklist, demo script, issue seeding plan, and first-week roadmap—not advice to manufacture engagement. ## Scope ICP Scout is good at **company discovery and qualification from public first-party web evidence**. It is deliberately not: - a people-search or consumer-data tool; - a bulk email sender; - a guarantee that a public address accepts mail; - a replacement for legal review, consent, suppression lists, or outreach rules; - a stealth crawler or access-control bypass. ## Contributing Small, inspectable contributions are preferred: a better extractor, a well-tested technology signature, a provider adapter, a scoring improvement with fixtures, or a reproducible crawler edge case. Start with [CONTRIBUTING.md](CONTRIBUTING.md) and the [roadmap](docs/ROADMAP.md). ## License MIT © ICP Scout contributors. See [LICENSE](LICENSE).