aeo-advisor

# AEO Advisor [](LICENSE) [](https://github.com/sponsors/kailichama) [](https://nodejs.org) **A self-hostable platform for getting your brand cited in AI answers.** Scan your visibility across ChatGPT, Perplexity, Gemini, and Google AI Overviews; generate citation-optimized content; publish it; and measure whether it moved the needle. > **Why this exists.** Search is shifting from ten blue links to AI-generated answers, and the tactics that earn citations are not the same as classic SEO. AEO Advisor is an open, end-to-end reference implementation of those tactics — built to **teach the patterns** as much as to run them. > **Local-first — no hosting required.** Run it on your own machine with `npm run dev` and your own AI provider keys (entered in Settings, stored encrypted in a local SQLite file). The database creates and migrates itself on first run — there is no database server to install and no cloud dependency. Hosting only matters for an always-on or team-shared deployment; in that case read [SECURITY.md](SECURITY.md) first — **the app has no built-in login** and must sit behind an authenticating proxy (production builds fail closed without one). ## What You'll Learn Even if you never deploy the app, these are the AEO patterns it demonstrates in working code: - **The closed-loop pipeline** — Scan → Create → Publish → Measure → re-Scan, so content is driven by measured visibility. - **Citation drivers** — research-backed signals (grounded in the GEO paper, KDD 2024) that make content extractable and citable: specificity, snippet-extractability, authority signals, standalone comprehensibility. - **LLM-as-Judge scoring** — a structured rubric that scores content for citability and feeds an auto-revision loop. - **BAVT (Budget-Aware Value Tree Search)** — tree-structured, backtracking research that matches baseline answer quality at roughly 4× lower tool-call cost. - **The machine-readability kit** — generating `llms.txt`, `robots.txt`, `sitemap.xml`, and feeds that make a site legible to AI crawlers. - **BYOK + encryption-at-rest** — a bring-your-own-key architecture with AES-256-GCM, so there are no shared server secrets. ## Screenshots | Home — visibility dashboard | Scan — multi-engine audit | |---|---| |  |  | | Measure — KPIs, cost & ROI | Site Health — Machine Readability Kit | |---|---| |  |  | ## What It Does AEO Advisor runs a **closed-loop pipeline** for brand visibility in AI-generated answers: <p align="center"> <img src="docs/assets/aeo-loop.png" alt="The closed loop: Scan, Create, Publish, Measure — then re-scan" width="440"> </p> 1. **Scan** — Query 4 answer engines (Claude, GPT-4o, Gemini, Perplexity) to audit brand visibility across target queries 2. **Create** — Generate optimized content via research --> brief assembly --> content generation --> citability scoring --> auto-revision loop 3. **Publish** — Push content to your CMS (GitHub Pages, WordPress, Webflow) with one click 4. **Measure** — Track KPIs, verify indexing, and monitor AI-referral traffic arriving at your published pages 5. **Loop** — Re-scan to detect shifts, feeding new gaps back into the pipeline (scheduled re-scans are on the roadmap, behind the experimental `ENABLE_SCHEDULED_SCANS` flag) ### Key Capabilities What works today, by pipeline phase: - **Scan** — 4 answer engines queried in parallel (Claude, GPT-4o, Gemini, Perplexity Sonar); AI query seeding from brand config, competitors, and research; competitor visibility tracking; response enrichment (URL classification, brand sentiment, citation context); per-project scan preferences (engine selection, Compete/Elevate strategy) - **Create** — deep per-topic research via BAVT tree search at a selectable depth (Quick / Standard / Deep) with embedding-based dedup; structured brief assembly; content generation; LLM-as-Judge citability scoring; draft edit history and snapshots - **Publish** — 3 CMS adapters (GitHub Pages, WordPress, Webflow); machine-readability kit (robots.txt, sitemap.xml, llms.txt, RSS) generated per brand site and deployable to GitHub; IndexNow + sitemap pings on publish; post-publish verification via the Google Indexing and Bing Webmaster APIs - **Measure** — visibility trends and per-query performance across scans; AI referral tracking via a privacy-safe snippet on published pages (platforms that strip referrers aren't detected, so counts understate actual AI traffic); site health audits; ROI forecasting - **Platform** — BYOK key management with AES-256-GCM encryption at rest; stdio MCP server for headless audits; SQLite with automatic migrations One feature is functional but off by default: the **auto-revision loop** (`ENABLE_REVISION_LOOP=true`) re-generates and re-scores content that fails a scoring dimension, up to 3 iterations. Scheduled re-scans are not yet functional — that's the top item on the [Roadmap](#roadmap). ## Tech Stack | Layer | Technology | |-------|-----------| | Framework | Next.js 16 (App Router) | | UI | React 19, Tailwind CSS 4 | | Language | TypeScript 5 | | Database | SQLite via better-sqlite3 (WAL mode, schema v46 — see `ui/src/lib/db.ts`) | | AI | Anthropic SDK (Claude), Google Generative AI SDK (Gemini); OpenAI and Perplexity via REST | | Embeddings | Gemini `gemini-embedding-001` | | CMS | GitHub REST API, WordPress REST, Webflow API | | Testing | Vitest | ## Project Structure ``` aeo-advisor/ ├── skills/ # Claude skill definitions (Anthropic convention) │ ├── README.md │ ├── ai-visibility-scanner/SKILL.md │ ├── brief-assembler/SKILL.md │ ├── citability-scorer/SKILL.md │ ├── content-generator/SKILL.md │ ├── content-topics-planner/SKILL.md │ ├── entity-authority-auditor/SKILL.md │ ├── combined-analyst/SKILL.md │ ├── domain-investigator/SKILL.md │ ├── serp-ranking-advisor/SKILL.md │ └── dev-search-first/SKILL.md └── ui/ # Next.js dashboard application └── src/ ├── app/ │ ├── page.tsx # Home — launchpad with activity feed │ ├── scan/page.tsx # Scan & audit (deep-linkable tabs) │ ├── create/page.tsx # Content creation pipeline │ ├── publish/page.tsx # Strategy planning & CMS publishing │ ├── measure/page.tsx # KPIs, query performance & attribution │ ├── site-health/page.tsx # Machine readability audit │ └── api/ # API routes (run `ls ui/src/app/api`; see CLAUDE.md) ├── components/ # Reusable UI components └── lib/ ├── skills/ # Colocated skill modules │ ├── index.ts # Barrel re-exports │ ├── types.ts # SkillPromptResult, ProjectContext │ ├── shared.ts # Citation driver constants, shared dimensions │ ├── scanner/ # prompt.ts, parser.ts, __tests__/ │ ├── assembler/ # prompt.ts, parser.ts, __tests__/ │ ├── generator/ # prompt.ts, parser.ts, __tests__/ │ ├── scorer/ # prompt.ts, parser.ts, __tests__/ │ ├── topics/ # prompt.ts, parser.ts, __tests__/ │ ├── mapper/ # prompt.ts, parser.ts, __tests__/ │ ├── revision/ # prompt.ts │ ├── query-seeding/ # prompt.ts │ ├── entity-auditor/ # prompt.ts, parser.ts │ ├── serp-advisor/ # prompt.ts, parser.ts │ ├── mr-prose/ # prompt.ts, parser.ts │ ├── mr-scorer/ # prompt.ts, parser.ts │ └── answer-engine/ # parser.ts ├── db.ts # SQLite schema + version-gated migrations ├── db-queries.ts # Typed query helpers for all tables ├── types.ts # Shared TypeScript interfaces ├── pipeline.ts # Pipeline stage orchestration ├── similarity.ts # Embedding similarity service ├── claude-client.ts # Anthropic SDK wrapper ├── openai-client.ts # OpenAI REST client (raw fetch) ├── perplexity-client.ts # Perplexity API client ├── brand-config.ts # Brand context management ├── enrichment/ # Multi-source enrichment pipeline (7 modules) ├── bavt/ # Budget-Aware Value Tree Search (6 modules) └── cms/ # CMS adapter layer (Strategy pattern) ``` ### Machine Readability Kit The MR Kit generates AI-optimized technical assets (robots.txt, sitemap.xml, llms.txt, RSS) for brand sites. An iterative LLM generation loop produces prose content for llms.txt with cache-through freshness. Assets are versioned in `mr_artifacts` / `mr_artifact_versions` tables and can be deployed to GitHub repos. ``` mr_briefs (structured brief per project) | mr_artifacts (one per asset type: robots, sitemap, llms-txt, rss) | mr_artifact_versions (version history per artifact) ``` ## Pipeline Architecture ``` ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ SCAN │───>│ CREATE │───>│ PUBLISH │───>│ MEASURE │───>│ RE-SCAN │ │ │ │ │ │ │ │ │ │ │ │ 4 AI │ │ Research│ │ GitHub │ │ Index │ │ Detect │ │ engines │ │ Assemble│ │ WP │ │ Verify │ │ shifts, │ │ per │ │ Generate│ │ Webflow │ │ Traffic │ │ new │ │ query │ │ Score │ │ │ │ ROI │ │ gaps │ │ │ │ Revise │ │ │ │ │ │ │ └─────────┘ └─────────┘ └─────────┘ └─────────┘ └─────────┘ ``` ### Scan Phase Queries 4 answer engines in parallel per target query. Strategy-aware: **Compete** mode tracks competitor displacement, **Elevate** mode tracks authority landscape. Enrichment pipeline classifies URLs, analyzes brand portrayal, and contextualizes citations. Expert gap detection surfaces opportunities where no named authority exists. ### Create Phase 1. **Topic Researcher** — BAVT-powered deep research per topic with similarity dedup 2. **Brief Assembler** — Structures content requirements from scan + research data (prescriptive citation sources with expert names, credentials, and quotable claims) 3. **Content Generator** — Produces optimized content with expert quotes, comparison tables, and cross-platform extractable claims 4. **Citability Scorer** — Scores content on 6-7 dimensions (research-backed PASS/FAIL criteria) 5. **Auto-Revision** — If score < threshold, revises and re-scores (up to 3 iterations) with evidence-based fix patterns. Opt-in via `ENABLE_REVISION_LOOP=true` — off by default until budget pre-flight lands ### Publish Phase Pushes approved content to configured CMS via adapter pattern. Machine readability kit generates robots.txt, sitemap.xml, llms.txt, and RSS feed. IndexNow auto-pings search engines on publish. ### Measure Phase Tracks visibility over time with line charts, scan history, and narrative insights. Submits URLs to Google/Bing indexing APIs and reports AI-referral visits arriving at your published pages. Query performance tracking with per-query detail drawers and archetype filters (gained/lost/visible/invisible). ## BAVT — Budget-Aware Value Tree Search BAVT is the research engine behind per-topic deep research in the Create phase. Based on the paper *"Spend Less, Reason Better"* (Li et al., UBC, March 2026), it optimizes how LLM agents solve multi-step questions by modeling reasoning as a **branching tree** instead of linear chains. ### Why not just chain tool calls? Linear reasoning chains commit fully to each path — if step 3 of 8 goes in the wrong direction, the remaining 5 calls are wasted. BAVT treats each step as a tree node and can **backtrack** to explore alternatives from any prior state, focusing compute where it matters most. **Headline result from the paper:** BAVT on 5 tool calls consistently matches or beats baseline on 20 tool calls — a 4x efficiency gain. ### Three core mechanisms 1. **Tree-Structured Search** — Nodes can branch from any prior state. Dead-end paths are abandoned, not followed to completion. 2. **Step-Level Critic (Residual Delta Scoring)** — After every action, an LLM critic scores the *marginal gain* of that step on a [-4, +4] scale. By evaluating change rather than absolute state, it sidesteps LLM overconfidence. 3. **Budget-Aware Node Selection (Power-Law Weighting)** — Selects which node to expand using: `weight = node_value ^ (1 / remaining_budget_ratio)`. Early on, this explores broadly. As budget depletes, it concentrates on the best node. ### How it runs in AEO Advisor ``` Question in --> Planner (abstract 2-5 hop plan) | Tree Search Loop: 1. Select node (budget-aware weighting) 2. Decide instruction (widen / deepen / answer / backstop) 3. Run Generator (one web search or reasoning step) 4. Run Critic (score marginal gain) 5. Update node value, branch or continue 6. Repeat until budget exhausted or answer found | Best answer from highest-confidence terminal node ``` **Research depth** is selectable per run: Quick (3 web searches / 25K tokens), Standard (6 / 50K), or Deep (10 / 100K). **Alternative engine:** setting `RESEARCH_ENGINE=single` swaps BAVT for a single-pass engine (`ui/src/lib/research/`) that makes one budget-aware request using the provider's web-search tool. It implements the same contract and is being A/B evaluated against the tree search. **Integration:** The topic research API (`/api/projects/[id]/topics/[topicId]/research`) triggers BAVT with SSE progress streaming. Results are cached per-topic (7-day freshness window) and deduplicated via Gemini embeddings. ### Implementation Six modules in `ui/src/lib/bavt/`: | Module | Role | |--------|------| | `types.ts` | Tree node structure, budget config, progress events | | `bavt-planner.ts` | Abstract plan generation (Sonnet, no tools) | | `bavt-generator.ts` | Action execution — web search or reasoning per instruction | | `bavt-critic.ts` | Step scorer — LLM-as-judge for marginal gain | | `bavt-orchestrator.ts` | Main loop — node selection, budget tracking, instruction dispatch | | `index.ts` | Public API (`runBAVT`) | ## Environment Variables > **AI provider keys are _not_ environment variables.** Anthropic, OpenAI, Gemini, and > Perplexity keys are entered per-user in the in-app **Settings** page (BYOK — bring your > own key) and stored encrypted in the database. Putting them in `.env.local` has no effect. Create a `.env.local` file in the `ui/` directory (copy `ui/.env.example`): ```env # Required for local dev — stands in for the Cloudflare Access identity header DEV_USER_EMAIL=you@example.com # Required to save any BYOK key + for CMS encryption. 64-char hex; encryption # throws if missing/malformed (no fallback, no derivation): openssl rand -hex 32 CMS_ENCRYPTION_KEY= # Optional — post-publish verification GOOGLE_INDEXING_SERVICE_ACCOUNT= # Google Indexing API service account JSON BING_WEBMASTER_API_KEY= # Bing Webmaster API key # Optional — feature flags ENABLE_REVISION_LOOP= # "true" enables the Auto-Revise loop (off by default; route 503s) ENABLE_SCHEDULED_SCANS= # "true" shows scan-scheduling UI + alert bell (experimental; needs external cron) RESEARCH_ENGINE= # Topic research engine: bavt (default) or single (one budget-aware request) # Optional — infrastructure CRON_SECRET= # Bearer token for /api/cron/scan DATABASE_PATH= # Custom database directory (defaults to ./data/) INDEXNOW_SALT= # Salt for IndexNow key derivation (recommended to override) LOG_LEVEL= # debug | info | warn | error (default: info) NEXTAUTH_URL=http://localhost:3000 # Base URL for the application # Production self-hosting only — read SECURITY.md before setting DISABLE_AUTH_GATE= # "true" disables the fail-closed production auth gate ``` Then start the app, open **Settings**, and paste your AI provider key(s) to begin. ## Database SQLite database at `ui/data/aeo-advisor.db` (auto-created on first run). - **Schema version**: v46 (auto-migrates on startup; the authoritative version is the last `PRAGMA user_version` in `ui/src/lib/db.ts`) - **Mode**: WAL (Write-Ahead Logging) for concurrent reads - **Tables & views**: see `ui/src/lib/db.ts` for the current, authoritative schema (projects, audit_results, citations, topics, topic_research, content_drafts, cms_configs, user_api_keys, brand_knowledge, mr_artifacts, competitor_mentions, … plus SQL views such as `v_source_authority` and `v_citation_patterns`) ## Getting Started ```bash cd ui npm install cp .env.example .env.local # set DEV_USER_EMAIL and CMS_ENCRYPTION_KEY (openssl rand -hex 32) npm run dev ``` Opens at [http://localhost:3000](http://localhost:3000). Then open **Settings** and paste your AI provider key(s) (BYOK) to start scanning. The SQLite database is created at `ui/data/` and migrated automatically on first run; there is no database server to install or configure. ### Deploying (optional) `ui/Dockerfile` + `ui/docker-compose.yml` build a standalone production image with a persistent volume for the database — useful for an always-on or team-shared instance. **Production deployments require an authenticating reverse proxy** (e.g. Cloudflare Access): the app has no built-in login, identity comes from the proxy's header, and production builds return 403 without it (`DEV_USER_EMAIL` is dev-only). Read [SECURITY.md](SECURITY.md) before deploying. ### Commands ```bash npm run dev # Development server npm run build # Production build npm test # Run unit tests npm run test:watch # Watch mode ``` ### Pages | Route | Page | Purpose | |-------|------|---------| | `/` | Home | Project launchpad, headline metrics, activity feed | | `/scan` | Scan | Multi-engine visibility audits, narrative insights, gap detection, drill-down (deep-linkable tabs) | | `/create` | Create | Content pipeline: research --> brief --> generate --> score --> revise | | `/publish` | Publish | Strategy planning, CMS config, publish & verify indexing | | `/measure` | Measure | Visibility timeline, scan history, query performance, ROI tracking | | `/site-health` | Site Health | Machine readability audit (robots.txt, sitemap, llms.txt, RSS) | ## Use Without the UI (MCP Server) AEO Advisor ships an [MCP](https://modelcontextprotocol.io) server, so you can run audits from any MCP-capable agent (Claude Code, Claude Desktop, Cowork) — no browser needed. It reuses the same libraries and your local database, and reads **your own** provider keys from its environment (BYOK). > Note the asymmetry with the web app: the **web app** reads keys only from Settings (database), never from env; the **MCP server** has no Settings UI, so it reads keys only from the env you pass it here. Same BYOK principle, two different homes. **Add it** (from Claude Code): ```bash claude mcp add aeo-advisor \ --env DATABASE_PATH=/absolute/path/to/aeo-advisor/ui/data \ --env ANTHROPIC_API_KEY=sk-ant-... \ -- npx tsx /absolute/path/to/aeo-advisor/ui/src/mcp/server.ts ``` Add `OPENAI_API_KEY` / `GEMINI_API_KEY` / `PERPLEXITY_API_KEY` to audit more engines (a single key gives a single-engine audit). | Tool | Does | Cost | |------|------|------| | `audit_brand` | Multi-engine visibility audit: does your brand appear for a set of customer questions, and who gets cited instead? | uses API credits | | `list_projects` · `get_visibility` · `get_scan_history` · `list_gaps` | Review projects, KPIs, past scans, and gaps | free | | `generate_machine_readability` | Generate robots.txt / sitemap.xml / llms.txt for a domain | free | Example: *"Audit whether **Acme** shows up when people ask 'best project management tools' and 'AI note-taking apps', and tell me who gets cited instead."* ## Design System "Bloomberg Terminal meets Luxurious Magazine" — dark galaxy aesthetic. | Element | Value | |---------|-------| | Background | `#06060e` (deep space navy) | | Cards | `#0c0c18` with `#1e1e35` borders | | Accent triad | Violet `#8b5cf6` / Cyan `#06b6d4` / Rose `#e879a0` | | Display font | Instrument Serif | | Body font | Geist Sans | | Data font | Geist Mono | ## Claude Skills The `skills/` directory contains the Claude skill definitions organized in the Anthropic subfolder convention (each skill is a named folder with a `SKILL.md` inside). See [`skills/README.md`](skills/README.md) for the full list. ## Roadmap Direction, in rough priority order. Larger themes are tracked as GitHub Issues labeled `roadmap`, where the detail and discussion live; everything else is a regular Issue (`good first issue` marks self-contained starting points). 1. **Scheduled scans for self-hosters** — headless BYOK key resolution and honoring the saved schedule config, so re-scans run on a schedule without a browser open 2. **Close the measurement loop** — correlate published content with visibility changes (a unit-tested match ladder is already scaffolded in `ui/src/lib/attribution.ts`), and track which content characteristics correlate with gains 3. **One-click workflows** — deterministic Scan / Create / Publish orchestrators (plain TypeScript, no agents) with progress, cost tracking, and cancel 4. **Deeper machine access** — MCP server v2 (generate/score/publish tools, SSE transport), an OpenAPI spec from the existing Zod schemas, and API token auth 5. **Channel integrations** — cross-posting (LinkedIn, Medium, Dev.to), social sharing, and Google Search Console / Bing Webmaster feedback loops 6. **Intelligent agents & autopilot** *(experimental)* — adaptive orchestrators on top of the workflows, and eventually a guardrailed end-to-end mode with dry-run and cost ceilings. Today the pipeline always stops between stages for human approval. ## Contributing We welcome contributions! See [CONTRIBUTING.md](CONTRIBUTING.md) for setup instructions, coding standards, and PR guidelines. ## License AEO Advisor is licensed under the [Apache License 2.0](LICENSE) — free to use, modify, and self-host, including commercially. See [`LICENSE`](LICENSE) and [`NOTICE`](NOTICE). ## Support If AEO Advisor is useful to you, consider [sponsoring the project](https://github.com/sponsors/kailichama) to help fund development and API costs.