agentglass

<div align="center"> # ✦ AgentGlass ### A glass box for your AI agents. **Capture, inspect, replay, and analyze every agent run — no SDK, no code changes.** Point your agent's base URL at AgentGlass and see every LLM call, tool invocation, token, and dollar it spends. Live, on your own machine.      _{Live dashboard → a run torn open (waterfall, context growth, every step's I/O) → analytics → ask your runs a question.} </div> --- ## Why AgentGlass exists Your agent just made 14 LLM calls, invoked 6 tools, burned 180k tokens, and cost you \$2.30 — and you have **no idea** what happened inside. Which step blew the context window? Which tool call failed and got silently retried? Which model ate your budget? Modern AI agents are **black boxes**: you see the final answer, not the messy, expensive path they took to get there. **AgentGlass makes them glass.** It sits transparently between your agent and the model provider (Anthropic or OpenAI), records everything that flows through, and turns it into a dashboard you can actually read — trace by trace, span by span, token by token, dollar by dollar. - 🛰️ **No instrumentation.** No SDK to install, no wrapper to import. Set one environment variable and you're capturing. Works with Claude Code, the OpenAI SDK, LangChain — anything that speaks the Anthropic or OpenAI wire format. - 🔒 **Local-first.** Everything runs on your machine and stays there. One process, one SQLite file, no cloud, no account, no telemetry. - 🧱 **No native modules.** Built on Node's built-in `node:sqlite` — nothing to compile, installs clean on every platform. - 🎛️ **Reads like an instrument.** A dense, dark, purpose-built console: tool-call waterfalls, per-step token & cost meters, a context-window chart, run diffs, spend analytics, and an assistant that answers questions about your runs. ``` your agent ──▶ AgentGlass ──▶ Anthropic / OpenAI │ ▼ records + streams │ ▼ 📊 the dashboard ``` ## Quickstart ```bash npx agentglass ``` That's it. Open **http://localhost:4319** — the dashboard is live with a week of **simulated** agent runs, so every screen has something to show before you've wired up anything at all. Then point a real agent at it: ```bash # Claude / Anthropic SDK export ANTHROPIC_BASE_URL=http://localhost:4319 # OpenAI SDK — also OpenRouter, LiteLLM, Groq, Together, vLLM, Ollama… export OPENAI_BASE_URL=http://localhost:4319/v1 ``` Run your agent as usual. Every call, every tool, every token, every cent shows up — **no SDK, no decorators, no code changes.** <details> <summary>Other ways to run it</summary> ```bash # Container docker run -p 4319:4319 -v agentglass:/app/data ghcr.io/vugarfamiloglu/agentglass # From source, with hot reload (dashboard on :4318, server on :4319) git clone https://github.com/vugarfamiloglu/agentglass.git cd agentglass npm install && npm --prefix web install npm run dev ``` </details> --- ## A tour of every screen AgentGlass is organized into three groups in the left rail — **Observe** (what's happening), **Analyze** (what it cost and how it behaved), and **Setup** (how to connect and configure). Here's what each screen is for. ### Overview — the command center  The landing page answers "how are my agents doing right now?" at a glance. Six **stat cards** show total runs, all-time spend, tokens used (and how many were served from cache), tool calls, error rate, and average latency. Below them, a **spend chart** plots hourly cost over the last day, and a **recent runs** table lists the latest agent runs with their model, status, tokens, cost, and age — updating **live** as new runs stream in. On the right sits the **assistant** (more on that below), always a question away. ### Live — watch runs arrive  A real-time feed of runs the instant they start. Backed by a WebSocket stream, it shows in-flight runs (status `RUNNING`) turning into `OK` or `ERROR` as they finish, with tokens and cost ticking up. This is the screen you leave open on a second monitor while your agents work. ### Traces — every run, searchable  The complete, filterable log of every agent run. Search by run name or model, filter by status (`all` / `ok` / `error` / `running`), and page through the history. Each row is one agent run — its span count, total tokens, cost, duration, and when it happened. Click any row to open the **inspector**. ### Run inspector — open the black box  This is the heart of AgentGlass. Open a single run and see exactly what happened inside: - **Summary chips** — duration, span count, tool calls, tokens, cached tokens, and total cost for the whole run. - **Context window chart** — a stacked bar per LLM call showing how the input context *grows* call-by-call (violet = fresh input, cyan = cached). This is how you spot the step that blew your context budget. - **Timeline waterfall** — every span (LLM call, tool execution, event) as a time-positioned bar, colored by type, so you can see the agent's `plan → tool → observe → act` loop unfold. Each row shows that step's tokens, cost, and latency. - **Click any span** to expand its full input and output JSON, model, and exact token/cost breakdown. ### Run diff — compare two runs  Pick any two runs and compare them side by side. The **metrics** rows show cost, tokens, spans, tool calls, and duration for each, with a colored delta (cyan = better, rose = worse) — so you can answer "was opus actually worth 100× the cost of mini here?" at a glance. Below, the two runs' span sequences sit column by column. ### Analytics — spend, patterns, and breakdowns  The big-picture view. A **spend chart** over time, a GitHub-style **activity heatmap** (runs per day), and two breakdown tables: **by model** (cost, calls, and tokens per model, with proportional bars) and **by tool** (call counts and failure counts per tool). One click **exports every trace to CSV** for your own analysis. ### Models & Tools — per-entity detail  Dedicated pages that zoom into a single dimension. **Models** ranks every model by cost with its call count and token usage; **Tools** ranks every tool by how often it's called and how often it fails — the fastest way to find the flaky tool that's quietly costing you retries.  ### Connect — point your agent here  Copy-paste setup for the recording proxy. It shows the exact `ANTHROPIC_BASE_URL` / `OPENAI_BASE_URL` values, an optional `x-agentglass-session` header for grouping a task's calls into one trace, and a diagram of how requests flow through AgentGlass to the real provider and back — untouched. ### Settings — connect a model, and mind your data  The assistant answers questions about your runs locally with **nothing connected at all**. Point it at a model to unlock open-ended analysis: | | | |---|---| | **Hosted** | Anthropic · OpenAI · Google Gemini · xAI Grok · DeepSeek · Mistral · Groq · Together · OpenRouter · Cohere · Qwen | | **Local** | Ollama · LM Studio — **no key, no cost, nothing leaves the machine** | | **Anything else** | Custom base URL — vLLM, LiteLLM, Azure, your own gateway | Nearly every vendor now serves an OpenAI-compatible `/v1/chat/completions` endpoint, so adding one is a base URL rather than a new client; Anthropic's native Messages API is supported directly. **Load models** asks the provider what it actually serves right now — including which models you've pulled into Ollama — so you're picking from reality instead of a list that rots. Keys are sealed with **AES-256-GCM** in `data/.vault-key`, **one per provider**, so switching back and forth doesn't lose them. The **Data** panel shows how many runs are stored and what they cost on disk, and lets you set a retention window (7 / 30 / 90 days, or keep everything — the default). Anything past the window is pruned on boot and every few hours after. Clearing traces and disconnecting a provider both go through a confirmation dialog. ### Assistant — ask your runs  The right-hand rail (toggle it with the ✦ in the top bar) is a chat that **reads your traces**. Ask "how much have I spent?", "what's my error rate?", "which models cost the most?", or "summarize my tool usage" and it answers instantly by computing over your data — no API key required. On a specific run's page it can answer about *that* run ("why did this cost so much?"). Connect a model in Settings and open-ended questions get routed to it, streamed token by token, with your run context attached — including to a local Ollama model, so you can ask about your agents without a single byte leaving your machine. --- ## How the recording proxy works AgentGlass exposes two endpoints that mirror the providers' own: | Endpoint | Mirrors | Set | |----------|---------|-----| | `POST /v1/messages` | Anthropic Messages API | `ANTHROPIC_BASE_URL=http://localhost:4319` | | `POST /v1/chat/completions` | OpenAI Chat Completions | `OPENAI_BASE_URL=http://localhost:4319/v1` | When your agent calls one of these, AgentGlass **forwards the request to the real provider** (preserving streaming via a stream tee), reads the token usage from the JSON or SSE response, prices it, and records it as a span — then returns the provider's response to your agent **completely untouched**. Recording is fully guarded: a bug in AgentGlass can never change or break the response your agent receives. Successive calls are grouped into a single **trace** (one agent task) either by an explicit `x-agentglass-session` header or by a short idle window, so a multi-step agent run shows up as one coherent tree instead of scattered calls. ### Tool calls, without instrumenting anything The proxy sees only LLM requests — it never watches a tool execute. It gets the tool tree anyway, by reading both sides of the conversation: ``` ┌── request ──────────┐ ┌── response ─────────────┐ │ "fix the auth test" │ → │ text + tool_use(read_file) ← call opens └─────────────────────┘ └─────────────────────────┘ ⋮ agent runs the tool ┌── next request ─────────────────┐ │ …, tool_result(read_file) → ok │ ← call closes └─────────────────────────────────┘ ``` A tool call is parsed out of each response (Anthropic `tool_use` blocks, OpenAI `tool_calls`, streamed or not) and held until its result shows up on the agent's next request. That's when the **tool span** gets written, running from the moment the LLM response finished to the moment the next request landed — precisely the window the agent spent executing it. Parallel tool calls share that window, because that's what happened. So pointing Claude Code — or any tool-using agent — at AgentGlass gives you the full `llm → tool → llm` waterfall in the inspector. **Zero code changes, zero SDK, zero decorators. Just a base URL.** ### Which models it knows Around 90 models across **Anthropic, OpenAI, Google, xAI, DeepSeek, Meta, Mistral, Qwen, Cohere, and Amazon**, matched by longest-prefix so a dated id (`claude-opus-4-8-20260101`) or a Bedrock-prefixed one (`anthropic.claude-sonnet-5`) resolves correctly, and an unreleased version still prices from its family. Cache reads and writes are billed at their own rates — which matters, because an agent loop writes cache constantly and Anthropic charges 1.25× input to do it. Anything that speaks `/v1/chat/completions` — **OpenRouter, LiteLLM, Groq, Together, Gemini's OpenAI-compatible endpoint, vLLM, Ollama** — records through the OpenAI route today and prices from the same table. ## Features | | | |---|---| | 🛰️ **Recording proxy** | Anthropic- & OpenAI-compatible endpoints that transparently forward and record — streaming preserved. | | 🔧 **Tool calls, uninstrumented** | Reconstructs the full `llm → tool → llm` tree from proxy traffic alone. No SDK, no decorators. | | 🌲 **Trace explorer** | Every run as a tree of spans: LLM calls, tool executions, events. | | 🔬 **Run inspector** | Waterfall timeline, context-window chart, per-step token & cost, full I/O. | | 💸 **Token & cost accounting** | ~90 models across 10 providers, cache reads and writes priced separately, rolled up per step and per run. | | ⚡ **Live stream** | Runs light up in the dashboard as they happen, over WebSocket. | | 🔀 **Run diff** | Compare two runs side by side — cost, tokens, latency, spans. | | 📈 **Analytics** | Spend over time, activity heatmap, breakdowns by model and tool, CSV export. | | 🤖 **Ask-your-runs assistant** | Answers about your traces locally; or connect any of 13 providers — including local Ollama — streamed token by token. | | 🧹 **Retention** | Keeps everything by default; pick a window and old runs are pruned automatically. | | 🔒 **Local & sealed** | Runs on your machine; the one secret (assistant key) is AES-256-GCM vaulted. | ## 🛠 Tech Stack | Layer | Technology | |-------|-----------| |  | Server runtime — one process serves API, WebSocket, proxy, and UI | |  | Tiny, fast HTTP + WebSocket framework | |  | Built-in trace store (WAL) — no native modules | |  | Dashboard SPA | |  | Frontend build + dev server | |  | End to end | All charts and visualizations are hand-rolled SVG — no charting library. ## Architecture ``` agentglass/ src/ server (Node + Hono + node:sqlite) index.ts entry — API + WebSocket + proxy + SPA hosting proxy.ts the recording proxy — correlation + tool reconstruction providers.ts provider wire formats (what flows *through* the proxy) llm.ts the assistant's LLM clients (what it calls *out* to) db.ts trace store (traces → spans, WAL, rollups) sim.ts simulator — a week of realistic runs, streamed live assistant.ts ask-your-runs (local answers + optional LLM) vault.ts AES-256-GCM key sealing pricing.ts the model catalog + token pricing retention.ts pruning old traces hub.ts / api.ts live event fan-out / REST surface web/ dashboard (Vite + React 19 + TypeScript) src/pages/ Overview, Traces, TraceDetail, Diff, Analytics, … src/components/ hand-rolled SVG charts, the assistant rail, … scripts/ record-demo.mjs — re-cuts the GIF at the top of this file deploy/ fly.toml for a read-only public demo docs/ demo.gif + the screenshots in this README ``` ## Docker ```bash docker run -p 4319:4319 -v agentglass:/app/data ghcr.io/vugarfamiloglu/agentglass ``` Open **http://localhost:4319**. The container serves the built dashboard, the API, the WebSocket stream, and the recording proxy — all on one port. Images are built and smoke-tested on tag; `docker build -t agentglass .` works locally too. ## Hosting a public demo A hosted AgentGlass can't be the one you run locally: strangers shouldn't be able to clear your traces, write a key into your vault, or proxy their agents through your box. `AGENTGLASS_READONLY=1` serves reads and refuses the rest — no proxy, no writes, and no model-discovery call, which fetches a URL the caller supplies and on a public host would let anyone probe your network from the inside. Asking the assistant still works: with no key it's computed from the store, so it costs nothing and reaches nothing. ```bash fly deploy --config deploy/fly.toml # read-only + simulator, no volume ``` No volume is deliberate — the data directory stays ephemeral, so every cold start seeds a fresh week of runs and the demo can never grow stale or unbounded. ## Configuration | Env | Default | Purpose | |-----|---------|---------| | `AGENTGLASS_PORT` | `4319` | Dashboard / API / proxy port | | `AGENTGLASS_DATA` | `data` | SQLite store + vault key directory | | `AGENTGLASS_SIMULATE` | `1` | Seed & stream simulated runs (set `0` to disable) | | `AGENTGLASS_READONLY` | `0` | Serve reads only — for a public demo (see above) | | `ANTHROPIC_API_URL` | `https://api.anthropic.com` | Upstream to forward Anthropic calls to | | `OPENAI_API_URL` | `https://api.openai.com` | Upstream to forward OpenAI calls to | Turn the simulator off (`AGENTGLASS_SIMULATE=0`) once you're feeding it real runs. ## Development ```bash npm run dev # server (:4319) + Vite dev server (:4318) together npm run typecheck # type-check server and web npm test # run the test suite npm run build # build the SPA + compile the server to dist/ npm start # run the production build ``` ## License [Apache-2.0](LICENSE)