harness-scope

# HarnessScope <!-- mcp-name: io.github.moongioh/hscope -->  *Every frame above is `hscope demo`: a synthetic corpus, generated locally. Real transcripts carry full prompt text and are never demo material.* Turn-level **governance observability** for Claude Code sessions. HarnessScope parses your local transcript JSONL files into SQLite, judges every turn against your workspace's own operating rules (plan-before-code, verify-before-done, context budget, path boundaries, …), and serves a local web viewer that shows *how* each session traversed that governance structure — passes included, not just violations. Screens: **Session Replay** (turn timeline with context chips + rule verdicts) · **Harness Circuit** (fixed-topology schematic of one turn's context sources → agent → action surfaces → governance outputs, with round-trip/off-host edges) · **Rule Health** (compliance × coverage per rule) · **Token/Loop** (per-turn token burn + loss pinpoint markers) · **Setup** (every hook, MCP server, skill and subagent you registered, next to what your transcripts show was actually called — `active` / `unused` / `unregistered-call`) · Plans hierarchy (work items above sessions — backend shipped, UI in progress). Everything is **local-first**: no transcript data leaves your machine (the one exception — the opt-in Tier B LLM judge — is off by default and fires only on an explicit per-session click). The tool never writes to your projects, your `.mcp.json` or your harness settings: a tool that observes governance must not reshape the surface it observes. The single exception is hscope's own `~/.hscope/<workspace>/workspace-model.json`, which the Setup page and the MCP server can edit — schema-checked, path-confined, and gated (a session token in the URL hscope opens; `hscope --no-edit` turns it off entirely). ## Requirements - **Python 3.11+** (3.10 works; developed on 3.13) - `pip install -r requirements.txt` — PyYAML, FastAPI, uvicorn. Nothing else. - **No Node.js needed.** The React UI ships pre-built in `web/dist` (committed on purpose); Node 20+ is only required if you want to hack on the UI itself (`cd web && npm install && npm run build`). - A machine with Claude Code transcripts under `~/.claude/projects/` (the default scan root). Optional, only for the Tier B intent judge: `anthropic` (default backend, needs `ANTHROPIC_API_KEY`) or `google-genai` (`HSCOPE_TIER_B_BACKEND=gemini`, Vertex ADC). ## See it without giving it your transcripts ```bash hscope demo --serve # builds a synthetic corpus, judges it, opens the viewer ``` Three invented sessions: one compliant, one wasteful (re-reads, dead context, a boundary read, a completion declared with nothing verifying it), and one fan-out (two subagents — one cites what it read, one does not; one spawned in the background — plus an off-host MCP round trip). It is held to the same standard as real data: the corpus must satisfy `hscope doctor`, so the generator writing a shape the parser does not expect is caught by the invariants. It already earned that twice — a `requestId` reused across sessions (which hscope correctly read as one API call replayed by a fork), and a padded `agentId:` that the parser's regex swallowed whole. ## Zero-config first run ```bash pip install hscope # or: pipx install hscope · npx -y hscope (Python 3.10+ still required) hscope # scan ~/.claude/projects → ingest → judge → check → open the viewer ``` No flags, no config file. Two things it will not get wrong: - **The port is found, not assumed.** 8000 is a reasonable default and on some Windows machines it is a reserved range (WinError 10013). A first impression should not be a stack trace. - **It says that transcripts expire.** The harness deletes them after about 30 days, and **ingest is what makes them permanent**. Ingest less often than that and you lose sessions silently — a session never ingested is indistinguishable from one that never happened. The first run prints how old your oldest transcript is and prescribes a cadence. It also announces which rules it is about to judge with — neutral presets, or the workspace model(s) it found on disk. Claiming "presets" while quietly applying a model would be the same lie as a silent fallback. ## Setup, delegated to the LLM you already have Nobody should hand-write rules to try a tool. When the first run finds no workspace model, it seeds one from this directory's structure and leaves a page for your model to finish: ```bash hscope # ...and, with no rules of your own, writes ~/.hscope/<id>/HSCOPE-SETUP.md hscope mcp --install # prints an MCP snippet; paste it into .mcp.json hscope setup # regenerate the page whenever your canon moves ``` `HSCOPE-SETUP.md` is one markdown page with two readers. For you: the canon, hooks, MCP servers, skills and subagents hscope found, and the canon clauses no rule cites yet. For your LLM: the instructions to call `get_setup_guide` → `suggest_rules` → `set_rule` and write those rules. You approve; it drafts. The whole journey is **three manual actions** — run `hscope`, paste the snippet, hand over the page. The snippet carries `HSCOPE_MCP_WRITE=1`, so pasting it *is* the write consent; there is no second step. And hscope never edits `.mcp.json` itself: a tool that observes governance must not reshape the surface it observes. ## Quickstart ```bash pip install -r requirements.txt # 1. Parse transcripts into SQLite (default scan: ~/.claude/projects) python -m hscope.cli ingest --db harness_scope.db # 2. Run the rule engine (add --model to overlay your workspace model) python -m hscope.cli rules --db harness_scope.db \ --model ~/.hscope/<workspace>/workspace-model.json # 3. Serve the viewer (API + built UI) HSCOPE_DB=harness_scope.db \ HSCOPE_WORKSPACE_MODEL=~/.hscope/<workspace>/workspace-model.json \ python -m hscope.server # → http://127.0.0.1:8000 ``` Steps 1–2 are idempotent — re-run them any time to refresh; step 2 wipes and rebuilds all derived judgments (rule evaluations, work-item links) from scratch. ### Environment variables | Variable | Default | Meaning | |---|---|---| | `HSCOPE_DB` | `./harness_scope.db` | SQLite path the server reads | | `HSCOPE_WORKSPACE_MODEL` | *(unset)* | Path to `workspace-model.json`; unset ⇒ neutral built-in presets | | `HSCOPE_HOST` / `HSCOPE_PORT` | `127.0.0.1` / `8000` | Server bind. Some Windows machines reserve 8000/8085–8184 (WinError 10013) — pick e.g. `9137` | | `HSCOPE_WORKSPACE_ROOT` | repo parent | Root the work-item convention's relative paths resolve against | | `HSCOPE_TIER_B_BACKEND` / `HSCOPE_TIER_B_MODEL` | `anthropic` / backend default | Tier B judge provider/model (opt-in feature) | | `HSCOPE_MCP_WRITE` | *(unset)* | `1` lets the MCP server edit **its own** workspace model. Unset ⇒ read-only | ## Sessions are not one model talking to itself A session spawns subagents, and they run on models the parent does not. HarnessScope collects that structure instead of flattening it: - **A subagent belongs to the turn that spawned it** — and to the turn that *resumed* it. The harness stamps every record of a prompt with a `promptId`; a subagent's header record carries its parent turn's, and a `SendMessage` continuation carries the later turn's. So the link is read, never inferred from timestamps. 15% of this workspace's subagents were resumed at least once; their work is charged to the turn that asked for it. - **A turn is one user prompt.** Not "any user record with text": interrupt notices, `<command-name>` echoes and `<bash-input>` blocks are user records too, and counting them inflated turn totals by 28% on this workspace's own transcripts. - **Each context window counts separately.** A subagent reading a file the parent already read has not re-read anything — it never had those tokens. `tool_uses.agent_id` keeps the boundary, so re-read waste (R3/R8) is charged to the context that actually paid it. On this workspace that removed **42% of the reported re-read waste** as false. - **The model dimension survives.** `session_report` returns which models ran, which of them only ever ran inside subagents, and what each spent. - **The circuit draws the round trip.** A subagent is a node with a spawn edge (who ran, on which model) and a return edge — labelled with what came *back*, which is not what the subagent *spent*. One real turn: three `general-purpose` agents on Opus burned 16.5k / 6.3k / 5.3k output tokens and handed the parent 1.5k / 2.2k / 2.0k. A background spawn's tool_result is only a launch acknowledgement, so its return size is reported as unmeasured rather than invented. Work a subagent did is tagged `sub`, never folded into the parent's. - **One usage record per request — the last one.** A streamed response writes several records for one `requestId`, and the early ones carry a partial `output_tokens` (2, where the finished response says 31,979). Deduplicating on the *first* occurrence undercounted this workspace's output by 3.6%, and far more inside subagents. (Two independent keys agree on the answer: max-per-`message.id` and last-per-`requestId` both give 32,348,242 where the naive sum of every record gives 90,957,147.) - **A shared request is disclosed, not silently subtracted.** A fork, resume or compact replays an assistant turn into a second transcript. Each session's own total is right; summing sessions counts that API call once per session. `/api/summary` reports `shared_requests` and `shared_request_tokens` (here: 28 calls, 27,059 output tokens, 0.08% of the total) so a workspace figure never pretends to be a clean sum. ## The workspace model (optional but recommended) Without a model, HarnessScope runs on **neutral presets**: generic rules with empty domain values, path labels fall back to file names, sessions render as a flat list. Crash-free in any repo — that's the de-domaining contract. With a model (`~/.hscope/<workspace>/workspace-model.json`) you declare, in one JSON document, how *your* workspace is governed: - `rules[]` — archetype instances with your params (keywords, thresholds, markers) - `roles[]` — what each path / MCP server / subagent *means* (label, circuit node, host boundary) - `gates[]` — your governance pipeline (e.g. context-load → plan-approval → implement → verify → ledger) - `unmeasurable[]` — canon clauses no detector can check (surfaced honestly instead of faked) - `work_items` — optional upper grouping above sessions (this workspace: `plans/**` files with a status frontmatter; other repos can map issues/PRs into the same slot). Absent ⇒ flat sessions. Schema and a neutral example live in `hscope/schema/`. Validate yours with: ```bash python -m hscope.schema.validate_model ~/.hscope/<workspace>/workspace-model.json ``` ## MCP surface Any MCP client can read what HarnessScope observed — and the loop closes: an agent opens a session by reading how the last one scored, instead of re-earning the finding. ```jsonc // .mcp.json { "mcpServers": { "harness-scope": { "command": "python", "args": ["-m", "hscope.cli", "mcp", "--db", "/path/to/harness_scope.db"] } } } ``` | Tool | | |---|---| | `list_sessions` · `session_report` · `token_hotspots` · `rule_health` | the viewer's numbers | | `get_workspace_model` | how this repo is governed, as one document | | `get_setup_guide` · `suggest_roles` · `suggest_rules` | evidence for authoring rules | | `set_role` · `set_rule` | write them back | **No model runs inside this server.** The client *is* the model: `get_setup_guide` hands it the closed archetype library and each archetype's param schema, `suggest_rules` hands it the canon clauses no rule cites, and the judgment comes back through `set_rule` — where the params are validated against that archetype's schema before anything is written. So the MCP path needs no provider, no credential and no off-host consent. `hscope compile` does the same job in one batch LLM call; this is its conversational, incremental counterpart. Writes are **off by default** (`HSCOPE_MCP_WRITE=1`) and can only ever touch `~/.hscope/<workspace>/workspace-model.json`. There is no tool that writes a project file or steers another agent — HarnessScope observes and judges; it does not drive. ## Verification harnesses (Definition of Done) Synthetic-transcript test suites, no fixtures required. Run them all with: ```bash python -m hscope.verify_all # discovers every hscope/verify_*.py; CI runs this on 3 OSes ``` Individually, for when you want one harness's report (this list is illustrative — the runner above is the source of truth, so a new harness never goes unrun): ```bash python -m hscope.verify # P1 parser (takes transcript paths; not in verify_all) python -m hscope.verify_rules # rule engine (archetypes, evaluations, de-domaining) python -m hscope.verify_circuit # circuit API (nodes, round-trips, gate lane) python -m hscope.verify_work_items # plan hierarchy (inference, lifecycle, guards) python -m hscope.verify_context_tiers # declared context-injection tiers python -m hscope.verify_workspace # workspace -> session model resolution python -m hscope.verify_compile # `hscope compile` (offline; stub provider) python -m hscope.verify_defense # parser defense / graceful degradation python -m hscope.verify_mcp # MCP surface (protocol, write gate, confinement) python -m hscope.verify_agents # multi-agent / multi-model collection (spawn turns, contexts) python -m hscope.verify_doctor # each invariant fires on its own break, and only its own python -m hscope.verify_firstrun # zero-config run (port discovery, retention notice) python -m hscope.verify_demo # the demo corpus: synthetic, ASCII, passes doctor python -m hscope.verify_ingest # incremental ingest: what must NOT be skipped python -m hscope.verify_setup # the handoff page + an LLM closing the loop over MCP python -m hscope.verify_i18n # viewer strings are English-canonical python -m hscope.verify_scrub # nothing publishable carries domain values ``` ## `hscope doctor` — drift is detected, not tuned away The transcript format is undocumented and it moves: this corpus alone declares **27 harness versions in one month**. hscope reads it anyway, because it is the only surface that knows *meaning* — which turn, which agent, which file. Metrics know quantities; a bill knows money; neither knows the word "subagent". So the answer to drift is not endless tuning. It is a check that fails loudly, naming the harness version that broke it, instead of quietly emitting a wrong number: ```bash python -m hscope.cli doctor --db harness_scope.db # exit 1 if any invariant is violated ``` Nine invariants, each measured true before it shipped — a turn-start record carries its `promptId`; a subagent row carries its `agentId`; the counted usage row is the request's *final* usage; a request never spans a main and a subagent file; a subagent sits on a turn its parent really had; and so on. Two things it refuses to do: - **Judge rows an older hscope wrote.** Afterwards, "the harness omitted the field" and "we never stored it" are the same absence. Sessions are stamped with the schema that read them (`ingest_schema`) and older ones are excluded *and counted*, never mixed in. - **Report a vacuous pass.** Zero violations over zero verifiable rows is not a green check. Alongside them it prints **metrics, never verdicts**: the share of spawn calls that name the agent they started (skills and workflows spawn without one), the requests replayed into more than one session, and the share of assistant records that carry text while reporting ≤1 output token — the reason a per-agent *spend* is a floor rather than a measurement. ## Failure behavior (graceful degradation) A governance tool that quietly checks nothing is worse than one that stops. So: | Situation | What happens | |---|---| | A transcript line will not parse | Counted and reported as `unparseable lines: N`. Never dropped in silence. | | A record type this build does not know | Counted per type and printed (`unknown rec types`). Ingest continues. | | A database written by a **newer** hscope | Refused with an upgrade message. A column this build ignores is a rule that silently stops checking. | | A database from before schema versioning | Migrated forward and stamped. | | `--model` points at a missing file | Hard error. A typo must not quietly hand you the neutral preset's numbers as if they were yours. | | A workspace model is unreadable, non-object, or from a future `schema_version` | Refused by name. During auto-resolution the *other* models still load and each skip prints why. | | `$HSCOPE_WORKSPACE_MODEL` is bad | The viewer keeps serving with no roles, and says so once. (An empty circuit otherwise looks exactly like a compliant one.) | | No LLM provider for `hscope compile` | Deterministic structure + the neutral preset rules, with the provider's absence reported. | | An MCP tool call is refused (unknown tool, bad params, writes off) | Returned as `isError` **content**, not a protocol fault. The client reads the reason and keeps its session. | | An MCP `set_rule` / `set_role` would produce an invalid model | Nothing is written. A governance tool that corrupts its own config judges every later session against garbage. | | A transcript has no `promptId` (pre-dates the field) | Turns fall back to text heuristics, and the run reports how many turns were detected that way. | | A subagent file names a spawn turn the main file never had | Parked on turn 0 **and listed** as `unlinked subagents`. Silently parking it would charge a turn that spawned nothing. | | A session is in the DB but its transcript is gone (~30-day retention) | Re-ingest cannot rebuild it. It is counted and reported as `not rebuilt`, so a DB of two vintages never looks like one. | | A subagent was spawned in the background | Its returned payload size is unknown (the tool_result is a launch ack). The circuit says "background", never a fabricated token count. | | A subagent is resumed under a prompt the main transcript never recorded | Its rows keep the turn they were on, and the run reports `unlinked resumes: N`. | ## Data & privacy - The SQLite DB contains **full prompt text** from your transcripts. It is gitignored (`*.db`) and must stay local — never commit or upload it. - The server binds `127.0.0.1` by default and is read-only over the DB. - Tier B (LLM judge) sends a per-session digest off-host **only** when you click the judge button for that session; the rule (`B1`) ships disabled. ## License & open-core boundary HarnessScope is **Apache-2.0 open core**. Everything an individual user touches is free forever, with no feature limits — paid tiers (if/when they exist) only ever add things an *organization* needs, never remove things from individuals: | Layer | Free (Apache-2.0 core) | Enterprise (`ee/`, future) | |---|---|---| | Capture | Transcript ingestion + hooks | MCP gateway (cross-harness capture) | | Analysis | Rule engine · circuit · token loss · coverage — all of it | Multi-user aggregation, org dashboards | | Security | Local security-flow rules (self-audit) | Org compliance reports, audit logs | | Accounts | Local, single user | SSO / RBAC | See `LICENSE` (Apache-2.0), `ee/README.md` (boundary), and `CONTRIBUTING.md` (DCO). ## Repo layout ``` hscope/ # Python package: cli, parse (JSONL→SQLite), db, rules engine, # work_items (plan hierarchy), server (FastAPI), mcp (stdio), # compile (canon→model), tier_b, verify_* hscope/schema/ # workspace-model JSON Schema + neutral example + validator web/ # React viewer source (Vite); web/dist = committed build docs/ # design notes per work package ee/ # enterprise-edition boundary (empty placeholder — see ee/README.md) ```