grimoire

# Grimoire Personal Knowledge Base MCP Server. Inspired by Andrej Karpathy's LLM Wiki approach, using a markdown KB as a single source of truth, allowing Claude Code to reuse context without repetitive explanations (RAG-lite), and writing outputs according to classification conventions. ## Why Create Grimoire? Existing approaches did not match our workflow patterns. - **VectorDB-based RAG**: Knowledge is replicated in VectorDB, disconnected from the original markdown, and the embedding infrastructure (VectorDB + embedding/summarization model) has significant operational burdens. - **Tool-embedded wiki (git-ignored)**: Karpathy's model fits well but has a storage location within the tool, causing disconnections in mobile/Obsidian/git synchronization. Grimoire uses the user's markdown KB as the single source of truth. It preserves the Obsidian + git synchronization workflow and operates without a separate embedding infrastructure. ## Core Principles 1. The original source is the markdown KB. No separate storage is created. 2. No embeddings are used. The search routing subject is Claude (Karpathy: "Delegate search to LLM"). The index is SQLite FTS5. 3. AI access boundary = pull blocking / push allowed. The blocking path (config `boundary.locked_dirs`/`private_dirs` + code hard guard `HardLockedDirs` + taxonomy unregistered directory fail-safe) is excluded from search/indexing/automatic injection but allows writing and explicit single-item reading. **Unclassified (taxonomy unregistered directory) is private by default**, and exposure is only allowed through note frontmatter `ai_access: shared` explicit opt-in. `HardLockedDirs` is a last safety net that remains unchanged even if the config is empty or edited incorrectly. 4. Scalability = fixed engine + externalized config (`kb.config.json`). Other users only need to replace the config. 5. Frontmatter is utilized if present; if not, it is inferred from the path/title/body (fallback), with gradual backfilling. ## Stack | Component | Choice | |---|---| | Language/Runtime | Go (single binary, no cgo) | | Deployment | stdio per-session | | MCP | Official `modelcontextprotocol/go-sdk` v1.6.1 | | Index | `modernc.org/sqlite` FTS5 (pure Go) | | Frontmatter | `gopkg.in/yaml.v3` | | Glob | `bmatcuk/doublestar/v4` | > Started with Bun + TypeScript, but switched to Go due to memory/deployment reasons. Refer to [docs/design.md](./docs/design.md) §8 for rationale and measurements. ## Structure ``` grimoire/ kb.config.example.json # KB configuration template (copied to kb.config.json; actual config is gitignored) README.md / docs/design.md / docs/configuration.md # documentation/design background/configuration guide cmd/ grimoire/ # stdio MCP server (9 tools, incremental Sync on start + periodic mtime) reindex/ # full reindexing CLI (verification/recovery) grimoire-context/ # SessionStart hook context injection helper (optional) lint/ # batch health check CLI (no Ollama required) internal/ config/ # kb.config.json loading/validation boundary/ # AI access boundary guard (pull blocking/push allowed) frontmatter/ # frontmatter parsing + fallback inference index/ # SQLite FTS5 index + indexer (Reindex / Sync) writer/ # write_note: classification remapping/file name/frontmatter/atomic write/redact contextsig/ # cwd → jump registry remapping (project inference) compiler/ # lint(structural inspection) + suggest/apply(frontmatter backfill) ollama/ # optional Ollama client (generation only, default inactive) bin/ # build output (gitignored) ``` ## Configuration ```bash cp kb.config.example.json kb.config.json # modify kb.config.json's kb.root to your KB path and taxonomy/boundary to your directory structure mkdir -p ~/memo # KB root is only verified for existence (not created) by config.Load → create beforehand ``` The actual `kb.config.json` contains personal paths/structures, so it is gitignored. Only the generic template `kb.config.example.json` is committed to the repository. ## Build / Run ```bash # build go build -o bin/grimoire ./cmd/grimoire # MCP server (incremental Sync on start) go build -o bin/reindex ./cmd/reindex # full reindexing CLI (verification/recovery) go build -o bin/grimoire-context ./cmd/grimoire-context # SessionStart hook helper (optional) go build -o bin/lint ./cmd/lint # health check CLI (no Ollama required) # full reindexing + verification statistics output (recovery path if index is damaged) ./bin/reindex kb.config.json # run MCP server (stdio) directly — synchronize changes based on mtime on start ./bin/grimoire kb.config.json ``` ## Claude Code Registration ```bash claude mcp add grimoire -- ~/tools/grimoire/bin/grimoire ~/tools/grimoire/kb.config.json ``` The config **file** path is specified by CLI argument or `GRIMOIRE_CONFIG` environment variable (argument takes precedence). The note **root** is specified by `kb.root` in the config. ## MCP Tools | Tool | Function | Blocking Path | |---|---|---| | `get_index` | routing for table of contents (title/tags/summary/path) | excluded | | `search` | FTS5 keyword + tag/directory filter | excluded | | `read_note` | read body | allowed for explicit single item only | | `links` | `[[link]]` graph (outgoing/incoming) | excluded | | `write_note` | save with classification convention (remapping/frontmatter/atomic/redact) | write allowed | | `get_context` | cwd→project inference + related runbook/note candidates | excluded | | `get_runbook` | return repetitive procedure (`type:runbook`) | excluded | | `lint` | health check (frontmatter missing/dangling link) — no Ollama required | excluded | | `suggest_frontmatter` | Ollama frontmatter backfill suggestion (apply records only missing keys) | excluded/denied | ## Usage (Workflow) Tools are automatically called by Claude Code. The following are the intentions and triggers for each flow. ### 1) Context Inquiry (RAG-lite) The search subject is Claude. It receives the KB map through `get_index`/`search`, selects a page to read, and uses `read_note` for the body and `links` to track connected notes. ``` "Find and summarize the dev cluster installation procedure in the sre guide" → search({query:"dev cluster installation", dir:"sre-guides"}) → read_note(...) ``` ### 2) Output Storage (write_note) Analysis/documents are stored in a location that matches the classification convention. The directory is remapped by `type`/`domain` (or directly specified by `dir`), and the file name and frontmatter are automatically generated. ``` "Save this analysis as a backend analysis" → write_note({title, content, type:"analysis", domain:"backend"}) → backend/<topic>-<detail>.md with frontmatter included ``` If the classification is unclear, `ok:false` + candidate directories are returned. The `redact:true` directory (e.g., `personal-blog`) rejects storage if internal identifiers are mixed. ### 3) Runbook: Write repetitive procedures once → reuse every time Write a `type:runbook` **once** for repetitive procedures like deployment/cluster access, and Claude will reuse the context through `get_runbook` thereafter (removing repetitive explanations). ``` # (once) Write runbook "Organize the proj-a deployment procedure as a runbook, including alpha/beta kubeconfig paths and deployment commands" → write_note({title:"proj-a deployment runbook", type:"runbook", tags:["proj-a","deploy"], content:...}) → runbooks/proj-a-deploy.md # (later) Reuse "Proceed with proj-a alpha deployment" → get_runbook({name:"proj-a"}) → use the runbook body (procedure/cautions) for the task ``` > `get_runbook` only considers `type:runbook` notes. If there are zero runbooks in the current KB, an empty list is returned, so the (once) writing is necessary beforehand. The `runbooks` directory is defined in config `taxonomy.directories`, so `type:runbook` write is remapped to it. ### 4) Working Path Context (get_context) Infer the project by mapping cwd to the jump registry (`context_signals.jump_registry`) and narrow down related runbooks/notes. ``` get_context({cwd:"/Users/you/projects/proj-a"}) → infer project:"proj-a" → return related runbooks/notes candidates ``` If there is no signal or no match, `enabled:false`/empty candidates are returned, and routing depends solely on the explicit `scope` argument. #### (Optional) Automatic Injection at Session Start Connecting `bin/grimoire-context` to Claude Code's `SessionStart` hook automatically injects runbooks/notes for the current working path at session start. If there is no match, nothing is output. ```jsonc // add group to ~/.claude/settings.json's hooks.SessionStart { "hooks": [ { "type": "command", "command": "~/tools/grimoire/bin/grimoire-context ~/tools/grimoire/kb.config.json" } ] } ``` ### 5) Health Check & Frontmatter Backfill (Compiler) As the KB grows, frontmatter missing/dangling links accumulate. Use `lint` for inspection, and optionally, Ollama suggests frontmatter. ```bash # structural health check (no Ollama required) — report top 50 issues ./bin/lint kb.config.json # top 50 ./bin/lint kb.config.json --all # all ``` `suggest_frontmatter`(MCP tool) suggests frontmatter candidates using Ollama. **Default inactive**, enable with config: ```jsonc "ollama": { "enabled": true, "host": "http://localhost:11434", "model": "qwen3:8b" } ``` ``` "Fill in the frontmatter for this note: backend/redis-penetration.md" → suggest_frontmatter({path:"backend/redis-penetration.md"}) # suggestion only (for review) → suggest_frontmatter({path:"...", apply:true}) # record after review ``` Safeguards (design §11): - **suggestion → review → record** order. `apply:true` only inserts **missing core keys** and never overwrites **existing keys** (idempotent, atomic write). - If the file name has no date, do not arbitrarily generate `date`. - Blocking paths are not **transmitted to or modified by** Ollama. - If Ollama is inactive/disabled, handle gracefully with `ok:false` (engine's other functions are normal). > Any text generation model from Ollama can be used. Backfill is a short JSON classification, so a lightweight model suffices (e.g., `qwen3:8b`, `gemma3:4b`, `llama3.2:3b`). This is not embedding. ## Current Status Functional tools: RAG-lite 4 tools (`get_index`/`search`/`read_note`/`links`) + `write_note` (classification convention storage) + incremental indexing + `get_context`/`get_runbook` (working path/runbook routing) + `lint`/`suggest_frontmatter` (health check/Ollama backfill). Future (optional): add embedding layer when documents grow to tens of thousands and exploratory questions increase. ## License Pending (personal project).