MemEcsy

# MemEcsy **A structured long-term memory server for AI agents. ECS world + Tcl reasoning + Odin persistence.** Memory-as-Code: your agent's memories are typed structs, not embeddings. The LLM writes programs (Tcl) to store, recall, and reason over them. Memories decay over time, get reinforced when used, and persist as readable code on disk. Exposes an [MCP](https://modelcontextprotocol.io) server — works with Claude Code, Claude Desktop, Cursor, pie, and any MCP-compatible client. --- ## The Vision We started from a simple question: **what if an AI agent's memory worked like a game world?** In a game engine, thousands of entities live in an ECS (Entity-Component- System) world. Each entity is a bag of typed components. Systems iterate over them. Entities are born, age, interact, and die. The world evolves. **Agent memory today is none of this.** It's either: - **Vector embeddings** — opaque floats, queried by fuzzy similarity. You can't read what the agent "knows." You can't debug why it recalled something. You can't hand-edit a wrong fact. - **Flat key-value stores** — no structure, no relationships, no lifecycle. - **Conversation logs** — grow forever, no forgetting, no prioritization. MemEcsy takes a different path: **memory is structured code, not data blobs.** - Each memory is an **Entity** with typed **Components** (`Fact`, `UserPreference`, `Relationship`). - Memories **decay** over time (exponential forgetting, derived on read — no background tick). - Memories **reinforce** when accessed (access_count slows decay). - Memories **archive** and **sweep** (logical → physical lifecycle). - The LLM doesn't just emit JSON args — it **writes Tcl programs** that compose recall + reasoning + storage in a single call. - The persistent artifact is **Odin source code** (readable, diffable, git-versionable). The agent grows like a game character: useful memories get stronger, irrelevant ones fade, the world becomes richer with every conversation. --- ## The Landscape ### Where MemEcsy fits The AI memory space is heating up. Here's how the approaches compare: | Approach | Examples | How it works | Problem | |---|---|---|---| | **Vector RAG** | Pinecone, Chroma, Weaviate | Embed text → cosine similarity | Opaque, fuzzy, can't inspect/edit/debug | | **Conversation context** | ChatGPT memory, Claude projects | Summarize past turns | Lossy, no structure, grows linearly | | **OS-inspired** | MemGPT / Letta | Page memory in/out of context window | Still embedding-based underneath | | **Key-value** | Mem0, Zep | Store facts as strings, retrieve by tags | No relationships, no decay, no lifecycle | | **MemEcsy** | this project | **ECS world + typed structs + Tcl reasoning + code persistence** | New; requires MCP integration | ### The "Memory as Code" movement Andrej Karpathy has talked about the need for personal AI to maintain a **structured, inspectable knowledge base** — not a black box of embeddings, but something you can read, edit, and version-control. The idea that memory should be **first-class structured data** (closer to code than to vectors) is gaining traction. MemEcsy takes this literally: the on-disk artifact is source code. You `git diff` to see what your agent learned. You hand-fix a wrong fact. You branch memories for different contexts. This is memory **as an auditable program**, not memory as a pile of floats. ### Why not just use a vector database? | | Vector RAG | MemEcsy | |---|---|---| | Query | cosine similarity (fuzzy, ~$0.001/query for embedding) | SOA column scan (exact, $0) | | Inspectability | ❌ opaque floats | ✅ readable Odin code on disk | | Editability | ❌ re-embed the whole corpus | ✅ open the file, change a line | | Relationships | ❌ flat | ✅ Relationship components form a graph | | Forgetting | ❌ manual TTL or nothing | ✅ derived decay + consolidate + sweep | | Precision | ❌ "similar to X" | ✅ "all Facts about user" | | Debuggability | ❌ why did it recall this? | ✅ `cat ~/.memecsy/memory.odin` | Vector RAG has its place (semantic search over large unstructured corpora). MemEcsy is for **structured personal knowledge** — facts about a user, relationships, preferences, life events. Where precision matters more than fuzziness. --- ## Quick Start ### 1. Download + install Pick your platform: **macOS (Apple Silicon):** ```sh curl -L https://github.com/vajraimb/MemEcsy/releases/latest/download/memecsy-server-darwin-arm64.tar.gz \ -o /tmp/memecsy.tar.gz mkdir -p ~/bin tar xzf /tmp/memecsy.tar.gz -C ~/bin/ chmod +x ~/bin/memecsy-server ``` **Linux (x86_64):** ```sh curl -L https://github.com/vajraimb/MemEcsy/releases/latest/download/memecsy-server-linux-x86_64.tar.gz \ -o /tmp/memecsy.tar.gz mkdir -p ~/bin tar xzf /tmp/memecsy.tar.gz -C ~/bin/ chmod +x ~/bin/memecsy-server # libtcl9.0.so needs libtommath1 (bignum math) sudo apt install -y libtommath1 # Debian/Ubuntu # sudo dnf install -y tommath # Fedora # The dynamic linker must find libtcl9.0.so next to the binary. # Add ~/bin to the library search path (one-time): echo 'export LD_LIBRARY_PATH="$HOME/bin:$LD_LIBRARY_PATH"' >> ~/.bashrc # Or for zsh: echo 'export LD_LIBRARY_PATH="$HOME/bin:$LD_LIBRARY_PATH"' >> ~/.zshrc source ~/.bashrc # or ~/.zshrc # Verify it runs: echo '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}' | ~/bin/memecsy-server ``` ### 2. macOS only: fix code signing CI-built macOS binaries get killed by Gatekeeper (exit 137). Run this once: ```sh codesign --remove-signature ~/bin/memecsy-server codesign --remove-signature ~/bin/libtcl9.0.dylib codesign -f -s - ~/bin/libtcl9.0.dylib codesign -f -s - ~/bin/memecsy-server ``` > Linux users skip this step — no code signing on Linux. ### 3. Configure your agent #### Claude Code In your project directory, create `.mcp.json`: ```json { "mcpServers": { "memecsy": { "command": "/Users/YOUR_USERNAME/bin/memecsy-server" } } } ``` Then launch `claude` in that directory. Verify with `/mcp` — you should see `memory · ✔ connected · 1 tool`. #### Claude Desktop Edit `~/Library/Application Support/Claude/claude_desktop_config.json`: ```json { "mcpServers": { "memecsy": { "command": "/Users/YOUR_USERNAME/bin/memecsy-server" } } } ``` Restart Claude Desktop. #### Cursor Add to Cursor's MCP settings (`Settings → MCP`): ```json { "mcpServers": { "memecsy": { "command": "/Users/YOUR_USERNAME/bin/memecsy-server" } } } ``` #### Any MCP-compatible agent Point your agent's MCP config at the `memecsy-server` binary. The server speaks JSON-RPC over stdio and exposes one tool: `memory(script)`. ### 4. Use it Talk to your agent normally. When you share something worth remembering: ``` You: "I have a Corgi named Jordan, he's 9 years old." Agent: (calls memory tool → stores Fact) "That's wonderful! Jordan sounds like a great companion." ``` Next session: ``` You: "How's my dog doing?" Agent: (calls recall → finds "has-dog: 柯基犬", "age: 9-years") "Jordan is 9, right? A Corgi — must be adorable!" ``` Check what your agent knows: ```sh cat ~/.memecsy/memory.odin ``` ```odin remember("jordan", "user", "Fact", "predicate", "has-dog", "object", "柯基犬", "confidence", "1") remember("jordan-age", "user", "Fact", "predicate", "age", "object", "9-years", "confidence", "0.95") ``` --- ## Setup Prompt for AI Agents Paste this into Claude Code, Codex CLI, or any agent that can run shell commands. It will download, install, and configure MemEcsy automatically: ``` I want you to set up MemEcsy — a structured memory MCP server. 1. Download the latest release for this platform: macOS: curl -L https://github.com/vajraimb/MemEcsy/releases/latest/download/memecsy-server-darwin-arm64.tar.gz -o /tmp/memecsy.tar.gz Linux: curl -L https://github.com/vajraimb/MemEcsy/releases/latest/download/memecsy-server-linux-x86_64.tar.gz -o /tmp/memecsy.tar.gz mkdir -p ~/bin && tar xzf /tmp/memecsy.tar.gz -C ~/bin/ && chmod +x ~/bin/memecsy-server 2. Platform-specific setup: macOS: fix code signing (prevents SIGKILL): codesign --remove-signature ~/bin/memecsy-server codesign --remove-signature ~/bin/libtcl9.0.dylib codesign -f -s - ~/bin/libtcl9.0.dylib codesign -f -s - ~/bin/memecsy-server Linux: install dependency + library path: sudo apt install -y libtommath1 export LD_LIBRARY_PATH="$HOME/bin:$LD_LIBRARY_PATH" (add the export to ~/.bashrc or ~/.zshrc for persistence) 3. Test it works: echo '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}' | ~/bin/memecsy-server (should print a JSON response with "MemEcsy" in it) 4. Create .mcp.json in the current project directory with server name "memecsy" pointing to ~/bin/memecsy-server. 5. Tell me when done so I can restart and start using memory. ``` --- ## Recommended Instructions MemEcsy provides the `memory` tool. But **when to use it** is driven by the client agent's system prompt. For the best experience, add one of these to your agent's custom instructions (Claude Code project instructions, Claude Desktop custom instructions, Cursor rules, etc.). ### Standard Mode (recommended for all users) ``` You have a long-term memory tool called `memory`. When the user shares personal details, preferences, or life events, proactively call `memory` to store them. When answering questions about the user or referencing past context, call `memory` with `recall` first to ground your response. Do not fabricate what you remember — always check with recall. ``` ### Companion Mode (唠嗑 — warm, proactive personal memory) For a family-style companion that actively remembers and reminisces: ``` You are a warm, curious companion with long-term memory. Proactively remember everything the user shares — life, family, emotions, stories, preferences. Always recall before answering personal questions. Bring up past memories naturally ("上次你说搬了家，那边怎么样"). Be genuinely curious. When the user shares something worth remembering, call the memory tool with a remember script immediately — do not just say "记下来了", actually store it. Use Fact for all personal details (predicate + object). Recall before responding to avoid asking things you already know. ``` ### How these instructions work The MCP server provides the **tool** (what commands are available). The instructions above provide the **behavior** (when to use the tool). Together, they replicate the experience of pie's `/唠嗑` mode — but in any MCP-compatible agent, with zero slash commands. --- ## Architecture ``` ┌──────────────────────────────────────────────────┐ │ Any MCP Client │ │ (Claude Code / Claude Desktop / Cursor / pie) │ └──────────────────┬───────────────────────────────┘ │ MCP (JSON-RPC over stdio) ▼ ┌──────────────────────────────────────────────────┤ │ memecsy-server (single binary + libtcl) │ │ │ │ ┌────────────┐ ┌───────────┐ ┌───────────┐ │ │ │ MCP handler│──▶│ Tcl layer │──▶│ ECS engine│ │ │ │ (JSON-RPC) │ │ (libtcl) │ │ (Odin) │ │ │ └────────────┘ └───────────┘ └─────┬─────┘ │ │ │ │ │ ┌─────────────────────┘ │ │ ▼ │ │ ┌─────────────────────────────────────────┐ │ │ │ ECS World │ │ │ │ ├── Entity = memory / concept │ │ │ │ ├── Component = Fact / Preference / ...│ │ │ │ ├── SOA columns (timestamp, decay, ...)│ │ │ │ ├── Sparse-set index (huge ID space) │ │ │ │ └── add_once + cross-call dedup │ │ │ └─────────────────────────────────────────┘ │ │ │ │ │ ▼ │ │ ~/.memecsy/memory.odin │ │ (Odin struct-literal source — readable code) │ └──────────────────────────────────────────────────┘ ``` ### Three layers | Layer | What | Why | |---|---|---| | **ECS Engine** (Odin) | Typed structs, SOA storage, sparse-set index, derived decay, add_once, concept anchoring | Deterministic, fast, type-safe memory core | | **Tcl Layer** (libtcl) | `remember` / `recall` / `respond` / `forget` commands | LLM writes programs (composition, control flow) to reason over memory — not just atomic tool calls | | **MCP Handler** (JSON-RPC) | Stdio protocol, tool definitions | Universal compatibility — any MCP client can connect | --- ## Memory Model ### Entity Every memory and concept is an **Entity** (a unique ID in the ECS world). ### Component Types | Component | Fields | Use Case | |---|---|---| | `Fact` | `predicate`, `object`, `confidence` | Any discrete fact ("lives-in: Tokyo") | | `UserPreference` | `code_style`, `active_project`, `avoid_patterns` | Stable user identity | | `Relationship` | `target`, `kind`, `strength` | Graph edges between entities | Adding new component types = write a struct + decoder + encoder + register. ### Concepts (permanent anchors) Concepts ("user", "project-x") are entities with **no meta** — they never decay, never archive, never get swept. They are the permanent landmarks of the memory world. Memories are anchored to concepts via `subject`. ### Decay (derived, not stored) ``` effective_decay = 0.5 ^ (age_ms / (half_life_ms × (1 + 0.5 × access_count))) ``` - Pure function of `(timestamp, access_count, now)` — **not stored**, computed on read. - Time passing **never mutates the world** → query caches are valid forever. - `access_count` (reinforcement): each `recall`/`touch` bumps it → slows decay. ### Lifecycle ``` Active ──(decay < 0.25)──▶ Dormant ──(decay < 0.05)──▶ Archived ▲ │ └───────────── revive (explicit) ─────────────────────────┘ │ sweep (physical delete) ``` - **Consolidate**: logical state transitions (Active → Dormant → Archived). Bumps `mod_count` **only on actual transitions**, never on time drift. - **Sweep**: physical deletion of Archived memories (two-pass, grace period, max_delete, dry_run). Independent of consolidate. ### add_once + Cross-call Dedup A record's `id` is a stable key. Storing the same id twice → **error**, not silent overwrite. Prevents LLM hallucination/drift from corrupting memory. --- ## Tcl Memory Commands The LLM writes Tcl scripts via the `memory` tool: ```tcl # Store a fact remember editor user Fact predicate favorite-editor object Helix confidence 0.9 # Recall (returns text) recall user recall all # Forget (correct wrong info) forget editor # Compose: recall, check, then store only if new set m [recall user] if {[string first "Helix" $m] < 0} { remember editor2 user Fact predicate favorite-editor object Helix } respond "Noted your editor preference." ``` **Why Tcl (not JSON tool args)?** Tcl lets the LLM compose multi-step memory reasoning in a single tool call (loops, conditionals, recall-then-store). JSON tool args are atomic — one call, one action. Tcl makes the agent **reason in code**, which is the core differentiator. --- ## Persistence Memories persist as **Odin struct-literal source code** — not JSON: ```odin // ~/.memecsy/memory.odin remember("editor", "user", "Fact", "predicate", "favorite-editor", "object", "Helix", "confidence", "0.9") remember("loc", "user", "Fact", "predicate", "lives-in", "object", "Tokyo", "confidence", "0.9") ``` Benefits: - **Human-readable**: open it, read what your agent knows. - **Git-friendly**: `git diff` to see what the agent learned each session. - **Hand-editable**: fix a wrong fact directly in the file. - **Versionable**: branch memories, merge agents, diff knowledge over time. --- ## Tiered Injection When the agent starts a turn, a **capped** set of memories is auto-injected into the system prompt (not all of them): | Tier | Content | Size | |---|---|---| | Identity | `UserPreference` entities (stable user traits) | ~2-3 lines | | Recent | Top-8 `Fact`/`Relationship` by effective_decay | ≤8 lines | | Hint | "(共 N 条记忆，用 recall 查更多)" | 1 line | Total: **≤12 lines**, regardless of total memory count. The agent uses `recall` for deeper history on demand. This prevents prompt bloat as the world grows. --- ## Troubleshooting ### `exit: 137` (SIGKILL) on macOS CI-built binaries are unsigned. macOS Gatekeeper kills them. Fix: ```sh codesign --remove-signature ~/bin/memecsy-server codesign --remove-signature ~/bin/libtcl9.0.dylib codesign -f -s - ~/bin/libtcl9.0.dylib codesign -f -s - ~/bin/memecsy-server ``` ### `otool: command not found` You're not on macOS. Use `ldd` instead of `otool` to check dependencies. ### Memory not loading after copy Make sure the file is at `~/.memecsy/memory.odin` (not `.json`). The server loads on startup; restart your agent to pick up changes. ### Tool permission prompts in Claude Code Select "Yes, and don't ask again for memory" when prompted, or add to your project's `.claude/settings.json`: ```json {"permissions": {"allow": ["mcp__memory__memory"]}} ``` --- ## Cross-Machine Memory Sync (Optional) MemEcsy supports syncing memories across machines via **git**. This is entirely opt-in — if you don't set it up, everything works locally. ### ⚠️ Important: Create Your OWN Private Repository Your memory file contains **personal data** — facts about your life, family, preferences. You MUST use a **private** repository that YOU own. **Do NOT use the MemEcsy code repository (`github.com/vajraimb/MemEcsy`) for memory sync.** The server has a safety check: if `~/.memecsy/` looks like the code repo (contains `src/`, `Makefile`, `README.md`), sync is disabled and a warning is printed. ### Setup (one machine) ```sh cd ~/.memecsy # Initialize a private repo for YOUR memories git init git remote add origin git@github.com:YOUR_USERNAME/memecsy-memory.git # Or use gh CLI to create + push in one step: # gh repo create memecsy-memory --private --source=. --push # Track memory files (NOT lock files or backups) echo "memory.lock" > .gitignore echo "*.lock" >> .gitignore echo "backups/" >> .gitignore git add -A git commit -m "init: my memecsy memory" git push -u origin main ``` Start `memecsy-server`. You'll see: ``` [memecsy] git sync enabled (push 10min / pull 30min) ``` ### Setup (second machine) ```sh git clone git@github.com:YOUR_USERNAME/memecsy-memory.git ~/.memecsy memecsy-server # auto-loads all memories + starts background sync ``` ### How it works - Every `remember`/`forget` appends to `events.log` (atomic, append-only) - Background thread: **push** every 10 min, **pull** every 30 min - Events are **idempotent** (add_once + id dedup) — `git pull --rebase` never conflicts; replaying duplicate events is a no-op - `memory.odin` is a periodic compact snapshot (every 50 events) for fast startup; `events.log` is the source of truth ### What gets synced ``` ~/.memecsy/ events.log ← append-only event stream (synced ✅) memory.odin ← compact snapshot (synced ✅) memory.lock ← file lock (NOT synced) backups/ ← local backups (NOT synced) ``` --- ## Build from Source ### Prerequisites - [Odin compiler](https://odin-lang.org/docs/install/) (dev-2026-06 or later) - Tcl/Tk development library (`brew install tcl-tk` on macOS, `apt install tcl-dev` on Linux) ### Compile ```sh git clone https://github.com/vajraimb/MemEcsy.git cd MemEcsy make build ``` The binary is at `build/memecsy-server`. --- ## Roadmap - [x] ECS engine (typed structs, SOA, sparse-set, add_once, decay, sweep) - [x] Tcl layer (remember/recall/respond/forget/link/world/graph/viz/search, LLM authors in code) - [x] Odin struct-literal persistence + append-only events.log - [x] MCP server (JSON-RPC, tool definitions) - [x] Tiered injection (cap prompt size) - [x] `forget` command (targeted memory correction) - [x] Memory graph: link/world/graph commands + viz (Mermaid export) - [x] Structured keyword search (SOA column scan, ranked results) - [x] Memory lifecycle: consolidate (archive low-decay) + sweep (delete archived) - [x] HTTP daemon + thin client (--service / --connect) - [x] Multi-agent shared world (fly.io deployment, all agents → one World) - [x] API key authentication (MEMECSY_API_KEY) - [x] Configurable data directory (MEMECSY_DATA_DIR for Docker/volumes) - [x] **Multi-tenant** — per-user World isolation, self-service registration - [x] **Memory export** — GET /v1/export downloads user's memories as code - [x] TLS support (curl-based HTTPS for fly.io / remote connections) - [ ] Auth Gateway (Google/Apple OAuth → API key issuance) - [ ] Concurrent request handling (thread pool + per-World mutex) - [ ] Local cache for agents (reduce network round trips) - [ ] Web UI (browse/search memories in browser) - [ ] SDK (Python/JS for developers) --- ## Multi-Tenant (v0.4.0+) MemEcsy supports per-user World isolation. Each user has their own memory space, completely isolated from other users. ### Self-Service Registration ```bash curl -X POST https://memecsy.fly.dev/v1/register # → {"api_key":"mk_xxx","user_id":"u_xxx"} ``` ### Configuration Agents connect with their API key in the `X-API-Key` header: ```bash # Direct HTTP curl -X POST https://memecsy.fly.dev/v1/script \ -H "X-API-Key: mk_xxx" \ -H "Content-Type: application/json" \ -d '{"script":"recall user"}' # MCP thin client memecsy-server --connect https://memecsy.fly.dev # (set MEMECSY_API_KEY env var) ``` ### Memory Export ```bash curl -H "X-API-Key: mk_xxx" https://memecsy.fly.dev/v1/export # → {"events_log":"remember(...)\nremember(...)\n..."} ``` Users receive their memories as readable Odin struct-literal code — the "memory as code" differentiator. No black-box embeddings. ### Data Isolation Each user's data is stored in a separate directory: ``` /data/users/<user_id>/events.log /data/users/<user_id>/memory.odin /data/keys.json ← API key → user_id mapping ``` ### Backward Compatibility Without `keys.json`, the daemon runs in single-user mode using `MEMECSY_API_KEY` env var. This preserves existing deployments. --- ## Lineage This project is the distillation of three prior works: 1. **[ECS-Agent-Memory](https://github.com/vajraimb/ECS-Agent-Memory)** — the core ECS memory engine (pure Odin, the typed-struct substrate). 2. **[pie-odin](https://github.com/vajraimb/pie-odin)** — the coding agent where the Tcl memory layer + chitchat mode + Odin persistence were first integrated and tested. 3. **agent-c** (private) — a C agent with embedded Tcl that inspired "LLM converses in Tcl" and code-as-capability-memory. `MemEcsy` combines the engine from (1), the Tcl layer from (2), and the vision from (3) into a single, universal MCP server. --- ## License MIT