Content
# PG-Git (Semantic Memory Engine)
<p align="center">
<img src="assets/banner.png" alt="PG-Git Banner" width="100%">
</p>
A persistent, PostgreSQL-backed repository management system and semantic memory engine for AI IDEs. Instead of storing Git objects loosely on the file system, PG-Git stores the entire Directed Acyclic Graph (DAG) natively in PostgreSQL, complete with automatically generated, temporally-decayed semantic vector embeddings.
[](https://badge.fury.io/js/pg-git-mcp)
[](https://opensource.org/licenses/ISC)
## 🧠 Why PG-Git?
In the standard AI coding agent ecosystem, searching codebases relies on rigid grep searches or expensive AST parsing. PG-Git fundamentally changes this by bridging Git directly with Vector Databases:
1. **Semantic Code Search**: Find code based on what it *does*, not just its syntax.
2. **Exponential Temporal Decay**: PG-Git mathematically decays older vectors. Your agent will prioritize code you wrote yesterday over highly similar dead code written 6 months ago.
3. **Local-First Purity**: No cloud APIs. It uses Ollama with `bge-large` for 100% private, on-device vectorization at 1024 dimensions.
4. **ACID Compliant**: Native transactions ensure complete safety for multi-node accessibility and concurrent AI swarm agents.
## ⚠️ Not a Replacement for Git
It is critical to understand that PG-Git **does not replace Git** or services like GitHub/GitLab. It does not handle branch merging, rebasing, or pull requests.
Instead, PG-Git is an **agentic augmentation layer**. You continue to use standard Git for your human-facing source control and team collaboration. PG-Git sits alongside it in your workflow, automatically ingesting your standard Git history to provide your AI agents with a mathematically optimized, semantically searchable clone of your codebase.
## 🔒 Reliability Audit & Pointer Resolution
To guarantee production-grade stability and complete continuity across complex monorepos, PG-Git underwent a thorough architectural audit and reliability overhaul:
1. **Deterministic Pointer Resolution**: In standard `'pointer'` storage mode (used to keep the database lightweight), file content is stored as `NULL` in PostgreSQL. We introduced a dynamic resolver inside `server/git-engine.js` that maps relative blob paths to the local filesystem monorepo root. This automatically streams raw file contents from disk on demand, resolving previous `null` value crashes on the Express `/api/blobs/:id` route and `pg_git_read_blob` MCP tool.
2. **Safe Semantic Search Previews**: Standard semantic search tool calls throw TypeErrors if they attempt to stringify nullable database columns. PG-Git now safely extracts the pre-compiled `summary` column (generated using `qwen2.5-coder:14b` inline summaries) to serve as the context preview.
3. **Starvation-Proof Queue Management**: The fleet priority queue (`PriorityQueue.process`) inside `lib/llm-queue.js` is secured using a strict `try ... finally` block. This guarantees that concurrent slots are cleanly released (`this.active--`) under all task outcomes, preventing memory and slot leaks that would otherwise block multi-agent swarm operations.
4. **Fidelity-Preserved Embedding Backfills**: Upgraded `backfill_embeddings.js` to automatically resolve local disk paths when processing pointer-mode blobs. This enables the embedding backfill process to compute full-fidelity, chunked centroid embeddings from the original code rather than falling back to lossy, shortened summary snippets.
## 🤝 The Agentic Brain (Synergy with Krusch Context MCP)
PG-Git is designed to be used as the **codebase engine** underneath the **[Krusch Context MCP](https://github.com/kruschdev/krusch-context-mcp)** — a unified IDE context server that collapses semantic code search, episodic memory, and holographic nuggets into a single MCP process.
While PG-Git can be run standalone as its own MCP server (via `server/mcp.js`), its full power is realized when consumed by Krusch Context MCP:
| Layer | Source | Purpose |
|-------|--------|---------|
| **Codebase Memory (The "What" & "How")** | PG-Git `blobs` table | Semantically embedded source files across your entire codebase |
| **Episodic Memory (The "Why")** | Krusch Context `ide_agent_memory` | Architectural decisions, bugs encountered, project goals |
| **Holographic Nuggets (The "How to Behave")** | Krusch Context `ide_agent_nuggets` | Lightweight steering facts, user preferences, project conventions |
**Infinite Continuity**: By running Krusch Context MCP (which imports PG-Git's database pool and embedding logic directly), your agent can cross-reference the *intent* (episodic memory) with the *implementation* (codebase blobs). It remembers *why* you chose a specific architecture, and instantly sees *how* it's currently implemented, creating a deeply contextualized and autonomous coding workflow that persists across infinite sessions.
> 🔗 **See the full unified server documentation:** [Krusch Context MCP README](https://github.com/kruschdev/krusch-context-mcp)
## ⚡ Quick Start
You **must** have [Ollama](https://ollama.com/) running with the `bge-large` embedding model pulled:
```bash
ollama pull bge-large
```
**1. Clone & Migrate Database**
You will need a running PostgreSQL instance with `pgvector` enabled.
```bash
git clone https://github.com/kruschdev/pg-git.git
cd pg-git
npm install
cp .env.example .env
# Edit .env with your PostgreSQL credentials
node db/migrate.js
```
**2. Import Your GitHub History**
> [!WARNING]
> **Choose your embedding model carefully.** You must set your preferred model (via the Web UI Settings tab or `.env`) *before* running your first import or snapshot. If you change models later, vector dimensions will collide and you will be forced to manually wipe the database and re-embed all repositories from scratch.
You can instantly import any local `.git` repository. PG-Git will natively parse the Git history, generate semantic embeddings for all blobs, and securely deduplicate them into PostgreSQL:
```bash
npm run import
```
**3a. Use via Krusch Context MCP (Recommended)**
The recommended way to use PG-Git is through the [Krusch Context MCP](https://github.com/kruschdev/krusch-context-mcp) unified server, which wraps PG-Git's database pool and embedding logic alongside episodic memory and nuggets. See the [Krusch Context MCP Quick Start](https://github.com/kruschdev/krusch-context-mcp#-quick-start) for setup instructions.
**3b. Use Standalone MCP Server**
If you prefer to run PG-Git as an isolated MCP server, you can execute it directly via NPM. Add it to your agent/IDE configuration (e.g., `mcp_config.json`):
```json
{
"mcpServers": {
"pg-git-mcp": {
"command": "npx",
"args": ["-y", "pg-git-mcp"],
"env": {
"PG_CONNECTION_STRING": "postgres://user:pass@localhost:5434/kruschdb",
"OLLAMA_URL": "http://localhost:11434",
"EMBED_MODEL": "bge-large"
}
}
}
}
```
**4. Start the Web UI (Optional)**
PG-Git includes a sleek, dual-pane IDE interface for browsing your semantic repositories.
```bash
npm run dev
```
---
## 🔧 MCP Tools (Standalone Mode)
When running PG-Git as a standalone MCP server, it exposes the following tools:
| Tool | Description |
|------|-------------|
| `pg_git_list_repos` | List all available PG-Git repositories stored in the database |
| `pg_git_read_tree` | Read the directory structure (DAG node) of a specific repository |
| `pg_git_read_blob` | Read the file contents of a specific blob |
| `pg_git_semantic_search` | Semantically search all indexed files with temporal decay scoring |
> [!NOTE]
> When used through **Krusch Context MCP**, these capabilities are exposed under unified tool names (`krusch_context_search_code`, `krusch_context_read_tree`, `krusch_context_read_blob`, `krusch_context_list_repos`) alongside 14 additional memory and nugget tools.
---
## 🚀 Real-World Usage Examples
### Standalone Mode
Speak to your IDE agent normally. It will use the standalone MCP tools to interface with the database:
**Example 1: Finding specific logic**
> **You:** "Where do we handle the temporal decay for the memory MCP?"
> **Agent:** *[Calls `pg_git_semantic_search`]* "I found the logic in `server/git-engine.js`. It uses the `exp(-0.01 * age_in_days)` formula in `searchBlobs()`."
**Example 2: Reading a repository tree**
> **You:** "What is the folder structure for the pg-git project?"
> **Agent:** *[Calls `pg_git_read_tree`]* "Here is the root directory structure..."
**Example 3: Filtering by project**
> **You:** "Search for authentication logic in the pocket-lawyer project only."
> **Agent:** *[Calls `pg_git_semantic_search` with `project: 'pocket-lawyer'`]* "Found 3 matches in the auth module..."
### How Does Temporal Decay Work?
When calling `pg_git_semantic_search`, PG-Git returns the highest cosine-similarity matches. However, it applies **Exponential Temporal Decay** based on the blob's `last_seen_at` timestamp:
```
score = cosine_similarity × exp(-0.01 × age_in_days)
```
If you have two very similar pieces of code, the *newer* one will have a significantly higher score, preventing your agent from hallucinating based on outdated implementations.
---
## 🤖 The Autonomous Agent Workflow
You can integrate PG-Git into your agentic workflow to ensure your semantic memory is always up to date.
### Snapshot (Single Project)
Whenever you step away from a task, tell your agent to run the snapshot script. The agent will autonomously:
1. Hash the current project folder into Git Blobs and Trees.
2. Ping Ollama to embed any new or modified files.
3. Commit the state directly into PostgreSQL.
```bash
npm run snapshot
```
### Sync All Projects (Fleet-Wide)
To re-index all active project codebases across the fleet:
```bash
npm run sync-all
```
### External Documentation Sync
PG-Git can ingest external documentation (e.g., `llms.txt` manifests) for hallucination-free framework knowledge:
```bash
node scripts/sync_external_docs.js
```
---
## 📂 Project Structure
```
pg-git/
├── server/
│ ├── index.js # Express API + Web UI server
│ ├── mcp.js # Standalone MCP server (StdioServerTransport)
│ └── git-engine.js # Git DAG operations + semantic search with temporal decay
├── db/
│ ├── schema.sql # PostgreSQL schema (repos, commits, branches, trees, blobs + pgvector)
│ ├── pool.js # Shared pg.Pool connection
│ ├── migrate.js # Schema migration runner
│ ├── create-db.js # Database creation helper
│ └── list-dbs.js # List available databases
├── lib/
│ └── embedding.js # Shared Ollama embedding client with fleet load balancing
├── scripts/
│ ├── sync_to_pg.js # Snapshot a single project into PostgreSQL
│ ├── sync_all_projects.js # Fleet-wide project sync
│ ├── sync_external_docs.js # External documentation ingestion (llms.txt)
│ ├── import_github.js # Import native .git history
│ ├── import_hf_dataset.js # Import HuggingFace datasets
│ ├── backfill_embeddings.js # Backfill missing embeddings
│ ├── migrate_to_1024.js # Dimension migration helper
│ ├── migrate_to_pointer.js # Storage mode migration
│ └── scaffold_nesting_dolls.js # Nesting-doll chunking scaffold
├── client/ # Vite + React Web UI
├── config/
│ └── external_docs.json # External documentation manifest
├── config.js # Unified configuration (env + config.json merge)
├── assets/ # Banner and social preview images
├── Dockerfile # Multi-stage production build
├── docker-compose.yml # Container orchestration
├── AGENTS.md # Agent context rules for AI IDEs
└── spec.md # Original project specification
```
---
## 🛠️ Configuration & Environment Variables
PG-Git uses a layered configuration system: environment variables override `config.json`, which overrides built-in defaults.
| Variable | Description | Default |
|----------|-------------|---------|
| `PORT` | Express server port | `4890` |
| `DB_HOST` | PostgreSQL Host address | `localhost` |
| `DB_PORT` | PostgreSQL Port | `5434` |
| `DB_NAME` | Database Name | `postgres` |
| `DB_USER` | Database User | `postgres` |
| `DB_PASSWORD` | Database Password | *(empty)* |
| `OLLAMA_URL` | The endpoint for your local Ollama instance | `http://localhost:11434` |
| `EMBED_MODEL`| The Ollama text-embedding model to use | `bge-large` |
### Database Schema
PG-Git's PostgreSQL schema maps Git objects directly into SQL tables:
- **`repositories`** — Project registries
- **`commits`** — SHA-1 identified commit objects with tree and parent references
- **`branches`** — Named references to commit heads
- **`trees`** / **`tree_entries`** — Directory structures mapping names to blob/tree object IDs
- **`blobs`** — File content with `embedding vector(1024)` and `last_seen_at` temporal tracking
Semantic search uses the `pgvector` HNSW index with `vector_cosine_ops` for sub-millisecond approximate nearest-neighbor lookups.
---
## 🗺️ Related Projects
| Project | Role |
|---------|------|
| **[Krusch Context MCP](https://github.com/kruschdev/krusch-context-mcp)** | Unified IDE context server — wraps PG-Git + episodic memory + nuggets into a single MCP process |
| [PG-Git MCP on NPM](https://www.npmjs.com/package/pg-git-mcp) | This project published to the NPM registry |
| [NeoVertex Nuggets](https://github.com/NeoVertex1/nuggets) | Original Holographic Nuggets MCP architecture adapted in Krusch Context |
## License
ISC License. Created by [kruschdev](https://github.com/kruschdev).
