knowledge-base-system-oss

# Tool List [](https://opensource.org/licenses/Apache-2.0) [](https://www.python.org/) []() A local knowledge memory system for engineering research and development, allowing Claude Code/Codex to retrieve historical decisions, troubleshooting records, and engineering specifications during coding.  > Claude Code recalls the bug repair memory that was just filed on the same day - human-written knowledge, AI can search on the same day. ## Why use this? - 🔒 **Local privacy priority**: All data (notes/vector index/backup) is stored locally and does not leave the device; embedded services have three options (built-in infinity/external OpenAI compatibility/disabled), which are completely controllable. - 🤖 **MCP native integration**: Out-of-the-box connection to Claude Code/Codex, allowing AI coding assistants to retrieve your historical decisions, troubleshooting notes, and engineering specifications - turning "AI forgets" into "AI gets smarter about you". - 📦 **Double-click to use**: Mac DMG/Windows EXE direct installation version, zero Docker zero Python environment configuration threshold, tray/menu bar App management start and stop. - 🎯 **Hybrid retrieval**: BM25 keyword recall + Qdrant vector semantic recall + optional Rerank, accurate matching and semantic retrieval full coverage. - 💾 **Data protection**: Upgrade automatic backup + active export/import + maintenance mode serialization, avoiding data corruption; backup recovery supports overwrite/merge dual modes + strict confirm token to prevent misuse. ## Deployment Modes ### Direct Installation (Main Line) No Docker/Python environment required, installation package double-click to use. - Storage: SQLite + Qdrant embedded mode - Tray/menu bar App management start and stop - Business configuration written to database (`/settings` page modification), port guided by `config/config.toml` - Installation structure (Windows, installed to `%LocalAppData%\KnowledgeBase\`): ``` bin\kb-api.exe FastAPI service process bin\kb-tray.exe Tray program config\config.toml Guided configuration (port, data path) scripts\local-restart-direct.ps1 /v1/system/restart called direct installation restart script agent-integration\ Claude/Codex access toolkit (MCP proxy + Skill + installation script) app.ico Desktop shortcut icon 使用说明.md User manual data\ Runtime data (knowledge.db + qdrant_local, reserved after uninstallation) logs\ Runtime log ``` - Installation structure (macOS, installed to `/Applications/KnowledgeBase/`): ``` KnowledgeBaseMenuBar.app Menu bar App (Swift binary, startup/status badge/knowledge base management menu) bin/kb-api FastAPI service binary (PyInstaller single file) config/config.toml Guided configuration (port, data path) scripts/ kb-start/kb-stop/kb-status + knowledge base management script (kb-import-* etc.) agent-integration/ Claude/Codex access toolkit 使用说明.md User manual data/ Runtime data (knowledge.db + qdrant_local, reserved after uninstallation) logs/ Runtime log (api.log/api.err.log) ``` Upgrade automatic backup stored in `~/Library/Application Support/KnowledgeBase/auto-backup/` (reserved after uninstallation). ### Docker Version (Planned) The Docker version is on the roadmap (v1.x plan), and the direct installation version is delivered first. The code level has reserved `KB_BACKEND=postgres` path and `app/repository_postgres.py` implementation, which is convenient for future Dockerization/external Postgres orchestration. Description: EXE/Mac App belongs to the direct installation version delivery, not the Docker version. ## Download and Installation ### macOS (v1.3.10) ➡️ [KnowledgeBase-mac-direct-1.3.10.dmg](https://github.com/SliverSucks/knowledge-base-system-oss/releases/latest) Download DMG → Double-click to open → Double-click `Install.command`, install to `/Applications/KnowledgeBase/`. After installation, the knowledge base icon appears in the menu bar. Click "Start Knowledge Base" to start. ### Windows (Coming Soon) The code is ready (`windows-app/` + `scripts/build_direct_install.ps1`), **signature installation package is being constructed**. If you need it urgently, refer to [docs/11-windows-tray-exe.md](docs/11-windows-tray-exe.md) to build it yourself (need Anaconda + Inno Setup). ### All Versions [Releases page](https://github.com/SliverSucks/knowledge-base-system-oss/releases) - contains historical version archives and release notes. ## Quick Start (Dev) > Prerequisites: Python 3.11+ installed, `.venv` created (see `requirements-local.txt`). macOS/Linux: ```bash ./scripts/kb-start.sh # Start in the background, PID written to data/.local_api.pid ./scripts/kb-stop.sh # Stop ./scripts/kb-status.sh # View status + health check ``` Windows: ```powershell powershell -ExecutionPolicy Bypass -File scripts\local-start.ps1 powershell -ExecutionPolicy Bypass -File scripts\local-stop.ps1 ``` After formal installation, the tray/menu bar App directly manages start and stop, no need to type commands. | Address | Purpose | |------|------| | http://127.0.0.1:18000/console | Web console | | http://127.0.0.1:18000/settings | System configuration (LLM/Embedding/Rerank) | | http://127.0.0.1:18000/docs | Swagger API documentation | | http://127.0.0.1:18000/health | Health check | Environment variables (built-in scripts, no need to set manually): ``` KB_BACKEND=sqlite SQLITE_PATH=data\knowledge.db VECTOR_ENABLED=1 QDRANT_MODE=local QDRANT_LOCAL_PATH=data\qdrant_local ``` ## System Configuration Business configuration (LLM, Embedding, Rerank, theme) is written to the database, no `.env` file needed. Port and data path are managed by the patterned startup entrance (direct installation version `config/config.toml`; Docker version port mapping/environment variable). Open `/settings` page to modify, configuration will be persisted. - Direct installation version modification `service_port` will rewrite `config/config.toml`, return `restart_required=true`, restart with new port. - Docker version returns `runtime_port_managed_by=docker`, need to update port mapping in orchestration layer and restart container. ## Embedding Service Semantic retrieval is based on vector embedding model implementation. This project provides three enablement methods: ### 1. Built-in Local Service (Recommended, Direct Installation Sweet Spot) After the first startup, visit `/setup`, select "Local Built-in" to install with one click: - The system will automatically download the selected model + install `infinity-emb` subprocess, and keep it alive in the background - Data does not leave the local machine, no external API call - Subprocess abnormal crash automatic restart ≤3 times, super limit fall back to keyword retrieval ### 2. External OpenAI Compatible Service `/setup` select "External Service", fill in 4 fields (base_url/api_key/model) to use. Suitable for: - Existing Azure OpenAI/private embedding deployment - Want to use larger models (e.g., `text-embedding-3-large`) ### 3. Skip / Keyword Retrieval No embedding service can also be used, automatically degraded to BM25 keyword retrieval. Suitable for: hardware resource tension / only need accurate matching scene. ### Model Selection Guide | Model | dim | Disk | Resident memory | Chinese and English | Recommended scene | |------|-----|------|---------|-------|---------| | **BGE-M3** ⭐ | 1024 | ~2.3 GB | ~1.5 GB | Multilingual | Default choice, long text + Chinese and English mixed scene sweet spot | | BGE-large-zh v1.5 | 1024 | ~1.3 GB | ~0.8 GB | Pure Chinese | Pure Chinese scene, smaller resource occupation | | Qwen3-Embedding 0.6B | 1024 | ~1.2 GB | ~0.8 GB | Multilingual | Lightweight alternative when resources are tight | ### Operation API - `/setup` first installation guide page (three options → model grid → installation progress SSE) - `/settings` Embedding service partition (mode switching / index reconstruction / log viewing) - **macOS tray** → Knowledge base management → Rebuild vector index (v1.3 added, native NSAlert secondary confirmation + background running + completion notification) - Model switching / index reconstruction must input `I-CONFIRM-OVERWRITE` confirmation (AC22 prevent misuse) - During reconstruction, you can click "Abort and roll back" to restore the old index (AC23) ### v1.3 Behavior Upgrade - **Mode switching automatic linkage**: local↔external↔disabled switching, the backend automatically `bump_desired`, shell layer reconcile loop really stops / really starts infinity subprocess, no need for user manual management - **Warmup self-healing**: infinity first loading slow > 120s, reconcile loop detects `/health` 200 automatic clear "loading" stuck state - **Upgrade reserved data**: Install.command automatic backup + injection `data/` + `models/` (1-4 GB) + `embedding-service/` three directories, APFS clonefile instantaneous completion; upgrade does not re-download 4GB model does not re-pip install - **Install skip downloaded**: detect local model weight complete (config.json + ≥50MB weight file) skip `snapshot_download` entire phase ### Troubleshooting embedding service is managed by shell layer ProcessManager (Mac Swift/Windows Python): - Status file: `{data_root}/runtime/{install_status.json,pid,port,owner_token}` - Log: `{data_root}/logs/{pip.log,infinity.log}` - Detailed architecture and troubleshooting process see [docs/15-embedding-service.md](docs/15-embedding-service.md) ## Agent Integration Integration method: put the installation directory `agent-integration/Installation Instructions.md` to Claude Code/Codex, let AI assistants complete MCP registration + Skill installation + permission whitelist consolidation according to the document. Example prompt: ``` Please follow /Applications/KnowledgeBase/agent-integration/Installation Instructions.md to help me access the knowledge base. ``` or Windows: ``` Please follow C:\Users\<your username>\AppData\Local\KnowledgeBase\agent-integration\Installation Instructions.md to help me access the knowledge base. ``` `agent-integration/` three core files: | File | Purpose | |---|---| | `kb-mcp-proxy.py` | MCP server implementation, stdio ↔ kb-api HTTP | | `SKILL.md` | Skill behavior rules mainstay, cross-platform universal | | `Installation Instructions.md` | AI-readable installation guide, containing platform path placeholders and fine-tuning points | Detailed integration process: [docs/10-agent-integration.md](docs/10-agent-integration.md) ### Skill Manual Installation (when not letting AI automatically access) ```bash # Claude mkdir -p ~/.claude/skills/knowledge-base-first cp agent-integration/SKILL.md ~/.claude/skills/knowledge-base-first/SKILL.md # Codex mkdir -p ~/.codex/skills/knowledge-base-first cp agent-integration/SKILL.md ~/.codex/skills/knowledge-base-first/SKILL.md ``` ## Mac Direct Installation ```bash # 1) Build kb-api binary (dependent on .venv + pyinstaller) ./scripts/build_mac_kb_api.sh # 2) Generate direct installation dmg (default version 1.0.0) ./scripts/build_mac_direct_install_dmg.sh # or: build api + package in one step ./scripts/build_mac_direct_install_dmg.sh --build-api 1.0.0 ``` Product location: `dist/KnowledgeBase-mac-direct-<version>.dmg` ## Safe Upgrade Direct installation version DMG built-in data protection: - **Process detection**: find `kb-api` running, prompt to exit App and then install (prevent SQLite/Qdrant getting inconsistent snapshot) - **Automatic backup**: before upgrade, cp current `data/` to `~/Library/Application Support/KnowledgeBase/auto-backup/{ts}/`, containing minimal manifest (trigger=install + before/after version number) - **Automatic restore**: after installation, restore data (not restore config, let new version default guide take effect) auto-backup historical snapshot **permanent retention**, recommend manual cleaning every quarter. See [`docs/07-user-manual.md`](docs/07-user-manual.md) "Full backup and recovery" chapter. Console/HTTP API also provide active `export`/`import`, support `overwrite` and `merge` dual modes + strict confirm token (`I-CONFIRM-OVERWRITE`/`I-CONFIRM-MERGE`). Backup contains LLM/Embedding credentials, **do not upload to public repositories or send through unencrypted channels**. ## Data Import ```bash # Import single Markdown file python scripts/import_markdown.py --file ~/notes/api-spec.md --project proj-a --domain work # Batch import directory python scripts/import_directory.py --dir ~/docs --project proj-a --domain work # Incremental import python scripts/import_incremental.py --project proj-a --domain work ``` ## Core API | Method | Path | Description | |------|------|------| | POST | `/v1/knowledge/search` | Hybrid retrieval (keyword + vector) | | GET | `/v1/knowledge/items/{id}` | Get item by ID | | POST | `/v1/knowledge/items/upsert` | Write/append new version | | POST | `/v1/knowledge/ask` | Intelligent Q&A based on knowledge base | | POST | `/v1/knowledge/import-incremental` | Incremental import directory | | POST | `/v1/knowledge/export-package` | Export knowledge package | | POST | `/v1/knowledge/import-package` | Import knowledge package (dangerous) | | POST | `/v1/knowledge/clear` | Clear knowledge base (dangerous) | | POST | `/v1/knowledge/cleanup-expired` | Clean up expired knowledge (dangerous) | | GET/PUT | `/v1/system/config` | System configuration read/write | | POST | `/v1/system/restart` | Restart according to mode/platform distribution | > Console internal interface (like soft delete `DELETE /v1/console/knowledge/items/{id}`) not in the table, only for local console call, see [docs/04-retrieval-api.md](docs/04-retrieval-api.md) section 4.5. Complete interface documentation: [docs/04-retrieval-api.md](docs/04-retrieval-api.md) ## Layout ``` app/ # FastAPI service main.py # Entrance, route repository_sqlite.py # SQLite data layer repository_postgres.py # PostgreSQL data layer (Docker version) vector_index.py # Qdrant integration service.py # Business orchestration mcp_server.py / mcp_tools.py # MCP service model_settings.py # LLM/Embedding/Rerank configuration static/console/ # Web console frontend static/settings/ # System configuration frontend scripts/ # Start/stop, import, installation scripts windows-app/ # Windows tray App source code mac-app/ # Mac menu bar App source code docs/ # Design and operation and maintenance documents tests/ # Unit test data/ # Runtime data (.gitignore) knowledge.db qdrant_local/ ```