Content
# Recall
Based on [zvec](https://github.com/alibaba/zvec) vector library, a personal local knowledge base - index your GitHub stars, Chrome bookmarks, and codex/claude-code session records with a single command for hybrid search, and directly access Claude Code/Codex in MCP format.
## Features
- **Local/Offline**: All data and indexes are stored in `~/.recall`, with no cloud uploads.
- **No torch embedding**: Using `fastembed` + `BAAI/bge-small-zh-v1.5` (512 dimensions), no torch/transformers required; approximately 100MB model download to HF cache on first run. Embedder is pluggable (with `ApiEmbedder` stub).
- **Hybrid search**: Vector recall (HNSW) + full-text (FTS) recall + RRF reranking + scalar filtering.
- **MCP + CLI dual forms**: Can be used as both a command-line tool and an MCP server (stdio, pure stdlib JSON-RPC).
- **Three data sources**: GitHub stars / Chrome bookmarks / session logs.
- **Incremental sync**: Based on `~/.recall/state.json` watermark, only processing new content.
## Quick Start (from scratch)
> The following `<REPO_DIR>` refers to the directory where you cloned the repository.
**Prerequisites**
- Python >= 3.11
- [uv](https://github.com/astral-sh/uv) (recommended) or pip
- `gh` CLI and logged in - only `stars` data source requires: `gh auth login`
- macOS - only `bookmarks` data source and launchd scheduled task require (other platforms can skip these two)
**Installation**
```bash
git clone https://github.com/tjcjc/recall.git
cd recall # = <REPO_DIR>
uv venv
uv pip install -e . # or: uv pip install zvec fastembed
```
> The first `sync` / `search` will download approximately 100MB of the bge model to the HuggingFace cache directory.
**First Run**
```bash
python -m recall.cli sync # Incremental collection + embedding +入库
python -m recall.cli search "rust async runtime"
```
You can also use the `recall` command directly (equivalent to `python -m recall.cli`) after installing as editable.
## Architecture / Data Flow
Responsibility hierarchy: **fastembed** only performs one-time text-to-vector conversion (replaceable); **zvec** is the knowledge base本体, responsible for storage, dual indexing (HNSW + FTS), hybrid recall, RRF fusion, scalar filtering, and persistence.
```
Text ──[fastembed]──► Vector ──┐
├──[zvec: storage/indexing/retrieval/fusion]──► Result
Metadata/Original text ────────────────┘
Preprocessing (replaceable) Knowledge base本体 (core)
```
### Building the Index (sync)
```
GitHub stars (gh) Chrome bookmarks codex+claude session jsonl
stars.py bookmarks.py sessions.py (filtering harness noise)
└──────────────────┴──────────────────┘
│ Unified record {id,title,url,source,lang,ts,text}
▼
sync.py orchestration (incremental: state.json, only processing new/changed)
│ Batch texts
▼
fastembed (bge-small-zh) Text → 512-dimensional vector
│ record + vector
▼
┌─────────────────── zvec ───────────────────┐
│ upsert (≤1024/batch) │
│ HNSW vector index (embedding) + FTS full-text index (text)│
│ Scalar fields title/url/source/lang/ts │
│ Persistence → ~/.recall/db (WAL) │
└─────────────────────────────────────────────┘
```
### Search (search)
```
query (+ optional --source / --topk)
├──────────────► fastembed encode ──► 512-dimensional query vector ─┐ VectorQuery
└─ Original text ─────────────────────────────────────────────┐ │ FtsQuery
▼ ▼
┌──────────────────── zvec.query() ───────────────────────┐
│ FTS recall (BM25-like) ‖ HNSW recall (vector neighbors) two paths in parallel │
│ └────────► RrfReRanker fusion ranking │
│ filter: source='stars' scalar filtering │
└──────────────────────────┬──────────────────────────────┘
▼
top-k {id,score,title,url,source,...}
├─► CLI pretty print (human use)
└─► MCP knowledge_search (Claude Code / Codex auto-tune)
```
Entry point: `recall` or `python -m recall.cli` (builds to single-file binary `dist/recall`).
## Usage
Four subcommands, two forms (binary and venv) are equivalent:
```bash
# Sync (incremental collection + embedding +入库). Optional --source limits to a single data source
recall sync
recall sync --source stars # stars | bookmarks | sessions
python -m recall.cli sync
# Hybrid search
recall search "rust async runtime" --topk 10
recall search "vector database" --source bookmarks
python -m recall.cli search "rust async runtime"
# Knowledge base document count
recall stats
python -m recall.cli stats
# Run as MCP server (stdio)
recall mcp
python -m recall.cli mcp
```
## Building a Single-File Binary
To build a single-file binary with zvec/onnxruntime/fastembed native libraries:
```bash
.venv/bin/pyinstaller recall.spec --noconfirm
# Output: dist/recall
```
Entry point is `build_entry.py`, spec is `recall.spec`.
## MCP Integration
Exposes two tools: `knowledge_search(query, source?, topk?)` and `knowledge_stats()`. Replace the path with your actual repository path `<REPO_DIR>`.
**Claude Code** - Project-level `.mcp.json`:
```json
{
"mcpServers": {
"recall": {
"command": "<REPO_DIR>/.venv/bin/python",
"args": ["-m", "recall.cli", "mcp"],
"env": {
"PYTHONPATH": "<REPO_DIR>"
}
}
}
}
```
**Codex** - `~/.codex/config.toml`:
```toml
[mcp_servers.recall]
command = "<REPO_DIR>/.venv/bin/python"
args = ["-m", "recall.cli", "mcp"]
env = { PYTHONPATH = "<REPO_DIR>" }
```
> After building the binary, you can replace `command` with `<REPO_DIR>/dist/recall`, `args` with `["mcp"]`, and remove PYTHONPATH.
## Scheduled Update (macOS only)
Use macOS launchd to automatically incrementally sync daily at 9:00 AM, template see [`launchd.plist.example`](./launchd.plist.example) (including installation / immediate run / view / uninstall instructions). launchd requires absolute paths, please replace placeholders in the template with your actual paths (e.g., `/Users/<YOUR_USERNAME>/recall`).
## Configuration
`~/.recall/config.toml` (automatically created on first run; takes effect after modification and re-running any command):
| Field | Description |
| --- | --- |
| `db_path` | zvec collection directory, default `~/.recall/db` |
| `model_name` | fastembed model, default `BAAI/bge-small-zh-v1.5` (512 dimensions) |
| `embedder` | `fastembed` (local) or `api` (stub) |
| `state_path` | Incremental watermark, default `~/.recall/state.json` |
| `chrome_bookmarks_path` | Bookmark file path (optional override) |
| `[sources]` | `stars` / `bookmarks` / `sessions` respective switches |
Storage located in `~/.recall/db` (zvec collection); embedding model cached in HuggingFace default cache directory.
## Data Source Description
| Data Source | Indexed Content | Dependency |
| --- | --- | --- |
| `stars` | GitHub starred repositories (name / description / language, etc.) | `gh` CLI (requires `gh auth login`) |
| `bookmarks` | Chrome bookmark titles and URLs | macOS only: `~/Library/Application Support/Google/Chrome/Default/Bookmarks` |
| `sessions` | codex and claude-code session jsonl logs | `~/.codex/sessions`, `~/.claude/projects` |
