Content
-code-mcp-built code semantic retrieval MCP Server - Natural language query → snippets.
**"Effectiveness First" Tech Stack:** **itter AST splitting + voyage-code-3 embedding (int8 + sqlite-vec vector database + BM25 full-textRF fusion + Voyage rerank-2.5 re-ranking + Call Graph graph extension**. One `VOYAGE_API_KEY` runs through embedding + rerank full link.
**Engineering Featureswatchdog file real-time monitoring + multi-workspace LRU management + MCP progress notification + cancel signal**.
## Architecture
```
code, dir)
│
Workspace─ LRU(max 8) ── FileWatcher(w 2s debounce)
│
▼
er ── Full scan/watcher increment(O(Change)) Chunker(Tree-sitter AST) ── Embedder-code-3)
│ │
│ ite-vec + FTS5 + file hash)
Retriever ── Vector recall + BM25 recall RRF fusion ── rerank(Voyage/Cohere)── Call Graph extension ── top_n code snippets
│
▼
MCP Server ── async tools + progress notification + cancel signal
```
| Module | Responsibility |
|---|chunker.py` | Tree-sitter splits by (16 languages + md/yaml/json config files split by extracts call relationships to build call graph |
| `store.py` | sqlite-vec vector (float/int8) + F full-text + file fingerprint + call graph edge table |
embedder.py` | voyage-code-3, document/query dual mode, int8 quantization, rate-limited retries |
| `retriever.py` | Vector + BM25 → RRF → rerank → intent-aware scoring (test/document/call chain intent) → file diversity truncation → graph extension |
| `indexer.py` | Directory scan + file hash incremental sync + watcher dirty incremental sync |
| `watcher.py` | watchdog file monitoring + 2s debounce + 50 changes immediate flush |
| `workspace.py` | Multi-workspace LRU manager (create/evict/status tracking) |
| `server.py` | MCP encapsulation, async tools + progress + cancel + WorkspaceManager |
## Installation
The release package name is `semcode-mcp` (PyPI `semantic-code-mcp` is occupied by a project with the same name):
```bash
# Method 1 (recommended): uv environment zero-installation, MCP client configuration directly uvx startup
uvx semcode-mcp
# Method 2: npx (requires uv installed, this npm package is a thin shell of uvx)
npx -y semcode-mcp
# Method 3: pip
pip install semcode-mcp
# Source code development
pip install -e .
```
## Configuration
Copy `.env.example` to `.env` and fill in the key:
| Variable | Required | Description |
|---|---|---|
| `VOYAGE_API_KEY` | Yes | Voyage embedding + rerank (same key for both), apply: https://dash.voyageai.com/ |
| `COHERE_API_KEY` | No | Optional rerank backup backend; default use Voyage rerank-2.5, no need for this key |
### Advanced Configuration
| Variable | Default Value | Description |
|---|---|---|
| `SCM_EMBED_MODEL` | `voyage-code-3` | Embedding model |
| `SCM_EMBED_DTYPE` | `float` | Vector precision: `float` (highest precision) / `int8` (saves 71% storage) |
| `SCM_EMBED_MAX_RETRIES` | `4` | API rate limit/error automatic retry times |
| `SCM_EMBED_MIN_INTERVAL` | `0` | Minimum request interval seconds (unbound card 3 RPM limit set to `21`) |
| `SCM_EMBED_CONCURRENCY` | voyage `3` / local `1` | Indexing embedding concurrent request number (measured 3 lines ≈ speed up 3 times) |
| `SCM_EMBED_CHAR_BUDGET` | `200000` | Single batch maximum character number (unbound card can be adjusted) |
| `SCM_RERANK_BACKEND` | `auto` | Rerank backend: `auto` (Voyage with VOYAGE key, followed by Cohere) /age` / `cohere` / `off` |
| `SCM_RERANK_MODEL By backend | `rerank-2.5`, cohere → `rerank-v3.5` |
| `SCM_DB_DIR` | `~/.semantic-code-mcp` | Index database storage directory |
| `SCM_TOP_K` | `50` | Vector recall candidate number |
| `SCM_TOP_N` | `10` | Final return result number |
| `HTTPS_PROXY` | — | Cohere API proxy (required for domestic use) |
| `SCM_MAX_WORKSPACES` | `8` | Maximum concurrent workspace number (LRU eviction) |
| `SCM_WATCH` | `1` monitoring: `1 (default), `0` disable |
## Register in
```json
{
"mcpServers": {
"": {
"command": "uvx",
"args": ["semcode-mcp"],
env": {
"VOYAGE_API_KEY":-voyage-key"
}
}
}
}
px usage replaces `command/args` with `" ["-y", "semcode-mcp"]`;
source code development uses `pip install -e .` and then configures `"semcode-mcp", []` (the relative import, does not support `python server.py` direct run, requires console script or `python -m semantic_code_mcp.server`).
## Tools
### codebase_search
| Parameter | Type | Description |
|---|---|---|
| `information_request` | string | Natural language query, e.g., "Where is the user login authentication logic?" |
| `` | string | Codebase absolute path |
The only tool, first call automatically indexes the entire volume, and then watcher drives incremental synchronization (O(Change), extremely fast). Supports MCP progress notification real-time return index progress. To force re-indexing, delete the corresponding `.db` file under `~/.semantic-code-mcp/`.
## Working Principle
1. **Splitting**: Tree-sitter parses AST, splits by function/class boundaries, and retains `file_path / symbol / line number` metadata.
2. **Vectorization**: voyage-code-3 generates 1024-dimensional vectors for code blocks (`input_type=document`).
3. **Storage**: vectors are stored in sqlite-vec (vec0 virtual table), and simultaneously in FTS5 full-text index; `blob_hash = SHA256(path+code)` achieves content addressable deduplication.
4. **Retrieval**: queries simultaneously walk vector KNN and BM25, use RRF to fuse rankings, and then use rerank (default Voyage rerank-2.5) rank top_n.
5. **Graph Extension**: main results extend along the call graph for 1 hop, recalling callers/callees with `relation` tags.
6. **Real-time Monitoring**: watchdog monitors file changes (2s debounce), **background actively increments synchronization** (O(Change)), queries with zero wait; background failures do not lose changes, and make up before the next query.
7. **LRU Management**: multi-workspace concurrency (up to 8), exceeding limits automatically evicts the least recently used workspace.
## Index Storage
Each workspace is stored as `~/.semantic-code-mcp/<hash>.db` by directory path hash (can be modified with `SCM_DB_DIR`). Deleting this directory clears all indexes.
## Privacy and Data Boundary
This tool's complete data flow is used to assess whether it can be used for sensitive/closed-source repositories:
### Data Leaving the Network (Default Configuration)
| Receiver | Sent Content |
|---|---|---|
| Voyage AI | Code block full text (embedding + rerank candidates) + query text | During indexing + each retrieval |
| Cohere (optional backend) | Query text + top-50 candidate code block full text | Only whenM_RERANK_BACKEND=cohere`; default zero sending |
| Self-equipped LLM endpoint (optional, default off) | Only query text (excluding code) | When `SCM_QUERY_EXPANSION=on` |
Vendor data retention/training strategies are evaluated based on current terms (Voyage/Cohere both state API data is not used for training by default, this project does not endorse).
### Data Staying Local
- Index database entirely stored on the local machine `~/.semantic-code-mcp/` (vectors + original code + call graph), no data uploaded to any service
- No telemetry, no statistical reporting, zero network requests except API
### Index Range Control
- Entire hierarchy follows `.gitignore` (including negation patterns): `.env`, build artifacts, etc., ignored files will not be indexed
- Exceptions: `AGENTS.md` / `.windsurfrules`, etc., agent guidance files are indexed even if gitignored (intentional design, they have high value for retrieval architecture issues)
- ⚠️ **Hardcoded secrets in code will be sent to suppliers with code blocks**, sensitive repositories should clean or use full local mode before indexing
### Full Local Mode (Zero Code Leaving the Network)
```bash
pip install sentence-transformers
SCM_EMBED_BACKEND=local # local embedding (default Qwen/Qwen3-Embedding-0.6B)
not configure COHERE_API_KEY # rerank skips automatically, using RRF fusion sorting
```
Retrieval quality will decrease (refer to public evaluation ofed 0.6B scheme differences), but data does not leave the local machine.
## Roadmap
- **Phase 1** ✅: Tree-sitter + voyage-code-3 + hybrid + rerank — First-tier RAG
- **Phase 2** ✅: Call Graph graph extension (caller/callee recall) + int8 quantization vector (saves 71% storage, 95% consistency)
- **Phase 3** ✅: Engineering — watchdog file monitoring + multi-workspace LRU + progress + cancel
- **Phase 4** ✅ (partial): Intent-aware sorting (call chain direct query edges / test intent / document intent) + document configuration indexing (AGENTS.md exempt from gitignore) + 50 query evaluation sets
- **Phase 5** ✅: Single tooling + background active incremental synchronization + phased progress bar logs + Java call chain (receiver edge/ construction type name/ generic stripping) + configuration intent (yaml reversal boost)
+ Concurrent embedding pipeline (worker only runs network calls, sqlite write-back stays in the call thread, sequential write-back; CLIProxyAPI 9067 blocks full volume 77s, ≈ 3 times serial)
- **HyDE evaluated, conclusion defaults off**: Top-1 86%→82%, delay 2.9s→10.4s, Recall@5 98%→100%; architecture class +40 but configuration/SDK class -40 (variant dilution precise identifier signal).
Improvement line: selective extension — only open HyDE for fuzzy queries without precise identifiers Evaluation (eval_golden.py, CLIProxyAPI / 9067 blocks)
50 queries × 10 categories (symbol location/ call chain/ architecture responsibility/ configuration schema/ same name interference/ SDK boundary/ protocol contract/ test/ Chinese/ details),
mechanized scoring is reproducible (expected file + must_contain keyword):
| Metric | Voyage rerank-2.5 (default) | Cohere v3.5 baseline | Reference (self-hosted 0.6B scheme public data) |
|---|---|---|---|
| Top-1 Precision | **88%** | 84% | 32% |
| Recall@5 | 98% | 100% | 54% |
| Answer Availability | 100% | 100% | 80% |
| MRR | **0.916** | 0.897 | — |
| Average Delay | **0.96s** | 6.2s (trial key throttling) | — |
Additional **K class private business repository probes** (not included in main score): verify compound intent splitting/ subquery score mixing/ bottom line, trigram Chinese recall, implementation intent reduction, `expand_graph=True` full link.
Probe definitions are in gitignore `eval_local_probes.py` (`PROBE_TARGET` + `PROBE_GOLDEN`,
written in `eval_golden K class loading section), non-existent automatic skip (explicit skip with `--skip-java`) —
you can build a set of probe formats for your own repository as a private regression security network.
Re-measurement: `python eval_golden.py --json`; regression test: `python test_fixes.py`
### Model Selection Description
- **Why voyage-code-3 embedding instead of Voyage 4 series**: 4 series (2026-01) is a general text model,
**no voyage-code-4**; official documentation still recommends code-3 for code retrieval scenarios.
Try general flagship `SCM_EMBED_MODEL=voyage-4-large` (need to rebuild index),
comparison evaluation with code-3 included in follow-up.
- **Why rerank defaults to Voyage**: same key as embedding without extra configuration;
rerank-2.5 open benchmark better than Cohere v3.5 (+7.94% official data, +4 this project evaluation Top-1);
200M token free quota and no trial key 10 times/minute speed limit (evaluation speed up 6.5×).
Cohere reserved as optional backend Engineering Boundary (Known Cost)
- **Rerank speed limit**: unbound/ trial key speed limitohere trial 10 times/minute).
Dense calls set `SCM_RERANK_MIN throttling; 429 automatic backoff retry bottom line.
- **Compound query extra delay**: Chinese dual-action query ("generate… and clean…") each subquery sends one more
rerank (+200~/times, up to 2 times). Only compound wording triggers, zero overhead for normal queries.
- **Threshold calibration range**: subquery score threshold 0.2, two-stage splitting, etc.,
parameters calibrated on **Cohere score distribution** in Chinese Java repository + Go repository double evaluation sets;
switch rerank backend (different score distribution) or other languages/ repository performance anomalies,
prioritize review `_COMPOUND_SUB_MIN_SCORE`.
## Performance Data (42 blocks / 9 files)
| Metric | float32 | int8 |
|---|---|---|
| Indexing Time | 2.9s | 2.4s |
| DB Size | 4328 KB | 1252 KB (saves 71%) |
| Retrieval Delay | 0.3–0.8s | 0.3–0.8s |
| Top5 Consistency | — | 95% (19/20) |
| Rerank Score | score 0.1–0.7 | — |
MCP Config
Below is the configuration for this MCP Server. You can copy it directly to Cursor or other MCP clients.
mcp.json
Connection Info
You Might Also Like
No recommendations available