askrepo

# AskRepo A local codebase assistant. Point it at a git repository and ask questions like *"where is authentication handled?"* or *"what does this function do?"* - it answers with citations to exact `file:line` locations. <p align="center"> <img src="docs/demo.svg" alt="askrepo ask, answering a question with a code citation" width="820"> </p> Two things make it work well: - **AST chunking.** Code is split by structure using tree-sitter - one chunk per function, method, or class - instead of fixed line windows. Chunks stay semantically whole and every chunk knows its exact line range, so citations are precise. - **Deterministic citations.** The model only refers to numbered snippets (`[1]`, `[2]`). The `file:line` for each is resolved from chunk metadata, not written by the model, so it can't hallucinate a line number. Snippets it doesn't cite are dropped from the answer. Everything runs locally: - `qwen3:14b` via [Ollama](https://ollama.com) generates the answer, - `jina-embeddings-v2-base-code` (via [fastembed](https://github.com/qdrant/fastembed), ONNX) embeds the code - no torch, no transformers, - `jina-reranker-v2-base-multilingual` (cross-encoder, ONNX) is available as an optional reranking pass - off by default, because the ablation shows it hurts here, - [LanceDB](https://lancedb.com) stores the vectors and does hybrid dense + full-text search. ## Install ```bash python -m venv venv && source venv/bin/activate pip install -e . # or: pip install -r requirements.txt ``` You need Ollama running with the answer model pulled: ```bash ollama pull qwen3:14b ``` Embedding and reranker models are downloaded automatically on first use. ## Usage ```bash # Index a repository (incremental - only changed files are re-embedded) askrepo index /path/to/repo # Ask askrepo ask "where is authentication handled?" --repo /path/to/repo askrepo ask "what does the login function do?" --repo /path/to/repo ``` If you installed with `pip install -e .` the `askrepo` command is on your PATH; otherwise use `python -m askrepo ...`. ### What gets searched Tests and prose are excluded from results by default. They describe the implementation in the same words the question uses, so they outrank the code that actually answers it - on the labeled query set they took rank 1 in **36%** of queries and ate nearly a third of the top-5 slots. Dropping them is the single largest ranking win in the stack (see the ablation). The filter turns itself off when the question is about tests or docs, so this still works: ```bash askrepo ask "how are citations mapped to file and line?" # implementation askrepo ask "where is citation mapping tested?" # tests kept automatically askrepo ask "..." --include-tests --include-docs # force everything back in ``` ### Use it as an MCP tool (Claude Code, etc.) AskRepo can run as an [MCP](https://modelcontextprotocol.io) server, exposing a read-only `search_code` tool. Point it at an already-indexed repo and an agent can search your codebase by meaning - semantic hits with exact `file:line`, which complements plain grep. ```bash pip install -e ".[mcp]" askrepo index /path/to/repo # index first ``` Add it to your MCP client. For Claude Code, in `.mcp.json`: ```json { "mcpServers": { "askrepo": { "command": "askrepo-mcp", "env": { "ASKREPO_MCP_REPO": "/path/to/repo" } } } } ``` The agent then calls `search_code(query, k=8, rerank=..., expand=..., include_tests=..., include_docs=...)`; `expand=true` also returns the definitions of functions the matches call. The tool never writes anything. Every flag falls back to the server's environment when the agent leaves it unset, so the `env` block sets the default for each call while the agent can still override it per search. A search costs ~20 ms with the defaults, which is what makes it usable as a tool an agent reaches for repeatedly. ### Evaluate retrieval ```bash askrepo eval --repo /path/to/repo --k 5 ``` Ablates the retrieval stack - `dense`, `fts`, `hybrid`, `hybrid+scope`, `+rerank` - and reports hit@1, hit@k, recall@k, MRR, symbol-hit@k, the share of slots wasted on tests/prose, and latency, against a labeled query set (`<repo>/eval/queries.json`). It's retrieval-only (no LLM), so it's fast and deterministic. `--with-llm` additionally ablates HyDE. See `eval/queries.json` for the format. Results on this repo (35 queries, k=5), showing what each stage adds: | config | hit@1 | hit@5 | MRR | test% | latency | |---|---|---|---|---|---| | dense | 0.657 | 0.857 | 0.744 | 38% | 11 ms | | fts | 0.371 | 0.743 | 0.499 | 22% | 5 ms | | hybrid | 0.600 | 0.914 | 0.726 | 23% | 17 ms | | hybrid+rerank | 0.514 | 0.914 | 0.695 | 15% | ~5.3 s | | **hybrid+scope** | **0.829** | **0.943** | **0.863** | **5%** | **18 ms** | | hybrid+scope+rerank | 0.743 | 0.943 | 0.833 | 6% | ~2.7 s | Hybrid beats either retriever alone on recall. The big ranking win is scope filtering: keeping tests and prose out of the candidate set lifts hit@1 from 0.60 to 0.83 and cuts wasted slots from 23% to 5%, for free. The cross-encoder reranker is **off by default**. It leaves recall untouched (hit@5 0.943 either way - the right chunk was already in the set) and moves rank 1 the wrong way: across 35 queries it fixed 2 and broke 5. Seven discordant queries is far too few to call that a real regression (exact McNemar p = 0.45), but that is the point - it cannot demonstrate a gain, and it costs ~2.7 s per query to not demonstrate it. The failure mode is consistent enough to name. Every query it broke promoted a *longer* chunk over a shorter one - 1034 vs 491 characters on average - and the promoted chunk was usually one that mentions the topic's vocabulary rather than one that implements it: | question | reranker demoted | reranker promoted | |---|---|---| | how are code embeddings produced with fastembed? | `embedder.py` `Embedder` | `config.py` `Config` (lists every `ASKREPO_*` name) | | how are citations mapped to file and line numbers? | `answer.py` `_cited_indices` | `chunker.py` `_line_fallback` | A cross-encoder trained on prose QA rewards surface overlap and has a length bias, so a big enumerative chunk outscores the small function that actually does the work. It also discards the RRF ordering entirely instead of blending with it, so a weak signal overwrites a good prior. Swapping in the lighter `ms-marco-MiniLM` model does not change this (hit@1 0.714, MRR 0.807) - it is the reranking step, not the model. It stays available behind `--rerank` / `ASKREPO_RERANK=1` for corpora where it does pay off. This is the same way the old symbol-boost heuristic was cut - the ablation decides. ## How it works ``` git repo -> ingest -> AST chunker -> embedder -> LanceDB | question -> retriever (dense + FTS, RRF, scope filter) -> context -> qwen3 -> answer + citations ``` | Module | Responsibility | |--------|----------------| | `ingest.py` | Walk the repo, honor nested `.gitignore`, detect language, hash files for incremental indexing. Skips symlinks and oversized/binary files. | | `chunker.py` | AST chunking with tree-sitter; overlapping line-window fallback for unsupported files. | | `embedder.py` | jina (fastembed/ONNX) or ollama/nomic embeddings. | | `store.py` | LanceDB table + file-hash manifest. | | `retriever.py` | Hybrid retrieval: RRF over dense + full-text, scope filtering, call-graph expansion. | | `reranker.py` | Optional cross-encoder reranking of the candidate set (off by default). | | `answer.py` / `llm.py` | Prompt assembly, generation, deterministic `[n]` citation mapping. | | `cli.py` | `index` / `ask` / `eval`. | | `mcp_server.py` | Read-only `search_code` MCP tool for agents. | ## Configuration All settings can be overridden via environment variables: | Variable | Default | Meaning | |----------|---------|---------| | `ASKREPO_EMBED_BACKEND` | `jina` | `jina` (fastembed) or `ollama` (nomic) | | `ASKREPO_LLM` | `qwen3:14b` | Ollama model for answers | | `ASKREPO_RERANK` | `0` | Cross-encoder reranker (`1` to enable; see the ablation first) | | `ASKREPO_RERANKER` | `jinaai/jina-reranker-v2-base-multilingual` | Reranker model (lighter: `Xenova/ms-marco-MiniLM-L-6-v2`) | | `ASKREPO_TOPK` | `8` | Chunks put into the answer context | | `ASKREPO_EXCLUDE_TESTS` | `1` | Drop test files from results (auto-off when the query is about tests) | | `ASKREPO_EXCLUDE_DOCS` | `1` | Drop markdown/config files from results (auto-off for doc questions) | | `ASKREPO_ENRICH` | `1` | Embed a path/symbol header with each chunk | | `ASKREPO_CONTEXTUAL` | `0` | Add an LLM one-line summary per chunk at index time (slow) | | `ASKREPO_HYDE` | `0` | Expand the query with a hypothetical snippet before embedding | | `ASKREPO_EXPAND` | `0` | Follow the call graph and retrieve called-function definitions | | `ASKREPO_EXPAND_DEPTH` | `2` | How many call levels `--expand` follows | | `ASKREPO_EXPAND_CALLERS` | `1` | With `--expand`, also add a few call sites of the results | | `ASKREPO_HOME` | `~/.askrepo` | Where indexes are stored | | `OLLAMA_HOST` | `http://localhost:11434` | Ollama endpoint | ### Retrieval quality - **Enrichment** (on by default): the embedded text is prefixed with the file path and symbol, which helps natural-language questions match code. - **Contextual retrieval** (`askrepo index --contextual`): generates a one-line LLM summary per chunk and embeds it too. Higher quality on conceptual questions, but one LLM call per chunk makes indexing slow - opt-in. - **HyDE** (`askrepo ask --hyde`): turns the question into a hypothetical code snippet and embeds that alongside it. One extra LLM call per question. - **Callee expansion** (`askrepo ask --expand`): after retrieval, follows the call graph - the functions the matched chunks call, then what those call, up to `--expand-depth` levels (default 2) - and adds their definitions to the context, so an answer can trace down to where the work actually happens and cite the whole chain. Each of these is a separate index (or query mode), so use `askrepo eval` to see whether it actually helps on your repo before leaving it on. Indexes are keyed by repo path *and* embedding model, so switching backend or model builds a separate index rather than mixing incompatible vectors. `jina` and `nomic` are both 768-dim but not interchangeable - reindex after switching. ## Development ```bash pip install -e ".[dev]" pytest ``` The tests cover the chunker (AST boundaries, decorators, arrow functions, fallback), the deterministic citation mapping, and ingest security (symlink and size handling). They don't require Ollama or any model downloads. ## Scope v1 is read-only Q&A. There's no code modification - the retriever and answer layer are kept separate so a change-proposal step could be added later without a rewrite. ## License MIT - see [LICENSE](LICENSE).