Content
# devitway-rag-starter
Minimal but complete E2E stack for local RAG: documents → vectors → search,
connectable to any agent via **MCP**. RAG here is a **standalone layer**,
not part of a specific agent: the same server can be connected to both `qwen-code`
and `Claude Code` without modifications.
```
OFFLINE (cron / watch — «self-updating»):
docs/ ─► Reader ─► Splitter ─► ollama(bge-m3) ─► Qdrant
│ │ ▲ ▲
└── docstore (hash by doc_id) ─┘ didn't change→skip,
changed→upsert, deleted→delete
QUERY (runtime, any agent):
agent ─MCP─► search(q) ─► ollama(bge-m3) ─► Qdrant top-k ─► fragments ─► agent
```
**Roles:** `ollama` runs only the embedding model (vectorization, locally) ·
`Qdrant` stores and searches vectors · `MCP Server` — a single entry point for agents.
The agent generates the response using its own model (Qwen / Claude).
---
## Requirements
- **Docker** (for Qdrant)
- **Python 3.10+**
- **[ollama](https://ollama.com)** installed and running
## Quick Start
```bash
# 1. Start Qdrant
docker compose up -d
# 2. Download the embedding model (multilingual, RU/EN/code)
ollama pull bge-m3
# 3. Python environment
python -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install -r requirements.txt
# 4. Index documents (example is already in ./docs)
python ingest.py ./docs
# 5. Check that the server starts (Ctrl+C to exit)
python mcp_server.py
```
Ready — RAG is working. All that's left is to connect the agent (below).
## How Incremental Updates Work
`ingest.py` stores state in `pipeline_storage/` (which files are already
indexed + their hashes). On restart:
| What happened to the file | Action |
|------------------------|----------|
| didn't change | skip (by hash) |
| changed | re-embed and upsert in Qdrant |
| deleted from `./docs` | its chunks are removed from Qdrant |
There is no full reindexing — only delta. So you can run it every minute.
### Auto-Update
**Cron** (every 15 minutes):
```cron
*/15 * * * * cd /path/to/devitway-rag-starter && .venv/bin/python ingest.py ./docs >> ingest.log 2>&1
```
**Watch** (react to changes immediately, requires `inotify-tools`):
```bash
while inotifywait -r -e modify,create,delete ./docs; do
.venv/bin/python ingest.py ./docs
done
```
## Connecting Agents
Both agents connect to the **same** server. Test on the one that's already
working, then add the second — no modifications needed.
> ⚠️ In `command`, specify the **absolute path to python from .venv** — the agent runs
> the server without activating the environment, otherwise dependencies won't be found.
### qwen-code
In `settings.json` (usually `~/.qwen/settings.json` or `.qwen/settings.json` in the project):
```json
{
"mcpServers": {
"rag": {
"command": "/path/to/devitway-rag-starter/.venv/bin/python",
"args": ["/path/to/devitway-rag-starter/mcp_server.py"]
}
}
}
```
### Claude Code
```bash
claude mcp add rag -- /path/to/devitway-rag-starter/.venv/bin/python /path/to/devitway-rag-starter/mcp_server.py
```
Verification: ask the agent something about the contents of `./docs` — it should call
the `search` tool.
## Main Invariant (don't break)
**Indexing and search must use the same embedding model.** If you index with one
(1024d) and search with another (e.g., 384d), you'll get garbage or a dimensionality error.
So the model and addresses are defined in one place — `config.py`, — from where both
scripts take them. If you change the model, update it in `config.py` and do a full reindex
(delete `pipeline_storage/` and the collection in Qdrant).
## Troubleshooting
| Symptom | Cause / Solution |
|---------|-------------------|
| Error about `OpenAI API key` | LlamaIndex defaults to OpenAI embedder. In this repo, `Settings.embed_model` is set explicitly — check that you didn't override it. |
| Search returns garbage / empty | Model during indexing ≠ during search. Keep the same (see invariant above). |
| `Connection refused` to Qdrant | `docker compose up -d` not running or port 6333 occupied. |
| `model not found` | Forgot `ollama pull bge-m3`. |
| Agent doesn't see MCP server | `command` must point to `.venv/bin/python` (absolute path), not system python. |
| Server connected, but agent doesn't call `search` (even on explicit request) | Usually the local model is to blame, not RAG. Qwen3 via ollama doesn't return tool calls in streaming ([ollama#14601](https://github.com/ollama/ollama/issues/14601)) — an empty response comes, and it can't be treated with a prompt. Check the layer separately: `python -c "from mcp_server import search; print(search('ollama'))"` — if fragments are returned, change the model: `ollama pull gpt-oss:20b` (tool calling is stable) or cloud (in qwen-code — default Qwen OAuth). For qwen-code, version ≥0.18.4 is required — older versions flap tool calling with any local model. |
## Structure
```
devitway-rag-starter/
├── docker-compose.yml # Qdrant
├── requirements.txt
├── config.py # unified configuration (embed model, addresses) — source of invariant
├── ingest.py # incremental ingest docs -> Qdrant
├── mcp_server.py # read-only MCP: search() tool
├── env.example # cp env.example .env
└── docs/ # put your documents here
└── example.md
```
## Future Growth
- **Reranker** as a second stage (`bge-reranker-v2-m3`) — significantly improves accuracy.
- **Redis-docstore** instead of file-based — for a large corpus and parallel ingest.
- **More formats** — `SimpleDirectoryReader` already reads pdf/docx/md/txt; connectors
(Google Drive, Notion, DB) — separate packages `llama-index-readers-*`.
