Content
# study_rag
> Enterprise knowledge base retrieval service built on LlamaIndex, exposed to AI agents via MCP.
[](LICENSE)
[](https://www.python.org/)
[](https://fastapi.tiangolo.com/)
[](https://modelcontextprotocol.io/)
[](https://milvus.io/)
[](https://react.dev/)
## Project Introduction
study_rag is a plugable, multi-tenant enterprise-level RAG (Retrieval Augmented Generation) knowledge base service. It packages "knowledge base management + document processing + multi-strategy retrieval" into a set of capabilities, providing both **MCP Tool** for AI agents to call directly and **Admin REST + Web console** for human management.
Core values:
- **Agent-ready**: Compliant with MCP specifications, Claude / Cursor / any MCP client can connect and retrieve your knowledge base
- **Multi-strategy retrieval**: Dense / Sparse / Hybrid / Milvus native BM25 / Hybrid-Milvus five strategies can be switched per request
- **Pluggable capabilities**: Embedding / VectorStore / Reranker / Parser are all interface-designed, configurable and swappable
- **Production-ready**: structlog structured logging, Prometheus metrics, rate limiting, circuit breaking, Docker Compose one-click deployment
## Technology Stack
| Layer | Technology |
|---|---|
| Backend | Python 3.10+, FastAPI, Pydantic, structlog |
| Retrieval Framework | LlamaIndex (NodeParser chunking), self-developed RetrievalEngine abstraction |
| Vector Database | Milvus 2.5+ (native BM25), scalable Qdrant / InMemory |
| Embedding | OpenAI / Ollama / BGE, interface-designed and swappable |
| Reranker | BGE / Cohere / TEI, interface-designed and swappable |
| MCP | FastMCP, Streamable HTTP transport |
| Frontend | React + Vite + TypeScript + shadcn/ui |
| Observability | Prometheus metrics, structlog logging, rate limiting, circuit breaking |
| Deployment | Docker Compose (admin + mcp dual-process), tini, non-root |
| Engineering | ruff (lint), pytest, justfile, TypeScript typecheck |
## Features
- **Multi-knowledge base management**: Supports multiple knowledge bases configured by department/theme
- **Multi-tenant isolation**: Based on api_key user ↔ knowledge base permission matrix
- **MCP protocol (Streamable HTTP)**: 10 core Tools, providing complete capabilities for discovery/ retrieval/ document management
- **Pluggable capabilities**: Embedding / VectorStore / Reranker can be replaced with implementations
- **LlamaIndex integration**: NodeParser chunking, dual retrieval paths (self-developed + LlamaIndex)
- **Document management enhancement**: Supports text input / file upload (txt / md / html / pdf / docx),
multiple chunking strategies (whole / sentence / token / semantic through named parser),
chunk preview (showing each chunk's content/size before submission), strategy configuration through `configs/llamaindex.yaml`
- **Chunk viewing**: Management UI "document" list "view chunk" button → Drawer displays all chunks of the document
with `#index / character count / full text / source metadata`, paginated 20/page
(corresponding to `GET /admin/kbs/{kb_id}/documents/{doc_id}/chunks` endpoint)
- **Parser Registry**: Named parser entities, similar to embedder / reranker registry design,
providing `get_parser(name)` / `list_parsers()` interfaces
- **New API endpoints**: `GET /admin/parsers` / `POST /documents/preview-chunk` / `POST /documents/upload`
- **Unified management interface**: FastAPI provides knowledge base CRUD + health check
## Documentation
| Document | Content |
|---|---|
| [Deployment Guide](docs/deployment.md) | Docker Compose, image build, Milvus, production checklist, operation and maintenance |
| [Configuration Reference](docs/configuration.md) | All YAML configuration files, environment variables, provider comparison table |
| [This document](#mcp-tool-complete-list) | MCP Tool complete API, client examples, error codes |
## Quick Start
### Docker Compose (Recommended)
```bash
# 1. Prepare environment variables
cp .env.example .env
# Edit .env as needed (configure at least one embedding backend API_KEY)
# 2. Build and start admin + mcp
docker compose -f docker/docker-compose.yml up -d --build
# Or: just docker-up
# 3. Start with Milvus vector database integration
docker compose -f docker/docker-compose.yml --profile vector up -d --build
# Or: just docker-up-vector
# Endpoints:
# Management UI http://localhost:8765/admin/ui/
# OpenAPI documentation http://localhost:8765/docs
# MCP endpoint http://localhost:8001/mcp
# Health check http://localhost:8765/health / http://localhost:8001/health
```
See [Deployment Guide](docs/deployment.md) for details.
### Local Python
```bash
# Install dependencies
pip install -e ".[dev,llamaindex,vector-milvus,embedding-bge,reranker-bge]"
# Start admin REST (port 8765)
study-rag
# Start MCP standalone server (port 8001, streamable_http transport)
study-rag-mcp
# MCP endpoint
http://localhost:8001/mcp
# Admin REST endpoint
http://localhost:8765/admin/docs
```
### Local Development
```powershell
# One-click script (PowerShell)
pwsh scripts/dev.ps1 mcp # Start MCP server
pwsh scripts/dev.ps1 inspector # Start MCP Inspector (browser debugging)
pwsh scripts/dev.ps1 admin # Start admin REST
pwsh scripts/dev.ps1 all-verify # Run lint + typecheck + full verification
```
## Architecture
```
study_rag/
├── capabilities/ # Capability layer (embedding/vector_store/reranker/llamaindex/retrieval)
├── knowledge_bases/ # Knowledge base management (registry/manager/models/config_store)
├── auth/ # Authentication (placeholder implementation, including writable_kbs)
├── mcp/ # MCP service (10 Tools + 3 Resources + 2 Prompts + filter)
├── api/ # FastAPI management interface (admin/health)
├── jobs/ # Asynchronous tasks (document chunking pipeline)
├── observability/ # Logging (structlog)/metrics (Prometheus)/rate limiting/circuit breaking/middleware
├── web/ # Admin UI SPA build product mounting (/admin/ui/)
├── mcp_standalone.py # MCP standalone deployment entry (streamable_http)
├── app.py # Admin REST entry
├── settings.py # Global configuration (pydantic-settings)
├── frontend/ # React + Vite + shadcn/ui management console source code
├── configs/ # YAML configuration (supporting ${ENV_VAR} placeholders)
│ ├── knowledge_bases.yaml
│ ├── embeddings.yaml
│ ├── vector_store.yaml
│ ├── reranker.yaml
│ ├── retrieval.yaml
│ ├── llamaindex.yaml
│ └── llm.yaml
├── docker/ # Containerization (Dockerfile + docker-compose + entrypoint)
├── docs/ # Documentation (deployment.md / configuration.md)
└── scripts/ # Development scripts (dev.ps1 / inspector.ps1)
```
**Deployment Topology** (two-process independent scaling):
```
Agent / LLM ──MCP──▶ mcp:8001 ──▶ ┌─ embeddings (OpenAI/BGE/Ollama)
├─ rerankers (BGE/Cohere/TEI)
Browser ──HTTP──▶ admin:8765 ──▶ └─ vector_store (Milvus/Qdrant/mock)
└─ /admin/ui/ (React SPA)
└─ /admin/* (KB/document CRUD)
└─ /metrics (Prometheus)
```
See [Configuration Reference](docs/configuration.md) for details.
## Environment Variables
Core variables (complete list see [`.env.example`](.env.example) and [Configuration Reference](docs/configuration.md#environment-variables-reference)):
| Variable | Default | Description |
|---|---|---|
| `STUDY_RAG_PORT` | `8765` | admin REST port |
| `MCP_PORT` | `8001` | MCP port |
| `STUDY_RAG_LOG_LEVEL` | `INFO` | Log level |
| `STUDY_RAG_ADMIN_TOKEN` | *(empty)* | admin Bearer Token (required for production) |
| `STUDY_RAG_MCP_REQUIRE_API_KEY` | `false` | Force MCP api_key |
| `MILVUS_URI` | `http://milvus:19530` | Vector database address |
| `OPENAI_API_KEY` | *(empty)* | OpenAI embedding key |
YAML configuration `${VAR}` placeholders are resolved to environment variables at runtime, allowing backend switching without code changes.
---
# MCP Tool Complete List
study_rag exposes **10 core Tools** under Streamable HTTP transport.
All Tools accept `api_key: str` as the first parameter (currently placeholder authentication).
Return values are JSON strings (avoiding FastMCP's side effects on list[dict]), parsed by clients with `json.loads(text)`.
## 1. `list_accessible_kbs` — Discover accessible knowledge bases
**The first step of any retrieval operation**. Call this Tool to understand which KBs are available and their descriptions.
| Parameter | Type | Required | Description |
|---|---|---|---|
| `api_key` | str | ✓ | User credential |
Return: `list[KBInfo]`, each element containing:
```json
[
{
"kb_id": "rd_frontend",
"name": "Frontend R&D Documentation",
"description": "React / Vue component library, state management, performance optimization",
"department": "rd",
"enabled": true,
"document_count": 42
}
]
```
Exceptions:
- Authentication failed → `permission_denied`
Applicable scenarios:
- Mandatory before any retrieval
- Uncertain about available KBs
- Need to determine relevant KB based on description
---
## 2. `get_kb_info` — Get knowledge base details
Confirm KB's embedding / reranker / document count, etc., to decide if it's retrievable.
| Parameter | Type | Required | Description |
|---|---|---|---|
| `api_key` | str | ✓ | |
| `kb_id` | str | ✓ | Knowledge base ID (naming convention `{dept}_{name}`) |
Return:
```json
{
"kb_id": "rd_frontend",
"name": "Frontend R&D Documentation",
"description": "...",
"department": "rd",
"enabled": true,
"document_count": 42,
"embedding": "bge-large-zh",
"reranker": "bge-reranker-large",
"extra": {}
}
```
Exceptions:
- KB not found / user unauthorized → `kb_not_found`
---
## 3. `search_kb` — Single knowledge base retrieval (core)
Retrieve top-k relevant documents in a specified KB. Optional rerank and metadata filter.
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
| `api_key` | str | ✓ | | |
| `kb_id` | str | ✓ | | Knowledge base ID |
| `query` | str | ✓ | | Retrieval query |
| `top_k` | int | | 5 | Return count, range (0, 50] |
| `use_rerank` | bool | | true | Enable reranking |
| `filter_expr` | dict | | null | Metadata filtering conditions |
**filter_expr operators:**
| Suffix | Meaning | Example |
|---|---|---|
| None / `__eq` | Equal | `{"source": "wiki"}` |
| `__ne` | Not equal | `{"status__ne": "draft"}` |
| `__in` | In list | `{"tag__in": ["api", "db"]}` |
| `__nin` | Not in list | `{"tag__nin": ["internal"]}` |
| `__gt` / `__gte` | Greater than / ≥ | `{"year__gte": 2024}` |
| `__lt` / `__lte` | Less than / ≤ | `{"score__lt": 0.5}` |
| `__contains` | String contains | `{"title__contains": "API"}` |
| `__exists` | Field existence | `{"draft__exists": false}` |
**Multiple conditions connected with AND**: `{"year__gte": 2024, "tag__in": ["api"]}` represents `year >= 2024 AND tag in [api]`.
Return:
```json
[
{
"doc_id": "doc-123",
"title": "React 18 Concurrent Rendering Mechanism",
"text": "React 18 introduced concurrent rendering...",
"score": 0.87,
"metadata": {
"source": "wiki",
"year": 2024,
"tag": "frontend"
}
}
]
```
Exceptions:
- KB not found / unauthorized → `kb_not_found`
- `query` empty / `top_k` out of range → `invalid_parameter`
- `filter_expr` field name invalid → `invalid_parameter`
---
## 4. `search_all_accessible_kbs` — Cross-knowledge base comprehensive retrieval
Fallback entry when unsure which KB to query. Parallel retrieval in all accessible KBs, merged and sorted by relevance.
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
| `api_key` | str | ✓ | | |
| `query` | str | ✓ | | |
| `top_k` | int | | 5 | Final return count |
| `use_rerank` | bool | | true | |
| `filter_expr` | dict | | null | Pushed down to each KB |
Return: Same as `search_kb`, but each hit's `metadata._source_kb` identifies the source KB ID.
Implementation details: Each KB internally reranks with its own reranker, and cross-KB layer only merges by score (different KBs' reranker score dimensions differ, strict merging is not rigorous) ---
## 5. `get_document` — Get Complete Document Content
`search_kb` returns document fragments. Use this Tool to view the full content.
| Parameter | Type | Required | Description |
|---|---|---|---|
| `api_key` | str | ✓ | |
| `kb_id` | str | ✓ | |
| `doc_id` | str | ✓ | Document ID |
Returns: Document metadata + complete `content`.
Exceptions:
- Document does not exist → `document_not_found`
---
## 6. `list_documents` — List Documents in KB
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
| `api_key` | str | ✓ | | |
| `kb_id` | str | ✓ | | |
| `limit` | int | | 100 | Maximum number of returned documents, range (0, 1000] |
| `offset` | int | | 0 | Number of skipped documents (pagination) |
Returns: `list[DocumentSummary]`, sorted by creation time in descending order.
---
## 7. `add_document` — Add a Document (a single chunk)
Write permission required (`writable_kbs` includes `kb_id`).
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
| `api_key` | str | ✓ | | |
| `kb_id` | str | ✓ | | |
| `doc_id` | str | ✓ | | Unique within KB |
| `title` | str | ✓ | | |
| `content` | str | ✓ | | |
| `source` | str | | null | Source; automatically stored as `metadata['source']` |
| `metadata` | dict | | null | Additional metadata |
| `overwrite` | bool | | false | Overwrite if document exists |
`source` / `metadata` fields can be referenced in subsequent `search_kb` `filter_expr`.
Exceptions:
- No write permission → `permission_denied`
- `doc_id` already exists + `overwrite=False` → `document_already_exists`
- Invalid parameter → `invalid_parameter`
---
## 8. `add_document_chunked` — Add a Document (NodeParser chunks)
Write permission required. The entire document will be chunked and stored using LlamaIndex NodeParser.
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
| `api_key` | str | ✓ | | |
| `kb_id` | str | ✓ | | |
| `doc_id` | str | ✓ | | |
| `title` | str | ✓ | | |
| `content` | str | ✓ | | |
| `source` | str | | null | |
| `metadata` | dict | | null | |
| `parser_config` | dict | | null | NodeParser configuration |
| `overwrite` | bool | | false | |
`parser_config` example:
```json
{
"strategy": "sentence",
"chunk_size": 512,
"chunk_overlap": 50
}
```
Recommended for long documents (> 1000 words).
---
## 9. `add_documents_batch` — Batch Add Documents
Write permission required. Failed documents do not affect others (logged).
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
| `api_key` | str | ✓ | | |
| `kb_id` | str | ✓ | | |
| `documents` | list[dict] | ✓ | | Each item: `{"doc_id", "title", "content", "source"?, "metadata"?}` |
| `overwrite` | bool | | false | |
Limit: 500 documents per batch.
---
## 10. `delete_document` — Delete a Document
Write permission required. Vector and metadata are deleted together.
| Parameter | Type | Required | Description |
|---|---|---|---|
| `api_key` | str | ✓ | |
| `kb_id` | str | ✓ | |
| `doc_id` | str | ✓ | |
Exceptions:
- No write permission → `permission_denied`
- Document does not exist → `document_not_found`
---
## Typical Workflow
```text
1. list_accessible_kbs(api_key) # List accessible KBs
2. (Optional) get_kb_info(api_key, kb_id) # Confirm KB scope
3. search_kb(api_key, kb_id, query, top_k=5) # Search
4. (Optional) get_document(api_key, kb_id, doc_id) # Get complete document
```
Fallback: `search_all_accessible_kbs(api_key, query)` cross-KB search.
---
## Client Examples
### Python (using official MCP SDK)
```python
import asyncio
import json
from mcp import ClientSession
from mcp.client.streamable_http import streamablehttp_client
async def main():
async with streamablehttp_client("http://localhost:8001/mcp") as (
read_stream,
write_stream,
_get_session_id,
):
async with ClientSession(read_stream, write_stream) as session:
await session.initialize()
# List tools
tools = await session.list_tools()
# Search
result = await session.call_tool(
"search_kb_tool",
arguments={
"api_key": "test",
"kb_id": "rd_frontend",
"query": "How to use concurrent rendering with React 18?",
"top_k": 5,
},
)
# content[0].text is a JSON string
hits = json.loads(result.content[0].text)
for h in hits:
print(f"[{h['score']:.2f}] {h['title']}")
asyncio.run(main())
```
### TypeScript
```typescript
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js";
const client = new Client({ name: "my-agent", version: "1.0.0" });
const transport = new StreamableHTTPClientTransport(
new URL("http://localhost:8001/mcp")
);
await client.connect(transport);
const result = await client.callTool({
name: "search_kb_tool",
arguments: {
api_key: "test",
kb_id: "rd_frontend",
query: "How to use concurrent rendering with React 18?",
top_k: 5,
},
});
const hits = JSON.parse(result.content[0].text);
```
### Claude Desktop Configuration
`~/Library/Application Support/Claude/claude_desktop_config.json` (macOS) or corresponding platform path:
```json
{
"mcpServers": {
"study-rag": {
"url": "http://localhost:8001/mcp",
"transport": "streamable_http"
}
}
}
```
---
## Error Codes
| code | Meaning | Triggering scenario |
|---|---|---|
| `permission_denied` | Authentication failed | Invalid api_key / user does not have access to the KB |
| `kb_not_found` | KB does not exist | kb_id does not exist or is disabled |
| `document_not_found` | Document does not exist | doc_id does not exist |
| `document_already_exists` | Document already exists | add_document with duplicate doc_id and overwrite=False |
| `invalid_parameter` | Invalid parameter | query is empty, top_k out of range, filter_expr field name is invalid |
All errors are returned through MCP's standard `error.code` field (JSON-RPC format).
---
## Debugging
```powershell
# Start MCP server
pwsh scripts/dev.ps1 mcp
# Start MCP Inspector (browser)
pwsh scripts/dev.ps1 inspector
# Open browser at http://localhost:5173
# Select transport: streamable_http
# URL: http://localhost:8001/mcp
# Connect → List Tools → Select Tool → Fill parameters → Call Tool
```
---
## Protocol Version
- MCP protocol: `2025-03-26`
- Transport: Streamable HTTP
- Python: `>=3.10`
- FastMCP: `>=1.0`
MCP Config
Below is the configuration for this MCP Server. You can copy it directly to Cursor or other MCP clients.
mcp.json
Connection Info
You Might Also Like
No recommendations available