atlas-agentic-rag

# Atlas — Agentic RAG Platform      > Ingest a knowledge base, then ask an **agent** that retrieves with **hybrid > search + reranking + a knowledge graph**, reasons with **tool-calling over > MCP**, routes across **Claude / OpenAI / local models**, is wrapped in > **guardrails** (prompt-injection defense, PII redaction), and is measured by a > first-class **evaluation harness** — with **Langfuse** observability, **Redis** > semantic caching, and full **Docker / Kubernetes / CI-CD**. <p align="center"> <img src="docs/demo.svg" alt="Atlas answering a question with grounded citations in the terminal" width="820"> </p> <p align="center"> <b>See the whole system in 20 seconds — no keys, no services:</b><br> <code>pip install pydantic pydantic-settings fastapi "uvicorn[standard]" httpx rich && python -m atlas.cli demo</code> </p> Atlas is a single, coherent agentic-RAG system rather than a folder of demos. Its subsystems are decoupled through Python `Protocol` interfaces, so every hosted dependency has a lightweight local fallback: **the default configuration runs fully offline with no API keys and no external services** (in-memory vector store, a deterministic embedder, and a mock model), and each hosted integration — Claude, OpenAI, pgvector, Qdrant, Langfuse, Redis — activates automatically the moment you configure its key or DSN. --- ## Architecture ```mermaid flowchart LR subgraph Ingest["atlas.ingestion"] SRC[Docs / URLs / PDFs] --> ETL[ETL + clean] ETL --> CHUNK[Chunking] CHUNK --> EMB[Embeddings] EMB --> VS[(Vector store)] CHUNK --> KG[Entity/Relation extract] --> GRAPH[(Knowledge Graph)] GOWORKER[Go ingestion worker] -. high-throughput fetch .-> ETL end subgraph Retrieve["atlas.retrieval"] Q[Query] --> HS[Hybrid search: vector + BM25] VS --> HS GRAPH --> GR[Graph retrieval] HS --> RRF[Reciprocal-rank fusion] GR --> RRF RRF --> RERANK[Cross-encoder rerank] end subgraph Agent["atlas.agent (LangGraph)"] RERANK --> PLAN[Plan / route] PLAN --> ROUTER{Model router} ROUTER -->|synthesis| CLAUDE[Claude] ROUTER -->|cheap triage| GPT[OpenAI] ROUTER -->|local/private| OLLAMA[Ollama / vLLM] PLAN --> MCP[[MCP tool server]] end subgraph Safety["atlas.guardrails"] GIN[Injection + PII in] --> Agent Agent --> GOUT[PII + content filter out] end U[User] --> API[FastAPI + Next.js UI] API --> GIN GOUT --> API Agent --> OBS[(Langfuse traces)] Agent --> CACHE[(Redis semantic cache)] Agent --> EVAL[atlas.evals: LLM-judge + A/B] ``` See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the request lifecycle and design deep-dive. --- ## Features **Retrieval** - Hybrid search (dense vector + BM25 keyword) fused with reciprocal-rank fusion - A knowledge-graph layer: entities/relations extracted at ingest feed a multi-hop graph retriever whose hits join the RRF fusion as a third, `graph_weight`-ed ranking - Cross-encoder reranking of the fused candidates - Pluggable vector stores behind one `VectorStore` interface — in-memory (offline default), Postgres/pgvector, and Qdrant - Configurable, token-aware chunking with markdown/heading awareness **Agent** - A LangGraph state machine: `guard_input → retrieve → plan → act → synthesize → guard_output` - Function/tool calling with Pydantic-derived JSON schemas; a safe AST calculator - Model Context Protocol (MCP) — Atlas is both an MCP **server** (exposes its KB) and an MCP **client** (consumes external tools) - Cost-aware multi-model routing across Claude, OpenAI, Ollama, vLLM, and Bedrock - Token-budgeted context packing with near-duplicate dedupe, and grounded answers that cite every claim back to a source chunk - **Faithful by design** — answers are grounded strictly in the retrieved context; when nothing relevant is found, a relevance gate makes the agent refuse honestly ("I don't have anything relevant in the current sources") and suppress citations, rather than hallucinating an answer from off-topic chunks **Safety & governance** - Guardrail pipeline on both input and output: prompt-injection detection, PII redaction (Presidio with a regex fallback), and content filtering - A red-team / jailbreak-resistance suite ([data/redteam](data/redteam)) **Evaluation & LLMOps** - Eval harness with RAG metrics (faithfulness, answer relevancy, context recall) via an LLM-as-judge that degrades to deterministic heuristics offline - A/B testing across agent variants - Every chat turn runs through one seam (`deps.run_chat`): semantic-cache lookup (Redis, in-memory offline) → agent → cost accounting — so `GET /metrics` reports live hit rates and spend, and repeated questions come back in milliseconds without touching a model - Langfuse tracing on the same seam, plus prompt caching in the Claude provider **Interfaces & infra** - FastAPI backend (REST + GraphQL + SSE streaming) and a Next.js/TypeScript chat UI - A concurrent Go ingestion worker for high-throughput URL fetching - Docker Compose for local dev; Kubernetes manifests and AWS/GCP/Azure deploy notes for production; GitHub Actions CI - An optional LoRA/PEFT fine-tuning module (Hugging Face Transformers + PEFT) --- ## Quickstart ### Offline in under a minute (no API keys, no services) ```bash python3 -m venv .venv && source .venv/bin/activate # Python 3.11+ pip install pydantic pydantic-settings fastapi "uvicorn[standard]" httpx pytest pytest-asyncio rich # 20-second guided tour of the whole platform (ingest → retrieve → guardrails → cache → eval) python -m atlas.cli demo # Ask the agent a question (auto-loads the bundled sample knowledge base) python -m atlas.cli chat -m "How long is the TR-1 warranty?" # Run the test suite (offline) pytest -q # Measure RAG quality — and export a shareable Markdown report python -m atlas.cli eval --suite rag_golden --report report.md python -m atlas.cli redteam # Serve the API + interactive docs (auto-loads the sample corpus in dev) uvicorn atlas.api.main:app --port 8000 # -> http://localhost:8000/docs ``` In development the API loads the bundled sample corpus on startup, so `POST /chat` returns grounded, cited answers immediately — no manual ingest step. `atlas demo` walks through, in one command: sample-corpus ingestion, a grounded answer with citations, the three hybrid-retrieval signals side by side, a prompt-injection attack getting blocked, a semantic-cache MISS→HIT speedup, and an eval scorecard. The one-shot `chat` output is the screenshot at the top of this README; `atlas eval --report` produces a report like [docs/sample-eval-report.md](docs/sample-eval-report.md). Ask the same question twice and the second answer is served by the semantic cache — check `GET /metrics` for live hit rates, chunk counts, and spend. ### Full stack (hosted models + real datastores) ```bash cp .env.example .env # add your API keys / DSNs make up # Postgres+pgvector, Redis, Qdrant, Langfuse, Ollama make install # full dependency set make ingest # ETL -> chunk -> embed -> index -> graph make api # http://localhost:8000/docs ``` Run `make help` to see all targets. ## Configuration All configuration is environment-driven (12-factor); see [.env.example](.env.example) for the full surface. Nothing is required to run offline. Set any of the following to light up the corresponding integration: | Variable | Enables | |---|---| | `ATLAS_PROVIDERS__ANTHROPIC_API_KEY` | Routing to Claude | | `ATLAS_PROVIDERS__OPENAI_API_KEY` | Routing to OpenAI | | `ATLAS_DB__DSN` | Postgres/pgvector store | | `ATLAS_REDIS__URL` | Redis semantic cache | | `ATLAS_OBSERVABILITY__LANGFUSE_*` | Langfuse tracing | ## Repo layout ``` atlas/ core/ shared Pydantic models, Protocol interfaces, config, logging ingestion/ ETL, chunking, embeddings, knowledge-graph extraction retrieval/ hybrid search, RRF, reranking, graph retrieval, vector stores providers/ Claude / OpenAI / Ollama / vLLM / Bedrock adapters agent/ LangGraph agent, model router, tools, prompts, context builder mcp/ MCP tool server + client guardrails/ prompt-injection, PII redaction, content filter pipeline evals/ eval harness, RAG metrics, A/B testing, red-team observability/ Langfuse tracing, Redis semantic cache, cost/latency tracking finetune/ LoRA/PEFT trainer for a reranker/intent classifier api/ FastAPI app (REST + GraphQL + streaming) frontend/ Next.js + TypeScript chat UI web/worker/ Go concurrent ingestion worker infra/ Docker, Kubernetes, SQL, cloud (AWS/GCP/Azure) deploy configs tests/ pytest suite docs/ architecture and responsible-AI docs ``` ## Development ```bash pip install -e ".[dev]" make test # pytest make lint # ruff make typecheck # mypy ``` Contributions welcome — see [CONTRIBUTING.md](CONTRIBUTING.md). Security policy in [SECURITY.md](SECURITY.md). ## License [MIT](LICENSE).