multimodel-rag

# Multimodel Hybrid RAG Platform > A dual-mode AI assistant — a full-featured **online web application** with agentic RAG capabilities, and a **downloadable offline desktop app** for fully local, internet-free usage. --- ## 🧠 Overview This platform combines the power of cloud-based agentic AI with the privacy and reliability of a fully offline desktop experience. Users can interact with an online ChatGPT-like interface with web search, multi-agent orchestration, and multimodal document understanding — or download a self-contained desktop bundle to run everything locally on their own machine. --- ## 🏗 Architecture ``` ┌──────────────────────────────────────────────────────────────┐ │ ONLINE (Web App) │ │ │ │ Next.js Frontend ──► FastAPI Backend │ │ │ │ │ LangGraph │ │ ┌────┴────┐ │ │ CRAG Agent Web Search Agent (Tavily) │ │ │ │ │ Qdrant (Vector DB) MongoDB (Chats/Users) │ │ Supabase Auth MCP Servers │ │ Whisper (Audio) LangChain (File Processing) │ └──────────────────────────────────────────────────────────────┘ │ [ Want Offline? ] Download Bundle │ ┌──────────────────────────────────────────────────────────────┐ │ OFFLINE (Desktop App) │ │ │ │ Electron.js ──► FastAPI (local) ──► Ollama (LLM) │ │ │ │ │ LangGraph │ │ Self-RAG Pipeline │ │ SQLite (local DB) │ │ Whisper (Audio) LangChain (File Processing) │ └──────────────────────────────────────────────────────────────┘ ``` --- ## ✨ Features ### 🌐 Online Mode - **Agentic RAG** — Corrective RAG (CRAG) with LangGraph state machine orchestration - **Web Search Agent** — Real-time retrieval via Tavily integrated as a LangGraph node - **MCP Servers** — Extensible tool use via Model Context Protocol - **Multimodal Input** — PDF, DOCX, images, and audio transcription via Whisper - **Streaming Responses** — Real-time output via FastAPI SSE - **User Authentication** — Supabase JWT auth with user-scoped data - **Persistent Chat History** — Stored per-user in MongoDB - **Vector Search** — Qdrant for fast, scalable semantic retrieval ### 🖥 Offline Mode (Desktop) - **One-click Install** — Downloadable bundle with automated dependency setup - **Fully Local** — No internet required after installation - **Self-RAG Pipeline** — Self-correcting RAG without external verification - **Local LLM** — Ollama-powered inference (runs on CPU or GPU) - **Local Storage** — SQLite database, all data stays on device - **Same Multimodal Support** — PDF, DOCX, images, audio via Whisper --- ## 🛠 Tech Stack ### Online | Component | Technology | |---|---| | **Frontend** | Next.js (App Router) | | **Backend** | FastAPI | | **Orchestration** | LangGraph + LangChain | | **RAG Strategy** | Corrective RAG (CRAG) | | **Vector DB** | Qdrant | | **Chat / User DB** | MongoDB | | **Auth** | Supabase | | **Web Search** | Tavily | | **Audio** | OpenAI Whisper | | **Tool Protocol** | MCP Servers | ### Offline | Component | Technology | |---|---| | **Desktop Shell** | Electron.js | | **Backend** | FastAPI (local) | | **Orchestration** | LangGraph + LangChain | | **RAG Strategy** | Self-RAG | | **Local LLM** | Ollama | | **Local DB** | SQLite | | **Audio** | OpenAI Whisper | --- ## 🚧 Status > **This project is currently in active development.** | Module | Status | |---|---| | Online — Auth + Chat API | 🔄 In Progress | | Online — CRAG Pipeline | 🔄 In Progress | | Online — Web Search Agent | 🔄 In Progress | | Online — MCP Servers | 📅 Planned | | Online — Frontend (Next.js) | 📅 Planned | | Offline — Electron Bundle | 📅 Planned | | Offline — Self-RAG Pipeline | ✅ Foundation Ready | --- ## 📁 Current Codebase (Offline Foundation) The current codebase is the **offline RAG foundation** that the full platform is being built on top of: ``` src/ ├── config.py # Configuration with Pydantic ├── processors/ # Document processing (PDF, DOCX, images, audio) ├── embeddings/ # Text (Ollama) + Image (CLIP) embeddings ├── storage/ # Vector store ├── retrieval/ # Cross-modal search + citations ├── graph/ # LangGraph RAG workflow └── api/ # FastAPI REST API main.py # API server entry point requirements.txt # Dependencies ``` --- ## 📄 License Licensed under the MIT License — see `LICENSE` file. --- *Built with LangChain · LangGraph · FastAPI · Ollama · Electron.js · Qdrant · MongoDB · Supabase*