MCP_building

# Blink — Personal AI Assistant Blink is a Telegram-based personal AI assistant powered by a local LLM (LM Studio), an MCP file server, and a vector memory system. It routes messages intelligently, executes filesystem tools, and remembers context across conversations. --- ## Architecture ``` Telegram User ↓ Telegram Bot (aiogram) ↓ Orchestrator — coordinates all agents ↓ RouterAgent — decides: task or chat? ↓ ↓ TaskAgent ChatAgent (file tools via MCP) (conversation with memory) ↓ tool_executor → MCP Server (FastMCP) (list_files, read_file) ``` --- ## Features - **Intelligent routing** — LLM decides whether to use a tool or chat normally - **Filesystem tools** — list and read files via MCP protocol - **Vector memory** — remembers conversations across sessions using ChromaDB - **Semantic search** — finds relevant past context by meaning, not just recency - **File delivery** — sends files as Telegram document attachments - **Local LLM** — runs fully on-device via LM Studio, no OpenAI API needed --- ## Project Structure ``` blink/ ├── app/ │ ├── agents/ │ │ ├── router_agent.py # Routes to task or chat │ │ ├── task_agent.py # Handles tool calls │ │ └── chat_agent.py # Handles conversation │ ├── core/ │ │ └── orchestrator.py # Connects all agents │ ├── llm/ │ │ └── client.py # LM Studio client │ ├── memory/ │ │ └── store.py # ChromaDB vector memory │ ├── services/ │ │ └── tool_executor.py # MCP JSON-RPC client │ └── bot/ │ └── handlers.py # Telegram message handler ├── mcp_server.py # FastMCP server ├── main.py # Entry point └── requirements.txt ``` --- ## Tech Stack | Layer | Technology | |---|---| | Bot framework | aiogram 3 | | LLM | LM Studio (local, OpenAI-compatible API) | | Tool protocol | MCP via FastMCP (streamable-http) | | Vector memory | ChromaDB (persistent, local) | | Language | Python 3.11+ | --- ## Setup **1. Clone the repo** ```bash git clone https://github.com/yourusername/blink.git cd blink ``` **2. Create a virtual environment** ```bash python -m venv .venv source .venv/bin/activate ``` **3. Install dependencies** ```bash pip install -r requirements.txt ``` **4. Set environment variables** ```bash TELEGRAM_BOT_TOKEN=your_token_here LM_STUDIO_BASE_URL=http://localhost:1234/v1 LM_STUDIO_API_KEY=lm-studio ``` **5. Start the MCP server** ```bash python mcp_server.py ``` **6. Start the bot** ```bash python main.py ``` --- ## How It Works ### Routing Every message goes through the `RouterAgent`, which asks the LLM to classify intent: - `task` → user wants to interact with the filesystem - `chat` → general conversation ### Tool Execution `TaskAgent` builds a prompt telling the LLM what tools are available. The LLM responds with JSON specifying which tool to call and with what arguments. The request is sent to the MCP server using the JSON-RPC 2.0 protocol with session handshaking. ### Memory Every message is stored in ChromaDB as a vector embedding. On each request, Blink retrieves: - **5 semantically similar** past messages (by meaning) - **4 most recent** messages (for conversational flow) These are merged, deduplicated, and passed as context to the LLM. ### MCP Protocol Flow ``` 1. POST /mcp — initialize (get session ID from response header) 2. POST /mcp — tools/call with Mcp-Session-Id header 3. Parse SSE response (data: {...}) ``` --- ## Bot Commands | Command | Action | |---|---| | Any message | Routed to task or chat automatically | | `/reset` | Clears your conversation memory | --- ## Example Interactions ``` You: get me the conference_paper.docx Blink: [sends file as attachment] You: list my files Blink: Found 12 files in Downloads: resume.pdf, notes.txt, ... You: what did we discuss about the drone project? Blink: [searches vector memory and summarizes past context] ``` --- ## Key Design Decisions **Why local LLM?** Full privacy and Rate limit — While using chatGPT, groq it hit limit so I used LM studio. **Why MCP?** Standard protocol for tool use. Swapping tools or adding new ones requires no changes to the agent layer. **Why ChromaDB?** Semantic search — Blink can find relevant past conversations by meaning, not just recency. Asking "what is my favourite video game?" returns contextually relevant messages even without exact keyword matches. **Why two memory queries per request?** Recent history maintains conversational flow. Semantic search surfaces relevant long-term context. Both together give the LLM the right information.