tech-radar-agent

# AI/Agent Tech Radar **English** · [English](README.en.md) A local knowledge assistant for AI Agent technology research: batch collecting arXiv papers, using hybrid retrieval and reranking to find evidence, and then generating cited Chinese answers through a fixed RAG pipeline or a bounded ReAct Agent. This repository is also a progressive Agent engineering practice project, focusing on building a complete and runnable chain of data ingestion, retrieval, tool calling, failure degradation, citation constraints, and execution observability, rather than simply stacking frameworks.  ## Current Capabilities - Manual batch scraping of arXiv, saving traceable JSONL snapshots, and idempotent import into SQLite. - Using multilingual E5, BM25, RRF, and Cross-encoder for cross-lingual hybrid retrieval and reranking. - Supporting a fixed Agentic RAG pipeline: query rewriting, evidence sufficiency judgment, limited secondary retrieval, and refusal to answer due to insufficient information. - Supporting bounded tool-calling ReAct: the model directly selects local paper retrieval, optional web search, or outputs final text, with a maximum of 5 tool calls. - Tavily web search is only used to clarify new terms and form more accurate paper queries; web content does not enter the answer evidence and cannot be cited. - Tavily authentication failure will disable web tools within the current request; errors are returned as tool observations, allowing the model to choose other tools, clarify, or end. Temporary network or service errors are retried once. - Final research answers undergo deterministic citation verification: must cite real papers from this round, unknown arXiv IDs, fictional versions, and answers without citations are safely refused; zero retrieval results also return clear explanations of insufficient information. - React chat interface displays answers in chunks through SSE after citation verification; verified inline arXiv IDs can directly open paper pages. Traces default to merging model and tool start/completion events, with raw parameters, usage, and tool outputs retained in folded technical details. - Complete original rounds are persistently saved in SQLite, with automatic batch compression of earliest rounds when the token threshold is reached, no longer using a fixed six-round window. - Providing FastAPI HTTP API and read-only Streamable HTTP MCP Server with Bearer Token. ## Project Thoughts: Clarifying the Agent Loop Boundary This project does not introduce LangGraph or complex Agent frameworks first, but instead uses a small handwritten harness to make the core loop transparent: ```text User message → Reorganize system, compress summary, uncompressed history, active evidence, and current question → LLM chooses tools or outputs final text → Execute a tool → Append assistant tool calls and tool observations to the message list → Call LLM again with the complete message list → Until the model no longer calls tools ``` The goal of this design is not to reduce code volume but to make each responsibility clear: - LLM decides what to do next but cannot bypass tool budgets and evidence rules. - Harness is responsible for executing tools, validating calls, maintaining message state, and terminating the loop. - Retrieval tools are internally responsible for stable recall and reranking, without exposing storage details to the Agent. - Paper tool results are the only technical fact source; web results, historical answers, and session summaries cannot become citation evidence. - Traces record model and tool boundaries, helping explain why the Agent continues, stops, fails, or degrades. ## Tool-Calling Harness The core loop is located in `rag/research_agent.py`, conceptually close to the following pseudocode: ```python messages = build_initial_messages(question, context, active_evidence) while True: tools = available_tools(call_budget, web_search_state) assistant = llm(messages=messages, tools=tools) messages.append(assistant) if not assistant.tool_calls: return grounded_final_answer(assistant.content) call = validate_first_tool_calls(assistant.tool_calls, tools) observation = execute_tool(call) messages.append(observation) ``` The tools currently provided to the model are: - `search_papers(query, top_k)`: calls local paper retrieval. Internally completes E5, BM25, RRF, and Cross-encoder; the Agent only needs to decide when to retrieve and what query to use. - `web_search(query, max_results)`: only used to explain ambiguous, new, or productized terms and help form subsequent paper queries; web content is untrustworthy, cannot be cited, and will not directly enter the final evidence set. ### Specific Engineering Handling in Harness 1. **Execute one tool per round.** Even if the vendor returns parallel tool calls, only the first one is executed; the remaining calls receive structured error observations without producing hidden side effects. 2. **The tool menu is dynamic.** After Tavily is not configured or an unrecoverable authentication error occurs, `web_search` is removed from the tool list for subsequent rounds, and the model cannot continue to call tools that have failed. 3. **The loop has a deterministic budget.** Each request executes at most 5 tools, with web search limited to 2 times. After the budget is exhausted, a tool-free model call is made to answer, clarify, or refuse with existing evidence. 4. **Tool failure is also an Observation.** Timeouts, rate limiting, authentication failures, and parameter errors are converted into tool messages containing `error_type`, `retryable`, and `tool_available`, allowing the model to change strategy based on the real state. 5. **Evidence boundaries are maintained by code.** The final research answer must contain at least one bracketed citation from the evidence of this round; unknown arXiv IDs are refused. Paper cards, persistent IDs, active evidence, and inline hyperlinks only use verified actual citations; web fragments, model memory, old assistant answers, and compressed summaries are not paper evidence. 6. **Agent failure does not equal request failure.** When an unrecoverable error occurs in the model or harness, it degrades to a reliable fixed Agentic RAG pipeline and retains the degradation trace. 7. **Observability stops at meaningful boundaries.** ReAct Traces retain model calls, tool lifecycles, errors, usage, and final output, but React merges start/completion events into product-level steps by default; complete raw events can still be expanded in technical details. Internal details like E5, BM25, RRF are kept in retrieval implementation, avoiding splitting a logical tool into pseudo-tools that the model cannot control. ## Session Context: Complete Original Text and Working Memory Separation Sessions no longer use a fixed recent 6-round window or set a 100-round storage limit: ```text SQLite complete event log: all user utterances, assistant answers, paper IDs, permanently retained ↓ Working context: structured summary + all uncompressed original rounds ↓ Batch compress earliest complete rounds when token threshold is reached ``` Compression is triggered by default when the estimated context reaches 12,000 tokens, aiming to reduce to about 8,000 tokens. Long histories are divided into multiple bounded batches, gradually merged in memory, and only updated atomically in SQLite after all batches succeed. Summaries only save user goals, explicit requirements, decisions, important context, and unsolved problems. Original paper records do not enter the compressed model; summaries also cannot serve as technical facts or citation sources. Compression failures do not overwrite original messages or silently return truncated windows. ## System Flow ```mermaid flowchart LR subgraph Offline[Offline Knowledge Base Construction] A[arXiv API] --> B[JSONL Snapshot] B --> C[Normalization and Deduplication] C --> D[(SQLite)] D --> E[Multilingual E5] E --> F[(ChromaDB)] end subgraph Online[Online Agent Loop] U[User Question] --> CTX[Load or Compress Session Context] CTX --> P{Pipeline / ReAct} P -->|ReAct| L[LLM Chooses Tools or Final Answer] L -->|search_papers| H[BM25 + Vector] H --> R[RRF + Cross-encoder] R --> O[Tool Observation] O --> L L -->|web_search| W[Tavily Term Clarification] W --> O L -->|No Tool Calls| G[Candidate Chinese Answer] P -->|Reliable Pipeline| J[Retrieval Judgment and Up to Secondary Retrieval] J --> G G --> V{Deterministic Citation Verification} V -->|Passed| UI[React + Verified SSE] V -->|Failed or No Evidence| X[Safe Refusal] X --> UI end D --> H F --> H D --> MCP[Read-only MCP tools] F --> MCP ``` The core evidence boundary of the system is: final answers can only be based on local SQLite/ChromaDB arXiv papers. External web results, model existing knowledge, and execution traces cannot become citation sources. ## Tech Stack | Layer | Technology | |---|---| | Data Source | arXiv API | | Current State Storage | SQLite | | Vector Index | ChromaDB | | Retrieval | Multilingual E5 + BM25 + RRF | | Reranking | Cross-encoder | | Model Interface | OpenAI-compatible API | | Backend | FastAPI + Server-Sent Events | | Agent | Handwritten Bounded ReAct Loop | | Tool Protocol | MCP Streamable HTTP | | Frontend | React 19 + TypeScript + Vite | | Testing | pytest + Vitest | ## Quick Start The current development environment is based on Windows, PowerShell, and Python 3.12. ### 1. Install Dependencies ```powershell python -m venv .venv .\.venv\Scripts\Activate.ps1 python -m pip install -r requirements.txt cd frontend npm install cd .. ``` It may take some time to download model files when loading embedded models and Cross-encoders for the first time. ### 2. Configure and Run Create a `.env` file in the repository root directory that will not be tracked by Git. The project intentionally does not provide `.env.example`, so please configure the following variables as needed: - `LLM_API_KEY`: required, OpenAI-compatible model service key. - `LLM_BASE_URL`: required, model service API address. - `LLM_MODEL`: required, chat model name. - `TAVILY_API_KEY`: optional; if missing, ReAct's `web_search` tool is automatically disabled. - `CONVERSATION_CONTEXT_TOKEN_THRESHOLD`: optional; estimated token threshold that triggers batch compression of session context, default `12000`. - `CONVERSATION_CONTEXT_TARGET_TOKENS`: optional; target token number for uncompressed context after compression, default `8000`, must be less than the trigger threshold. - `MCP_AUTH_TOKEN`: required only when running MCP Server, at least 16 characters. - `MCP_HOST`, `MCP_PORT`, `MCP_ALLOWED_HOSTS`: optional MCP network settings. The frontend connects to `http://127.0.0.1:8000` by default. If you need to modify it, set `VITE_API_BASE_URL` in the ignored `frontend/.env.local`. ### 3. Prepare Local Paper Data List preset arXiv queries: ```powershell python -m ingestion.run_arxiv_ingestion --list-queries ``` Fetch a small batch and use the snapshot path output by the command for subsequent import: ```powershell python -m ingestion.run_arxiv_ingestion --query-name agent_core --max-results 3 python -m ingestion.import_snapshot data/raw/<snapshot>.jsonl python -m rag.indexer ``` Fetching is manual batch processing; online questioning will not call arXiv in real-time. Snapshots, SQLite databases, and ChromaDB indexes in `data/` are not submitted to Git by default. ### 4. Start Web Application After installation, you can run in the repository root directory: ```powershell .\start_services.ps1 ``` The script will start FastAPI and Vite and open `http://127.0.0.1:5173`. You can also start them separately: ```powershell python -m uvicorn api.main:app --reload ``` ```powershell cd frontend npm run dev ``` API documentation is located at `http://127.0.0.1:8000/docs`. ## HTTP API | Method | Path | Purpose | |---|---|---| | `GET` | `/health` | Process health check | | `GET` | `/knowledge-base/stats` | Return SQLite paper and vector counts | | `POST` | `/conversations` | Create a persistent session | | `GET` | `/conversations` | List sessions by recent update time | | `GET` | `/conversations/{id}` | Return complete text and paper citation history | | `DELETE` | `/conversations/{id}` | Delete session and its rounds | | `POST` | `/conversations/{id}/chat` | Return complete question and answer results and save to database | | `POST` | `/conversations/{id}/chat/stream` | Send status, tool events, verified text chunks, usage, and final results through SSE | Session-level chat interfaces support `pipeline` and `react` modes, with requests only containing questions, `top_k`, and mode. ReAct tool errors are returned as observations to the model first; only when the model or harness encounters an unrecoverable error, it degrades to a reliable fixed pipeline and marks `fallback_used` in the response. ## MCP Server The project also provides a standalone read-only MCP service: ```powershell python -m mcp_server.main ``` Current tools include: - `query_knowledge_base(query, top_k=3)` - `get_paper_by_arxiv_id(arxiv_id)` - `get_knowledge_base_stats()` Clients must send a Bearer Token when connecting to `/mcp`. Complete data boundaries, configuration, and deployment instructions are in [docs/mcp.md](docs/mcp.md). ## Verification Run backend tests in the repository root directory: ```powershell python -m pytest ``` Run frontend tests and production build in `frontend/`: ```powershell npm test npm run build ``` Tests cover data normalization, idempotent import, retrieval, deterministic citation verification, zero-evidence refusal, token threshold compression, original history retention, tool-calling harness, verified SSE sequence, trustworthy citation links, trace display merging, web search failure, and HTTP/MCP boundaries. ## Project Structure ```text api/ FastAPI routes, request contracts, and runtime lifecycle config/ Query, model, MCP, and web search configuration boundaries frontend/ React chat interface, trustworthy citation links, citation cards, and simplified traces ingestion/ arXiv scraping, normalization, snapshots, and SQLite import mcp_server/ Read-only Streamable HTTP MCP adapter rag/ Retrieval, reranking, fixed pipeline, ReAct Agent, answer generation, and citation verification tests/ Python tests that run offline by default docs/ ADR decision logs and MCP usage instructions first.md Complete scope, learning route, and phased acceptance records ``` FastAPI, MCP, and command-line entries reuse domain capabilities in `rag/`; network, storage, retrieval, and UI logic are kept separate. ## Current Limitations and Roadmap - The knowledge base currently only uses arXiv titles and abstracts, without reading full PDF content. - Data updates are triggered manually, without scheduled scraping or online incremental updates. - ReAct calls at most 5 tools and records token usage for each model call, without a request-level token limit. - Web search brings indirect prompt injection surface, currently limited by "not entering answer evidence"; complete Guardrails are yet to be implemented. - MCP's shared token is suitable for local development and controlled invitations; OAuth 2.1, user-limited current limiting, and HTTPS reverse proxy are required before public service. - Session compression uses vendor-agnostic conservative token estimation, not equivalent to the model's official tokenizer; threshold calibration is needed based on real usage distribution. - Compression only maintains user goals, constraints, decisions, and unsolved problems, without serving as paper fact evidence; original user messages, assistant answers, and paper data are still retained in trustworthy storage. - Deterministic verification ensures citation IDs come from this round's evidence but cannot prove each conclusion is fully implied by corresponding abstracts; finer-grained claim-level groundedness still requires subsequent evaluation or controlled Reflection. - Cross-session long-term memory, LangGraph, Human-in-the-Loop, or multi-Agent orchestration have not been introduced. The next phase plan is to calibrate compression thresholds with real long sessions and verify early user constraints can still be stably retained after multiple batch compressions. Complete milestones are in [first.md](first.md), and important architectural choices are in [docs/decision-log.md](docs/decision-log.md). ## Development Conventions The project uses Conventional Commits and requires important data sources, storage, models, frameworks, data contracts, deployment, and security decisions to be recorded in ADR. Please run backend tests, frontend tests, and frontend build before submitting; detailed collaboration constraints are in [AGENTS.md](AGENTS.md).