InsightGap-Agent

# Tool List > Gap Intelligence Workbench · Discover information gaps · Verify information gaps · Generate actionable insights · Precipitate long-term cognition       A full-stack Agent workbench built around "bridging information gaps". The system captures noteworthy signals from information streams, interprets them, and points out the information gaps. It then advances these gaps to in-depth research, document verification, outcome generation, and long-term memory and reusable skills. The project is based on LangChain's official `langchain-ai/open_deep_research` and complements it with Web App, LangGraph Runtime, MCP tool governance, RAG retrieval, Memory/GSSC context management, manual approval recovery, audit logs, and frontend visualization. Its mainline is not ordinary chat but making "discovering information gaps -> verifying information gaps -> forming judgments -> generating outcomes -> precipitating memory and skills -> reusing next time" a complete loop.  ## Capability Overview | Capability | How it helps bridge information gaps | Key Modules | |---|---|---| | Information Discovery | Sifts high-value signals from GitHub, arXiv, search sources, and manual seeds, labeling explicit relevance, proximal opportunities, and distant inspirations | FeedService, FeedCard, Source adapters | | Information Gap Explanation | Explains for each card "what's the gap", "why it's relevant to you", and "what to do next" | Feed scorer, card generator, Profile | | In-depth Research | Conducts systematic research on information gaps, generating evidence-based, risk-assessed, and actionable reports | LangChain's official Open Deep Research, `research_agent` | | Document Verification | Uploads user documents, uses RAG to retrieve evidence, and verifies the existence of information gaps | DocumentService, RagService, Qdrant | | Tool Action | Enters tool governance for actions that need execution, with low-risk direct execution and high-risk approval | MCP Registry, ToolRouter, ToolExecutor | | Outcome Precipitation | Saves research conclusions, product plans, and Markdown reports as persistent Artifacts | ArtifactService, Artifacts Page | | Long-term Memory | Saves user preferences, judgment bases, and historical fragments, making next discoveries more user-centric | MemoryService, GSSC, conversation summary | | Skill Reuse | Precipitates frequently occurring research/verification processes into Skill drafts, reducing repetitive labor | SkillService, Skills Page | ## Table of Contents - [Project Positioning](#project-positioning) - [Capability Overview](#capability-overview) - [In-depth Research Sources](#in-depth-research-sources) - [Core Capabilities](#core-capabilities) - [Architecture Overview](#architecture-overview) - [Core Business Loop](#core-business-loop) - [Functional Module Details](#functional-module-details) - [Technical Stack](#technical-stack) - [Local Launch](#local-launch) - [Environment Variables and Key Management](#environment-variables-and-key-management) - [Database and External Services](#database-and-external-services) - [Testing and Verification](#testing-and-verification) - [System Screenshot List](#system-screenshot-list) - [Pre-Submission Security Checklist](#pre-submission-security-checklist) ## Project Positioning The core of this project is not "letting Agent answer questions" but "letting Agent help users continuously discover and bridge information gaps". Information gaps here refer to high-value signals that users have not noticed, verified, or converted into actions. The system is designed around four questions: - **Discovery**: Which information is worth noting? Where does it come from? How is it related to the user's current domain? - **Interpretation**: What specific gap does this information have? Why might it affect the user's research, product, or decision? - **Verification**: Can it be further confirmed through in-depth research, RAG document evidence, or external search? - **Precipitation**: How can verified conclusions be saved as Artifacts, Memory, or Skills for future reuse? Under this mainline, the project also addresses issues encountered in Agent productization: - Tools cannot be directly exposed to LLMs without parameter verification, risk grading, approval, and auditing. - High-risk actions cannot be synchronously blocked in memory, waiting for user confirmation, and must be recoverable across requests and processes. - RAG cannot only perform vector similarity but must handle document structure, keywords, numbers, and parent context and assessable recall quality. - Long conversations cannot just stuff recent messages into prompts; they need summarization, segmentation, memory, and context selection. - The frontend cannot just display final answers but must show the operation trajectory, tool calls, approval status, Artifacts, Memory, and historical tasks. Thus, it is not just an information gap discovery and research workbench but also an Agent application base with runtime, tool governance, memory, and auditing capabilities. ## In-depth Research Sources The in-depth research capability of this project is integrated from LangChain's official open-source project [`langchain-ai/open_deep_research`](https://github.com/langchain-ai/open_deep_research). In this repository: - `src/open_deep_research/` retains the official Open Deep Research core research graph and configuration. - `src/web_app/agent/adapters/open_deep_research_adapter.py` integrates the official research capability into the Web App's Agent Runtime. - `research_agent` is responsible for calling this capability when users initiate in-depth research and integrating the results into the project's Artifact, Memory, Skill, and operation audit loop. That is, in-depth research itself is not a temporary prompt written from scratch but based on LangChain's official Open Deep Research workflow; the project focuses on complementing productized runtime, frontend workbench, tool governance, approval recovery, context management, and outcome precipitation. ## Core Capabilities ### 1. Recoverable LangGraph Agent Runtime The project splits the Agent into multiple nodes with clear responsibilities rather than letting a single large function complete all logic: - `planner`: Understands user intent and generates a route plan. - `dispatcher`: Schedules specific agent nodes based on the route plan. - `tool_agent`: Handles tool calls, approvals, and tool results. - `rag_agent`: Handles document retrieval and evidence answering. - `memory_agent`: Handles long-term memory writing and recall. - `research_agent`: Integrates LangChain's official Open Deep Research for in-depth research. - `artifact_agent`: Generates persistent outcomes. - `skill_agent`: Identifies reusable workflows. - `post_agent_gate`: Performs quality gate judgment after each agent, deciding to continue, retry, or degrade. - `evaluator`: Performs global consistency checks before final responses. - `final_response`: Aggregates structured results and generates user-readable answers. Engineering value: - Each node's input and output can be recorded for debugging. - Agent failures are not direct crashes but write state, events, and AgentResult. - High-risk tools can be paused with LangGraph `interrupt()`, then recovered via checkpoint. - `post_agent_gate` prevents erroneous results from polluting subsequent nodes.  ### 2. MCP Tool Governance: Registration, Verification, Grading, Approval, Auditing Tools are not directly exposed to LLMs but registered as Tool specs: ```text ToolSpec - name - description - input_schema - output_schema - permission_level - approval_required - enabled ``` Tool call flow in the project: ```text tool_agent -> ToolRouter standardizes tool names -> JSON Schema parameter verification -> ToolExecutor performs pre-execution verification -> Risk grading -> ToolCall auditing -> Low-risk execution / L3 approval / L4 blocking ``` Highlights: - `input_schema` is the tool parameter contract. - ToolRouter standardizes tool names and parameter verification. - ToolExecutor performs verification before execution or approval creation. - JSON Schema verification covers `required`, types, enumerations, ranges, array/object structures, `additionalProperties`, and common `format`. - L3 external writes enter manual approval. - L4 high-risk operations are blocked by default. - All tool calls are written to ToolCalls for auditing. The tool audit page displays tool call details, including tool name, input parameter summary, permission level, status, error messages, and execution time. ### 3. L3 Manual Approval Loop, Not Just Simple Confirmation Box Many tools only display a confirmation box on the frontend. This project's focus is on enabling the entire Agent execution chain to recover after approval. Approval process: ```text tool_agent judges L3 -> Creates or reuses ToolCalls / Approval -> LangGraph interrupt pauses -> PostgresSaver saves checkpoint -> Frontend displays approval_required -> User approve / reject -> Command(resume={action: approved}) -> Graph returns to interrupt point -> tool_agent executes execute_approved_tool_once -> Writes back tool results and continues final_response ``` Engineering value: - Approval is an asynchronous process involving human participation, not dependent on process memory waiting. - Services can recover after restart through checkpoints. - Actual side effects are placed after interrupt returns to avoid repeated execution. - ToolCalls / Approval can track "who approved what and whether it was executed". The approval page displays tools waiting for approval, including tool names, risk levels, parameter previews, security prompts, and approve/reject operations. ### 4. RAG: Structured Document Retrieval Chain, Not Simple Vector Library The project's RAG goal is not "chunking documents and embedding" but solving real document question-answering issues: - Chunks that are too small lose context. - Chunks that are too large have inaccurate recall. - Dense embedding struggles with numbers, field names, amounts, and table column names. - BM25 struggles with semantic rewriting. - No evaluation means optimization effectiveness is unknown. Current design: ```text Upload documents -> DocumentService saves files -> DocumentParser parses -> StructuredChunker generates Overview / Parent / Child -> Child writes to Qdrant -> Parent / Overview write to PostgreSQL -> Query Analyzer -> Dense + Sparse / BM25 retrieval -> RRF fusion -> Hits child -> Recalls parent context -> Evidence is passed to final_response ``` Highlights: - Child is for precise recall. - Parent is for final answer context supplementation. - Overview is for document summary and general question answering. - Dense handles semantic similarity, sparse/BM25 handles keywords, numbers, field names. - Qdrant Hybrid + RRF uses ranking fusion to reduce incomparable score issues. - Synthetic evaluation compares hit@k, keyword hit rate, latency, and fallback for different backends. The document upload page and RAG Q&A page display uploaded documents, parsing status, chunk numbers, retrieval status, answer citations, and chunk/source information. ### 5. Memory / GSSC: Context Isn't Everything, but Selection Long conversations and long-term memory challenges aren't "storing everything" but selecting what to include in prompts. There are several context sources in the project: - Recent conversation messages. - Running conversation summary. - Conversation segments. - Semantic / episodic / working memory. - RAG evidence. - Feed card/page context. - Skill match results. GSSC's role is organizing these contexts into controllable prompt sections, avoiding irrelevant information being fed to the model. Highlights: - Supports long-term memory writing and recall. - Supports historical conversation segment recall. - Supports memory and RAG evidence entering context simultaneously. - Supports configuring recall quantity and token budget limitations. - Allows debugging `gssc_debug` to observe context selection reasons.  The context debugging page can further display how recent messages, summary, memory, and RAG evidence are combined. ### 6. Information Gap Radar -> In-depth Research -> Outcome Precipitation Product Loop The project not only answers user questions but also actively organizes external information into "information gap cards". Each FeedCard is not just a news summary but includes: - What the information is. - Why it's relevant to the user's current domain. - What information gap it reveals. - What opportunities, risks, or next actions it may bring. - Users can choose to bring it into conversation, initiate in-depth research, view details, save, or ignore. The overall process is: ```text External information sources / manual seeds -> Feed generates information gap cards -> Users filter noteworthy signals -> Agent initiates in-depth research / RAG verification / tool calls -> Generates research reports, product plans, or Markdown Artifacts -> Writes to Memory, records user preferences and judgment bases -> Forms Skill drafts, precipitates reusable research processes -> Next information gap discovery is closer to the user ``` Highlights: - Feed can access arXiv, GitHub, DuckDuckGo, Tavily, SerpAPI, and manual seeds. - Each card provides scoring, relevance types, evidence sources, information gap explanations, and suggested actions. - Users can directly enter in-depth research from cards, turning fuzzy signals into structured reports. - Research results can generate Artifacts, avoiding conclusions staying in a single conversation. - Memory saves user preferences, historical judgments, and task conclusions, making subsequent recommendations closer to the user. - Skill records reusable workflows, allowing similar information gap verification processes to be reused. > Screenshot suggestions 9: Feed page. >     ### 7. Frontend Isn't Just a Shell but an Agent Operation Workbench Frontend pages include: - Home / Agent Chat - Agent Run Detail - Approvals - MCP Tool Calls - Research Runs - Research Run Detail - Artifacts - Memory - Feed - Feed Card Detail - Skills - Profile - Settings These pages let users see not just final answers but also why the Agent did it, what tools were called, where approvals were needed, and which results were persisted. > Screenshot suggestions 11: Complete navigation sidebar. >  ## Architecture Overview > Screenshot suggestions 12: System architecture diagram. > > Supplemental file: `images/architecture.jpg` > > Screenshot content: Relationships between frontend, FastAPI, LangGraph Runtime, MCP, RAG, Memory, PostgreSQL, Qdrant, Checkpoint, and external LLM services. ```mermaid flowchart TD U["User / Browser"] --> FE["React Frontend"] FE --> API["FastAPI Web App"] API --> RT["LangGraph Agent Runtime"] RT --> PLAN["Planner / Supervisor"] PLAN --> DISPATCH["Dispatcher"] DISPATCH --> TOOL["tool_agent"] DISPATCH --> RAG["rag_agent"] DISPATCH --> MEM["memory_agent"] DISPATCH --> RESEARCH["research_agent"] DISPATCH --> ART["artifact_agent"] DISPATCH --> SKILL["skill_agent"] TOOL --> MCP["MCP Registry + ToolExecutor"] MCP --> APPROVAL["ToolCalls / Approval"] MCP --> PROVIDER["Local / Email / Web / File Providers"] RAG --> QDRANT["Qdrant"] RAG --> PG["PostgreSQL"] MEM --> QDRANT MEM --> PG RT --> CHECKPOINT["LangGraph Checkpoint"] CHECKPOINT --> PG RT --> FINAL["Evaluator + Final Response"] FINAL --> API ``` ## Core Business Loop ```mermaid flowchart LR SRC["External Signals<br/>GitHub / arXiv / Search / Manual Seed"] --> FEED["FeedCards<br/>Information Gap Cards"] FEED --> GAP["Gap Explanation<br/>Where's the Gap / Why Relevant / Next Steps"] GAP --> RESEARCH["Deep Research<br/>Systematic Research"] GAP --> RAG["RAG Verification<br/>User Documentation Verification"] GAP --> TOOL["MCP Tools<br/>Controlled Actions"] RESEARCH --> ART["Artifact<br/>Reports / Plans / Markdown"] RAG --> ART TOOL --> AUDIT["ToolCalls / Approval Audit"] ART --> MEM["Memory<br/>Preferences / Judgments / Conclusions"] ART --> SKILL["Skill Draft<br/>Reusable Processes"] MEM --> NEXT["Next Gap Discovery<br/>More User-Centric Gap Discovery"] SKILL --> NEXT ``` ## Functional Module Details ### Information Gap Feed Key Files: ```text src/web_app/services/feed_service.py src/web_app/feed/sources/ src/web_app/feed/scorer.py src/web_app/feed/card_generator.py src/web_app/db/repositories/feed_repository.py src/web_app/api/v1/feed.py ``` Feed module converts external information sources into actionable information gap cards: - Retrieves candidate information from arXiv, GitHub, DuckDuckGo, Tavily, SerpAPI, and manual seeds. - Deduplicates, normalizes, and scores candidate information. - Judges relevance based on user profiles, historical memory, and current focus directions. - Categorizes information into explicit relevance, adjacent opportunities, and distant inspiration. - Generates explanation fields such as "information gap," "why relevant to you," and "suggested actions." - Supports direct entry into conversation, deep research, details page, or saving/ignoring from FeedCard. This part is the entry point for the project "bridging information gaps": it's not just a simple news stream but transforms external signals into verifiable, researchable, and precipitable task starters. ### Agent Runtime Key Files: ```text src/web_app/agent/runtime/graph_builder.py src/web_app/agent/runtime/dispatch.py src/web_app/agent/runtime/node_groups/ src/web_app/agent/runtime/graph_registry.py src/web_app/agent/runtime/graph_manifest.py src/web_app/agent/runtime/recovery.py ``` Implementation Focus: - StateGraph handles recoverable processes. - Runtime state serves as a data bus between nodes. - Each node writes its structured results. - Post-agent gate performs local quality control. - Evaluator performs global consistency checks. - Service layer handles SSE events and run state persistence. ### MCP Tool Governance Key Files: ```text src/web_app/mcp/registry.py src/web_app/mcp/tool_router.py src/web_app/mcp/tool_executor.py src/web_app/mcp/local_provider.py src/web_app/services/mcp_service.py src/web_app/api/v1/mcp.py ``` Tool Risk Levels: | Level | Type | Example | Strategy | |---|---|---|---| | L0 | Pure computation / internal read | calculation, time, formatting | direct execution | | L1 | Read public or local information | web search, read file | execute and record | | L2 | Local write / draft | create artifact, draft | restricted execution | | L3 | External write | send email, submit form | human approval | | L4 | High-risk irreversible | delete, dangerous command, payment | default block | ### RAG Key Files: ```text src/web_app/services/document_service.py src/web_app/services/rag_service.py src/web_app/rag/document_parser.py src/web_app/rag/structured_chunker.py src/web_app/rag/vector_store.py src/web_app/rag/retriever.py src/web_app/rag/bm25.py ``` Chunk Division: | Chunk | Role | Whether to Qdrant | Whether for Answer Context | |---|---|---:|---:| | Overview | document summary | no | optional | | Parent | complete paragraph / chapter context | no | yes | | Child | precise retrieval unit | yes | check parent after hit | ### Memory / GSSC Key Files: ```text src/web_app/services/memory_service.py src/web_app/context/builder.py src/web_app/context/packets.py src/web_app/context/compression.py src/web_app/services/conversation_summary_service.py ``` Context Sources: - recent messages - running summary - historical segments - user memory - RAG evidence - feed card / page context - skill match ### Frontend Key Directories: ```text frontend/src/pages/ frontend/src/components/ frontend/src/api/ ``` Page Capabilities: - Run Agent. - View approvals. - View tool call audit. - View research tasks. - Manage artifacts. - View memory. - Browse feed. - Manage skills. ## Tech Stack ### Backend - Python 3.10+ - FastAPI - SQLAlchemy / Alembic - PostgreSQL - LangGraph - Qdrant - Redis (optional) - Neo4j (optional) - DashScope / OpenAI-compatible LLM API ### Frontend - React - TypeScript - Vite - React Router - React Markdown ## Local Startup Taking Windows + PowerShell as an example. macOS / Linux only need to replace the virtual environment activation command with `source .venv/bin/activate`. ### 1. Prepare Basic Dependencies Need to install: - Python 3.10 or higher, recommended Python 3.11. - Node.js 20.19+ or 22.12+. - PostgreSQL 14+. - Git. - `uv` for Python dependency installation. Install `uv`: ```powershell pip install uv ``` Check version: ```powershell python --version node --version npm --version uv --version git --version ``` ### 2. Clone Project ```powershell git clone https://github.com/<your-github-name>/<your-repo-name>.git cd open_deep_research ``` If the project is already on the local machine, directly enter the project directory: ```powershell cd D:\pythonproject\open_deep_research ``` ### 3. Create and Activate Python Virtual Environment ```powershell uv venv .venv\Scripts\activate ``` Install backend dependencies: ```powershell uv sync ``` If `uv sync` is not suitable for the current environment, you can also use: ```powershell pip install -e . ``` ### 4. Configure Environment Variables Copy template: ```powershell copy .env.example .env ``` Edit `.env`. The minimum runnable configuration usually requires: ```env APP_ENV=local SECRET_KEY=<generate a strong random string> POSTGRES_HOST=127.0.0.1 POSTGRES_PORT=5432 POSTGRES_USER=postgres POSTGRES_PASSWORD=<your local database password> POSTGRES_DATABASE=agent_os AGENT_LLM_ENABLED=true AGENT_LLM_PROVIDER=aliyun DASHSCOPE_API_KEY=<your DashScope Key> ALIYUN_BAILIAN_API_KEY=<your Aliyun Baizhan Key, can be the same as DashScope or configured as needed> ALIYUN_BAILIAN_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1 AGENT_CHECKPOINTER_BACKEND=postgres AGENT_CHECKPOINTER_REQUIRE_DURABLE=true EMAIL_PROVIDER=mock LOCAL_TOOLS_ENABLED=true LOCAL_TOOLS_WORKSPACE_DIR=./agent_workspace LOCAL_TOOLS_ALLOW_DELETE=false ``` Optional services can be closed or left blank: ```env ENABLE_NEO4J=false QDRANT_URL= QDRANT_API_KEY= TAVILY_API_KEY= SERPAPI_API_KEY= GITHUB_TOKEN= SMTP_PASSWORD= ``` Description: - `.env` is only saved locally and should not be submitted to Git. - When `EMAIL_PROVIDER=mock`, no real emails will be sent. - If there is no Qdrant, RAG can use the fallback capability first; complete vector retrieval requires Qdrant configuration. - Do not use the default `SECRET_KEY` in the production environment; it must be replaced with a strong random value. ### 5. Initialize PostgreSQL Database Enter PostgreSQL and create a database: ```sql CREATE DATABASE agent_os; ``` Confirm that the database connection information in `.env` is consistent with the local PostgreSQL: ```env POSTGRES_HOST=127.0.0.1 POSTGRES_PORT=5432 POSTGRES_USER=postgres POSTGRES_PASSWORD=<your local database password> POSTGRES_DATABASE=agent_os ``` Execute Alembic migration: ```powershell alembic upgrade head ``` If the migration fails, prioritize checking: - Is the PostgreSQL service started? - Does the database exist? - Is `POSTGRES_PASSWORD` correct? - Is the current virtual environment activated? ### 6. Start Backend Service Execute in the project root directory: ```powershell python run_server.py ``` Default backend address: ```text http://127.0.0.1:8000 ``` API documentation: ```text http://127.0.0.1:8000/docs ``` Health check: ```text http://127.0.0.1:8000/api/v1/health ``` If the port is occupied, you can modify `port=8000` in `run_server.py` or specify the port with uvicorn: ```powershell uvicorn src.web_app.main:app --host 127.0.0.1 --port 8001 ``` ### 7. Start Frontend Service Open a new PowerShell window: ```powershell cd D:\pythonproject\open_deep_research\frontend npm install npm run dev ``` Default frontend address: ```text http://127.0.0.1:5173 ``` The frontend will make API calls to the backend. The default CORS configuration is in `.env`: ```env CORS_ORIGINS=http://localhost:5173,http://127.0.0.1:5173 ``` ### 8. First Run Check After startup, it is recommended to check in order: 1. Open `http://127.0.0.1:8000/docs` and confirm that the FastAPI documentation is accessible. 2. Open `http://127.0.0.1:5173` and confirm that the frontend page is accessible. 3. Register or log in to an account. 4. Initiate a normal question in Home / Agent Chat. 5. Check if the Agent Run Detail has operation steps. 6. Open the MCP Tool Calls page and confirm that the tool call audit records are viewable. 7. If document upload is configured, upload a desensitized test document and test RAG Q&A. ### 9. Common Startup Issues | Issue | Possible Cause | Solution | |---|---|---| | Backend cannot connect to database | PostgreSQL not started or password incorrect | Check PostgreSQL service and `.env` | | `alembic upgrade head` fails | Database does not exist or migration environment not activated | Create `agent_os` database and activate `.venv` | | Frontend request fails | Backend not started or CORS mismatch | Check backend port and `CORS_ORIGINS` | | Agent cannot call model | API Key not configured or model name unavailable | Check `DASHSCOPE_API_KEY`, `ALIYUN_BAILIAN_API_KEY`, and model configuration | | Approval recovery fails | checkpoint backend unavailable | Locally confirm PostgreSQL availability, production keeps `AGENT_CHECKPOINTER_REQUIRE_DURABLE=true` | | Email tool does not send real emails | `EMAIL_PROVIDER=mock` | This is the default safe behavior; configure smtp variables when needed | ## Environment Variables and Key Management Do not submit `.env` to GitHub. The repository only submits `.env.example`, and real keys are only placed in local, server environment variables, or GitHub Actions Secrets. ### Required Basic Items | Variable | Purpose | Whether Sensitive | Suggestion | |---|---|---:|---| | `SECRET_KEY` | JWT / application signature key | yes | production must be replaced with a strong random string | | `POSTGRES_HOST` | PostgreSQL address | no | local can use `127.0.0.1` | | `POSTGRES_PORT` | PostgreSQL port | no | default `5432` | | `POSTGRES_USER` | PostgreSQL username | depending on the situation | do not use production root/superuser | | `POSTGRES_PASSWORD` | PostgreSQL password | yes | do not submit | | `POSTGRES_DATABASE` | database name | no | default `agent_os` | ### LLM / Embedding | Variable | Purpose | Whether Sensitive | |---|---|---:| | `DASHSCOPE_API_KEY` | DashScope / Qwen call | yes | | `ALIYUN_BAILIAN_API_KEY` | Aliyun Baizhan call | yes | | `AGENT_LLM_API_KEY` | Agent LLM custom API Key | yes | | `AGENT_LLM_BASE_URL` | OpenAI-compatible base URL | no | | `EMBED_API_KEY` | Embedding API Key | yes | | `EMBED_BASE_URL` | Embedding base URL | no | ### RAG / Memory | Variable | Purpose | Whether Sensitive | |---|---|---:| | `QDRANT_URL` | Qdrant address | possible | | `QDRANT_API_KEY` | Qdrant Cloud Key | yes | | `QDRANT_COLLECTION` | document vector collection | no | | `MEMORY_QDRANT_COLLECTION` | memory vector collection | no | ### Checkpoint / Cache | Variable | Purpose | Whether Sensitive | |---|---|---:| | `AGENT_CHECKPOINTER_BACKEND` | checkpoint backend, production recommends `postgres` | no | | `AGENT_CHECKPOINTER_DATABASE_URL` | independent checkpoint database URL | yes | | `REDIS_URL` | Redis address | yes, if it contains password | | `REDIS_PASSWORD` | Redis password | yes | ### Search / Feed | Variable | Purpose | Whether Sensitive | |---|---|---:| | `TAVILY_API_KEY` | Tavily search | yes | | `SERPAPI_API_KEY` | SerpAPI search | yes | | `GITHUB_TOKEN` | GitHub feed / API | yes | ### Email | Variable | Purpose | Whether Sensitive | |---|---|---:| | `EMAIL_PROVIDER` | `mock` or `smtp` | no | | `EMAIL_FROM` | sender | no | | `SMTP_HOST` | SMTP address | no | | `SMTP_USERNAME` | SMTP username | yes | | `SMTP_PASSWORD` | SMTP password | yes | ### Local Tools | Variable | Purpose | Suggestion | |---|---|---| | `LOCAL_TOOLS_WORKSPACE_DIR` | local file tool allowed access directory | use a dedicated directory, do not point to project root or user home directory | | `LOCAL_TOOLS_ALLOW_DELETE` | whether to allow file deletion | default `false` | | `LOCAL_TOOLS_MAX_READ_CHARS` | single read character limit | adjust as needed | | `LOCAL_TOOLS_MAX_WRITE_CHARS` | single write character limit | adjust as needed | ## Database and External Services ### PostgreSQL Production environment recommends using managed PostgreSQL and enabling backups. Approval recovery relies on durable checkpoint; do not use pure memory checkpoint in production. ### Qdrant Local development can use fallback retrieval first; configure Qdrant when complete RAG effect is needed. ### Redis Redis can be used for partial caching or experimental checkpoints in the current project. For production checkpoints, PostgreSQL is recommended. ### Neo4j Neo4j is an optional graph capability that can be disabled by default: ```env ENABLE_NEO4J=false ``` ## Test and Verification Backend testing: ```bash pytest ``` Running key module tests: ```bash pytest src/web_app/tests/test_mcp_stage7.py pytest src/web_app/tests/test_rag_stage3.py pytest src/web_app/tests/test_memory_system.py pytest src/web_app/tests/test_postgres_checkpoint_e2e.py ``` Frontend type checking and building: ```bash cd frontend npm run type-check npm run build ``` ## License This project retains the original Open Deep Research open-source license. Before releasing to GitHub, please confirm that `LICENSE` is consistent with your secondary development and release method.