ZSN.AgentBrook

**English** | [English](README_EN.md) <div align="center"> # Tool List **Enterprise-level AI Intelligent Agent Orchestration Platform** Visual Workflow Orchestration · Multi-model Intelligent Agent · RAG Knowledge Base · MCP Tool Integration [](https://dotnet.microsoft.com/) [](https://github.com/microsoft/semantic-kernel) [](https://www.postgresql.org/) [](https://github.com/pgvector/pgvector) [](https://github.com/apache/age) [](https://www.mysql.com/) [](https://redis.io/) [](https://react.dev/) [](https://playwright.dev/) [](https://www.docker.com/) [](https://github.com/modelscope/FunASR) [](https://github.com/searxng/searxng) [](https://www.quartz-scheduler.net/) [](LICENSE) [](https://github.com/ChenGan666/ZSN.AgentBrook) [](https://github.com/ChenGan666/ZSN.AgentBrook) </div> <div align="center"> ⭐ **If this project helps you, give us a Star!** </div> --- ## Platform Highlights - **Visual DAG Workflow Engine** — Drag-and-drop designer, 20+ node types, support conditional branches, parallel execution, manual approval, sub-workflow nesting, **AI auto-generates downstream node chains** - **Message Node** — Workflow-driven IM message sending node, supports Enterprise WeChat/DingTalk/Feishu/WhatsApp, Redis queue asynchronous decoupling, batch sending and receipt confirmation - **Message Gateway** — Unified IM message receiving and sending gateway, Webhook reception → routing rule matching → workflow triggering, supports multi-channel and multi-rule orchestration - **Voice Node** — Speech recognition + LLM post-processing integration, supports FunASR local deployment, speaker separation, multi-format output (SRT/VTT/JSON), long audio automatic segmentation, hot word enhancement - **ClawAI Intelligent Agent** — Plan-Execute-Reflect loop architecture, multi-layer memory system (short-term/long-term/scenario/personality/user portrait), supports task decomposition and dynamic re-planning - **ServiceDesk Node** — FunctionCall-driven knowledge base retrieval + generation integration, supports multi-turn dialogue, intention recognition, information collection - **Research Node** — Autonomous network research, based on SearXNG search + Playwright web crawling, multi-turn search-analysis-reflection iteration, automatically generates research reports - **RAG Knowledge Base** — Vector retrieval + full-text retrieval hybrid search, supports PDF/Word/Markdown and other document formats, automatic blocking and indexing, supports image recognition and image output - **MCP Tool Protocol** — Built-in MCP Server/Client, quickly access external tools and data sources - **Multi-model Support** — OpenAI/Claude/DeepSeek/Ollama/Zhisheng/Baidu and other mainstream models, can be configured independently for each node - **Real-time Streaming Output** — Based on Redis Stream's streaming response, front-end real-time display of LLM generation process - **Chat + Agent Dual-mode Client** — Vue3 + Tauri desktop client, Chat mode supports SSE streaming dialogue/workflow drive, Agent mode supports Plan-Act-Reflect intelligent agent orchestration, with voice input, file upload, Human-in-the-loop interaction, Web SPA mode out of the box - **Browser Automation** — Based on Playwright's Agent browser, supports web operation and data collection --- ## Architecture Overview [](https://agentbrook.com/) ### Data Flow [](https://agentbrook.com/) ### Core Module Dependencies ``` AI.Entity ────────────────────────────────────── (Basic data model, zero dependency) │ ├── AI.DAL ──── AI.DAL.MySql / AI.DAL.Postgres (Data access abstraction) │ │ │ └── AI.BLL ──────────────────────────── (Business logic layer) │ │ │ ├── AI.Service ───────────────── (Application service layer) │ │ │ └── AI.Node ◄── AI.Core ───────── (Workflow engine + AI kernel) │ │ │ │ │ ├── AI.Plugins (Function plugins) │ │ ├── AI.MCPClient (MCP client) │ │ └── AI.Functions (Built-in functions) │ │ │ └── AI.KnowledgeBase ──────── (Knowledge base service) │ ├── AgentBrook.API ────────────────────────── (API gateway, connects all modules) │ ├── AgentBrook.MessageGateway ───────────── (IM message gateway) │ ├── AgentBrook.AutoJob ────────────────────── (Background task scheduling) │ ├── AgentBrook.Client ──────────────────────── (Cross-platform client) │ ├── AgentBrook.Web / Web.Manage ───────────── (Front-end interface) │ └── Utils.Core / LitJSON / Cache ──────────── (General utility library) ``` --- ## Project Structure | Project | Description | Key Technology | |---|---|---| | **ZSN.AI.Core** | AI kernel, Semantic Kernel encapsulation, multi-model routing, streaming output | Semantic Kernel, Extensions.AI | | **ZSN.AI.Node** | Workflow node execution engine, ClawAI/ServiceDesk/LLM and other nodes | SK FunctionCall, Pipeline | | **ZSN.AI.Entity** | Data model, DTO, enumeration definition | SqlSugar annotation | | **ZSN.AI.BLL** | Business logic layer: workflow management, task scheduling, knowledge base operation | | | **ZSN.AI.DAL** | Data access abstraction interface | SqlSugar ORM | | **ZSN.AI.DAL.MySql** | MySQL data access implementation | SqlSugar + MySQL | | **ZSN.AI.DAL.Postgres** | PostgreSQL implementation (vector retrieval + knowledge graph) | Npgsql, pgvector, Apache AGE | | **ZSN.AI.KnowledgeBase** | Knowledge base service: document import, blocking, indexing, semantic retrieval, image recognition | Npgsql, pgvector, Apache AGE | | **ZSN.AI.MCPServer** | MCP tool server, exposing platform capabilities as MCP tools | ModelContextProtocol | | **ZSN.AI.MCPClient** | MCP client, connecting to external MCP services | ModelContextProtocol | | **ZSN.AI.Plugins** | Semantic Kernel function plugin collection | | | **ZSN.AI.Functions** | Built-in function library | | | **ZSN.AgentBrook.API** | REST API gateway, Swagger documentation | ASP.NET Core, SignalR | | **ZSN.AgentBrook.MessageGateway** | Unified IM message gateway, multi-channel Webhook reception and routing | ASP.NET Core, Redis Queue | | **ZSN.AgentBrook.Client** | Cross-platform client (Vue3 + Tauri/Web SPA) | Vue3, TypeScript, Element Plus, Tauri | | **ZSN.AgentBrook.Web** | Front-end interface (React + Ant Design Pro) | React, Ant Design Pro | | **ZSN.AgentBrook.Web.Manage** | Management backend (LayUI): application/model/knowledge base/conversation management/menu configuration, etc. | LayUI, jQuery | | **ZSN.AgentBrook.AutoJob** | Background task scheduler, polling execution of workflow tasks | Quartz.NET | | **ZSN.AgentBrook.Plugins** | Application-level plugins | | | **ZSN.AgentBrowser** | AI browser automation | Playwright | | **ZSN.Cache** | Distributed cache service | Redis, MemoryCache | | **ZSN.Utils.Core** | General utility library | log4net, NPOI | | **LitJSON** | Lightweight JSON library | | --- ## Core Capabilities ### 1. Visual Workflow Engine Based on DAG (directed acyclic graph) visual workflow editor, supports drag-and-drop design: [](https://agentbrook.com/) **Supports 20+ node types:** | Category | Node | |---|---| | Process Control | Start, End, AgentStart, AgentEnd | | AI Reasoning | MainAI, LargeModel, ClawAI, ServiceDesk, Research, Voice, Message | | Knowledge Retrieval | KnowledgeBase, FileToMarkdown | | Logic Routing | Selector (conditional branch), Merge (convergence), IntentionRecognition (intention recognition) | | Tool Integration | MCP, Plugins, Agent (sub-workflow) | | Human-Computer Collaboration | HumanInTheLoop (manual approval), Reporter (report generation) | | Trigger | TimeTrigger (timing trigger) | **AI auto-generates downstream node chains:** In the workflow editor, select any node, input natural language requirements, and AI will automatically plan, generate, and assemble complete downstream node chains: - **Three-stage pipeline**: plan workflow structure → generate node details in parallel → engineering assembly, through SSE real-time push progress - **Intelligent variable mapping**: automatically parse available variables in upstream context, accurately match sourceId references for downstream nodes - **Compatible with existing workflows**: keep upstream nodes unchanged, only generate new downstream nodes and connections, automatic layout and DAG loop detection - **Prompt template customization**: built-in AutoGeneratePlan and ModifyNodeDetail prompt templates, can customize AI behavior through md files ### 2. ClawAI — Advanced Intelligent Agent ClawAI is the core intelligent agent node of the platform, implementing a complete **Plan-Execute-Reflect** loop: [](https://agentbrook.com/) **Multi-layer memory system:** | Memory Layer | Function | |---|---| | Short-term Memory | Current session context, maintaining dialogue coherence | | Scenario Memory | Historical event records, cross-session experience accumulation | | Long-term Memory | Knowledge base retrieval results, persistent knowledge storage | | User Portrait | User preferences and behavior characteristics, personalized service | | AI Personality | AI role state, maintaining consistent interaction style | **Multi-model collaboration:** Each link can be configured independently (main model, planning model, reflection model, memory model, personality model), achieving flexible balance between cost and effect. ### 3. ServiceDesk — Customer Service Intelligent Agent A fast-response node for customer service scenarios, allowing LLM to autonomously call knowledge base retrieval through FunctionCall: [](https://agentbrook.com/) **Core Features:** - FunctionCall-driven: LLM autonomously decides whether to retrieve knowledge base - Multi-turn dialogue: automatically maintains session context - Information collection: detects intention and automatically asks for missing fields - Confidence level grading: high/medium/low confidence corresponds to different processing strategies - Source citation: answer with knowledge base source annotation ### 4. RAG Knowledge Base [](https://agentbrook.com/) - **Hybrid Retrieval**: vector semantic search + full-text keyword search, fusion sorting - **Knowledge Graph**: entity relationship graph based on Apache AGE - **Multi-format Support**: PDF, Word, Markdown, TXT, HTML, etc. - **Intelligent Blocking**: semantic-aware document blocking strategy - **Image Recognition**: automatically extracts images from documents, generates image descriptions through VLM (visual language model), OCR text recognition, supports PDF/Word/PPT documents - **Image Output**: knowledge base retrieval results support returning associated images, images and text blocks automatically associated, support hybrid text-image retrieval **Image Processing Pipeline:** ``` Document upload → Image extraction (PDF/Word/PPT) → Content deduplication (SHA256 hash) → VLM description generation (image description + OCR + tags) → Image storage + metadata storage → Image-block automatic association ``` ### 5. Research — Autonomous Research Node The Research node is an autonomous network research engine that can automatically conduct multi-turn searches, web crawling, and analysis-reflection based on research goals: [](https://agentbrook.com/) **Key Features:** - **Dual-mode Crawling**: Playwright web crawling priority, automatically degrades to search summary mode when unavailable - **Multi-turn Iteration**: up to 3 turns of search-analysis cycle, LLM automatically plans keywords - **Completeness Evaluation**: evaluates information coverage after each analysis (0.0-1.0), automatically stops when threshold is reached - **LLM Call Budget**: configurable maximum LLM call times, preventing cost loss - **Content Caching**: Redis-based web content caching, avoiding repeated crawling - **Timeout Protection**: global timeout protection, timeout returns acquired content - **Streaming Output**: real-time streaming push research progress ### 6. Voice — Transcription Node The Voice node is an intelligent voice processing node that integrates speech recognition and LLM post-processing, supporting automatic generation of structured text from audio files: **Core Capabilities:** - **Speech Transcription**: Localized speech recognition based on FunASR (WebSocket offline mode), with data remaining on the server - **Speaker Separation**: Automatic identification of different speakers, with support for custom speaker label mapping - **Multi-format Output**: Plain text, timestamped JSON, SRT subtitles, WebVTT subtitles - **LLM Post-processing**: Transcription results automatically connected to LLM, supporting custom prompts for text refinement, summary generation, etc. - **Long Audio Segmentation**: Audio exceeding the threshold (default 300 seconds) automatically segmented by silence detection, parallel transcription, and merging - **Hotword Enhancement**: Supports configuration of hotword lists to improve recognition rates for specific domain vocabulary - **Multi-format Input**: 12 audio/video formats such as WAV, MP3, M4A, OGG, FLAC, AAC, etc., with automatic conversion by FFmpeg **Processing Flow:** ``` Audio Input → Format Conversion (FFmpeg) → Long Audio VAD Segmentation (silencedetect) → FunASR WebSocket Transcription (chunked sending) → Speaker Label Mapping + Output Formatting → LLM Post-processing (optional) → Structured Result Output ``` **Configuration Example (appsettings.json):** ```json { "VoiceNodeOptions": { "DefaultProvider": "FunASR", "MaxConcurrentSegments": 4, "MaxFileSizeMb": 500, "AutoSegmentThresholdSeconds": 300, "TempFileDirectory": "", "FFmpegPath": "" }, "FunASROptions": { "ServerUrl": "ws://127.0.0.1:10095", "ChunkSize": 9600, "ConnectTimeoutSeconds": 5, "TranscribeTimeoutMinutes": 10 } } ``` ### 7. Message — Message Sending Node The Message node enables workflows to proactively send IM messages, supporting multi-channel integration and flexible sending strategies: **Core Features:** - **Multi-channel Support**: Enterprise WeChat, DingTalk, Feishu, WhatsApp, unified adaptation through MessageGateway - **Multi-user Mode**: Static (manual specification) / Dynamic (upstream variables) / Query (reserved), supporting batch independent sending - **Placeholder Replacement**: Message templates support placeholders like `{{input}}`, `{{variable name}}`, etc., automatically replaced with workflow context variables - **Send Confirmation**: Supports WaitForConfirmation mode, polling to wait for gateway confirmation of sending results before triggering downstream - **Redis Decoupling**: Asynchronous communication with MessageGateway through Redis queue, node does not directly call IM API - **Personalized Messages**: Supports covering message content for each target user, achieving personalized push **Processing Flow:** ``` Workflow → MessageNode → Redis Enqueue (msg_send_queue) → MessageGateway consumer dequeue → Find channel configuration → Provider instantiation → Token refresh / signature calculation → IM API sending → Result write back to tb_msg_send_record → (WaitForConfirmation mode) Node polling confirmation ``` **Configuration Example (appsettings.json):** ```json { "MessageNode": { "SendQueueName": "msg_send_queue", "WaitTimeoutSeconds": 30, "PollIntervalMs": 500 } } ``` ### 8. MessageGateway — Message Gateway MessageGateway is an independent IM message gateway service, responsible for receiving and sending IM messages: **Receiving Flow (IM → Workflow):** - Webhook receives IM platform callback → Channel signature verification → Message parsing → Idempotent deduplication → Routing rule matching → Create workflow task **Sending Flow (Workflow → IM):** - Redis queue consumption (msg_send_queue) → Channel configuration lookup → Provider acquisition / circuit protection → Retry mechanism → Result write back **Routing Rules:** - Supports All / Keyword / Regex multiple matching types - Priority sorting, the first matching rule takes effect - Automatic creation of sessions, temporary members (deterministic MemberID) - Supports custom inputs mapping to workflow variables **Circuit Protection:** - Configurable continuous failure threshold, automatic circuit breaking of Provider - Automatic recovery after circuit breaking (configurable recovery time) ### 9. MCP Tool Integration [](https://agentbrook.com/) Built-in MCP Server and Client, supporting: - Exposing platform capabilities (knowledge base retrieval, workflow triggering, etc.) as MCP tools - Connecting to external MCP services, extending LLM tool capabilities - Supporting bi-directional calling modes for client and server ### 10. Cross-platform Client (ZSN.AgentBrook.Client) Based on Vue3 + TypeScript + Element Plus + Tauri, the cross-platform client supports Web SPA and desktop application deployment modes, providing **Chat** and **Agent** dual-mode interaction: **Chat Mode** — Traditional conversational AI interaction, select workflow application and send message, receive workflow execution progress and AI response in real-time through SSE, supporting multi-session management, regenerate, node retry. **Agent Mode** — Client intelligent body orchestration, after user selects "orchestration brain" model, Agent automatically executes Plan-Act-Reflect cycle: planning → calling workflow App → reflecting results → continuing or completing, achieving autonomous orchestration of multi-step complex tasks. **Core Features:** - **Dual-mode Deployment** — Tauri desktop application (Windows/macOS) + Web SPA (.NET host), one frontend code for two deployments - **Chat + Agent Dual Mode** — Sidebar one-click switch, Chat workflow-driven dialogue / Agent Plan-Act-Reflect autonomous orchestration, independent session management - **SSE Streaming Dialogue** — Real-time display of LLM generation process, supporting workflow node status display - **Voice Input** — FunASR 2-pass real-time speech recognition, Tauri native audio collection - **File Upload** — Multi-format support, large file chunked upload, image compression - **Human-in-the-loop** — Workflow manual approval interaction - **Internationalization** — Built-in Chinese and English switching, expandable to more languages - **System Notification** — Desktop/Web system notification when session is completed, multi-session short window aggregation deduplication, cross-heartbeat and SSE anti-repeated push; Windows automatically adapts AppUMID to fix notification not displayed, one-click test in settings - **Local Cache** — IndexedDB stores session messages, offline queue support - **Security Mechanism** — Token encrypted storage, API request signature, XSS protection [](https://agentbrook.com/) [](https://agentbrook.com/) [](https://agentbrook.com/) **Technical Architecture:** ``` ZSN.AgentBrook.Client/ ├── Program.cs # .NET host (SPA static file service) ├── appsettings.json # Kestrel configuration ├── wwwroot/ # Vue3 build product (SPA mode) └── client-app/ # Vue3 frontend source code ├── src/ │ ├── views/ # Pages (Login, Chat, Agent, Settings, MeetingTranscribe, MiniChat) │ ├── components/ # Components (chat, agent, layout, settings, common) │ ├── composables/ # Composite functions (useChat, useAgentOrchestrator, useVoice, useFileUpload, etc.) │ ├── services/ # API services (http, auth, chat, agent, model, session, hitl, voiceApi) │ ├── stores/ # Pinia state management (app, chat, agent, etc.) │ ├── platform/ # Platform adaptation layer (Tauri / Web) │ └── utils/ # Utility functions (cache, crypto, db, markdown, sseRequest, agentPlan) └── src-tauri/ # Tauri desktop configuration ``` **Running Client:** ```bash # Web SPA mode (out of the box) dotnet run --project ZSN.AgentBrook.Client # Visit http://localhost:5006 # Development mode (frontend hot reload) cd ZSN.AgentBrook.Client/client-app npm install npm run dev # Tauri desktop development cd ZSN.AgentBrook.Client/client-app npm run tauri:dev ``` ### 11. Multi-model Support Connects to multiple AI providers through the unified `IChatService` interface: | Provider | Model Example | Access Method | |---|---|---| | OpenAI | GPT-4o, GPT-4, GPT-3.5 | OpenAI API | | Anthropic | Claude series | OpenAI compatible interface | | DeepSeek | DeepSeek-V3/R1 | OpenAI compatible interface | | Ollama | Qwen, Llama, Mistral, etc. local models | Ollama API | | Zhipu AI | GLM-4 | Zhipu API | | Baidu | Wenxin Yiyan | Baidu API | | Others | Any OpenAI compatible interface | Custom EndPoint | Each workflow node can independently configure model, Temperature, TopP, and other parameters. --- ## Technology Stack | Level | Technology | |---|---| | **Runtime** | .NET 10 | | **AI Framework** | Microsoft Semantic Kernel 1.74, Microsoft.Extensions.AI 10.4 | | **MCP** | ModelContextProtocol 0.3 | | **ORM** | SqlSugar 5.1 | | **Database** | MySQL (main library), PostgreSQL + pgvector + Apache AGE (knowledge base) | | **Cache** | Redis (StackExchange.Redis) | | **Document Processing** | PdfPig, OpenXml, Markdig, ImageSharp | | **Image Processing** | VLM image description, OCR text recognition, image-block association | | **Task Scheduling** | Quartz.NET | | **Frontend** | React + Ant Design Pro (user end), LayUI (management end), Vue3 + Element Plus + Tauri (client) | | **Browser Automation** | Playwright | | **API Documentation** | Swagger / OpenAPI | --- ## Quick Start ### Environment Requirements - .NET 10 SDK - MySQL 8.0+ - PostgreSQL 16+ (knowledge base function, need to install pgvector extension) - Redis 7.0+ ### Installation Steps ```bash # Clone repository git clone https://github.com/your-org/ZSN.AgentBrook.git cd ZSN.AgentBrook # Restore dependencies dotnet restore ZSN.AI.sln ``` ### Configuration Edit `ZSN.AgentBrook.API/appsettings.json`: ```json { "ConnectionStrings": { "DefaultConnection": "Server=localhost;Database=zsn_ai;Uid=root;Pwd=your_password;", "PostgresConnection": "Host=localhost;Database=zsn_kb;Username=postgres;Password=your_password;" }, "Redis": { "ConnectionString": "localhost:6379" }, "LLM": { "DefaultModelId": 1 } } ``` ### Run ```bash # Start API service dotnet run --project ZSN.AgentBrook.API # Start background task scheduling (workflow execution) dotnet run --project ZSN.AgentBrook.AutoJob # Start message gateway (IM message receiving and sending, optional) dotnet run --project ZSN.AgentBrook.MessageGateway # Start management backend (optional) dotnet run --project ZSN.AgentBrook.Web.Manage # Start client (Web SPA mode, optional) dotnet run --project ZSN.AgentBrook.Client ``` --- ## Demo ### Multi-image Tweet Generation [](https://agentbrook.com/) [](https://agentbrook.com/) [](https://agentbrook.com/) [](https://agentbrook.com/) ### Picture Book Generation [](https://agentbrook.com/) [](https://agentbrook.com/) [](https://agentbrook.com/) [](https://agentbrook.com/) [](https://agentbrook.com/) [](https://agentbrook.com/) ### AI Customer Service - Knowledge Base [](https://agentbrook.com/) [](https://agentbrook.com/) [](https://agentbrook.com/) --- ## License This project is open-sourced under the [MIT License](LICENSE).