MiMo-Forge

# Tool List An AI full-stack platform for individuals and teams. The project is based on frontend Vue 3 and backend NestJS, and provides intelligent dialogue, visual intelligent agent workflow, MCP tool calling, RAG knowledge base, TTS voice synthesis, and background management capabilities through unified backend proxy access to MiMo and other OpenAI Chat Completions compatible services.       ## Overview | Module | Implemented Capabilities | | --- | --- | | **Intelligent Assistant** | SSE streaming dialogue, session persistence, Markdown and code highlighting, multimodal attachments, deep thinking, internet search, function calling, role setting, knowledge base association | | **Visual Intelligent Agent** | Vue Flow-based workflow editor, supporting draft auto-save, graph verification, streaming debugging, version release, and running published versions in formal sessions | | **MCP Tools** | STDIO / SSE two transmission modes, tool discovery and caching, connection health check, sensitive parameter encryption, chat ReAct calling, and workflow node calling | | **RAG Knowledge Base** | Document upload and asynchronous processing, configurable text splitting, local Embedding, multi-model switching, sqlite-vec vector retrieval, and sharded viewing and quoting information | | **TTS Synthesis** | Pre-set voice, text voice design, audio voice replication, supporting style description / tags, streaming synthesis, WAV / PCM16 / MP3, and history records | | **User and Permission** | Registration login, JWT authentication, personal data and password modification, avatar upload, `admin` / `user` RBAC, and automatic promotion of the first registered user to administrator | | **Background Management** | User, role, login log, operation log, system configuration, chat configuration, audio tags, session audit, and ECharts statistical panel | ## Core Features ### 1. Intelligent Assistant - Consumes backend SSE using native `fetch + ReadableStream`, displaying answers, thinking processes, and tool calling trajectories in real-time. - Sessions and messages are persisted to SQLite by user, allowing creation, renaming, deletion, and continuation of historical sessions. - Dynamically reads model lists from the current AI service provider's `/models` interface; manual model name input is still possible if reading fails. - Supports text, image, audio, and video message content. - Can be used with administrator switches for: - Deep thinking - Internet search - Function calling - Model role setting (preset roles or custom prompts) - RAG knowledge base - After enabling MCP, chat enters tool calling loop, with the model selecting and executing enabled MCP tools for the current user. - Creating intelligent conversation sessions binds published intelligent agent versions; subsequent republishing does not change execution snapshots of existing sessions. ### 2. Visual Intelligent Agent Workflow The intelligent agent editor provides the following nodes: | Node | Function | | --- | --- | | **Start** | Provides user questions, session history, and other system variables | | **Large Model** | Calls specified models, with settings for system prompts, prompt templates, temperature, maximum tokens, and historical rounds | | **Knowledge Base** | Uses query templates to retrieve specified knowledge bases and output context and references | | **MCP Tool** | Selects MCP services and tools, binding tool parameters by literal, variable, or template | | **Template** | Combines upstream node outputs using `{{variable}}` | | **Condition** | Performs true/false branching based on rules like equality, containment, null value, existence, and numerical comparison | | **Reply** | Generates final reply and ends current execution path | The workflow supports: - Canvas dragging, connection, node copying, and deletion. - Draft auto-save and saving protection before leaving the page. - Static verification before publishing, including node configuration, connectivity, variable references, condition branches, and MCP parameter schema. - Independent debugging panel; debugging results are returned as SSE event streams, without writing to formal chat records. - Publishing generates immutable version snapshots; formal chat only runs published versions. ### 3. MCP Tool Calling - Supports local process `stdio` and remote service `sse`. - Supports configuration of commands, parameters, environment variables, URLs, and request headers. - Enables/disables servers, manually refreshes tool lists, and views connection health status. - Tool definitions are cached to the database; editing intelligent agents directly reads tool JSON schema to construct parameter forms. - Fields with names containing `key`, `token`, `secret`, `password`, `api`, `auth`, `credential`, or `private` in `env` and `headers` are encrypted and displayed as `***` when returned to the frontend. > STDIO MCP will launch local child processes by the backend. Please configure trustworthy commands only and ensure the system running the backend has the corresponding runtime or executable files installed. ### 4. RAG Knowledge Base Processing flow: 1. Uploads PDF, Word, Excel, CSV, or text files. 2. Splits documents using `RecursiveCharacterTextSplitter`. 3. Generates Embedding locally through `@huggingface/transformers`. 4. Establishes independent vector tables for each knowledge base using `sqlite-vec`. 5. Executes vector similarity retrieval during chat or intelligent agent node queries, injecting relevant context into the model. Each knowledge base can be configured independently: - Embedding model - `chunkSize`, default `500` - `chunkOverlap`, default `100` - `embeddingBatchSize`, default `8` Current built-in models: | Model | Vector Dimension | | --- | ---: | | `Xenova/all-MiniLM-L6-v2` | 384 | | `Xenova/all-mpnet-base-v2` | 768 | | `Xenova/bge-small-en-v1.5` | 384 | | `Xenova/gte-small` | 384 | | `Xenova/multilingual-e5-small` | 384 | Switching Embedding models or modifying parameters affecting splitting results will re-process existing documents. The model is downloaded from Hugging Face upon first use and uses local caching thereafter. ### 5. TTS Voice Synthesis | Mode | Model | Description | | --- | --- | --- | | **Pre-set Voice** | `mimo-v2.5-tts` | Generates voice using built-in premium voices | | **Voice Design** | `mimo-v2.5-tts-voicedesign` | Designs voices through natural language descriptions | | **Voice Replication** | `mimo-v2.5-tts-voiceclone` | Replicates voices by uploading audio samples | Pre-set voices include: MiMo-default, Bing Tang, Jasmine, Soda, White Birch, Mia, Chloe, Milo, Dean. Other capabilities: - Natural language style descriptions or audio tags maintained in the background. - `wav`, `pcm16`, `mp3` output formats. - Ordinary generation and SSE streaming generation interfaces. - Synthesis history saved to the database, including audio Base64, with playable addresses rebuilt after page refresh. - Uniform use of `new Audio()` for audio playback, avoiding browser or plugin misjudgment of `<audio>` preloading behavior for Blob URLs. ### 6. Multi-service Provider Compatibility API settings have built-in Base URL presets for: - MiMo ordinary API - MiMo Token Plan: China, Singapore, Europe - OpenAI - DeepSeek - Alibaba Cloud Baidu / Tongyi Qianwen: China, international station - Moonshot / Kimi - Zhipu GLM - OpenRouter - SiliconFlow - Custom OpenAI compatible interfaces Authentication methods support automatic judgment, `Authorization: Bearer`, `api-key`, `x-api-key`, and no authentication. The TTS, MiMo deep thinking, and MiMo internet search extension capabilities of this project still depend on whether the target service provider is compatible with the corresponding parameters. > MiMo Token Plan does not support `mimo-v2-flash` and `web_search`. The frontend and backend will perform compatibility restrictions; please use supported models, disable internet search, or switch to other endpoints. ## Technical Architecture ```mermaid flowchart LR Browser["Browser / Vue 3"] -->|"JWT + REST / SSE"| API["NestJS API"] API --> DB["SQLite / TypeORM"] API --> Vec["sqlite-vec vector table"] API --> LLM["MiMo / OpenAI compatible services"] API --> MCP["MCP STDIO / SSE services"] API --> Embed["Local Hugging Face Embedding"] API --> Files["Uploaded files and model cache"] ``` ### Frontend | Technology | Usage | | --- | --- | | Vue 3 + `<script setup>` | Pages and components | | TypeScript 6 | Strict type checking | | Vite 8 | Development server and build | | Element Plus + Tailwind CSS 4 | UI and styles | | Pinia + Vue Router | State management and routing | | Vue Flow | Intelligent agent workflow canvas | | Axios + Fetch Streams | REST requests and SSE streaming responses | | marked + DOMPurify + highlight.js | Markdown, security cleaning, and code highlighting | | ECharts | Background statistical charts | | Web Crypto API | AES-GCM sensitive request encryption | ### Backend | Technology | Usage | | --- | --- | | NestJS 11 | API, authentication, and modular services | | TypeScript 5.7 | Backend development | | TypeORM + better-sqlite3 | Data persistence | | sqlite-vec | Vector retrieval | | `@huggingface/transformers` + ONNX Runtime | Local Embedding | | `@langchain/textsplitters` | Document splitting | | Model Context Protocol SDK | MCP client and tool calling | | Passport JWT + bcryptjs | Identity authentication and password hashing | | pdf-parse + mammoth + exceljs | Document parsing | ## Quick Start ### Environment Requirements - Node.js **22 or higher** - pnpm **10.24.x**, recommended to be enabled via Corepack - Native module compilation environment: - Windows: Visual Studio Build Tools (including C++ toolchain) - Debian / Ubuntu: `build-essential python3 make g++` The project uses `better-sqlite3`, `sqlite-vec`, and `onnxruntime-node`, which may fail to install with low Node.js versions or missing native compilation tools. ### 1. Install Dependencies ```bash corepack enable pnpm install ``` The repository uses pnpm workspace; root directory commands will install dependencies for both frontend and `server/`. ### 2. Configure Environment Variables Create `.env` in the project root: ```env # NestJS backend address VITE_BACKEND_URL=http://localhost:3001 # Must be consistent with server/.env's AES_SECRET_KEY VITE_AES_SECRET_KEY=replace-with-a-strong-random-secret ``` Create `.env` in `server/`: ```env PORT=3001 CORS_ORIGIN=http://localhost:3000 # Consistent with frontend's VITE_AES_SECRET_KEY AES_SECRET_KEY=replace-with-a-strong-random-secret # JWT signature key JWT_SECRET=replace-with-another-strong-random-secret # Optional: Hugging Face mirror and model cache directory # HF_ENDPOINT=https://hf-mirror.com/ # TRANSFORMERS_CACHE=./models/transformers ``` | Variable | Required | Default Value | Description | | --- | --- | --- | --- | | `VITE_BACKEND_URL` | No | `http://localhost:3001` | Backend address accessed by frontend | | `VITE_AES_SECRET_KEY` | Yes | None | Frontend sensitive request encryption key | | `PORT` | No | `3001` | Backend listening port | | `CORS_ORIGIN` | No | `http://localhost:3000` | Allowed frontend origins, separated by commas | | `AES_SECRET_KEY` | Yes | None | Backend AES key, must be consistent with frontend | | `JWT_SECRET` | Yes | None | JWT signature key | | `HF_ENDPOINT` | No | Hugging Face official address | Model download mirror address | | `TRANSFORMERS_CACHE` | No | `./models/transformers` | Embedding model cache directory | > MiMo or other service providers' API Keys are not configured through environment variables. Log in and fill in the "API settings" in the top right corner; the backend will save the encrypted AES-GCM to the current user's database configuration. ### 3. Start Development Environment ```bash # Start frontend 3000 and backend 3001 simultaneously pnpm dev:all ``` You can also start them separately: ```bash pnpm dev pnpm -C server start:dev ``` Open `http://localhost:3000`. The database and default configuration will be automatically created when the backend starts for the first time; the first registered user will become an administrator. ### 4. Initial Use 1. Register the first account and log in. 2. Select the service provider, fill in the API Key, and authentication method in "API Settings". 3. Enter "Intelligent Assistant", select or input a model, and start a conversation. 4. If RAG is needed, create a knowledge base and wait for document processing to complete. 5. If tool calling are needed, add and enable MCP servers, and use them in chat or intelligent agent workflows. 6. If custom intelligent agents are needed, create workflows, verify, debug, publish, and then initiate sessions from the intelligent agent list. ## Common Commands ```bash # Development pnpm dev # Frontend only pnpm -C server start:dev # Backend only pnpm dev:all # Start both frontend and backend # Build pnpm build # Frontend type checking and build pnpm -C server build # Backend build pnpm build:all # Build both frontend and backend # Testing and checking pnpm test # Frontend Vitest pnpm typecheck # Frontend independent type checking pnpm -C server test # Backend Jest pnpm -C server test:cov # Backend test coverage pnpm -C server format # Backend Prettier # Production launch pnpm -C server start:prod pnpm preview # Local preview of frontend dist ``` Production build products are located in: - Frontend: `dist/` - Backend: `server/dist/` Please use Nginx, Caddy, or other static servers to host the frontend and forward `/api` requests to the NestJS backend during actual deployment. ## Project Structure ```text TTS-Project/ ├─ src/ # Vue 3 frontend │ ├─ api/ # REST and SSE API encapsulation │ ├─ components/ # Chat, TTS, audio, and workflow components │ ├─ composables/ # TTS, page data, and other combined logic │ ├─ router/ # Routing and permission guards │ ├─ stores/ # Pinia state management │ ├─ types/ # Frontend type definitions │ ├─ utils/ # Encryption, audio, workflow, and stream parsing tools │ └─ views/ # Assistant, intelligent agent, knowledge base, MCP, TTS, and background pages ├─ server/ │ ├─ src/ │ │ ├─ agent/ # Intelligent agent, version, graph verification, and workflow executor │ │ ├─ auth/ # Registration, login, and JWT authentication │ │ ├─ chat/ # Session, SSE conversation, MCP agent │ │ ├─ knowledge-base/ # Document processing, Embedding, RAG, and sqlite-vec │ │ ├─ mcp/ # MCP configuration, connection, and tool service │ │ ├─ tts/ # TTS proxy and streaming synthesis │ │ ├─ admin/ # User, log, chat audit, and statistical │ │ ├─ config/ # User API and TTS configuration │ │ ├─ common/ # Encryption, exception filters, guards, upload, and interceptors │ │ └─ ... # User, role, audio tags, system configuration, etc. modules │ ├─ public/uploads/ # Upload files │ ├─ models/transformers/ # Default Embedding model cache │ └─ data.sqlite # Default SQLite database ├─ public/ # Frontend static resources ├─ .github/workflows/ # CI and Release workflows ├─ pnpm-workspace.yaml └─ package.json ``` ## Security and Data Description - All external AI, TTS, and MCP requests are initiated by the backend, and the browser does not directly call the model service. - Sensitive requests such as login, registration, personal profiles, and password modifications use AES-256-GCM, with a format of `iv:authTag:ciphertext`; the backend still supports the old AES-CBC two-segment format. - API Key is saved to SQLite as AES-GCM ciphertext; the frontend must use the same AES key as the backend. - User passwords are saved using bcrypt hash. - JWT has a default expiration period of 7 days, and the current implementation is a single Access Token mode. - Management interfaces are protected using `@Roles('admin')` and `RolesGuard`. - The backend generates or passes through `X-Request-Id` for each request, and the global exception response includes the request identifier for easy troubleshooting. - Management operations are written to the operation log through a global interceptor, and sensitive fields are desensitized. - The current database configuration uses `synchronize: true` to automatically synchronize Entity, which is suitable for the development environment; a migration plan should be evaluated and a backup of `server/data.sqlite` should be made before formal production. ## API Overview All backend routes have the `/api` prefix, and most interfaces require: ```http Authorization: Bearer <JWT> ``` except for registration, login, and public configuration. | Route Group | Description | | --- | --- | | `/api/auth/*` | Registration, login, current user, profile, and password | | `/api/config` | API/TTS configuration for the current user | | `/api/chat/*` | Sessions, messages, model queries, and SSE conversations | | `/api/agents/*` | Intelligent agent CRUD, drafts, verification, release, and debugging | | `/api/mcp/servers/*` | MCP server management, tool refresh, and health check | | `/api/knowledge-base/*` | Knowledge base, documents, slices, models, and segmentation configuration | | `/api/tts/generate` | Non-streaming TTS | | `/api/tts/generate-stream` | SSE streaming TTS | | `/api/history/*` | TTS history records | | `/api/upload/*` | Avatar and general file upload | | `/api/admin/*` | Backend users, logs, statistics, and chat audit | | `/api/admin/roles` | Role management | | `/api/admin/system-config` | System configuration | | `/api/admin/chat-config` | Chat model compatibility configuration and feature switches | ## Testing and CI/CD - Frontend testing: Vitest + jsdom, covering workflow graph processing and intelligent agent SSE event parsing tool logic. - Backend testing: Jest, covering intelligent agent graph verification, workflow execution, intelligent agent services, chat role processing, and global exception filtering logic. - GitHub Actions CI uses Node.js 22 and pnpm 10.24, executing frontend and backend builds and backend tests on push and pull request of `main` / `master`. - Pushing `v*` tags triggers the Release workflow, building the frontend and backend and generating `release.zip`. ## Frequently Asked Questions ### Login or registration prompts "request data decryption failed" Check if the `VITE_AES_SECRET_KEY` in the root directory `.env` is exactly the same as the `AES_SECRET_KEY` in `server/.env`. After modifying the frontend environment variables, restart Vite. ### `better-sqlite3`, `sqlite-vec`, or `onnxruntime-node` installation fails Confirm that the Node.js version is not lower than 22, and install the local C/C++ compilation tools. After switching Node.js major versions, it is recommended to re-execute `pnpm install`. ### Embedding model download is slow or fails Configure in `server/.env`: ```env HF_ENDPOINT=https://hf-mirror.com/ ``` You can also specify the prepared model cache directory through `TRANSFORMERS_CACHE`. ### MCP tool list is empty or calls have no response - Confirm that the MCP server is enabled. - In STDIO mode, confirm that the command can be executed in the backend runtime environment. - In SSE mode, confirm that the URL, request headers, and network are reachable. - Click "Refresh tool" to rediscover the tool and view the health check results. - After editing the server configuration, the old connection will be disconnected and re-established at the next use. ### TTS history audio cannot be played after refreshing New records save `audioBase64` and rebuild Blob URL when the page loads. If old data only has temporary `audioUrl`, it may not be recoverable after refreshing. ### Data loss after deleting `data.sqlite` `server/data.sqlite` saves users, configurations, chats, knowledge base metadata, intelligent agents, and logs. After deletion, the backend will rebuild the table and write default data, but the original business data cannot be recovered. ## License The current repository does not provide an independent open-source license file. If you need to distribute or sublicense, please supplement a clear license.