fantasy

# Aether [English](README.en.md) | [English](README.en.md) > A smart home AI butler that integrates large models (LLM): control devices with natural language, visual perception of the environment, scheduled automation, and semantic knowledge graph retrieval. The frontend is a Vue 3 single-page application, and the backend is FastAPI. ## What it can do - **AI conversation control devices** — Connect to Home Assistant, control devices with natural language, open lights / adjust air conditioning / pull curtains, and verify actions with `verify_action` read-only verification. - **Camera visual perception** — RTSP / USB input, motion detection triggers visual reasoning, and configurable attention items. - **Scheduled tasks and automation rules** — Generate cron trigger times with natural language, automatically generate task names, and rule engine links devices according to conditions. - **Semantic knowledge graph (RAG)** — Document vectorization + faiss retrieval + entity co-occurrence graph construction, 3D visualization, embed model change detection + one-click rebuild. - **MCP tool ecosystem** — Built-in weather / web search / device control tools, can connect to external MCP Server. - **Multi-user + JWT authentication** — Independent LLM Key management for each user, session isolation, and support for one-click clearing of historical sessions. ## Architecture and ports | Service | Port | Description | |------|------|------| | Aether application | `8010` | FastAPI main service (REST + WebSocket + frontend SPA hosting) | | Startup progress | `8011` | Cold startup progress reporting, available for loading page polling (before main port is ready) | | Home Assistant | `8123` | Smart home brain, Aether controls devices through its REST API | | Mosquitto MQTT | `1884` | Virtual device simulator → HA message channel | | Vite development server | `5173` | Only for frontend local development, production environment hosted directly by 8010 | ``` ┌─────────────┐ REST ┌─────────────────┐ MQTT ┌───────────┐ │ Browser SPA │ ────────► │ Aether (8010) │ ◄──────► │ HA (8123)│ │ Vue 3 + Vite│ │ FastAPI+LLM │ └───────────┘ └─────────────┘ └────────┬────────┘ ▲ │ SQLite / faiss │ MQTT ▼ │ ┌──────────────┐ ┌──────────────┐ │ app/data,logs│ │ Mosquitto │ └──────────────┘ │ (1884) │ └──────────────┘ ``` ## Quick start (Docker, recommended) One `docker compose up` command to start all three services. Preparation: ```bash # 1. Configure secrets cp .env.example .env # Edit .env, fill in LLM / STT API Keys # 2. Copy configuration template cp config.example.json config.json # Leave ha.token empty, fill in later in the guided tour ``` Start: ```bash docker compose up -d --build # First time or after code update add --build docker compose ps # All four containers should be Up (aether, aether-ha, mosquitto, aether-simulator) ``` Open `http://localhost:8010` to enter the application, first-time users need to go through the guided tour: 1. Family information (name, owner title, region) 2. LLM model configuration (chat/vision/embed/summary, at least configure chat model) 3. Home Assistant connection — need to complete account registration and create a long-term access token at `http://localhost:8123` first > **HA initial initialization** (new users must read): > 1. Open `http://localhost:8123` → Create admin account (onboarding process, requires filling in name/password/location) > 2. **Configure MQTT integration** (let simulator devices report to HA): > ```bash > docker exec aether-ha python /config/add_mqtt_config.py > docker compose restart homeassistant > ``` > The script will automatically create an MQTT integration pointing to the mosquitto container (broker=`mqtt`, port=`1884`, user=`aether`). Can also manually configure in HA UI's "Settings → Devices and services → Add integration → MQTT". > 3. Log in to HA, left-bottom avatar → Long-term access token → Create token → Copy JWT and paste to guided tour step 3 > > The repository **does not include** HA's runtime state files (onboarding/auth/entity_registry, etc.), each clone is a clean HA, need to complete the above onboarding to use. `ha_config/.storage/core.config` retains default location/timezone, `ha_config/mqtt/*.yaml` are simulator device declarations, automatically loaded by HA at startup. > **Built-in demo devices**: The repository comes with 11 virtual devices (3 lights / 1 air conditioner / 1 curtain / 1 fan / 1 humidifier / 2 sensors / 2 sockets), declared in `ha_config/mqtt/*.yaml`, reported by `aether-simulator` container via MQTT. The purpose is to let users without real smart home devices experience complete AI control process (chat to turn on lights, adjust air conditioning, etc.). > > **How to handle demo devices when connecting real HA devices**: > - Method 1 (recommended): Disable corresponding entities in HA UI "Settings → Devices and services → MQTT", or directly delete `ha_config/mqtt/*.yaml` and restart HA container. > - Method 2: Comment out `simulator` service and `aether`'s `depends_on: simulator` in `docker-compose.yml`, then `docker compose up -d`. > - Method 3: Keep demo devices, Aether will see both real and demo devices, use device names to distinguish during chat. - Application logs: `docker compose logs -f aether` - Stop: `docker compose down` (data retained in Docker volume and `logs/` mount directory) > **Camera**: Default uses RTSP network stream (`vision.rtsp_url`), container does not require special device permissions, as long as the camera IP is reachable in the container network. If using a local USB camera, need to add `devices: ["/dev/video0:/dev/video0"]` to `aether` service (only Linux hosts). ## Local development (without Docker) Suitable for hot reloading when modifying code. Backend uses uvicorn, frontend uses Vite development server: ```bash # Backend dependencies python -m pip install -r requirements.txt # Frontend dependencies cd frontend && npm install # Frontend development server (5173, proxy /api /ws to 8010) npm run dev # Backend (project root directory, open another terminal) python -m uvicorn app.main:app --host 0.0.0.0 --port 8010 # Frontend build and sync to backend static directory (before production deployment) npm run build ``` > Production deployment uses Docker: `docker compose up -d`, then access `http://localhost:8010` in the browser. Stop with `docker compose down`. ## Remote access from outside (Tailscale) Aether defaults to only use within the home LAN. Want to access from outside with a mobile phone, **do not expose port forwarding to the public**, use Tailscale (based on WireGuard's peer-to-peer VPN) for more security — only devices in your Tailscale network can connect. Core approach: 1. Install Tailscale on both computer and mobile phone, log in with the same account, each assigned a `100.x.x.x` intranet IP. 2. Backend already bound `0.0.0.0:8010` (listen to all network cards), no need to modify startup command. 3. Add a Windows firewall rule, **only allow Tailscale network segment `100.64.0.0/10`** access 8010: ```powershell New-NetFirewallRule -DisplayName "Aether Backend 8010 (Tailscale only)" ` -Direction Inbound -Action Allow -Protocol TCP -LocalPort 8010 ` -RemoteAddress 100.64.0.0/10 -Profile Any ``` 4. Mobile phone browser access `http://<computer's Tailscale IP>:8010/` (use `tailscale ip -4` to check computer IP) > **Access 8010, not 5173**: 5173 is the Vite development server, only listens to `127.0.0.1`, external devices cannot connect. 8010 simultaneously hosts frontend pages and API, is the port for daily use and remote access. Detailed steps, firewall profile pitfalls, and troubleshooting see [`docs/01-安装部署/Tailscale远程访问与防火墙配置.md`](docs/01-安装部署/Tailscale远程访问与防火墙配置.md). Backend CORS has pre-released Tailscale `100.64.0.0/10` network segment ( `app/main.py`'s `allow_origin_regex`). ## Configuration files | File | Effect | Included in repository | |------|------|:---:| | `.env` | API keys (LLM / STT / HA token, etc.), injected through environment variables | ✗ | | `config.json` | Application runtime configuration (LLM keys mapping, HA connection, visual, weather, etc.) | ✗ | | `.env.example` / `config.example.json` | Templates for the above two | ✓ | Environment variable priority is highest, can override `config.json`: | Environment variable | Overridden configuration | Purpose | |----------|-----------|------| | `HA_URL` | `ha.url` | Container points to `http://homeassistant:8123` | | `HA_TOKEN` | `ha.token` | HA long-term access token | | `LLM_ENABLED` / `LLM_BASE_URL` / `LLM_MODEL` | `llm.*` | LLM global switch and model | | `LOG_LEVEL` | `logging.level` | Log level | | `STARTUP_PROGRESS_HOST` | — | Startup progress port binding address (container internally set `0.0.0.0`) | > LLM keys recommended to manage with "Advanced" page's API Keys card, automatically write to `.env` and persist to database. ## Project structure ``` app/ ├── main.py # FastAPI entry: lifecycle, middleware, routing registration, SPA hosting ├── bootstrap.py # Service initialization (construct all service instances) ├── container.py # DI container (AppContainer, eliminate from main import global) ├── core/ # Configuration, database, authentication, rate limiting, exception, tracing ├── clients/ # HA / LLM (chat/vision/embed) HTTP clients ├── services/ # Business services (rules, scheduling, visual, weather, session, RAG, semantic graph…) ├── agents/ # LangGraph Agent, Dispatcher, Validator ├── mcp/ # MCP tools (local tools, external Server, tool executor) ├── routes/ # REST/WS routing (divided by function modules) ├── sg/ # Semantic graph pipeline (entity extraction, vectorization, relationship analysis, graph construction) ├── schema/ # Request/response schema └── data/ # SQLite database, JWT key, emoji vector index (generated at runtime) frontend/ # Vue 3 + Vite frontend ha_config/ # Home Assistant configuration (mounted to HA container /config; only track configuration templates, runtime state generated by HA) mosquitto/ # Mosquitto MQTT configuration tests/ # Backend pytest (740+ tests) frontend/tests/ # Frontend vitest docs/ # User-level and technical-level documentation (classified by function) ``` ## Testing ```bash # Backend pytest # All pytest -m "not slow" # Skip slow tests that require real API calls pytest tests/test_dispatcher.py # Single module # Frontend cd frontend && npm test ``` ## Documentation Detailed documentation in `docs/`, classified by function: - `docs/01-安装部署/` —— Environment preparation, Docker deployment, HA connection, LLM keys, weather API, Tailscale remote access - `docs/02-AI聊天/` —— Chat entry, personality customization, model role, session management, slash commands - `docs/03-设备控制/` —— Natural language control, device controls, device panels - `docs/04-自动化规则/` —— Scheduled tasks, automation rules, rule maintenance, visual triggers - `docs/05-摄像头视觉/` —— Camera connection, attention item configuration, motion detection - `docs/06-集成扩展/` —— Exa search, MQTT connection, external MCP - `docs/07-个性化/` —— Emoji customization, family information and theme - `docs/08-运维排查/` —— API authentication, log inspection, health check - `docs/tech/` —— Architecture overview, API/MCP reference, scheduling/automation engine, visual subsystem, configuration reference ## Interface navigation Four sidebar entries, other functions accessible through slash commands: | Entry | Description | |------|------| | **Butler** | Chat main interface, input `/` to view all slash commands (device, scheduled, model, etc.) | | **Settings** | Family information, region, dark mode | | **Advanced** | System-level configuration page: weather API, Exa search, visual parameters, HA connection, assistant role, API Keys (click card to pop up modal editing), and emoji index rebuild, document vector rebuild | | **Monitor** | System monitoring page, also accessible by chatting `/monitor` |