Content
center">
<img src="https://img.shields.io/badge/Python-3.10+-blue?style=flat&logo=python" alt="Python">
<img src="https://img.shields.io/badge/MCP-Enabled-FF6B35" alt="MCP">
<img src="https://img.shields.io/badge/3D-Three.js-green?style=flat&logo=three.js" alt="3D">
<img src="https://img.shields.io/badge/Physics-pymunk-purple?style=flat" alt="Physics">
<img src="https://img.shields.io/badge/AI-MusicGen-red?style=flat" alt="AI Music">
<img src="https://img.shields.io/badge/AI-Stable%20Diffusion-orange?style=flat" alt="AI
<br>
<img src="https://img.shields.io/github/last-commit/hvrrgfe/aetherforge?style=flat&color=informational" alt="Last Commit">
<img src="https://img.shields.io/github/license/hvrrgfe/aetherforge?style=flat" alt="License">
</p>
<h1 align="center">⚡ AetherForge</h1>
<p align="center"><strong>A game engine for AI, for humans to watch</strong></p>
<p align="center">
AI like Codex and Claude interact with 53 tools via <b>MCP protocol</b><br>
Scene setup · Rule writing · Physics · Audio playback · 3D rendering · AI image and music generation
</p>
---
## 📋 Table of Contents
- [✨ Features](#-features)
- [📦 Installation](#-installation)
- [🚀 Quick Start](#-quick-start)
- [🛠 Tool List](#-tool-list)
- [🤖 Codex Integration](#-codex-integration)
- [🎨 AI Models](#-ai-models)
- [⚠️ FAQ](#️-faq)
- [📄 License](#-license)
---
## ✨ Features
| Module | Description |
|------|------|
| 🧠 **Semantic World Model** | Entities have types, descriptions, capabilities, states, and relationships - directly understandable by AI |
| ⚛️ **Physics Engine** | 2D physics driven by pymunk: gravity, collision, force, ray detection |
| 🔊 **Audio Engine** | pygame.mixer: sound effects, background music, volume control, fade in and out |
| 🎮 **3D Rendering** | Three.js WebGL: mesh, lighting, camera, scene export |
| 🖼️ **AI Image Generation** | 7 models: anything-v5 / SD / SDXLUX |
| 🎵 **AI Music Generation** | 5 models: MusicGen small / medium / large / melody / AudioGen |
🔄 **Transaction Rollback** | All operations can be, `commit_change` / `rollback_change` |
| 🧪 **Automated Testing** | One-click scene testing |
| 🔌 **MCP Protocol** | Native support for Codex / Claude Desktop and other MCP clients |
---
## 📦 Installation
### Requirements
- Python 3.10+
- 4GB+ memory (8GB+ for AI features)
### 1. Clone the Repository
```powershell
git clone https://github.com/hvrrgfe/aetherforge.git
cd aetherforge
```
### 2. Install Core Dependencies
```powershell
pip install -r requirements.txt
```
### 3. Install Optional Engines
<details>
<summary><b>Physics Engine</b> (Recommended)</summary>
```powershell
pip install pymunk
```
</details>
<details>
<summary><b>Audio Engine</b> (Recommended)</summary>
```powershell
pip install pygame
```
</details>
<details>
<summary><b>AI</b> (RequiresGB+ memory)</```powershell
pip --index-url https://download.pytorch.org/whl/cpu
pip install transformers
```
</details>
<details>
<summary><b>AI Music Generation</b> (Requires 8 memory, see FAQ)</summary>
```powershell
pip install torch torchvision --index-url https://download.pytorch.orgl/cpu
pip install audiocraft --no-build-isolation --no-deps --no-cache-dir
pipus flashy hyd hydra_colorlogwords tqdm
pip install spacy transformers huggingface
pip install librosa torchmetrics xformers
```
</details>
### 4. Update Project
Regularly update to get the latest features and fixes:
```powershell
# the latest code
# ② Synchronize and install new dependencies
-r requirements.txt
# ③ If necessary, update the configuration file (keep your custom configuration)
python -c "from aetherforge.config import get_config, save_config; save_config(get_config()); print('Config updated')"
```
After completing, restart the engine.
### ✅ 5. Verify Installation
```powershell
python -c "from aetherforge.api.engine_v2 import EngineToolsV2; from aetherforge.core.world_model import WorldModel; t =V2(WorldModel()); print('OK:', len(t.list_tools().data['tools']), 'tools')"
```
## 🚀 Quick### MCP Server (Interface)
```powershell
python -m aetherforge_server
# OutputetherForge MCP Server.0.0 started - waiting for MCP clientAfter starting, connect to the MCP client (Codex Desktop, etc.) to call all 56 tools.
### Web Application (for humans)
```powershell
python -m aetherforge.main --demo
```
| Endpoint | Address |
|----------|-----|
| Game Screen | http://localhost:7890/ |
| 3D Screen | http://localhost:7890/viewer-3d |
| Tool List | http://localhost:7890/api/tools/ |
| World State | http://localhost:7890/api/observe |
### Command Line
```powershell
# List all tools
python -m aetherforge.cli --tool list_tools
# Create an entity
python -m aetherforge.cli --tool create_entity --params '{"semantic_type":"npc","name":"Guard"}'
# Observe world state
python -m aetherforge.cli --tool observe
# Get engine information
python -m aetherforge.cli --tool get_engine_info
```
---
## 🤖 AI Integration Guide
AetherForge interacts with AI clients through the **MCP protocol**. Here are the configuration methods for different platforms.
### Codex / codex-cli
Configuration location:
- Codex Desktop: ~/.codex/mcp.json
- codex-cli: Project root directory .mcp.json
```json
{
"mcpServers": {
"aetherforge": {
"commandpython",
" ["-m", "aetherforge.mcp_server"],
"cwd": "D:\\aetherforge"
}
}
}
```
### Claude Desktop
Open Claude → Settings → Developer → Edit MCP Configuration:
```json
{
"mcpServers": {
"aetherforge": {
"command": "python",
"args": ["-m", "aetherforge.mcp_server"],
"cwd": "D:\\aetherforge"
}
}
}
```
### Cursor / Windsurf
Add to the project .cursor/mcp.json:
```json
{
"mcpServers": {
"aetherforge": {
"command": "python",
"args": ["-m", "aetherforge.mcp_server"],
"cwd": "D:\\aetherforge"
}
}
}
```
### Quick Test Connection
After configuration, ask in the AI client:
> “Connect to AetherForge engine and see the current world state”
AI will call the observe tool through MCP to get a world snapshot. Returns similar to:
```json
{
"tick": 0,
"game_time": "0.0s",
"weather": "clear",
"entity_count": 0,
"rule_count": 0,
"quest_count": 0
}
```
### Suggested AI Prompts
When using AetherForge with AI, you can give it this prompt:
> You are a user of the AetherForge game engine. Your goal is to create and manage a game world through MCP tools.
>
> **Workflow:**
> 1. Use observe to view the current world state
> 2. Use create_entity to create entities (players, NPCs, items, scenes)
> 3. Use set_weather to set the environment
> 4. Use create_rule to add game rules
> 5. Use create_quest to design tasks
> 6. Use set_behavior to give NPCs behavior
> 7. After each modification, use observe to confirm the effect
> 8. When satisfied, use commit_change to submit
## 🎨 AI Models
### Image Generation
The engine automatically scans the `models/` directory and HuggingFace cache, supporting 7 models:
Model | Parameters | Minimum VRAM | Description |
|------|------|----------|------|
| `anything-v5` | — | | Anime style, suitable for game assets |
|-1.5` | 860M |GB | Stable Diffusion 1.5 |
| `sd-2.1` | 860M | 4GB | SD 2.1 quality improvement |
| `sdxl` | 2.6B | 8GB | SDXL high quality |
| `sd3` | 2.6B | 8GB | Stable Diffusion 3 |
| `fluxell` | 3.5B |GB | FLUX fast version |
| `flux-dev` | 12B | 12GB | FL-quality version |
### Music Generation
| Model | Parameters | Description |
|------|------|------|
| `musicgen-small` | 300M | Fastest, suitable for sound effects |
| `musicgen-medium` | 1.5B | Balanced quality/speed `musicgen-large` | 3.3B | Best quality, requires 8GB+ |
| `musicgen-melody` | 1.5B | Melody conditional generation |
| `audiogen` | 1.5B | Focus on sound effect generation |
###
```powershell List available models
python -m aetherforge.cli --tool_models
python -m aetherforge.cli list_music_models
# Switch model (name path)
python -m aetherforge.cli --tool select_image_model --params '{"name_or_path":"sdxl"}'
python -m aetherforge.cli --tool select_music_model --params '{"name_or_path":"musicgen-large"}'
image
python -etherforge.cli generate_image --params":"a rusty iron door"}'
```
--- 📁 Local Model Placement Guide
The engine `models/` directory automatically; just put models in it.
### Directory Structurebash
aetherforge/
├── models/ # ← Engine scans this directory ├── anything-v5/ # Model name can be arbitrary; engine recognizes automatically
│ │ ├── model_index.json # Diffusers format must have this file
│ │ ├── unet/
│ │ ├── vae/
│ │ └── text_encoder/
│ ├── sdxl-base-1.0/ # SDXL model similarly
│ ├── my-custom-model/ # Your custom model
│ │ ├── model_index.json
│ │ └── *.s
│ └── musicgen-medium/ # Music model (directory name contains musicgen)
│ └── ...
└── ...
```
---
### Supported Model Formats
| Format | Description | Required Files |
| --- | --- | --- |
| **Diffusers** | Complete pipeline directory | `model_index.json` + `unet/`, `vae/`, `text_encoder/` subdirectories |
| **SafeTensors** | Single or multiple `.safetensors` files | At least one `.safetensors` file |
| **HuggingFace Cache** | HF automatically downloads to cache directory | No manual action required, `~/.cache/huggingface/hub/` auto-recognized |
### Placement Steps
**① Create `models/` Directory**
```powershell
# Create in project root directory
mkdir models
```
**② Place Model**
- **anything-v5**: Place complete model folder in `models/anything-v5/`
- **Download from HuggingFace**:
```powershell
# Method A: Download using huggingface-cli
pip install huggingface-hub
huggingface-cli download stablediffusionapi/anything-v5 --local-dir models/anything-v5
# Method B: Copy existing folder
Copy-Item -Recurse D:\path\to\your\model models\my-model
```
**③ Verify Recognition**
```powershell
python -m aetherforge.cli --tool list_image_models
python -m aetherforge.cli --tool list_music_models
```
The engine will output similar to:
```plaintext
Available image models: anything-v5 (local), sdxl (downloadable)...
Available music models: musicgen-medium (local), musicgen-small (downloadable)...
```
### Manual Path Specification
If the model is not in the standard path, specify it through configuration:
```yaml
# ~/.aetherforge/config.yaml
image_gen:
model_path: "D:/my_models/sd-model"
music_gen:
model_id: "facebook/musicgen-small"
```
Or pass the path directly in the tool:
```powershell
python -m aetherforge.cli --tool select_image_model --params '{"name_or_path":"D:/my_models/sd-model"}'
```
### Engine Auto-Recognition Logic
- **Image Models**: Scan `models/` subdirectories for `model_index.json` or `*.safetensors`
- **Music Models**: Scan folders with `musicgen` in the name
- **HF Cache**: Auto-recognize `~/.cache/huggingface/hub/models--*`
- **anything-v5**: Built-in recognition, place in `models/anything-v5/`
- **Fallback**: If not found locally, download from HuggingFace (requires internet)
---
## ⚠️ Frequently Asked Questions
<details>
<summary><b>audiocraft installation failure: av compilation error LNK1181</b></summary>
```plaintext
error: Failed building wheel for av
LINK : fatal error LNK1181: cannot open input file avformat.lib
```
**Cause:** `audiocraft` locks `av==11.0.0`, which has no Windows pre-compiled wheel.
**Solution A: Use pre-installed av + skip isolation compilation (recommended)**
```powershell
pip install av>=12.0.0
pip install audiocraft --no-build-isolation --no-deps --no-cache-dir
pip install julius flashy hydra-core hydra_colorlog num2words tqdm
pip install spacy transformers huggingface_hub
pip install librosa torchmetrics xformers
```
**Solution B: Install FFmpeg SDK (complete compilation)**
```powershell
choco install ffmpeg # requires admin privileges
pip install audiocraft
```
</details>
<details>
<summary><b>MCP Server reports Internal Server Error</b></summary>
```plaintext
Received exception from stream: 1 validation error for JSONRPCMessage
{"method":"notifications/message","params":{"data":"Internal Server Error"}}
```
**Cause:** HuggingFace network timeout causes MusicGen model download to stall (old version), or terminal sends empty line to stdin.
**Solution:** Update to the latest version:
```powershell
git pull
```
New version `init_music_gen` no longer blocks download. Returns error prompt if no network.
</details>
<details>
<summary><b>HuggingFace network timeout</b></summary>
If the environment cannot access huggingface.co, model auto-download will fail.
**Solution:**
1. Pre-download model on a machine with internet, copy to `models/` directory
2. Or use local path: `select_model("D:/path/to/local/model")`
</details>
---
## 📄 Open-Source License
MIT License
---
<p align="center">
<sub>AetherForge — Let AI not need to learn traditional engines, directly use intent, semantics, tools, and feedback to create games.</sub>
</p>
MCP Config
Below is the configuration for this MCP Server. You can copy it directly to Cursor or other MCP clients.
mcp.json
Connection Info
You Might Also Like
No recommendations available