Content
# TD-Codex Ultimate MCP
This repository contains the TD-Codex Ultimate MCP system - an open-source AI control layer for TouchDesigner that allows Claude, Codex, Cursor, VS Code, and other MCP-compatible agents to:
- Understand TouchDesigner
- Generate networks from creative prompts
- Modify existing TouchDesigner projects
- Debug errors and validate scene integrity
- Analyze and optimize performance
- Create show-control systems (Art-Net, sACN, OSC, MIDI)
- Build pixel mapping and LED display workflows
- Integrate with external media servers (Millumin, Resolume, Unreal, Notch)
- Self-correct using vision feedback from screenshots
- Build reusable components and templates
## What Has Been Implemented
Based on the implementation plan in `IMPLEMENTATION_PLAN.md`, the following components have been implemented:
### ✅ Already Present in td-codex-ultimate:
1. **Runtime Engine** (`td-bridge/TD_mcp/`)
- Hundreds of direct TouchDesigner control tools
- Node creation/deletion, parameter control, connections, execution, DAT access, layout/styling
2. **Knowledge Engine** (`packages/knowledge/`)
- Operator database and reference materials
3. **Compiler Engine** (`td-bridge/TD_mcp/modules/mcp/services/domain_builders.py`)
- Blueprint generation from high-level goals
4. **Validator** (`td-bridge/TD_mcp/modules/mcp/services/api_service.py`)
- Scene validation and checking
### 🔧 Recently Added Systems:
1. **Vision Feedback System** (`packages/vision/`)
- Screenshot capture and analysis
- Image comparison and correction suggestions
- Visual feedback loop for self-correction
2. **Performance Analyzer** (`packages/performance/`)
- Real-time GPU/CPU monitoring
- Bottleneck detection and optimization recommendations
- Performance reporting and trend analysis
3. **Show Control System** (`packages/showcontrol/`)
- Art-Net, sACN, OSC, MIDI tools
- Timecode (LTC/MTC) synchronization
- Network protocol managers
4. **LED & Pixel Mapping System** (`packages/led-mapping/`)
- LED wall and voxel grid creators
- Pixel mapping tools
- Universe management and channel map export
5. **Media Server Integration** (`packages/media-server/`)
- Millumin, Resolume, Notch, Disguise connectors
- Bidirectional communication bridges
### 🔄 Enhanced MCP Server
The enhanced MCP server (`packages/mcp-server/enhanced_server.js`) now implements the full architecture flow:
- Prompt → Knowledge → Compiler → Validator → Runtime
- Optional Vision Feedback for self-correction
- REST API endpoints for all services
- Standard MCP JSON-RPC endpoint for compatibility
## Files Created/Updated
### New Service Files:
- `packages/vision/vision-service.js`
- `packages/performance/performance-service.js`
- `packages/showcontrol/showcontrol-service.js`
- `packages/led-mapping/led-mapping-service.js`
- `packages/media-server/media-server-service.js`
### Updated Files:
- `packages/mcp-server/enhanced_server.js` - Added all service integrations
## Usage
To use the enhanced MCP system:
1. Start the MCP server:
```bash
cd packages/mcp-server
npm start
```
2. Connect your MCP client to:
```
http://127.0.0.1:3000/mcp
```
3. Use the creative prompt endpoint for full workflow:
```
POST http://127.0.0.1:3000/mcp/creative-prompt
```
## Vision Feedback Integration
The vision feedback system is now integrated into the creative prompt endpoint. To use it:
- Set `useVisionFeedback: true` in your request
- The system will analyze the output and suggest corrections if needed
## Performance Monitoring
Performance metrics can be collected and analyzed through:
- `POST /performance/collect-metrics` - Collect current metrics
- `POST /performance/analyze` - Analyze trends and detect issues
- `POST /performance/get-report` - Get comprehensive performance report
## Show Control
Create show control outputs:
- Art-Net: `POST /showcontrol/create-artnet-output`
- sACN: `POST /showcontrol/create_sacn_output`
- OSC Receiver: `POST /showcontrol/create-osc-receiver`
- OSC Sender: `POST /showcontrol/create-osc-sender`
- MIDI Input: `POST /showcontrol/create-midi-input`
- MIDI Output: `POST /showcontrol/create-midi-output`
- Timecode Sync: `POST /showcontrol/setup-timecode-sync`
## LED & Pixel Mapping
Create LED and pixel mapping systems:
- LED Wall: `POST /ledmapping/create-led-wall`
- LED Strip: `POST /ledmapping/create-led-strip`
- Voxel Grid: `POST /ledmapping/create-voxel-grid`
- Pixel Mapper: `POST /ledmapping/create-pixel-mapper`
## Media Server Integration
Connect to media servers:
- Millumin: `POST /media/connect-millumin`
- Resolume: `POST /media/connect-resolume`
- Notch: `POST /media/connect-notch`
- Disguise: `POST /media/connect-disguise`
## License
MIT
MCP Store