my-comfyui

# ComfyUI Workflow Tool ## What is this project for - Provides import and export tools to facilitate importing and exporting workflows in ComfyUI. Also provides a scanning tool to delete models not used by all workflows. - Based on the exported workflow, provides the ability to call workflows on ComfyUI through API, making it convenient for other applications to use. - Based on the exported workflow and written workflow description files, provides ComfyUI's calling capability to LLM/Agent through the MCP protocol, realizing image generation through dialogue. - Shares and backs up some practical workflows, see [Sharing](#tool-list) ## Using this project - Clone the project to local - Install dependencies (using `uv sync` to install) - Modify configuration (copy `.env.example` to `.env` and modify the configuration) - Run the project (using `uv run src.main parameter` to run the project) ### Workflow Management - `python -m src.main workflow list`: List all workflows - `python -m src.main workflow export`: Export workflow to backup directory - `python -m src.main workflow import`: Import workflow from backup directory - `python -m src.main workflow info XXX`: View models referenced by workflow XXX, including VAE, LoRA (may contain interference items) ### Model Management - `python -m src.main model scan`: Scan model directory, list all model files, and delete unused model files after confirmation through console ### API Adapter ComfyUI's API call requires providing workflow json. This project provides an API server to receive external requests. The program forwards it to ComfyUI service to realize ComfyUI workflow interface call. Note that ComfyUI does not provide API documentation. This project's implementation is based on web-side calls and ComfyUI code analysis, such as [server.py](https://github.com/Comfy-Org/ComfyUI/blob/master/server.py) The design is only suitable for calling some simple workflows with few input parameters, making it convenient for integration in Agent and other scenarios. For complex workflows and parameter passing, support is limited. If you need to use it, it is recommended to check [known issues](#known-issues) to ensure that your scenario can accept these issues. #### Steps to use this feature - Confirm that [known issues](#known-issues) can be accepted or solved - Ensure ComfyUI service is started - The workflow to be used has been tested and passed in ComfyUI, and exported to the backup directory through this program (孤 nodes do not need to be deleted, and the program will ignore them by default, such as markdown note nodes) - Open the `.json` file of the workflow in the API directory (not the backup file), replace the keyword, and refer to the interface part below for details - `python -m src.main server`: Start API forwarding server #### Supported Interfaces - `POST /submit`: Submit task request - Request body: `requestBody` - `workflow`: The workflow name to be used, i.e., the file name in the backup directory (without `.json` suffix) - `kwargs`: Workflow parameters, key-value pair format, `{{param_name}}` in workflow json file will be replaced with the corresponding field value in request parameter `.kwargs` object, regardless of its level (direct string replacement) - Response body: `responseBody`, returns immediately after submission, without waiting for task completion - `prompt_id`: ComfyUI task ID, used to query task progress and download output - `number`: ComfyUI returned sequence number - `node_errors`: Error message returned by ComfyUI when error occurs - `GET /progress/{prompt_id}`: Query task details - Request parameter (url): - `prompt_id`, ComfyUI task ID, from `prompt_id` field in response body of `/submit` interface - Response body: `responseBody`, returns immediately, regardless of task completion - `status`: Task status, `completed` indicates task completion, `in_progress` indicates task in progress, `failed` indicates task failure - `output`: A list containing all outputs of workflow, passing through ComfyUI return, may contain image, text and other types. You can remove redundant preview nodes to get more concise output. Each element contains the following fields: - `type`: Output type, such as `images`, `text`, etc. - `res`: Output result, if `text`, returns a string, if `images`, returns an object containing fields required by `download` interface - `outputs_count`: Number of task output materials - `GET /download`: Download task output - Request parameter (query): all from response body of `/progress` interface - `filename`, file name to be downloaded - `subfolder`, subfolder where file to be downloaded is located - `type`, type of file to be downloaded #### Known Issues 1. Missing parameters  - Problem phenomenon: successful submission, but query task progress is failed, and error message in ComfyUI is a field name, such as `select_style` in the figure, without any other information - Problem cause: used node contains parameters that cannot be controlled by connection, such as style selector in the figure. Due to the characteristics of such parameters, parameters are lost when saving workflow (parameter name is also lost, and default value cannot be written), and cannot be passed when converting to API, resulting in error - Problem solution: - Solution 1 (recommended): modify workflow to remove uncontrollable parameters or replace with other controllable nodes (if available) - Solution 2: manually export workflow in ComfyUI in API format, replace parameter values in json file generated by program, and replace seed parameter value with `{_SEED_}` to call (same seed, ComfyUI will skip execution), but need to re-export every time ### MCP Support This project provides MCP service implementation (HTTP way), enabling LLM/Agent to call ComfyUI workflow. #### Usage  1. Complete all steps of API way, because MCP way still needs to use API to call ComfyUI, so all pre-work is required. 2. Create Schema file for workflow to be used through MCP 1. Schema file must have same name as corresponding file in `api` directory, but use `yml` format 2. In yml file, must configure `name` (workflow name), `description` (workflow description), `arg_schema` (parameter schema) fields. `arg_schema` is a JSON schema object, directly passed to LLM/Agent, without explicit format requirements, just describe parameter requirements clearly. Note that parameter name must match parameter name used in `api`. Refer to `workflow/mcp/Nahida(NSFW).yml` file.  3. Start MCP service using `python -m src.main mcp-server`, listening port is `8181` (allow configuration), currently does not allow MCP service and API service to start simultaneously, need to modify if needed. 4. Configure MCP service in LLM/Agent, connect in `HTTP` way, address is `http://localhost:8181`. 5. Can use after connection. #### Provided Tools 1. Query available workflows: scan `mcp` directory, read each `yml` file, return names and descriptions of all workflows, directly provide to LLM/Agent. 2. Submit generation task: same as API way, response directly returned to LLM/Agent. 3. Query task progress: slightly different from API way. In MCP way, directly return ComfyUI's download link, no longer forward through server. LLM/Agent only gets download link. If network environment can access ComfyUI service, can directly download, otherwise need to expose ComfyUI service port or enable forwarding service.  ## Tool List ### Image Generation #### realistic > Note: model has been updated, recommended to use new model Workflow using [RealVisXL](https://civitai.com/models/139562/realvisxl-v50?modelVersionId=789646) Realistic style, suitable for landscape Has hand problem, relatively serious <img height="300px" src="doc/real.png" /> #### SD1.5(NSFW) Workflow using [cuteyukimixadorable](https://civitai.com/models/28169/cuteyukimixadorable-style) Unique painting style, suitable for generating cute cartoon-style character images, can visit model page to see more examples Has hand problem but controllable Does not recognize most ACG characters BaseModel: SD 1.5 <img height="300px" src="doc/sd1.5.png" /> #### Nahida(NSFW) Workflow using [zukiCuteILL_v60](https://huggingface.co/John6666/zuki-cute-ill-v60-sdxl) Not very "safe" model, also can be used for ordinary image generation scenarios. Model page on C station has been hung, [author's C station link](https://civitai.com/user/ZU_KI) Painting style is relatively cute, suitable for generating cute, cartoon-style character images, good at NSFW content Recognizes most ACG characters BaseModel: SDXL <img height="300px" src="doc/nahida.png" /> ### Image Editing #### Regional Redrawing Inpaint Use mask to redraw specific area of image Suitable for repairing, modifying, replacing small part of image Not suitable for completing complex drawing tasks in redrawn part The following is an effect of replacing ground with water surface full of flowers through mask guidance <img height="300px" src="doc/inpaint.png" /> #### Image Editing Qwen2509 Perform image editing through pure prompt guidance, effect is very good Can reference up to three images, but may have confusion problems Can perform slightly complex editing operations The following is an effect of removing door in image through prompt <img height="300px" src="doc/edit.png" /> #### Image Watermark Removal Automatically identify watermark through model and redraw through mask to remove watermark Watermark recognition effect is okay Redrawing effect is general, can try other models to redraw <img height="300px" src="doc/water.png" /> #### Image Magnification Magnify 4 times, simple and effective, magnification effect is good <img height="300px" src="doc/up.png"> ### 3D > 3D is less played, and no good workflow has been found #### Image to 3D Model Template workflow, effect as shown <img height="300px" src="doc/3d.png" /> ### Audio Processing #### Music Generation (ACE1.5) Currently, ACE1.5's effect sounds less good than Suno's V3.5 (earlier versions not tried), only for fun, not recommended In open-source solutions, HeartMula 3B effect is better (subjectively equivalent to Suno V3.5, 7B not open), but ComfyUI integration is not good #### Voice Cloning Complete voice cloning and speech generation process in one workflow Cloning effect is good, such as [**this audio**](doc/civilization.mp3) using Bellona's (from Arknights) voice, reading opening words of Civilization 6 ### Video Processing #### Wan2.2 Image to Video Template workflow, generate video with image + prompt Video may have static problem, weird scenes, etc. (relatively better than Wan2.1) Generation speed is slow, not recommended to generate too long video Video effect can refer to: [0 cost to get started? Make a beautiful but grassy MV for two-dimensional sisters](https://www.bilibili.com/video/BV18GQQBoEGx/?share_source=copy_web) (this video is output of this workflow, edited to get) #### Wan2.2 First and Last Frame Video Template workflow, generate video with first and last frame images + prompt Video easily has static, strange transitions, etc., relatively better than Wan2.1 On 16G显存 device, `640*640` 5s video needs 733s, also provides faster version (quality will decrease) The following is usage effect, transition is a bit weird:  Wan2.1 First and Last Frame Video effect can refer to: [Can music be used as programming language? Listen to code what melody](https://www.bilibili.com/video/BV12xKWz5EnJ/?share_source=copy_web) (video in bottom right corner is Wan2.1's first and last frame video spliced to end) #### hunyuan Prompt Directly to Video Template workflow, generate video with prompt only Video effect is general, not deeply used <img height="300px" src="doc/hunyuan.webp" /> #### Audio + Image + Prompt to Video Relatively practical workflow, can make person in image open mouth according to dubbing Also based on Wan2.2 workflow Real person image situation can also match lip Can make action according to prompt  Extend according to chunk, add one node to extend one chunk, to realize long video generation. But test found that generating too long video may have some weird situations, recommended not to generate video over one minute. [**Effect video**](doc/civilization.mp4) (using above audio, add green screen Bellona image to generate)