This ComfyUI workflow is designed to create a lip-synced video by combining a static character image with an audio file. Utilizing the power of the Wan2.1 InfiniteTalk model, the workflow analyzes the uploaded audio to generate mouth movements that match the spoken words, creating a realistic lip-sync effect. The process begins with the 'LoadAudio' node, which ingests the audio file and prepares it for analysis. Simultaneously, the 'LoadImage' node allows users to upload the character image that will be animated. The workflow leverages custom nodes like 'ComfyUI-WanVideoWrapper' and 'audio-separation-nodes-comfyui' to process the audio and synchronize it with the visual elements. The final output is a video file, saved using the 'SaveVideo' node, where the character's mouth movements are in perfect harmony with the audio track.
Technically, this workflow excels by utilizing advanced audio analysis and video generation techniques. The 'audio-separation-nodes-comfyui' node is crucial for breaking down the audio into components that can be mapped to specific mouth shapes, while the 'comfyui-kjnodes' handle the intricate task of blending these movements into the character's image. This workflow is particularly useful for content creators looking to produce animated videos without the need for complex animation software, making it an accessible tool for enhancing storytelling and engagement through visual media.
API
Use this workflow from code
Every Comfy workflow is a JSON graph. The payload below is this workflow, exactly as ComfyUI runs it — fetch it from the URL, keep it in version control, or load it in ComfyUI and run it node by node.
GET https://comfy.org/workflows/download/1eab7aa23f6a.jsonFetching workflow JSON…
Run it from Python or TypeScript with the Comfy SDK. The same code targets Comfy Cloud or a ComfyUI you host yourself — only the base URL changes.
# Install (beta)
pip install comfy-sdk # Python
npm i @comfyorg/sdk # TypeScript
# Run "Audio-Driven Character Lip Sync Video" (Python)
from comfy_sdk import Comfy
client = Comfy(api_key="comfyui-...")
# This workflow, exported in API format (see note below)
wf = client.workflows.from_file("templates-wan2_1_infinitetalk_music_api.json")
asset = client.assets.from_file("input.png")
wf.set_input("284", "image", asset) # LoadImage
job = client.run(wf)
for output in job.get_outputs("325"): # SaveVideo
output.to_file(output.name)The SDK takes a workflow in API format: open this workflow in ComfyUI and use File → Export Workflow (API). API access requires a Comfy Cloud plan with an API key. SDK docs · Get an API key
FAQ
