The Wan 2.1 ControlNet workflow is designed to generate videos that are guided by pose, depth, and edge controls. This workflow leverages the capabilities of the Wan2.1 model, which is known for its robust handling of video-to-video transformations. The core of the workflow is built around several key nodes such as KSampler, CLIPTextEncode, and VAEDecode, which work together to process input video frames and apply stylistic transformations based on user-defined prompts. The use of ControlNet allows for precise control over video characteristics, making it possible to manipulate aspects like pose and depth with high accuracy.
Technically, the workflow begins by loading models and preparing the initial frame using nodes like CLIPVisionLoader and LoadImage. The WanFunControlToVideo node is crucial as it integrates ControlNet's functionalities to guide the video transformation process. The Sampling & Decode group, which includes nodes like ModelSamplingSD3 and UNetTemporalAttentionMultiply, ensures that the output video maintains high quality and coherence across frames. This workflow is particularly useful for creators looking to produce stylized videos with specific visual effects, offering a high degree of customization and control over the final output.
API
Use this workflow from code
Every Comfy workflow is a JSON graph. The payload below is this workflow, exactly as ComfyUI runs it — fetch it from the URL, keep it in version control, or load it in ComfyUI and run it node by node.
GET https://comfy.org/workflows/download/524727db7cac.jsonFetching workflow JSON…
Run it from Python or TypeScript with the Comfy SDK. The same code targets Comfy Cloud or a ComfyUI you host yourself — only the base URL changes.
# Install (beta)
pip install comfy-sdk # Python
npm i @comfyorg/sdk # TypeScript
# Run "Wan 2.1 ControlNet" (Python)
from comfy_sdk import Comfy
client = Comfy(api_key="comfyui-...")
# This workflow, exported in API format (see note below)
wf = client.workflows.from_file("wan2.1_fun_control_api.json")
asset = client.assets.from_file("input.png")
wf.set_input("52", "image", asset) # LoadImage
wf.set_input("6", "text", "a women sticking out her toungue while mkaing wierd faces,(clay style:1.1)") # CLIPTextEncode
job = client.run(wf)
for output in job.get_outputs("78"): # SaveVideo
output.to_file(output.name)The SDK takes a workflow in API format: open this workflow in ComfyUI and use File → Export Workflow (API). API access requires a Comfy Cloud plan with an API key. SDK docs · Get an API key
FAQ
