The 'Grok: Reference to Video' workflow in ComfyUI is designed to transform static reference images into dynamic video content. This workflow leverages the Grok API to create videos that maintain high visual consistency across frames, even when using up to six reference images. The process begins with the 'LoadImage' node, where you can upload your reference images. These images are then processed by the 'GrokVideoReferenceNode', which utilizes the Grok model to interpret and transform the static visuals into a coherent video sequence. Finally, the 'SaveVideo' node allows you to export the generated video for use in various applications.
What sets this workflow apart is its ability to handle multiple reference images, ensuring that the resulting video maintains a consistent style and character portrayal. This is particularly useful for projects requiring character animation or consistent artistic style across video frames. The Grok model's advanced capabilities allow for nuanced interpretation of the input images, making this workflow a powerful tool for creators looking to convert static art into engaging video content.
API
Use this workflow from code
Every Comfy workflow is a JSON graph. The payload below is this workflow, exactly as ComfyUI runs it — fetch it from the URL, keep it in version control, or load it in ComfyUI and run it node by node.
GET https://comfy.org/workflows/download/88f014a33e2a.jsonFetching workflow JSON…
Run it from Python or TypeScript with the Comfy SDK. The same code targets Comfy Cloud or a ComfyUI you host yourself — only the base URL changes.
# Install (beta)
pip install comfy-sdk # Python
npm i @comfyorg/sdk # TypeScript
# Run "Grok: Reference to Video" (Python)
from comfy_sdk import Comfy
client = Comfy(api_key="comfyui-...")
# This workflow, exported in API format (see note below)
wf = client.workflows.from_file("api_grok_reference_to_video_api.json")
asset = client.assets.from_file("input.png")
wf.set_input("6", "image", asset) # LoadImage
job = client.run(wf)
for output in job.get_outputs("4"): # SaveVideo
output.to_file(output.name)The SDK takes a workflow in API format: open this workflow in ComfyUI and use File → Export Workflow (API). API access requires a Comfy Cloud plan with an API key. SDK docs · Get an API key
FAQ
