This workflow turns a single image into a music-backed, audio‑reactive video by combining text-to-music generation (ACE Step 1.5 XL Turbo) with image-to-video animation (LTX 2.3) enhanced by an Audio‑Reactive LoRA. You provide one image and a short creative brief; the workflow uses a GeminiNode to draft both "style tags" for the visuals and "lyrics" for the song in one pass. A RegexExtract node then splits that text at '---': everything before becomes visual style tags for LTX 2.3, and everything after becomes lyrics fed to ACE Step 1.5 XL Turbo. The generated song is saved via SaveAudioMP3 and then drives the reactivity of the video, which is saved with SaveVideo.
Technically, the pipeline is organized into groups: Load Image, Create ACE Step 1.5 Prompts, Create Audio‑Reactive LoRA Prompt, Regex Extract, ACE Step 1.5 XL, and LTX 2.3 with Audio‑Reactive LoRA. The Audio‑Reactive LoRA (see the linked model card) adds motion effects that follow the music; adjust its strength between 1.0 and 2.0 (1.4+ for stronger effects) for best results. PreviewAny helps you check intermediate outputs (prompts, audio, and frames) before final render, while MarkdownNote contains quick tips inside the graph. The result is a streamlined image → song → audio‑reactive video pipeline that’s easy to iterate: tweak style tags and lyrics, re‑generate the track, and re‑render the video until it matches your creative intent.
API
Use this workflow from code
Every Comfy workflow is a JSON graph. The payload below is this workflow, exactly as ComfyUI runs it — fetch it from the URL, keep it in version control, or load it in ComfyUI and run it node by node.
GET https://comfy.org/workflows/download/977fefa2b934.jsonFetching workflow JSON…
Run it from Python or TypeScript with the Comfy SDK. The same code targets Comfy Cloud or a ComfyUI you host yourself — only the base URL changes.
# Install (beta)
pip install comfy-sdk # Python
npm i @comfyorg/sdk # TypeScript
# Run this workflow (Python)
from comfy_sdk import Comfy
client = Comfy(api_key="comfyui-...")
wf = client.workflows.from_file("workflow_api.json")
job = client.run(wf)
for output in job.get_outputs("<output-node-id>"):
output.to_file(output.name)The SDK takes a workflow in API format: open this workflow in ComfyUI and use File → Export Workflow (API). API access requires a Comfy Cloud plan with an API key. SDK docs · Get an API key
FAQ














