AMS.

My first day with ComfyUI: from a selfie to a 20-second aquarium shot

5 min listen

Cover: A large school of silver fish with dark markings swimming through deep blue water in a spacious aquarium. by Faisal Ahammad, CC0 1.0.

Background

After installing ComfyUI for a friend I couldn't resist trying it myself. I don't work in VFX and knew nothing about nodes or models, so I started from ComfyUI's built-in templates: LTX-2.5 for image to video and FLUX.2 Klein for image editing. My graphics card has 16 GB of VRAM, and a 5-second clip took about 100 seconds.

What I tried

  • A selfie with the template's own prompt: the clip started with me at a restaurant door and then cut to a cyberpunk rooftop. Image to video means "make the first frame move", so when the prompt describes a different place, the AI does both.
  • Edit the photo first, then animate it: an image-edit prompt saying "turn him into a cyberpunk character" gave me a movie hero who looked nothing like me. Rewriting it as a list ("keep his face, change only these things") got much closer.
  • A full-body shot with new clothes and a new location: no likeness at all. In a full-body frame the face is only a few dozen pixels, far too little for the AI to keep the details that make it you.
  • Changing tack: a photo without a face: me from behind at an aquarium. With no face to keep, the AI could put all its effort into the fish and the camera. This worked best.

How I did it

1. The first frame decides everything

Location, distance and aspect ratio all come from the first frame. My first aquarium clip showed only my head and shoulders: the aspect ratio was still 16:9, so a very tall portrait photo was cut into a horizontal strip from the middle. I cropped the photo to 9:16 (trimming only the top of the tank and keeping my feet) and set the aspect ratio to 9:16.

2. Describe what's already in the frame first

First the person, clothes and setting, then one or two motions and one camera move, and finally "Same location throughout, no scene change, no cuts". LTX-2.5 also generates sound, so the prompt can end with what you want to hear.

3. Control the camera with words

For a wide shot, write "wide shot, the camera keeps the same distance". To stop it pushing in too far, write "stays at a medium distance, always showing the whole sardine ball", because fish seen too close fall apart.

4. For 20 seconds, move the camera instead of cutting

The AI tends to warp at a hard cut. I wrote four beats with "first... then... finally...": a slow push in on the wide view, then tilt up to follow the manta ray, then drift back down to the sardine ball as a shark swims through it, then pull back to finish. About five seconds each, so it never feels rushed.

One continuous take, smooth camera movement, no hard cuts.
At the start: <wide description of the first frame>. The camera drifts slowly toward <subject> but stays at a medium distance.
Then <second subject> passes; the camera gently tilts and pans to follow it.
Then the camera drifts back to <main subject>.
At the end, the camera slowly pulls back to the wide view.
The person does not move. Same location throughout, no scene change, no text.
<sound description>.

Things that bit me

  • Keeping a real face is the hardest part: asking the AI to "become a character" makes it invent a new person. "Keep the face, change only...", a waist-up framing and a higher resolution all help; for full-body likeness you need a dedicated face-swap tool or a LoRA trained on your face. That's also why VFX artists pair AI with Nuke.
  • Settings I forgot to change: I thought I had set 20 seconds and portrait, but the result was 5 seconds and landscape. Check duration and aspect ratio before every run.
  • Be specific about the sardines: a vague "fish" gave me a crowd of big fish. "A massive swirling ball of thousands of small silvery sardines with dark spots along their sides" gave me the ones I wanted.
  • Game screenshots as references: fine for playing around, but don't copy existing characters or locations for commercial work. And tick "Add AI label" when you post.

What improved

At first Later
Location Restaurant door, then a sudden cyberpunk rooftop The same place from start to end
Framing Cut down to head and shoulders Whole body in frame
Fish A crowd of big fish A sardine ball, a manta ray and a shark
Length and camera 5 seconds, pushing in the whole time 20 seconds, four natural camera beats

Let AI apply it for you

If you want to turn a photo into an AI video, paste this into an AI chat along with your photo:

Help me write an image-to-video prompt for ComfyUI (LTX-style models) from the attached photo:

1. Describe exactly what is already in the photo: subject, clothes, setting, lighting.
2. Add only one or two motions and one camera move. For clips over 10 seconds, use "first... then...
   finally..." beats with continuous camera moves instead of cuts.
3. Control distance with words: "wide shot, the camera keeps the same distance" or "stays at a medium
   distance"; avoid "close-up" or "push in" unless I ask.
4. End with: "Same location throughout, no scene change, no cuts, no text." and a short sound description.
5. Tell me which aspect ratio and duration to set so the photo is not cropped.

Comments