Week 5: Text to Video

The third generator — the four-sentence prompt that controls motion directly from words, a model's native clip length, and how to read a take that failed.

22 March 2027

Text to video generates motion from words alone, with nothing to anchor composition but the prompt itself. That’s weaker control than starting from a keyframe, but it’s the fastest way to test an idea before committing an image to it, and every video generator this course uses has a native clip length — the duration one pass reliably holds together before coherence starts to fall apart — that the prompt has to respect rather than fight.

The four sentences

t3four sentences

A usable video prompt collapses into four sentences, each doing one job: who or what the subject is and what it’s doing, how close the shot is — the shot size, from wide to close-up — what the camera itself is doing while the subject acts, and what the light is doing. Write those four in order, inside the model’s native clip length, and the prompt is usable before style ever enters the conversation.

A courier checks a delivery address on a phone. The shot is medium. The camera is static. The light comes from a window, from the side, and is soft.

Reading a bad take

t5contradictory prompt (counter-example)

A failed text-to-video take tends to fail in one of three specific ways: motion that reads as physically wrong rather than merely stylised, a cut where the model quietly changes what the subject looks like partway through the clip, or a camera move that contradicts what the prompt asked for. Naming which of the three happened is more useful than a general “this doesn’t look right,” because each one points at a different sentence in the prompt to fix.

The ladder

The ladder for a text-to-video prompt runs four rising tiers, plus one tier that is not a top but a warning. The lowest tier is one line naming only the subject. The next tier adds the subject’s action. The next tier writes out the full four sentences — subject, shot size, camera, light. The top tier keeps those four and adds a specific beat the action turns on, plus a negative prompt naming what the shot must not do. And a counter-example, not a fifth rung: the same four sentences buried under contradictory camera and style demands at once, which reliably produces a worse result than the plain four sentences alone — worth generating once, on purpose, to see what over-specifying actually costs.

one line

One line gives the model everything to invent: watch the camera wander and the framing change.

subject + action

Naming the subject and the action locks the pose; the light and the camera are still free.

four sentences

The four-sentence shape pins style, motion, camera and continuity - watch the shot stop cutting.

four sentences + beats + negatives

Timecoded beats put the blink and the gust where you asked; check 00:02 and 00:03.5.

contradictory prompt (counter-example)

Every camera move and every look contradicts the last one - watch the model pick for you.

Recorded on the convener's rig with MiniMax H3 (local weights), keyframes with NoobAI-XL, reference stills for The Ninth Head with ChatGPT's image generator. Nothing is generated in the browser. Run it in the Studio.

Before class

Pick one action you want to see performed and write it as a single sentence, with no camera or light information yet.

This week’s exercise

Build the ladder above for that action: one line, subject and action, the full four sentences, and the four sentences with a beat and a negative prompt. Generate the counter-example too, on purpose. Bring all five clips, in ladder order with the counter-example marked, to Wednesday’s Dailies.

Reading

Teaching team