Week 4: Image to Image

The second generator — pushing an existing image toward a finished keyframe with denoising strength, ControlNet, and IP-Adapter, on the turnaround from rough input to finished frame.

15 March 2027

Text to image starts from nothing; image to image starts from something, and this week is about how much of that something survives. Denoising strength is the single number that decides it: low strength keeps most of the input’s structure and repaints its surface, high strength keeps almost nothing and lets the prompt take over. Everything else this week is about giving the generator a better “something” to start from.

Reading the strength dial

t1rough sketch

Denoising strength runs from barely touching the input to nearly regenerating from scratch, and the useful range for turning a rough input into a finished keyframe sits well below the top of that scale — high enough to fix proportions and clean up linework, low enough that the composition the input already got right survives the pass. Pushing the number up because a result looks unfinished usually erases the one thing worth keeping.

Controlling structure without controlling colour

t5finished keyframe + ControlNet pose lock

ControlNet reads a separate signal from the input — a pose skeleton or a depth map — and holds that signal fixed while everything else in the image regenerates, which is the difference between “keep this composition” and “keep this exact picture.” IP-Adapter does the opposite job: it carries identity or style forward from a reference image into the new one, independent of the pose or depth signal entirely. Used together, a pose comes from one input and a face or style anchor comes from another, and the generator reconciles both.

The turnaround

t2clean lineart

A production turnaround is the distance between a rough input and a keyframe good enough to hand to the next stage, and this course measures it the same way a studio does: not by how good the first pass looks, but by how few passes it took to get somewhere usable. A short turnaround built on a clean sketch and one controlled pass beats a long one spent fighting a vague input with prompt changes alone.

The ladder

The ladder this week holds the prompt and the denoising strength fixed and climbs by improving the input image instead — five tiers, one prompt, one strength, across all five. The lowest tier starts from a rough sketch. The next tier starts from clean lineart instead of a rough one. The next tier starts from a flat-coloured version of that lineart. The next tier starts from an already finished keyframe, refined rather than built. The top tier starts from that same finished keyframe plus a ControlNet pose signal locking its composition in place.

rough sketch

Recorded output for rough sketch

The prompt is fixed from here to t5 and only a rough sketch is guiding this pass. The denoising strength was held constant too, but its value was never written down - this manifest carries no strength and no seed, in the one week whose whole point is that dial. Read it as the argument for the log rather than as a number to copy.

clean lineart

Recorded output for clean lineart

Cleaning the lines before the pass steadies the proportions the rough sketch left uncertain.

flat-coloured lineart

Recorded output for flat-coloured lineart

A flat colour guide pins the palette and the silhouette before a single gradient exists.

finished keyframe, refined

Recorded output for finished keyframe, refined

The input here is week03's own top-tier keyframe, refined rather than built again from a sketch.

finished keyframe + ControlNet pose lock

Recorded output for finished keyframe + ControlNet pose lock

A bare pose skeleton, with no colour or line detail at all, still holds this composition in place.

Recorded on the convener's rig with MiniMax H3 (local weights), keyframes with NoobAI-XL, reference stills for The Ninth Head with ChatGPT's image generator. Nothing is generated in the browser. Run it in the Studio.

Before class

Bring one keyframe from Phase 2 so far, and sketch three rougher versions of it at decreasing levels of finish, to use as this week’s ladder of inputs.

This week’s exercise

Run the same prompt and denoising strength against your four rough inputs plus one ControlNet-locked pass, producing all five ladder tiers. Bring all five outputs, their shared prompt, and the strength value used to Wednesday’s Dailies.

Reading

Teaching team