Your Own Workflow

Week 7 · SLOP8760

Phase 3 of 4 · Holding It Together

This week adds

Your own graph: the supplied workflow opened up, understood, and rebuilt as a version that is actually yours.

Photoreal is permitted from here

Building your own graph is what demonstrates the reference discipline photoreal work needs, now that stylisation is no longer covering for the generator.

One file, three components

A graph reduces to four recurring jobs: loaders bring things in, conditioning feeds the sampler, the sampler denoises, and everything after it is a fix pass. The first of those is one node, and the docs say what falls out of it.

Load Checkpoint gives	Which the docs describe as
MODEL	denoising latents
CLIP	encoding text prompts
VAE	encoding and decoding images

Source: ComfyUI docs, Load Checkpoint — https://docs.comfy.org/built-in-nodes/CheckpointLoaderSimple

A generation graph reduces to four recurring jobs, however many nodes it actually contains: loaders bring a model, a checkpoint, or a reference image into the graph; conditioning turns a prompt or a control signal into something the sampler can read; the sampler runs the actual denoising steps that turn noise into a result; and everything after the sampler is a fix pass — an upscale, a face restoration step, a colour correction — cleaning up what the sampler handed back. Recognising which of the four a node belongs to makes an unfamiliar graph readable in minutes instead of hours.

The docs’ own phrasing is that Load Checkpoint “decomposes [the checkpoint] into three core components”, and the VAE line is worth reading in full: it handles “encoding and decoding images to and from latent space”, which is both directions of the boundary the sampler works behind. The text side lands in CLIP Text Encode, whose output the docs call “a conditioning containing the embedded text used to guide the diffusion model”.

Save and load workflows as JSON, or recover complete workflows and seeds from supported generated media
ComfyUI, README

Source: ComfyUI — https://github.com/comfyanonymous/ComfyUI

The graph has an equivalent in JSON

The recorded tier two is a two-reference workflow. Exported, it is data: nodes keyed by number, each carrying a class_type, its inputs and a _meta, posted to /prompt, which answers with a prompt_id or with node_errors.

Source: ComfyUI docs, Workflow API Format — https://docs.comfy.org/development/api-development/workflow-api-format
Source: ComfyUI docs, Server routes — https://docs.comfy.org/development/comfyui-server/comms_routes

Every graph in a node-based workflow engine can be exported as a request body — the same loaders, conditioning, and sampler settings expressed as plain data rather than boxes and wires. That export is what an API-based rig sends instead of running the graph locally, and reading one is the same skill as reading the graph: find the four jobs inside the JSON the same way they’d be found inside the nodes.

The export is File → Export Workflow (API), and the format “omits UI metadata” — no positions, no colours, nothing about how the graph looked. Links appear as a pair, the source node’s id and an output index, written like ["4", 0]. The route that receives it “validates the prompt and adds it to an execution queue, returning either a prompt_id and number … or error and node_errors”, which is the whole contract between a graph and a rig that runs it somewhere else.

Everything after the sampler is a fix pass

The sampler hands back a latent. VAEDecode turns it into an image; an upscale node runs an upscale model over it. Neither revisits the decision. A LoRA loader looks like one and is not: it hands back adjusted model and clip. That is loader work, upstream of all this.

Source: ComfyUI docs, LoraLoader — https://docs.comfy.org/built-in-nodes/LoraLoader

Recorded your own workflow ladder, tier t3: t1 upscaled x2 to 1080x1920 — Ladder t3 · t1 upscaled x2 to 1080x1920

Most of what a graph needs is a small edit, not a rewrite: swapping one loader’s checkpoint, adding a second conditioning input, inserting one fix-pass node before the output. Rebuilding from an empty graph is rarely the right first move — it throws away a working arrangement of the four jobs to relearn something that was already solved. The one exception is when the supplied graph does something the shot in front of it doesn’t need at all, in which case removing that piece cleanly is worth more than working around it.

The LoRA loader is on the slide as the counter-example, because it is the node this taxonomy most often gets wrong: it is late in the list, it is optional, and it changes how the result looks, so it reads like a fix pass. The docs settle it. It takes model and clip and returns “The model with LoRA adjustments applied”, strength “typically used between 0~1 for daily image generation” — the same two wires in and out as the checkpoint loader, and the sampler has not run yet when it happens.

The still here is the ladder’s upscale tier, which is a fix pass and nothing else: the same take as the tier below it, at twice the size. Worth noticing that the upscale node’s docs state no scale factor — the factor belongs to the model you load into it, not to the node. Worth noticing too that the recording ran that upscale after the render rather than inside a graph, which is a shortcut the assessment does not allow: a fix pass in the graph is the one that reruns from the file.

The ladder

One shot, held constant, with more of the supplied graph replaced each time.

The bare supplied graph, unmodified, as a baseline
Plus reference anchoring — a loader bringing in a locked reference image
Plus an upscale and fix pass after the sampler
A two-stage pipeline: a keyframe in one graph, video in a second

Before class

Open the graph this course has been supplying and trace, node by node, which of the four jobs — loader, conditioning, sampler, fix pass — each one is doing.

This week’s exercise

Rebuild that graph across all four ladder tiers on one shot: the bare graph, plus reference anchoring, plus a fix pass, and finally the two-stage keyframe-then-video pipeline. Bring all four graph files and their outputs to Wednesday’s Dailies.

Reading

ComfyUI: Load Checkpoint
Comfy Org
encoding and decoding images to and from latent space
ComfyUI: Workflow API Format
Comfy Org
API format omits UI metadata (positions, colors, groups, node sizes) that is only needed for visual editing in the frontend.
ComfyUI: Server routes
Comfy Org
validates the prompt and adds it to an execution queue
ComfyUI
comfyanonymous
Save and load workflows as JSON, or recover complete workflows and seeds from supported generated media

Your Own Workflow#

Week 7 · SLOP8760#

This week adds#

Photoreal is permitted from here#

One file, three components#

The graph has an equivalent in JSON#

Everything after the sampler is a fix pass#

The ladder#

Before class#

This week’s exercise#

Reading#