Text to Image

Week 3 · SLOP8760

Phase 2 of 4 · Four Generators

This week adds

Text to image: a prompt that reliably produces one still frame good enough to build the rest of the week on.

Why anime, first

8 milliontagged images in the caption corpus

Flat colour and clean linework absorb a generator’s small errors as line style. They also arrive with a vocabulary: Illustrious reports training on Danbooru’s corpus of over eight million images, so the checkpoint answers in that corpus’s own words.

Source: Illustrious: an Open Advanced Illustration Model — https://arxiv.org/abs/2409.19946

This phase works in anime grammar on purpose, and it’s a technical decision, not an aesthetic one. A generator’s small mistakes — a slightly wrong proportion, an edge that drifts — read differently depending on the style they land in: flat colour and clean linework compress detail into shape and outline, so a minor error gets absorbed as “that’s the line style.” The same magnitude of error on a photorealistic face lands squarely in the range human perception is most sensitive to and reads as wrong rather than stylised.

The second reason is the one on this slide. Illustrious builds on the SDXL family — a “three times larger UNet backbone” than the previous generation, with a second text encoder, OpenCLIP ViT-bigG alongside CLIP ViT-L — and trains it on a tag-captioned dataset rather than a prose-captioned one. That is why the vocabulary matters more here than the sentence does.

Four things a keyframe prompt should say

Subject. Framing — size and angle. What the light is doing. A style anchor naming the checkpoint’s own register. In that order.

The checkpoint publishes its own settings

An anime checkpoint is not a general-purpose model with a style applied. Its card states the numbers it was tuned for, and they are not the defaults.

Setting	NoobAI-XL 1.1
CFG	5 to 6
Steps	25 to 30
Sampler	Euler a
Vertical resolution	768 x 1344

Source: NoobAI-XL 1.1 model card — https://huggingface.co/Laxhar/noobai-XL-1.1

Four numbers, all off the model card, and all lower than the generic defaults from last week: CFG 5 to 6 against 8.0, steps 25 to 30 against 20. A card that publishes its own settings is telling you the generic ones will cost you rolls.

The last row is a generation size and not a delivery frame, and the two are different things worth keeping apart: 768 x 1344 is 4:7, or 0.5714, while the 9:16 the episode ships in is 0.5625. What the card publishes is a shape the model was trained to produce, so asking it for a true 9:16 instead buys worse rolls, not a better frame. Meeting the delivery frame is a crop or a scale at the end of the pipeline, in week 11.

The same card is where the licence complication for this family lives. Its metadata declares fair-ai-public-license-1.0-sd, and the README adds a sentence of its own: “We prohibit any form of commercialization.” Those are two separate instruments and the second is not part of the first, so a shot built on this checkpoint has to satisfy both.

person count ||| character names ||| rating ||| general tags ||| artist ||| score range based rating ||| year modifier
Illustrious, on caption order

Source: Illustrious: an Open Advanced Illustration Model — https://arxiv.org/abs/2409.19946

The checkpoint’s own vocabulary

Tier four opens masterpiece, best quality, amazing quality, very aesthetic, absurdres, newest and its negative opens worst quality, low quality, lowres, bad hands. NoobAI-XL’s card recommends almost exactly that pair. These are trained tokens, not adjectives.

Source: NoobAI-XL 1.1 model card — https://huggingface.co/Laxhar/noobai-XL-1.1

Anime-trained checkpoints are largely trained on tag-style captions, not prose, so they respond more precisely to a checkpoint’s own tag vocabulary than to a naturally written sentence saying the same thing. Pairing that vocabulary with a negative prompt — naming what to suppress, not just what to include — removes far more bad rolls than adding qualifiers to the positive prompt ever does.

The card’s own recommended positive is “masterpiece, best quality, newest, absurdres, highres, safe,” and its negative is “nsfw, worst quality, old, early, low quality, lowres, signature, username, logo, bad hands, mutated hands”. The recorded ladder’s tier four is that pair with the shot’s own tags threaded through it. The card also documents the year tags — newest is 2021–2024, old is 2005–2010 — which is a date filter wearing a style tag’s clothes.

The ladder

One line naming the subject alone
Plus a line of setting and mood
The full four-part grammar — subject, framing, light, style anchor
Plus the checkpoint’s tag vocabulary and a negative prompt
Plus shot size, camera angle, and a colour script for the sequence

Before class

Pick one anime-trained checkpoint and read through its own documentation or model card for the tag vocabulary it expects.

This week’s exercise

Produce one keyframe at each of the five ladder tiers above, on the same subject, keeping a seed and parameter log for all five. Bring all five keyframes, in ladder order, to Wednesday’s Dailies.

Reading

Illustrious: an Open Advanced Illustration Model
Park et al.2024
person count ||| character names ||| rating ||| general tags ||| artist ||| score range based rating ||| year modifier
SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
Podell et al.2023
three times larger UNet backbone
NoobAI-XL 1.1 model card
Laxhar2024
We prohibit any form of commercialization
Danbooru wiki: Help:Tags
Danbooru (community wiki)
Spaces in tag names become underscores

Text to Image#

Week 3 · SLOP8760#

This week adds#

Why anime, first#

Four things a keyframe prompt should say#

The checkpoint publishes its own settings#

The checkpoint’s own vocabulary#

The ladder#

Before class#

This week’s exercise#

Reading#