Week 05 · Block II Simplification

Optimization versus obfuscation

The compiler flag is part of the experiment. Change it and a transformation can halve, double, or vanish — while the obfuscator's own output file stays exactly the same.

Block II · SimplificationStudio 24 March 2027Shape: Controlled comparison

Is this result about the obfuscator, or about the optimisation flag that came after it?

Week 3 ended with a transformation that mostly stopped existing when the optimiser was turned on. That was not an accident of that tool. Optimisation level is not a nuisance variable in this field — it is part of the treatment, and this week is about designing comparisons that survive it.

How the matrix controls for it

Two different problems, two different answers.

For the source-level lane the problem is easy to state and easy to get wrong: if you generate obfuscated C four times, once per optimisation level, you have four specimens and no control. So the matrix generates the C once and compiles that exact file at every level. Here is the check, in the two retained metadata records:

the O0 build

/Od /Ob0

  "optimization": "O0",
  "optimization_flag": "/Od /Ob0",
  "optimization_note": "",
  "dataset_role": "optimization_survival",
  "selection": "Tigress --Functions=demo_substitution",
  "canonical_source": "E:\\Workspace\\seeing_through_obfuscation\\src\\showcase.c",
  "canonical_sha256": "39FEF0BB2A9742870F3D6F7CC864F7CFFA129136F6D2A59C4FB37F25906B99D2",

lines 7–13 of 34open metadata.json(2 kB)

the maximum-optimisation build

/Ox, recorded as O3

  "optimization": "O3",
  "optimization_flag": "/Ox",
  "optimization_note": "MSVC has no /O3; /Ox is the maximum-optimization analogue.",
  "dataset_role": "optimization_survival",
  "selection": "Tigress --Functions=demo_substitution",
  "canonical_source": "E:\\Workspace\\seeing_through_obfuscation\\src\\showcase.c",
  "canonical_sha256": "39FEF0BB2A9742870F3D6F7CC864F7CFFA129136F6D2A59C4FB37F25906B99D2",

lines 7–13 of 34open metadata.json(2 kB)

input_source_sha256 is the same value in both records. One generated file, two compilations — so any difference between the two binaries is the compiler’s, and nothing else’s. Note also "optimization_note": "MSVC has no /O3; /Ox is the maximum-optimization analogue." The directory says O3; the flag was /Ox, and quoting it as /O3 would be a small lie that compounds.

For the IR-level lanes the problem is the opposite. If you run the obfuscation pass and then hand the result to a normal compilation, the middle-end runs twice and you cannot tell what the pass did from what the second run did. So the matrix emits post-pass IR once, then generates object and assembly from that exact IR with -Xclang -disable-llvm-passes. One middle-end run, per build, on purpose.

The same generated C, twice

target instructions at /Od — against a matched clean 249: 426target instructions at /Od — against a matched clean 249optimization_survival.md
target instructions at /Ox — against a matched clean 106: 226target instructions at /Ox — against a matched clean 106optimization_survival.md
the surviving ratio at maximum optimisation: ≈2.1×the surviving ratio at maximum optimisationselected_metrics.md
the clean /Ox comparator — not the older, larger O2 figure: 106the clean /Ox comparator — not the older, larger O2 figureobfuscator_comparison_summary.md

The absolute count nearly halved. The ratio against the correct comparator barely moved. Both statements are true, and only the second one is about the transformation.

That last number carries a warning worth repeating. An earlier measurement of the clean /Ox target overran the function boundary — it measured to the next export rather than to the return — and produced a larger clean figure and therefore a smaller, flattering ratio. The corrected comparator is 106 instructions. If you are ever tempted to compare against a number you did not scope yourself, that is the shape of the mistake.

What disappears

Now the result that ought to make you nervous about every -O0 finding in the literature.

Table 04

Opaque flow: what survives optimization?

BCF = bogus control flow. The observation is about the measured native target.

Treatment	At O0	O1 through O3 / Ox	Interpretation
OLLVM BCF	Clones and opaque branches	RESIDUESmall optimized opaque-loop structure	Optimization removes much, not everything.
Hikari BCF	Injected native gates already folded	REMOVEDMeasured target matches clean	IR injection does not imply a native predicate.
Polaris BCF	Strong cloned/opaque topology	REMOVEDMeasured target matches clean	A strong O0 example can be weak at O1+.
Tigress AddOpaque	Initialization-dependent opaque flow	PERSISTS/Ox: 6 conditional jumps vs 1 clean	Runtime state matters to proof and optimization.

Choose the optimization level deliberately: it can erase the phenomenon being taught.

Scope: demo_bogus_control_flow in the retained optimization study. Jump counts alone do not prove opacity.Open the original lecture figure (SVG)

Two of the four opaque-control-flow lanes are gone by -O1. Not reduced — the measured target is structurally identical to the clean target. One of them was already mostly folded at -O0, because its injected guards were literal icmp eq i32 1, 1 and the backend does not need an optimiser’s permission to resolve that.

For arithmetic, the same pattern with different winners:

Table 03

Arithmetic: complexity can shrink or survive

Optimization acts on the obfuscated program before we inspect its machine code.

Treatment	At O0	At higher optimization	Main lesson
OLLVM SUB	24 IR sites transformed; native XOR/SUB recognizable	WEAKENEDO1+: all 24 native operations reappear	Large IR trees need not survive code generation.
Hikari SUB	Partial coverage; 213 native instructions	REMOVEDO1+: measured target matches clean	Coverage and optimization survival are different.
Polaris SUB	XOR 8/8; ADD 8/8; SUB 3/8 transformed	WEAKENEDO1+: native operations recognizable	Classic SUB is not the stronger MBA treatment.
Tigress EncodeArithmetic	426 native instructions	PERSISTS/Ox: 226 vs 106 clean	Exact same generated C; substantial complexity remains.

IR expansion, native instruction count and semantic difficulty are different measurements.

Target: demo_substitution. Tigress uses MSVC /Ox (recorded as O3); the 106-instruction clean is also /Ox.Open the original lecture figure (SVG)

And for flattening, the opposite result entirely:

Table 05

Flattening can survive without an indirect jump

FLA = control-flow flattening. Look for state-driven dispatch, not one opcode.

Treatment	At O0	At higher optimization	What to look for
OLLVM FLA	State/switch dispatcher	Strong flattening persists	Repeated state tests and returns to dispatch.
Hikari FLA	State/switch dispatcher	Strong flattening persists	Dispatcher structure despite simpler blocks.
Polaris FLA	Strong state dispatcher	Smaller, but still flattened	Simplification is not full unflattening.
Tigress Flatten	Indirect jump table	O1: direct networkO2 and /Ox: indirect table	Same generated C, different native dispatch.

A direct chain of comparisons can implement the same flattened state machine as a jump table.

Scope: demo_flattening. Survival of obfuscation does not establish compatibility with an unflattener.Open the original lecture figure (SVG)

Flattening is the durable family. Two implementations keep essentially their whole dispatcher from -O1 through -O3; a third loses about half its structure and still has a dispatcher; the source-level one changes how the dispatch is lowered — an indirect jump table at some levels, a direct comparison network at another — without losing the state machine at all.

That last point is the one people get wrong. A missing indirect jump is not evidence that flattening disappeared. It is evidence about lowering.

What this does to course design, and to yours

The study’s own recommendation is blunt: teach bogus control flow with explicit -O0-versus--O2 pairs, because two of the four implementations vanish at -O1 and presenting their -O0 structure as “an obfuscated binary” would be straightforwardly false.

Which is why weeks 6 to 8 work on -O0 specimens and say so in every sentence, and why week 10’s flattening work can use higher levels.

This establishes

One generated C file compiled at two MSVC levels went from 426 to 226 target instructions, against matched clean comparators of 249 and 106 — a surviving ratio of roughly 2.1× at maximum optimisation.
Two of the four tested bogus-control-flow lanes are structurally identical to their clean comparator from -O1 onward, in the measured target.
Flattening survived every level tested in all four implementations, with one changing its native dispatch form between an indirect jump table and a comparison network.

This does not establish

That these figures rank the four tools. Each measures a different technique on one function, and two of the lanes are not even measured in the same units.
That a smaller instruction count means weaker obfuscation. Against the matched comparator, the source-level lane barely changed.
That absence of an indirect jump means flattening was removed. It means the backend lowered the switch differently.
That any of these survival results transfers to another program. Optimisation interacts with the specific code, and the corpus is one 483-line file.

Survival matrix: optimization_survival.mdComparator correction: obfuscator_comparison_summary.md

Studio: compare two builds fairly

Open the two metadata records and confirm for yourself that input_source_sha256 matches. Write down what that fact rules out.
Compare the two disassembly listings. Find one arithmetic site in each and say what the optimiser did to it.
Compute both ratios — obfuscated against clean, at each level, using the matched control. Then write the one sentence that is supported by them.
Now write the sentence that is not supported: the cross-tool comparison that this data does not license. Say precisely which control is missing.

A1 is due Friday at 17:00. Bring a draft of its “what this does not establish” section to the studio if you want it read.

demo_substitution_O0_disassembly.txt426 instructions at /Od28 kB
demo_substitution_Ox_disassembly.txt226 instructions at /Ox, from the same generated C16 kB
demo_substitution_disassembly.txtthe matched clean /Od control — 249 instructions16 kB
target_function_disassembly.txtthe matched clean /Ox control — 106 instructions7 kB
metadata.jsonbuild record for the /Od specimen2 kB
metadata.jsonbuild record for the /Ox specimen — same source hash2 kB
exact_generated_encode_arithmetic.cthe generated C itself, all 387 kB of it378 kB
optimization_metrics.jsonevery IR, machine and source metric in the matrix, per build205 kB
optimization_survival.mdthe full survival matrix, IR and machine, for every lane9 kB

Come to the studio with

The studio assumes you have done these. Each is written so you can tell whether you have, without asking.

you can say what `-O0` stops the compiler doing, in concrete terms
you have week 3's finding written down — what survived to machine code, and what did not
you can compute a ratio and say what the denominator has to be for it to mean anything