Week 10 · Block IV Limits

Unflattening and hidden assumptions

Four runs of one unmodified tool. Two working rewrites, one valid executable that returns the wrong answer, and one failure that never reached the part of the tool everybody talks about.

Block IV · LimitsStudio 12 May 2027Shape: Failure post-mortem

The tool exited zero and the graph got smaller. How would you find out whether it worked?

One tool, unmodified, pointed at four flattened functions from four implementations. It is a reasonable tool built on a reasonable idea: discover the dispatcher, recover each region’s successors by symbolic execution, then patch the binary so the blocks jump to each other directly.

Four runs. Four different kinds of outcome.

The scoreboard, before anything else

Table 09

Unflattening: the smaller graph can be wrong

Stock ollvm-unflattener results on four O0 implementations.

Input	Block counts	What happened?	Outcome
OLLVM FLA	60 flattened17 rewritten	25/25 semantic successor pairs recovered	PASSWorking rewrite
Hikari FLA	66 flattened20 rewritten	28/28 semantic successor pairs recovered	PASSWorking rewrite
Polaris FLA	56 flattened10 rewritten	Multiple legitimate switch cases now run sequentially	FAILWrong result
Tigress Flatten	38 actual6 stock-discovered	Indirect jump-table discovery is incomplete	BLOCKEDNo rewrite

Visual simplification is not proof of correct deobfuscation.

Miasm CFG counts split at calls. Tigress numbers compare reference vs discovery, NOT a before/after rewrite.Open the original lecture figure (SVG)

Two working rewrites, one wrong answer, one no-output. Keep the third and fourth rows apart in your head — they are different failures with different causes, and treating them as one is the most common misreading of this table.

The two that worked

successor pairs recovered in the first implementation — 60 blocks to 17: 25 / 25successor pairs recovered in the first implementation — 60 blocks to 17semantic_edge_score.json
successor pairs in a second, related implementation — 66 blocks to 20: 28 / 28successor pairs in a second, related implementation — 66 blocks to 20semantic_edge_score.json
bytes changed in the first rewrite, with the file size unchanged: 862bytes changed in the first rewrite, with the file size unchangedstock_result.json
rewritten blocks against the compiler's own clean build: 17 vs 21rewritten blocks against the compiler's own clean buildselected_metrics.md

Both produced working executables: the canonical run prints all six expected values and exits zero. The transfer is genuine — the tool was built against one implementation and worked unmodified on another — and it is bounded, because those two implementations share a lineage, a dispatcher shape and a state layout.

The fourth number is the interesting one. The rewritten function has 17 blocks; the compiler’s own clean build of the same source has 21. The recovered version is smaller than the original, keeps dead state writes that no longer control anything, and is correct on every retained test. Graph identity was never the goal and would have been the wrong goal.

A disassembler diff close-up. Both panes show loop and case operations; the right-hand pane retains state-variable stores and padding NOP instructions while its blocks branch directly to one another. — Clean on the left, the successful rewrite on the right. Different instructions, different layout, same tested behaviour. “Correct” and “identical” were never the same requirement.
analysis view — not correctness evidencedemo_flattening4_clean_to_rewritten_fla_1.jpg — open for full resolution

The one that produced a valid, wrong executable

the flattened original

PASS

SUB      = 08080000
BCF      = 6665BBBC
FLA      = DBEFFCE7
CALLS    = 12345F05
DATA     = 57977293
COMBINED = EEBB6D71
SHOWCASE PASS

7 linesopen input_runtime_stdout.txt(141 B)

the smaller, cleaner, valid rewrite

FAIL

SUB      = 08080000
BCF      = 6665BBBC
FLA      = F9CD8332
CALLS    = 12345F05
DATA     = 57977293
COMBINED = EEBB6D71
SHOWCASE FAIL: 1 mismatches

7 linesopen output_runtime_stdout.txt(155 B)

One line differs. FLA = DBEFFCE7 became FLA = F9CD8332, and the process exits 1 instead of 0. Every other value is untouched, because the tool was only ever asked to rewrite one function.

Here is why, and it is completely legible once you look at the right file.

the clean build: four alternatives

four blocks, each reconverging

0x1400013a7 -> 0x1400013ea
0x1400013a7 MOV        EAX, DWORD PTR [RSP + 0x8]
0x1400013ab ADD        EAX, 0x11111111
0x1400013b0 MOV        DWORD PTR [RSP + 0x8], EAX
0x1400013b4 JMP        loc_key_11

0x1400013b9 -> 0x1400013ea
0x1400013b9 MOV        EAX, DWORD PTR [RSP + 0x8]
0x1400013bd XOR        EAX, 0x22222222
0x1400013c2 MOV        DWORD PTR [RSP + 0x8], EAX
0x1400013c6 JMP        loc_key_11

0x1400013cb -> 0x1400013ea
0x1400013cb MOV        EAX, DWORD PTR [RSP + 0x8]
0x1400013cf SUB        EAX, 0x3333
0x1400013d4 MOV        DWORD PTR [RSP + 0x8], EAX
0x1400013d8 JMP        loc_key_11

0x1400013dd -> 0x1400013ea
0x1400013dd MOV        EAX, DWORD PTR [RSP + 0x8]
0x1400013e1 ADD        EAX, 0x4444
0x1400013e6 MOV        DWORD PTR [RSP + 0x8], EAX

lines 82–103 of 109open clean_cfg.txt(3 kB)

the rewrite: one block, four operations

no branch, no choice

0x140001348 -> 
0x140001348 NOP        
0x140001349 NOP        
0x14000134a NOP        
0x14000134b NOP        
0x14000134c MOV        EAX, DWORD PTR [RSP + 0x14]
0x140001350 XOR        EAX, 0x22222222
0x140001355 MOV        DWORD PTR [RSP + 0x14], EAX
0x140001359 MOV        DWORD PTR [RSP + 0xC], 0x24E
0x140001361 MOV        EAX, DWORD PTR [RSP + 0x14]
0x140001365 SUB        EAX, 0x3333
0x14000136a MOV        DWORD PTR [RSP + 0x14], EAX
0x14000136e MOV        DWORD PTR [RSP + 0xC], 0x24E
0x140001376 MOV        EAX, DWORD PTR [RSP + 0x14]
0x14000137a ADD        EAX, 0x4444
0x14000137f MOV        DWORD PTR [RSP + 0x14], EAX
0x140001383 MOV        DWORD PTR [RSP + 0xC], 0x24E
0x14000138b MOV        EAX, DWORD PTR [RSP + 0x14]
0x14000138f ADD        EAX, 0x11111111
0x140001394 MOV        DWORD PTR [RSP + 0x14], EAX
0x140001398 MOV        DWORD PTR [RSP + 0xC], 0x24E
0x1400013a0 MOV        EAX, DWORD PTR [RSP + 0x14]
0x1400013a4 XOR        EAX, 0xCAFEBABE
0x1400013a9 ADD        RSP, 0x20
0x1400013ad RET

lines 34–58 of 101open after_cfg.txt(4 kB)

On the left, the source’s four-way switch: four separate blocks — add 0x11111111, XOR 0x22222222, subtract 0x3333, add 0x4444 — each ending with a jump to the common successor. On the right, the same four operations in one straight line, executed one after another, with no branch between them. Then the final XOR 0xCAFEBABE and RET.

The four legitimate switch alternatives were grouped under a single incoming state. The recovery trace records one edge where the ground-truth region map has four, and the patcher faithfully emitted what the trace told it. Note the four MOV DWORD PTR [RSP + 0xC], 0x24E stores in the right-hand listing — the same “next state” written four times, which is the fingerprint of the collapse.

Three control-flow graphs side by side with block counts labelled: a clean structured graph, a wide flattened graph, and a small ten-block graph marked as the incorrect rewrite. — Clean, flattened, and the incorrect rewrite, at the same scale and in the same convention. The third graph is the most attractive one on the page.
Miasm recursive CFG — the rewritten graph is marked incorrect in the figurecourse_comparison.svg — open for the full graph

Two function graphs side by side. The left is a wide state-dispatch network. The right is a much smaller graph containing a long straight-line region and only a few branches. — The original dispatcher beside the incorrect rewrite. The long straight-line region on the right is the collapsed switch. Visual simplification is not semantic correctness — and this image is the reason that sentence is in every block of this course.
analysis view — not correctness evidencedemo_flattening5_polaris_to_rewritten_1.jpg — open for full resolution

The one that failed before it started

The fourth run is a different kind of negative result, and confusing it with the third would be a mistake.

stock_result.json — the whole outcome

read the stderr line

  "exit_code": 0,
  "timed_out": false,
  "elapsed_seconds": 0.36,
  "input": "E:\\Workspace\\seeing_through_obfuscation\\results\\showcase_matrix_v1\\optimization_survival\\tigress\\flatten\\O0\\showcase.exe",
  "input_sha256": "4201A5095EFE5D367C7A261ED612F108CEE85AEBA7D002E6B2831F1E43F0549F",
  "target_va": "0x140001020",
  "output": "E:\\Workspace\\seeing_through_obfuscation\\results\\deobfuscation\\unflattening\\tigress\\o0\\stock_rewritten.exe",
  "output_exists": false,
  "stdout": "",
  "stderr": "INFO:root:Unflattening function 0x140001020\r\nINFO:root:Function 0x140001020 is not flattened\r\n",

lines 16–25 of 35open stock_result.json(2 kB)

INFO:root:Function 0x140001020 is not flattened. No output file. No patch. No symbolic execution, no solver, no successor recovery — the tool’s discovery stage saw six blocks where the jump-table-aware reference has 38, decided this was not a flattened function, and stopped.

blocks the stock discovery stage found: 6blocks the stock discovery stage foundbefore_cfg.txt
blocks in the jump-table-aware reference for the same function: 38blocks in the jump-table-aware reference for the same functionreference_complete_cfg.txt
dispatch targets it never saw, behind one indirect jump: 21dispatch targets it never saw, behind one indirect jumpverified_jump_table.json
solver queries issued — the failure is upstream of the analysis: 0solver queries issued — the failure is upstream of the analysisstock_result.json

Six discovered blocks is not a six-block function. It is a graph with 32 missing blocks, because the edges out of the dispatcher go through an indirect jump and resolving those needs a jump-table model the discovery stage does not have. A disassembler with that model resolves them; this tool does not.

So the honest verdict is “blocked at discovery”, not “the symbolic execution failed” and certainly not “the solver could not handle it”. Lowering the acceptance threshold does not help either — a separately labelled scoring adapter was tried and also fails, which is what you would expect when the problem is 32 absent blocks rather than a score.

A very small control-flow graph of six blocks with few edges, ending abruptly at an indirect jump. — What the tool saw.
stock discovery — 6 blocksbefore_cfg.svg — open for the full graph

A control-flow graph of thirty-eight blocks fanning out from one indirect-jump block, with most blocks returning to it. — What was there.
jump-table-aware reference — 38 blocksreference_complete_cfg.svg — open for the full graph

The left graph is not simpler code. It is the same code, under an analysis that could not follow it. If you take one habit from this week: a small recovered graph is a claim about the recovery, not about the program.

What the four outcomes have in common

Each of the four runs used the same tool with the same defaults, and the difference between them is an assumption about the representation:

the two successes share the dispatcher shape and state layout the tool was written against;
the wrong rewrite shares the shape closely enough to be accepted, and differs in how a multi-way source construct is represented in its states;
the blocked run differs in the dispatch mechanism itself.

None of that is visible in a tool’s exit code, and none of it is visible in the output graph. It is visible in the successor score, in the ground-truth region map, and in eight hex digits of program output.

Table 10

Four tempting conclusions to avoid

Keep visual appearance, representation validity and semantic correctness separate.

Tempting conclusion	Retained counterexample	What the evidence says
Smaller CFG = correct	Polaris FLA: 56 to 10 blocks	WRONGF9CD8332 instead of DBEFFCE7; exit 1.
Valid IR = faithful lift	Mergen on valid OLLVM SUB input	WRONG3,917 / 4,118 mismatches in every lifted stage.
Six discovered blocks = six actual blocks	Tigress Flatten	INCOMPLETEThe jump-table-aware reference has 38 blocks.
Matching PDB ID = valid instruction map	Rewritten BCF / FLA PEs	STALE METADATAOld RSDS references survive. Use exports / RVAs.

Validate the output at the level of the claim: expression, CFG, lifted function or complete executable.

Original OLLVM SUB passes native checks. FLA whole-function rewriting moves internal instructions.Open the original lecture figure (SVG)

This establishes

Stock unflattening produced working canonical rewrites for two related -O0 specimens, recovering 25/25 and 28/28 ground-truth successor pairs.
On a third specimen the same tool produced a structurally valid PE with a 56→10 block graph that returns F9CD8332 instead of DBEFFCE7 and exits 1; its four legitimate switch alternatives now execute in sequence.
On a fourth it stopped at discovery, seeing 6 blocks where the jump-table-aware reference has 38, without reaching symbolic successor recovery at all.
A correct rewrite need not match the compiler's clean graph: the successful one has 17 blocks against the clean build's 21 and retains dead state writes.

This does not establish

That the tool is broken, or that unflattening does not work. Two of four runs produced working executables.
That the -O0 successes extend to -O2. For one of the two implementations, the higher-optimisation run fails before useful recovery.
That the blocked run is a solver failure or a semantics failure. It is a discovery failure, upstream of both.
That a successor-pair score proves correctness. The successful runs scored 25/25 and were also executed; the score alone does not establish branch polarity.

Paired examples: unflattening_course_examples.mdFull protocol and limits: unflattening_validation.md

Studio: audit a beautiful result

Static analysis only. Do not execute the incorrect rewrite — you already know what it prints, and the exercise is to find the fault without that.

In the clean listing, identify the four switch alternatives and the block they all reconverge to.
In the rewritten listing, find the block that contains all four. Walk it instruction by instruction and compute what it returns for an input whose low two bits are 01. Compare with the clean path for the same input.
Find the four identical state stores and say what their repetition tells you about how the recovery went wrong.
Now the assessment skill. Write the rejection: a paragraph that refuses the claim “this rewrite deobfuscated the function”, citing the successor score, the region map and the block listing — without citing the runtime output. That is the position an analyst is usually in.
Finally, the counterfactual. What would the tool have had to represent differently to get this right? One sentence.

after_cfg.txtthe incorrect rewrite, instruction by instruction4 kB
clean_cfg.txtthe clean build of the same function, for the diff3 kB
semantic_edge_score.json14 of 22, one false pair, eight unresolved — the warning nobody read418 B
provenance_region_map.jsonground truth: the four switch regions the rewrite collapsed32 kB
instrumentation.jsonthe recovery trace that grouped them under one state59 kB
output_runtime_stdout.txtwhat it actually prints155 B
stock_result.jsona successful run, for comparison: 862 bytes changed, exit 03 kB
stock_result.jsonthe blocked run: no output file, no recovery attempted2 kB
course_comparison.svgclean, flattened and incorrect rewrite, one figure82 kB
unflattening_course_examples.mdall four cases, with addresses, states and region maps9 kB

Come to the studio with

The studio assumes you have done these. Each is written so you can tell whether you have, without asking.

you can read a block listing and follow fall-through from one instruction to the next
you have week 9's state table for the parity branch written down
you can say what a precision and a recall of 1.0 do and do not tell you

Unflattening and hidden assumptions

The scoreboard, before anything else#

Unflattening: the smaller graph can be wrong

The two that worked#

The one that produced a valid, wrong executable#

The one that failed before it started#

What the four outcomes have in common#