What Is a File?

Identity is not a property. It’s a claim someone checks.

Test 1: Name

Two files are “the same” if they’re called the same thing.

Everyone’s default test. Trust it least.

Test 2: Metadata

Size, modified time, permissions, owner.

Rewritable with one command. Doesn’t survive a copy.

Test 3: Content

A hash of the bytes.

Identical hash, identical content. Full stop. Says nothing else.

Test 4: Reference

The inode a name resolves to.

The filesystem’s own test. Invisible to almost every other tool.

They disagree

                        name    metadata    content    reference
renamed in place        fail      pass       pass        pass
copied to another dir   pass      fail       pass        fail
hard-linked             fail      pass       pass        pass
edited, then reverted   pass      fail       pass        pass

Renaming and hard-linking are indistinguishable here. They are not the same event.

One pair of files put to four identity tests: name says different, metadata, content and reference all say same. A hard link scores identically.

Every tool picks one, silently

A file browser dedupes by size and date.

A sync client tracks a reference.

A backup tool hashes content.

Their disagreements are four questions answered correctly.

Pick the wrong test for the situation and you either merge two things that were never one, or split one thing you can no longer reconcile.

Test 3, in practice

Two PDFs. Different content on the page. Same SHA-1:

38762cf7f55934b34d179ae6a4c80cadccbb7f0a

Published 2017. Identity by content is the strongest test available, and “strongest available” is not the same as “true.”

Next week

Two files can pass all four tests and still become different works the moment someone starts editing one of them.

Copies vs. clones.

Sources

  • The First Collision for Full SHA-1 Marc Stevens, Elie Bursztein, Pierre Karpman, Ange Albertini and Yarik Markov, CRYPTO 2017, LNCS 10401, 570–596. Two PDFs displaying different content, sharing one SHA-1 digest, at a cost of roughly 2⁶³ compression-function calls. eprint.iacr.org