Sequentia Explores: The Mathematics of Crypto
Part III: Historical Ciphers & the Dawn of Cryptanalysis
Lecture 32: Collision Resistance: Why Two Fingerprints Can’t (Realistically) Match
In our last lecture, we introduced the cryptographic hash function as a “digital fingerprint” generator. We established several key properties: it’s deterministic, it’s a one-way street, and it produces a fixed-size output. Today, we’re going to zero in on the most crucial property for a hash function’s security: collision resistance.
A collision occurs when two different inputs produce the exact same hash output.
hash(Input_A) = hash(Input_B), where Input_A ≠ Input_B
If an attacker, Mallory, can find a collision, she can wreak havoc. Imagine Alice signs a legitimate contract (Input_A) by hashing it and signing the hash. If Mallory can create a fraudulent contract (Input_B) that has the exact same hash, she could attach Alice’s valid signature to the fraudulent contract, and it would appear mathematically valid.
For a hash function to be secure, it must be computationally infeasible to find such a collision. But how hard is it, really? The answer is tied to a famous and counter-intuitive concept from probability theory: the Birthday Problem.
The Birthday Problem: A Surprising Probability Puzzle
The classic Birthday Problem asks: “How many people do you need to have in a room for there to be a greater than 50% chance that at least two of them share the same birthday?”
Our intuition might suggest a large number, perhaps half the number of days in a year (around 183). The actual answer is surprisingly small: just 23 people.
Why is the number so low? It’s because we’re not looking for a match for a specific birthday (e.g., “who shares my birthday?”). We’re looking for any pair of people who share any birthday. With each new person who enters the room, the number of possible pairs to check for a match grows exponentially.
- With 2 people, there’s 1 pair.
- With 3 people, there are 3 pairs (A-B, A-C, B-C).
- With 23 people, there are (23 × 22) / 2 = 253 possible pairs to check.
This rapid growth in the number of pairs makes finding a random match much more likely than our intuition suggests.
The “Birthday Attack” on Hash Functions
This same principle applies directly to finding hash collisions.
Imagine a hash function that produces a very small, 32-bit output. The total number of possible hashes (the “birthdays”) is 2^32, which is about 4.3 billion.
- Pre-image Attack (Finding a match for a specific hash): If Mallory has a specific hash and wants to find an input that produces it, she would, on average, have to try half the total possibilities: 2^32 / 2 = 2^31 inputs. This is a huge number.
- Collision Attack (Finding any two inputs that match): But what if Mallory just wants to find any two inputs that collide? Because of the Birthday Problem, she doesn’t need to generate 2^31 hashes. She only needs to generate approximately the square root of the total number of possibilities.
√(2^32) = 2^16
2^16 is only 65,536.
This means an attacker could find a collision for a (terrible) 32-bit hash function by generating only about 65,000 random inputs and their hashes and then comparing the hashes for a match. This is trivial for a modern computer. This is called a Birthday Attack.
Why Hash Output Size is Everything
This is why the output size of a cryptographic hash function is so critical. The security against a collision attack is not the total number of outputs, but roughly the square root of the total number of outputs. The security level is effectively halved.
Let’s look at some real-world examples:
- MD5 (Broken):
- Output size: 128 bits.
- Total hashes: 2^128.
- Security against collision attack: √(2^128) = 2^64.
- 2^64 (about 18 quintillion) is a very large number, but it is now considered within the realm of possibility for well-funded organizations or large botnets. In fact, practical collision attacks against MD5 were demonstrated as early as 2004. MD5 is considered cryptographically broken and should never be used for security.
- SHA-1 (Deprecated):
- Output size: 160 bits.
- Total hashes: 2^160.
- Security against collision attack: √(2^160) = 2^80.
- 2^80 is significantly harder than 2^64, but a practical collision was demonstrated by Google in 2017. SHA-1 is also considered broken and should be phased out.
- SHA-256 (Current Standard):
- Output size: 256 bits.
- Total hashes: 2^256.
- Security against collision attack: √(2^256) = 2^128.
- As we discussed in our lecture on randomness, 2^128 is an astronomically large number. Finding a collision via a birthday attack on SHA-256 is computationally infeasible with current and foreseeable technology, requiring more energy than is available on the planet.
The move from MD5 and SHA-1 to the SHA-2 family (SHA-256, SHA-512, etc.) was driven almost entirely by the need to create hash functions with output sizes large enough to be secure against birthday attacks.
Collision resistance is not a theoretical nicety; it is the practical measure of a hash function’s strength against an attacker trying to forge data. The surprising math of the Birthday Problem teaches us that to build a secure system, our “fingerprints” need to come from a pool of possibilities so vast that even the square root of that number is beyond all astronomical comprehension.
In our next lecture, we’ll take a closer look at the SHA (Secure Hash Algorithm) family and see conceptually how it takes input data and scrambles it through multiple rounds to produce these secure, collision-resistant hashes.
