Lesson 3.7 · 20 min

Unique ID generator

Build and decode a Snowflake ID bit by bit, push 8,000 requests into one millisecond, step the clock back and watch a naive generator issue 9,000 duplicates, then spend a 63-bit budget.

The idea in plain words

Every order, message and user needs an ID no one else will ever get. On one database a counter does it; across hundreds of machines that can’t stop to ask each other, each machine must be able to make unique IDs on its own.

The problem it solves

Every record needs an ID no other record will ever get. One database counting 1, 2, 3 works until you have many machines creating records at the same time without time to coordinate.

How it works

1. Give each machine a unique number when it starts.
2. Build each ID from the current time in milliseconds, the machine number and a counter for IDs created in the same millisecond.
3. Two machines can’t collide because their machine numbers differ; one machine can’t collide because time or the counter always moves forward.
4. Because time comes first, sorting by ID roughly sorts by creation time.

Example

A Snowflake-style ID uses 41 bits of time, 10 bits of machine number and 12 bits of counter: 1,024 machines, each making up to 4,096 IDs per millisecond, for about 69 years, all in a 64-bit number.

Key terms

Auto-increment

The database counts 1, 2, 3…; simple, but only one machine can hand them out.

UUID

A 128-bit random (v4) or time-ordered (v7) ID that is unique without coordination.

Snowflake ID

A 64-bit ID made of a timestamp, a machine number and a counter.

Sortable

IDs that increase with time, so sorting by ID sorts by creation time.

Clock skew

Machines’ clocks disagreeing or jumping back, which can break time-based IDs.

In this lesson you'll learn

• The five ways to generate IDs at scale, and what each costs.
• How a Snowflake ID packs time, machine and sequence into 64 bits, and how to decode one.
• What happens on bursts and when clocks go backwards, and how ordered the IDs really are.

Sections

1. Theory: five options
2. Theory: Snowflake bits
3. Theory: bursts and clocks
4. Theory: sortable?
5. Theory: bit budget
6. Design: pick an ID
7. Check yourself

Cheat sheet

The interactive simulations and the quiz load with JavaScript.