Lesson 4.3 · 15 min
Message queues
Run a live broker: shrink the visibility timeout, crash consumers before the ack, inject a poison message, and watch duplicates, dead letters and backlogs appear.
The idea in plain words
A message queue is like a to-do tray between two teams: one team drops tasks in whenever it likes, the other picks them up at its own pace. Nobody has to wait for the other, and a rush just makes the tray fuller for a while.
The problem it solves
When one service calls another directly, both must be up and fast at the same moment. A slow step (sending email, resizing images) keeps users waiting, and a traffic spike hits everything at once.
How it works
- The producer writes a message to the queue and moves on immediately.
- The queue stores it safely until someone takes it.
- Consumers take messages at their own pace and process them.
- A message is removed only after the consumer confirms it is done; if the consumer crashes, the message is offered again.
- That means a message can be processed twice, so consumers must handle repeats safely.
Example
A sign-up request saves the user and puts “send welcome email” on a queue, then returns in 50 ms. A worker sends the email a second later. If the email service is down for 10 minutes, emails wait in the queue and go out when it is back; no sign-up fails.
Key terms
- Producer / consumer
- The producer puts messages on the queue; the consumer takes them off and does the work.
- At-least-once delivery
- Every message arrives, but some may arrive twice.
- Idempotent
- Doing the same thing twice has the same effect as doing it once.
- Visibility timeout
- How long a taken message is hidden from others before it is offered again.
- Dead-letter queue (DLQ)
- Where messages go after failing too many times, so they stop blocking the rest.
In this lesson you'll learn
- How visibility timeouts, acks and redelivery work, and why duplicates are normal.
- What poison messages, dead-letter queues and backlogs look like, and how to size consumers.
- When to use a work queue, pub/sub, an ordered partition, or no queue at all.
Sections
- Theory: run a broker
- Theory: five configurations
- Theory: work queue vs pub/sub
- Design: pick a pattern
- Check yourself
Cheat sheet
| Delivery | at-least-once: idempotent consumers |
|---|---|
| Visibility timeout | > longest processing time |
| Backlog growth | arrival rate − service rate |
| Poison message | after N tries → DLQ |
| Publish reliably | transactional outbox |
The interactive simulations and the quiz load with JavaScript.