All examples

Examples

A token bucket rate limiter, request by request

A burst of five requests meets a bucket of three tokens: three get through, two are refused with a 429, and the bucket refills so an idle client can burst again.

This video was made with the System design skill.

  • 1:05
  • System design
  • Narrated by Iapetus
  • English

Made from a prompt

Design a rate limiter with a token bucket. Show a burst of requests and which ones get a 429.

Video transcript

Five simultaneous requests can overwhelm your API unless you control the burst.

You start with a client calling your API directly, but sustained traffic pushes utilization to ninety-five percent.

We insert a token bucket limiter before the API, starting with three tokens and refilling one each second.

Each request spends one token, so five arriving together let three through and reject two with HTTP four twenty-nine.

Why reject instead of queueing? Immediate refusal bounds waiting, and clients should retry later with backoff.

The bucket refills continuously up to its capacity, so an idle client can burst again.

The cost is rejected work, and multiple API replicas must share an atomic bucket per client.

Use this at API entry points: the client sends, the limiter admits, and the API handles accepted work.

Your topic, drawn in about a minute.

Your first 2 videos are free, up to 1 minute each, watermarked. No card required. Full product, every voice and style included.