Designing Resilient Webhook Delivery Systems

When an automotive record updates, our system fires webhooks to thousands of subscribers. But what happens when their servers go down?

Exponential Backoff

We designed our webhook dispatchers to use exponential backoff algorithms with a randomized jitter. If a delivery fails, we wait 1 minute, then 5, then 20, up to 72 hours, preventing us from accidentally DDOS-ing a struggling client server while ensuring eventual delivery.