Resilience
4 engineering logs on Resilience. Newest first.
- Retry storms: how the resilience layer became the outage
Why retries at every layer multiply rather than add, why a dependency recovering into full retry load never actually recovers, and why a retry budget is the only control that bounds the total.
- Backpressure is not rate limiting, and treating them the same fills your queues
Why a rate limit set from a number you chose cannot protect a system whose capacity moves, how an unbounded queue converts an overload into an outage, and why the useful signal comes from the consumer rather than the producer.
- Jitter is the cheapest reliability fix you are not using
Why independent clients converge on the same instant without any coordination, how retries, cron schedules, health checks and reconnects all self-synchronise, and why adding randomness is a one line fix for a class of outage.
- Designing Resilient APIs: Failure-Handling Patterns for Distributed Systems
Practical resilience patterns for distributed APIs: fail-fast, retries with backoff, circuit breakers, bulkheads, fallbacks, rate limiting, failover, and observability.