Further Reading: Overload & Backpressure

Back to Overload & Backpressure

The Tail at Scale (Dean & Barroso, 2013)

Paper: The Tail at Scale

Why it matters: Explains how overload affects tail latency and provides techniques for handling overload gracefully.

Key Excerpts

On overload behavior:

"At scale, overload is inevitable. The question is not whether overload will occur, but how gracefully the system degrades when it does."

Key insight: Systems will experience overload. The goal is to degrade gracefully, not fail catastrophically.

On backpressure:

"Backpressure is essential for preventing cascading failures. When a component is overloaded, it must signal upstream components to slow down."

Relevance: Directly addresses the backpressure mechanisms we discussed. The paper provides techniques for implementing backpressure effectively.

Techniques for Handling Overload

  1. Load shedding: Drop requests when overloaded
  2. Request prioritization: Process important requests first
  3. Adaptive backpressure: Adjust backpressure based on load
  4. Graceful degradation: Reduce functionality instead of failing

Site Reliability Engineering (Google SRE Book)

Book: Site Reliability Engineering: How Google Runs Production Systems

Why it matters: Google's approach to handling overload and implementing backpressure in production systems.

Key Concepts

Overload Protection: - How to detect overload - When to shed load - How to prioritize requests

Circuit Breakers: - When to open circuit breakers - How to implement circuit breakers - Recovery strategies

Load Shedding: - What requests to drop - How to implement load shedding - Monitoring load shedding

Relevance: Provides real-world examples and best practices from Google's production systems.

Why Do Internet Services Fail? (Oppenheimer et al., 2003)

Paper: Why Do Internet Services Fail, and What Can Be Done About It?

Why it matters: Analysis of real-world service failures, including how overload contributes to failures.

Key Findings

Common failure causes: 1. Overload: 40% of failures due to overload 2. Cascading failures: Overload often triggers cascading failures 3. Insufficient capacity: Under-provisioning common cause

On cascading failures:

"Cascading failures are often triggered by overload. When one component fails, load shifts to other components, causing them to fail as well."

Relevance: Explains why backpressure and load shedding are critical for preventing cascading failures.

Prevention Strategies

  1. Capacity planning: Provision adequate capacity
  2. Load shedding: Drop requests when overloaded
  3. Circuit breakers: Stop calling failing services
  4. Rate limiting: Limit request rates
  5. Monitoring: Detect overload early

Circuit Breaker Pattern

Martin Fowler's Article

Article: Circuit Breaker

Why it matters: Classic explanation of the circuit breaker pattern, with implementation details.

Key Concepts

Three States: 1. Closed: Normal operation, calls downstream 2. Open: Fails fast, doesn't call downstream 3. Half-open: Testing if downstream recovered

Implementation: - Failure threshold: When to open circuit - Timeout: How long to stay open - Success threshold: When to close circuit

Relevance: Provides the foundation for implementing circuit breakers to prevent cascading failures.

Additional Resources

Papers

"The Datacenter as a Computer" (Barroso & Hölzle, 2018) - Chapter on overload handling - Link

"Delay-Tolerant Load Balancing" (Dean, 2009) - Techniques for handling variable load - Link

Books

"Release It!" by Michael Nygard - Chapter on circuit breakers and bulkheads - Real-world examples of overload handling

"Designing Data-Intensive Applications" by Martin Kleppmann - Chapter on reliability - Overload handling in distributed systems

Online Resources

Google SRE Book: Site Reliability Engineering - Chapter 21: Handling Overload - Chapter 22: Addressing Cascading Failures

Netflix Hystrix: Hystrix Documentation - Circuit breaker implementation - Load shedding strategies

Key Takeaways

  1. Overload is inevitable: Plan for overload, don't assume it won't happen
  2. Backpressure is essential: Components must signal when overloaded
  3. Load shedding prevents cascades: Better to drop some requests than fail completely
  4. Circuit breakers help: Stop calling failing services to prevent cascades
  5. Monitor and alert: Detect overload early, respond quickly