A (In)Cast of Thousands: Scaling Datacenter TCP to Kiloservers and Gigabits

Abstract

This paper presents a practical solution to the problem of high-fan-in, high-bandwidth synchronized TCP workloads in datacenter Ethernets-the Incast problem. In these networks, receivers often experience a drastic reduction in throughput when simultaneously requesting data from many servers using TCP. Inbound data overfills small switch buffers, leading to TCP timeouts lasting hundreds of milliseconds. For many datacenter workloads that have a synchronization requirement (e.g., file system reads and parallel data intensive queries), incast can reduce throughput by up to 90%. Our solution for incast uses high-resolution timers in TCP to allow for microsecond-granularity timeouts. We show that this technique is effective in avoiding incast using simulation and real-world experiments. Last, we show that eliminating the minimum retransmission timeout bound is safe for all environments, including the wide-area.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Feb 01, 2009
Accession Number
ADA512849

Entities

People

  • Amar Phanishayee
  • David G. Andersen
  • Elie Krevat
  • Garth A. Gibson
  • Gregory R. Ganger
  • Hiral Shah
  • Vijay Vasudevan

Organizations

  • Carnegie Mellon University

Tags

Communities of Interest

  • Space

DTIC Thesaurus Topics

  • Bandwidth
  • Clocks
  • Congestion
  • Data Centers
  • Discrete-Event Simulation (Model)
  • Estimators
  • High Resolution
  • Hypervelocity Flow
  • Measurement
  • Networks
  • Operating Systems
  • Packet Loss
  • Scheduling (Production)
  • Statistics
  • Test And Evaluation
  • Transport Protocols
  • Workload

Fields of Study

  • Computer science

Readers

  • Applied Combinatorial Optimization and Logic Circuit Design.
  • Computer Networking