Skip to content

A few notes on BBR queueing under bufferbloat#

2024-09-18

BBR (Bottleneck Bandwidth and Round-trip propagation time) tries to model the bottleneck link explicitly rather than react to packet loss. In theory this makes it immune to bufferbloat — it stops sending faster once the queue starts building. In practice, there are edge cases.

What BBR does#

BBR periodically probes for:

  • Bandwidth: the max throughput observed in a window (default 10 s).
  • RTT: the min RTT observed in a window (default 10 s).

Its sending rate is set to BW * gain / RTT, where gain cycles through values (1.25 in probe-up phase, 0.75 in probe-down, 1.0 otherwise). The idea is that periodically the sender pushes harder to see if there's more BW, then backs off to drain any queue that built.

When it works#

  • Path capacity is stable.
  • Buffers are reasonably sized.
  • Not many concurrent flows.

When it breaks down#

  • Very large buffers (bufferbloated home routers): the queue can grow by tens of MB during a probe-up phase before BBR notices RTT inflation and backs off. You still get high latency.
  • Multiple BBR flows: BBR flows compete somewhat aggressively; two BBR flows sharing a link can each estimate ~60% of the link BW and overshoot together.
  • CUBIC vs BBR fairness: BBR is generally unfair to CUBIC — it takes more than its fair share. BBRv2 (2018) tries to fix this via loss signaling.

Real numbers I've seen#

Testing over a residential 500 Mbps link (0.5 ms RTT to the CMTS, 5-15 ms to the ISP core):

  • Idle: 5 ms RTT.
  • Single BBR flow: 6-8 ms RTT sustained, 470 Mbps throughput.
  • Two BBR flows: 15-30 ms RTT, ~230 Mbps each.
  • BBR + CUBIC: BBR gets ~380, CUBIC gets ~90.

Takeaway#

BBR is a real improvement over CUBIC for individual flows on WAN paths. For a proxy VPS, it's basically always the right choice. For a residential router with mixed traffic, you probably want fq_codel on the interface regardless of what your endpoints run.

Reference#

  • Cardwell et al., BBR: Congestion-Based Congestion Control (ACM Queue 2016).
  • Cardwell et al., BBRv2: A Model-based Congestion Control (IETF ICCRG 2018).