HTTP/2 — Many requests on one connection

In one line

HTTP/2 overlays multiple requests as streams on one connection and compresses headers, but in exchange every stream shares that one connection's flow-control window and one TCP byte stream.

Why this was needed

An HTTP/1.1 connection handles only one request at a time. The specification has pipelining, which sends requests back to back without waiting for responses, but responses must come back in request order, so a slow response at the front holds up everything behind it. This is application-level head-of-line blocking. So browsers and client libraries chose to open several connections instead of pipelining, and each connection separately paid for a TCP handshake, a TLS handshake, and the slow start of the congestion window. The cookie and authentication headers repeated identically on every request were also resent in plain form every time.

HTTP/2 in RFC 9113 targets these two wastes. It keeps one connection and opens multiple independent streams on it, and it shrinks headers with compression that holds per-connection state (RFC 7541 HPACK). gRPC runs on top of this, and a good share of the failures where a streaming response suddenly stalls come from the flow-control window explained here.

How it works

Everything in HTTP/2 is a frame. The frame header in section 4.1 is fixed at 9 bytes.

Field	Size	Meaning
Length	24 bits	Length of the payload excluding the header. The default limit is 16,384 (SETTINGS_MAX_FRAME_SIZE)
Type	8 bits	DATA 0x0, HEADERS 0x1, RST_STREAM 0x3, SETTINGS 0x4, PING 0x6, GOAWAY 0x7, WINDOW_UPDATE 0x8 ...
Flags	8 bits	The meaning differs by type, as with END_STREAM and END_HEADERS
R + Stream Identifier	1 + 31 bits	The reserved bit is 0 when sending and ignored when receiving

Streams opened by the client have odd numbers and streams opened by the server have even numbers, and stream 0 is used for control of the whole connection (SETTINGS, PING, GOAWAY). One request starts with a HEADERS frame and ends with the END_STREAM flag, and frames of different streams can flow interleaved in any way within one connection. A connection always starts with the client's preface (24 bytes beginning with PRI * HTTP/2.0) and SETTINGS from both sides, and the way to decide to use HTTP/2 is ALPN's h2 on TLS and prior knowledge (section 3.3) on plaintext.

The number of streams that can be open at the same time is announced by the receiving side through SETTINGS_MAX_CONCURRENT_STREAMS. Section 6.5.2 says the initial value of this is unlimited, and recommends not setting it lower than 100. Here comes a common trap. Before the other side's first SETTINGS arrives, you do not know the limit, so if you rush requests out right after sending the preface, you can exceed the limit.

Flow control (sections 5.2 and 6.9) applies only to DATA frames. There is one window per stream and one for the whole connection, and both start at 65,535 bytes. The sender can send only as much as the smaller of the two windows, and it sends more only when the receiver gives the window back with WINDOW_UPDATE. What you can change with SETTINGS_INITIAL_WINDOW_SIZE is only the initial value of the stream window, and the connection window widens only through WINDOW_UPDATE. So even if you advertise a stream window of 100,000, the first response stops at 65,535 bytes. If there is one client that does not give the window back, a single large response uses up the connection window, and even a 3-byte response on the same connection has to wait.

HPACK uses a static table of 61 entries and a dynamic table (4,096 bytes by default) that builds up once per connection. When the same header is sent a second time on the same connection, it shrinks to a single table index. When you open a new connection, the table is empty too and grows again from scratch. The benefit of compression comes from using a connection for a long time.

What it looks like in the field

Reports are common that a slow page got faster after switching to HTTP/2 but actually got worse on a mobile network. The six HTTP/1.1 connections are independent, so even if a packet is lost on one, the other five keep flowing. An HTTP/2 connection is, from TCP's point of view, a single byte stream no matter how many streams there are, so while one segment is resent, every stream stands still. The next module continues this story.

Another is the failure where a gRPC stream stalls for seconds. If the receiving application takes messages out late, the library gives the window back late, and the sender waits with the window closed. It looks like the network is idle but throughput has hit the floor.

What you will do in the next lab

With the h2 library, you read a frame header, measure HPACK sizes, and build a client that overlays requests on one connection. Then you fix it to respect the concurrent stream limit, and measure the number of bytes at which the window closes in three cases, 16,384, an arbitrary value, and 100,000, confirming the existence of the connection window in numbers. At the end, you put in a relay where TCP stalls once, and compare when a small response ends over HTTP/2 and over HTTP/1.1.