HTTP — A Stateless Protocol and Connection Reuse

In a nutshell

HTTP is a stateless protocol that returns one response per request, and the history of performance has been the history of how to use TCP connections sparingly on top of it.

Why this was needed

HTTP/1.0 opened and closed a new TCP connection for every request. If you open a page with 30 images, you make 30 connections. Each connection adds one round trip (two or three if TLS is used), so the latency translated directly into what users felt.

How it works

HTTP/1.1 keep-alive keeps the connection open after the response and reuses it for the next request. It greatly reduced the round-trip cost, but it has limits. On one connection, responses must come out in the order of the requests. If an earlier request is slow, the later requests wait (head-of-line blocking at the application layer). Browsers worked around it by opening about 6 connections per domain, and so even the technique of scattering resources across several domains became popular.

HTTP/2 eliminated this problem by multiplexing requests as streams within one connection. It also compresses headers. However, it is still on top of a single TCP connection, so if one packet is lost, all the streams behind it stall together. It removed queuing at the application layer, but queuing at the transport layer remained.

HTTP/3 moved onto QUIC (UDP-based) and gained independent delivery per stream. Loss in one stream does not block other streams.

The meanings of methods are also part of the protocol. GET is safe (no side effects) and idempotent, PUT and DELETE are not safe but are idempotent, and POST is neither. Idempotency is directly tied to retryability. When a timeout occurs, you cannot tell whether the request reached the server, so carelessly retrying a non-idempotent request creates an order twice. That is why payment APIs require idempotency keys.

You must not lump status codes together either. 4xx means you must fix the request, and 5xx is a server problem, so a retry may be meaningful. In particular, 502, 503, and 504 have different causes. 502 means the gateway received a bad response from the backend, 503 means the service itself said it cannot cope, and 504 means the gateway grew tired of waiting for the backend. If you do not distinguish these three in the logs, you end up digging in the wrong place.

What it looks like in the field

Connection pool settings are a common accident point. The client tries to reuse connections with keep-alive, but if the idle timeout of the server or of an intermediate load balancer is shorter, a race arises in which the client sends a request on a connection the server has just closed. The symptom is intermittent connection resets, and it appears at a steady rate regardless of load. The standard practice is to set the client's idle timeout shorter than the server's.

What to check in the quiz that follows

Check whether you can explain what each version solved and what remained, and why idempotency is the premise of retry design.