TT Lab
Get started
Learn Learning paths Courses

One Slow Connection Froze Every Other One

Writable but Refused: Nonblocking Connection Setup

Continue in TT Lab

In one line

Write readiness is not a stamp of connection success. After observing completion readiness in the connecting state, read SO_ERROR once, and keep the first result in the application state.

Why this was needed

If you test only an example where the server accepts connections, code that marks connected as soon as EVENT_WRITE arrives also passes. Apply the same code to a closed port, and the green light comes on even though the connection was refused. That is because you used "no need to wait any longer" and "succeeded" to mean the same thing. You need the view that "failure also produces a completion event" before you can trust the diagnostic tool's success rate.

What you build here is a diagnostic tool that gathers the TCP connection establishment results of several targets. When a connection is made, it closes it right away, and it does not send an HTTP request or inspect a TLS certificate. So connected does not mean login is possible, the API is healthy, or data storage succeeded. If the name of the result shown to the user goes beyond the actual scope of verification, even correct code becomes an inaccurate operations tool.

How it works

According to connect(2), a non-blocking TCP connection on Linux may finish immediately or be in progress. You use the integer result of Python's connect_ex for state transitions. This lab is limited to AF_INET/SOCK_STREAM. Do not carry AF_UNIX's in-progress EAGAIN rule over as is to numeric IPv4 TCP.

Current state Observation Next state
new connect_ex result 0 or EISCONN connected, error 0
new EINPROGRESS or EALREADY pending, result undecided
new Any other error failed, that errno
pending SO_ERROR is 0 after WRITE connected
pending SO_ERROR is an error after WRITE failed
new/pending Absolute deadline or cancellation timed_out or cancelled

You do not call connect_ex again every time it is pending. You are waiting for the result of an attempt that has already started. Do not treat reusing a failed socket as a general retry method; close it and create a new attempt. This diagnostic tool does no automatic retries itself, so one result corresponds to one connection attempt.

SO_ERROR in socket(7) clears the error as it gets it. If the function that writes the log reads it first and the state decision function reads it second, their results can differ. Keep Dial.error and preserve the value of the first completion decision. Requesting the "latest error" every time you read is not a way to raise accuracy.

python3 /opt/fixtures/reactor/connect_probe.py

The experiment calls listen on one socket and only bind on the other. Both hold their ports, so there is no race of "find a free port, close it, then connect again." In the case of an in-progress connection observed in a Linux container, both had writable=true, but SO_ERROR was 0 and ECONNREFUSED respectively. When the refusal result was read again, it was 0. The value of an errno constant like the number 111 can differ by OS, so use errno.ECONNREFUSED in code. If it completed immediately, classify by the first return value of connect_ex.

Don't sneak in name resolution

If you put in a domain instead of a numeric address, a name resolution step arises before the connection. The fact that you switched the socket to non-blocking alone does not mean that name resolution also became non-blocking. This lab validates host as a numeric IPv4 and rejects localhost too. IPv6 address candidate racing, DNS caching, and Happy Eyeballs are separate topics that need additional design. Marking the scope is not to hide errors but to make clear which stage was measured.

The deadline is set not when the attempt starts but when the Dial is created. That is because a connection that stays long in the queue also consumes total time. If the deadline has already passed before calling connect_ex, it does not start a network attempt and ends with timed_out. A terminal state is not overwritten by a later cancel or expire. Even if a cancel-all arrives right after a success, it does not change an already confirmed success into a cancellation.

What it looks like in the field

The Cloudflare Software Engineer, Spectrum posting checked on 2026-09-13 deals with debugging Linux sockets, TCP connection states, and lifecycle problems. Here you practice connection establishment and preserving the cause of failure, among those. It is not that company's official training or a hiring guarantee, and you have to learn Go, Rust, and large-scale operations skills separately.

What you will do in the next check

In the next quiz, you check the difference between observation and decision. After that, in the combined lab, you build Dial's transition from new to pending to a terminal state and test the success and refusal of real loopback TCP. After explaining in your own words why the seemingly right "WRITE means success" implementation and the "keep re-querying SO_ERROR" implementation are wrong, confirm it with the quiz.