How to Read a Packet Capture (Concepts)
In one line
Packet capture is the last referee that decides "who is lying". However, this lab environment has no capture privileges, so you learn how to read as a concept and the labs substitute ss and /proc/net.
Why this exists
A log is an application's claim, and ss is the kernel's summary. Neither directly shows "how packets actually went back and forth". So in the following three situations you end up needing a capture.
- When the logs on both sides tell different stories (the client says it sent, the server says it did not receive)
- When you have to see something that happens only inside the kernel, such as retransmission and fragmentation
- When you have to check whether an intermediate device (proxy, LB, NAT) is modifying something
How it works
You can count the essential options of tcpdump on one hand.
tcpdump -nn -i any 'host 10.0.3.11 and tcp port 8080'
tcpdump -nn -i any 'icmp[icmptype] == 3 and icmp[icmpcode] == 4'
tcpdump -nn -i any -w /tmp/cap.pcap -c 2000
-nndoes not resolve host names and port names. It prevents the incident in which DNS lookups go out again during a capture.-i anyis all interfaces. It is especially useful in container environments.- Save to a file with
-wand open it in Wireshark. Reading on screen gives only a rough flow.
The core of the reading technique is the TCP flag notation.
| Notation | Meaning | What happened then |
|---|---|---|
[S] |
SYN | Connection attempt |
[S.] |
SYN+ACK | The peer accepted |
[.] |
ACK | Data acknowledgment |
[P.] |
PSH+ACK | Data transfer |
[F.] |
FIN+ACK | Start of a clean close |
[R] / [R.] |
RST | Refusal or forced close |
With just this table, two failures are told apart at a glance.
- refused: one
[S]goes out, one[R.]comes back, and it ends. Two lines and it is over. - timeout: the
[S]of the same sequence repeats at 1-second, 2-second, and 4-second intervals and there is no response line at all.
For diagnosing an MTU black hole, you capture ICMP type 3 code 4 (Fragmentation needed). This is why a policy of "block all ICMP for security" is itself a cause of outages. What may be blocked is only about echo request/reply, and type 3 code 4 must be let through. On IPv6, Packet Too Big (type 2) plays the same role, and there you do not even have a choice.
What it looks like in the field
Capturing is hard in containers. When capabilities are removed, as in this lab environment, tcpdump cannot open a socket. On Kubernetes, attaching a diagnostic Pod to the same network namespace with an ephemeral container (kubectl debug) is the standard workaround. If you can get onto the node, you specify the Pod's veth on the node and capture.
So the practical order hardens like this. Narrow things down as far as you can with ss and curl -w, and open a capture only when the two claims still differ. A capture is powerful but expensive.
What we will do next
The labs of this course all proceed without captures. Instead, there is a step where you read /proc/net/tcp directly to check the socket state the kernel holds — it is the way to get closest to the kernel's truth without a capture.