TT Lab
Get started
Learn Learning paths Courses

My TCP Parcel Arrived in Pieces

Rebuild the Boundaries of a Broken Parcel

Continue in TT Lab

Goal

Implement framing, deadlines, and half-close, and verify them on real TCP.

Why it matters

If you trust a received piece to be one whole message, it succeeds locally but breaks on real input. You express the length, the overall time budget, and socket ownership in code so that you can tell data loss apart from an infinite wait. You use only the standard library, and no internet access or installation is needed.

Steps

  1. The envelope holds a byte count, not a character count — In codec.py, create MAX_PAYLOAD=4096 and encode(payload). Prepend an unsigned 4-byte big-endian length to the bytes body and return bytes. An empty body is allowed, and anything that is not bytes or exceeds 4096 bytes is a ValueError.
  2. A header can be cut in half too — Add a Decoder to codec.py. The receive state is separate for each new instance, and feed(chunk: bytes) returns a list[bytes] of completed bodies. Even if you cut one frame at any byte and feed it in two calls, it restores the frame, and incomplete data is kept until the next call.
  3. Three parcels arrived in one bag — Improve Decoder.feed so that if one chunk holds several frames, it returns all of them in order. If part of the next frame remains, keep it. An empty body is also one message, and an empty chunk is not EOF.
  4. Delivered versus damaged in transit — Create Decoder.finish(). If there is an incomplete header or body, raise EOFError, and if there is no leftover, return None. Do not mistake a correctly interpreted length-0 body for a mid-frame close.
  5. Reject the 4 GiB parcel notice — The Decoder must raise a ValueError as soon as it receives a length header larger than 4096. Do not wait until the actual body arrives or allocate a buffer of that length first. A 4096-byte body is still allowed. After an error, discard that connection.
  6. Will it wait forever if you send one character at a time? — In transport.py, implement recv_frame(sock, timeout, clock=None). The header and body of a whole frame share the same time budget. clock is a clock function that takes no arguments and returns seconds, and the default is time.monotonic. Use only gettimeout/settimeout/recv of sock. A normal frame returns bytes, EOF before receiving even one byte of a new header is None, a mid-frame EOF is EOFError, an oversized length is ValueError, and a missed deadline is TimeoutError. Allow only a positive finite timeout, and restore the previous socket timeout on both success and failure.
  7. Only half of "it was sent" is true — In transport.py, add send_frame(sock, payload). Send the entire encoded frame and do not hide transmission errors. Even when send handles only some of the bytes, you must not drop part of the frame. The caller decides the socket's send timeout.
  8. Get the return slip over a real connection — In server.py, create handle_connection(sock, timeout). Take over ownership of the received socket and echo the same bytes for each frame. End on the normal EOF (None) of recv_frame, but echo a b"" body. Close the socket on every exit path and pass protocol errors to the caller. The checker creates a real TCP connection on an ephemeral port of 127.0.0.1, and even after the client half-closes only its sending side, it must still receive the remaining responses.

Notes

Put all files under /root/tcp-parcel. The example files contain function skeletons, not answers. Keep the functions from earlier steps as you go. Grading runs the submitted code and aborts if it takes more than 12 seconds. The send timeout of send_frame is decided by the caller. Files do not persist after the lab session ends, so keep them separately.

The envelope holds a byte count, not a character count

In codec.py, create MAX_PAYLOAD=4096 and encode(payload). Prepend an unsigned 4-byte big-endian length to the bytes body and return bytes. An empty body is allowed, and anything that is not bytes or exceeds 4096 bytes is a ValueError.

The !I of struct.pack is a 32-bit unsigned integer in network order. The caller does the UTF-8 conversion first.

A header can be cut in half too

Add a Decoder to codec.py. The receive state is separate for each new instance, and feed(chunk: bytes) returns a list[bytes] of completed bodies. Even if you cut one frame at any byte and feed it in two calls, it restores the frame, and incomplete data is kept until the next call.

Do not interpret the header before you have collected at least 4 bytes. Even when you know the length, the whole body may not be here yet.

Three parcels arrived in one bag

Improve Decoder.feed so that if one chunk holds several frames, it returns all of them in order. If part of the next frame remains, keep it. An empty body is also one message, and an empty chunk is not EOF.

After you remove a completed frame, keep checking for the next header with a while loop. If you create a new Decoder on every feed, the leftover disappears.

Delivered versus damaged in transit

Create Decoder.finish(). If there is an incomplete header or body, raise EOFError, and if there is no leftover, return None. Do not mistake a correctly interpreted length-0 body for a mid-frame close.

The external event of a connection closing and the mere input of "no bytes received this time" are separate. Call finish when you actually know EOF has occurred.

Reject the 4 GiB parcel notice

The Decoder must raise a ValueError as soon as it receives a length header larger than 4096. Do not wait until the actual body arrives or allocate a buffer of that length first. A 4096-byte body is still allowed. After an error, discard that connection.

Check the length right after reading the header. That a value can be represented as a 32-bit integer and that the service allows it are different things.

Will it wait forever if you send one character at a time?

In transport.py, implement recv_frame(sock, timeout, clock=None). The header and body of a whole frame share the same time budget. clock is a clock function that takes no arguments and returns seconds, and the default is time.monotonic. Use only gettimeout/settimeout/recv of sock. A normal frame returns bytes, EOF before receiving even one byte of a new header is None, a mid-frame EOF is EOFError, an oversized length is ValueError, and a missed deadline is TimeoutError. Allow only a positive finite timeout, and restore the previous socket timeout on both success and failure.

Decide deadline=clock()+timeout once at the start and, before every recv, give the remaining time to settimeout. recv(n) is not a promise to give you all n bytes.

Only half of "it was sent" is true

In transport.py, add send_frame(sock, payload). Send the entire encoded frame and do not hide transmission errors. Even when send handles only some of the bytes, you must not drop part of the frame. The caller decides the socket's send timeout.

sendall either sends everything or raises an exception. If you repeat the call yourself, advance only by the length that send returns and treat a return of 0 as a termination error.

Get the return slip over a real connection

In server.py, create handle_connection(sock, timeout). Take over ownership of the received socket and echo the same bytes for each frame. End on the normal EOF (None) of recv_frame, but echo a b"" body. Close the socket on every exit path and pass protocol errors to the caller. The checker creates a real TCP connection on an ephemeral port of 127.0.0.1, and even after the client half-closes only its sending side, it must still receive the remaining responses.

Tie the lifetime together with with sock. if not payload confuses None with an empty body. You can still respond to a peer that has closed only its sending direction.