TT Lab
Get started
Learn Learning paths Courses

SSE — How the Server Speaks First

Build the Parser the Spec Describes

Continue in TT Lab

Goal

You will implement the event stream parsing rules of the WHATWG HTML Standard yourself, and build on top of them a mechanism for resuming through reconnection.

Why it matters

SSE is "just HTTP," so you can use it without any library. That is why in practice a one-line parser of split("\n\n") is very common, and that parser collapses immediately on input that arrives in pieces. TCP does not guarantee that data arrives in the units you sent it.

Resuming is even harder. Depending on whether the id buffer is reset for each event and whether a truncated last event is dispatched, the screen after reconnection has gaps or overlaps. The rules are all written in the specification, and this lab is about turning those rules into code.

What to build

In /root/work/resume/client.py, provide the following.

Name Contract
StreamParser feed(조각) (the placeholder is the chunk) → the list of events completed this time. Attributes last_event_id · reconnection_time
replay(events, last_event_id) The server-side replay window — returns what comes after that id
resume_headers(parser) The header dictionary to send on reconnection

One event is a dictionary: {"event": ..., "data": ..., "id": ...}.

Steps

  1. Chunks and line endings — feed gives the same result however the input is cut
  2. Field rules — comments · lines without a colon · joining data
  3. The id buffer is not reset
  4. retry only for ASCII digits
  5. Replay window — replay
  6. Reconnection header — resume_headers
  7. Measure it yourself and write it down — /root/work/resume/07-gap.txt
  8. Wrap-up — /root/work/resume/08-notes.md

Notes

Chunks and line endings

In /root/work/resume/client.py, create StreamParser. feed(조각) (the placeholder is the chunk) returns the list of events completed by this chunk. The result must be the same wherever the input is cut when it arrives, and it must accept all three line endings: CRLF, LF, and CR. Drop a single BOM at the very start.

Run mkdir -p /root/work/resume. Leave the tail you haven't processed yet in an instance buffer, and process only as far as you can find a line ending. One pitfall — if the buffer ends with \r, it may be the first half of a \r\n, so you must wait for the next chunk. In this step you only need to handle data:.

Field rules

Add the specification's field rules. Ignore a line that starts with a colon; for a line with no colon, the whole line is the field name and the value is an empty string; data appends its value and adds one newline each time; and event changes the event type (the default is message). Ignore unknown fields.

Strip only one space before the value. When dispatching, check whether the data buffer is an empty string before removing the newline, and if it is not empty, remove just the one final newline. So a block with only a data line goes out as an event whose data is an empty string.

The id buffer and the last id string

Handle the id field. If the value contains U+0000 NULL, ignore that field; otherwise set the internal id buffer to that value. The last_event_id used for reconnection is updated to that buffer value at the point of dispatching an event, and it is not reset even after dispatching. Put that value in the id of the event you dispatch.

In the dispatch step, the specification says to ① set the last id string to the buffer value and ② return from there if the data buffer is empty. Because the order is as written, a block with only id: and no data also moves the reconnection position, but if no blank line arrives and dispatching itself never happens, it doesn't move. And the id is a string — don't convert it to an integer.

retry only for ASCII digits

Handle the retry field. Only when the value consists solely of ASCII digits, set reconnection_time to that integer; otherwise ignore it. The initial value is None. A block with only retry: and no data is not an event.

Python's str.isdigit() also returns true for full-width digits. Check both conditions together, as in value.isascii() and value.isdigit(). An empty string, +100, 3s, and 3.5 are all ignored.

The server-side replay window

Add replay(events, last_event_id). events is a list of dictionaries that have an id. If last_event_id is empty or None, return everything; if it is an id inside the window, return what comes after it; if it is an id not in the window, return None.

None is the signal "cannot resume, so fetch everything again from the beginning." If you send only what remains, the middle has a hole and no one knows it. Compare ids as strings, exactly — ids like b7-1042 are really used.

Reconnection header

Add resume_headers(parser). Only when the parser's last_event_id is not an empty string, return {"Last-Event-ID": 값} (the placeholder is the last id); otherwise return an empty dictionary.

The specification includes the header only when the last id string is not an empty string. If you send an empty value, the server can't tell whether it means "from after 0" or "from the beginning." Use the header name's capitalization and hyphen exactly as given.

Measure the truncated event yourself

Feed the whole of /opt/fixtures/sse-resume/cut.txt through your parser, and write the resulting values in /root/work/resume/07-gap.txt as three lines of 이름=값 (the placeholders are the name and the value). events — the number of events dispatched. last_event_id — the last id at that point. retry_ms — the reconnection_time at that point.

In that file, the last event is cut off without a blank line. The specification says not to dispatch such an event, so all three numbers depend on that rule. The grader feeds the same file through your parser again and compares with the values you wrote, so don't estimate by eye.

What prevented the gap

In /root/work/resume/08-notes.md, write at least three lines: what you see after reconnection without Last-Event-ID, what goes out of sync if you dispatch a truncated event, and what happens if you send only what remains for an id outside the replay window.

The text must contain Last-Event-ID, 재접속, and 잘린 (or 불완전) (the Korean words for "reconnection", "truncated", and "incomplete"). These three are the things that actually get people paged when SSE goes into production.