I renumbered a field and the old client silently read the wrong value

Change the schema in front of an old client

Goal

You leave the "old client", which has the v1 schema hard-coded in its code, as it is and change the schema to v2. You add new fields with new numbers, seal the deleted field with reserved, and reproduce how the old client wrongly reads, without any error, bytes whose numbers were deliberately changed. Then you measure the traps of default values and presence, and leave a final schema with optional and reserved and a compatibility table.

Why it matters

Clients and servers are never deployed at the same moment. There are rollbacks, and old bytes remain in logs. So the safety of a schema change must be judged not by "does it compile" but by "does an old binary read new bytes, and a new binary old bytes, correctly". The wire has no names and only numbers, so changing a number is the same as deleting a field and creating a new one — yet the parser raises no error at all and reads the value of the changed number under the old name. The purpose of this lab is to reproduce that incident and engrave it in memory.

Steps

Run python3 /opt/app/grpc/old_client.py /opt/app/grpc/order_v1.bin and save the four lines of output as they are to /root/grpc/evolve/01-old.txt.
Write /root/grpc/evolve/v2.proto. Start with syntax = "proto3";, in message Order keep v1's int32 id = 1; int32 qty = 2; string note = 3; unchanged, and add int64 unit_price and string currency with new numbers.
In v2.proto, delete note and leave reserved 3; and reserved "note";. The rest stays the same.
In /root/grpc/evolve/make_v2.py, use /opt/app/grpc/pbmini.py to encode id=9001, qty=2, unit_price=15000, currency="KRW" with the numbers of v2.proto and save it to /root/grpc/evolve/order_v2.bin, then save the output of reading that file with the old client to /root/grpc/evolve/04-old-reads-v2.txt.
/opt/app/grpc/renumbered.bin is the same order made with /opt/app/grpc/renumbered.proto (a schema with the numbers reassigned). Save the output of reading it with the old client to /root/grpc/evolve/05-renumbered.txt, and below it write one line truth: id=<진짜 id> qty=<진짜 qty> (the true id and qty) and one line why: <왜 오류 없이 틀린 값이 나왔는지> (why a wrong value came out without an error).
With pbmini, make an order with id=9002, currency="KRW" and qty=0 in two ways. /root/grpc/evolve/06-zero.bin uses implicit presence (0 is not sent), and /root/grpc/evolve/06-zero-explicit.bin uses explicit presence (0 is sent too). In /root/grpc/evolve/06-presence.txt, write four lines: implicit=<16진수>, explicit=<16진수>, implicit_qty=<옛 클라이언트가 읽은 qty>, and explicit_qty=<옛 클라이언트가 읽은 qty> (the hex of each file, and the qty the old client read from each).
Write the final /root/grpc/evolve/Order.proto. It includes optional int32 qty = 2;, reserved 3; + reserved "note";, unit_price and currency, and message Customer { string name = 1; int32 tier = 2; } and Order's Customer customer = <새 번호>; (with a new number).
Write a markdown table in /root/grpc/evolve/compat.md. The first column is the six keys add-new-number remove-and-reserve reuse-number renumber int32-to-string rename-only, the second column is safe or unsafe, and the third column is a one-line reason.

Notes

Source of the rules: proto3 language guide — updating a message type, field presence.
old_client.py is a program with the v1 schema hard-coded. Its output is four lines: id= qty= note= unknown=. In step 6, have it read each of the two files.
Using pbmini: import sys; sys.path.insert(0, "/opt/app/grpc"); import pbmini as pb; pb.encode_message([(1, "int32", 9001), (2, "int32", 2)]).
The grader reads .proto leniently (comments and whitespace are free). But number reuse, a missing reserved, and type changes always fail.
The rename-only in step 8 is on the basis of the binary wire — there are no names on the wire, so changing only a name has no effect on the bytes (it is different for JSON and text formats).

Read the v1 bytes with the old client

Save the four lines of output of python3 /opt/app/grpc/old_client.py /opt/app/grpc/order_v1.bin as they are to /root/grpc/evolve/01-old.txt.

After mkdir -p /root/grpc/evolve, just save the output with >. The old client reads believing field 1 is id, 2 is qty, and 3 is note, and writes any other number under unknown. Do not write it by hand; save the output — the grader runs the same command again and compares.

New fields get new numbers

In /root/grpc/evolve/v2.proto, keep v1's three fields as they are and add int64 unit_price and string currency with new numbers.

Do not change a single character of the number, name, or type of an existing field. A new field gets a number that has never been used so far — 4 and 5 are natural, but 6 and 7 are fine too. 19,000–19,999 is a range reserved by the implementation and cannot be used, and 1–15 take a one-byte tag, which is good for frequently used fields.

The grader looks for syntax = "proto3"; and message Order { ... }, and checks whether fields 1 and 2 are the same as v1, whether unit_price is int64 and currency is string, and whether their numbers are not 1, 2, or 3.

Seal the deleted number

In v2.proto, delete note and leave reserved 3; and reserved "note";.

Deleting itself is wire-safe. The danger is someone using 3 again later, and reserved is the device that makes protoc block that as a compile error. A number and a name cannot be mixed in one statement, so you write two lines.

reserved 3; may be anywhere inside the message block.

The old client reads the v2 bytes

In /root/grpc/evolve/make_v2.py, use pbmini to encode id=9001, qty=2, unit_price=15000, currency="KRW" with the numbers of v2.proto and save it to /root/grpc/evolve/order_v2.bin, then save the output of reading that file with the old client to /root/grpc/evolve/04-old-reads-v2.txt.

Use the numbers you chose in v2.proto as they are, like pb.encode_message([(1, "int32", 9001), (2, "int32", 2), (4, "int64", 15000), (5, "string", "KRW")]). note(3) is a deleted field, so do not include it.

The old client does not know 4 and 5 but knows the length from the wire type and skips them, and it reads id and qty correctly. Those two numbers must appear on the unknown= line.

The old client reads the bytes with swapped numbers

Save the output of reading /opt/app/grpc/renumbered.bin with the old client to /root/grpc/evolve/05-renumbered.txt, and append two lines, truth: id=<진짜> qty=<진짜> and why: <이유> (the true values and the reason).

Read /opt/app/grpc/renumbered.proto first — someone renumbered the fields. renumbered.bin is the same order (the same values as order_v1.bin) serialized with that schema.

The old client still believes number 1 is id. Since the wire types are the same (both VARINT), the values come out switched with no error or warning. The truth line is the real values read according to renumbered.proto.

Was qty=0 sent or not sent

Make an order with id=9002, currency="KRW", qty=0 as /root/grpc/evolve/06-zero.bin (implicit presence, 0 is not sent) and /root/grpc/evolve/06-zero-explicit.bin (explicit presence, 0 is sent too), and in /root/grpc/evolve/06-presence.txt write four lines: implicit=, explicit=, implicit_qty=, and explicit_qty=.

A proto3 unlabeled scalar does not serialize its default value (0, ""). So the implicit side does not include a field 2 record at all, and the explicit (optional) side includes (2, "int32", 0) — the two bytes 10 00 are carried in addition.

Have the old client read each of the two files and write the qty. Both come out 0 — the old client cannot tell "received 0" from "received nothing". This is why optional is needed.

The final schema with optional and reserved

Write /root/grpc/evolve/Order.proto — optional int32 qty = 2;, reserved 3; + reserved "note";, int64 unit_price and string currency, message Customer { string name = 1; int32 tier = 2; }, and Order's Customer customer = <새 번호>; (with a new number).

If you attach the optional label, qty gets explicit presence and you can tell sending 0 from not sending. On the binary wire, attaching the label itself does not change the bytes (it only means that 0 gets sent).

Keep Customer as a separate message and use it as a type in Order. The number of customer must be one that has never been used and is not reserved.

Write the compatibility table

Write a markdown table in /root/grpc/evolve/compat.md. The first column is add-new-number remove-and-reserve reuse-number renumber int32-to-string rename-only, the second column is safe/unsafe, and the third column is the reason.

This step sums up what you saw first-hand in this lab. Step 4 (adding with a new number), step 3 (delete and reserve), and step 5 (renumbering) are each the evidence for one row. For a type change, if the wire type changes (int32 VARINT → string LEN), an old parser reads the value with a different meaning. Changing only a name leaves the bytes the same, because there are no names on the binary wire.

Use only one of the verdict words, safe or unsafe — the grader looks at the first verdict word that appears in a row.