A Field Was Renamed and No Exception Was Raised

Summary

When attaching to someone else's system, what we must protect is not their entire schema but the list of fields we actually read, and that list must be pinned down not in a document but in a test that runs every time, so that a version upgrade cannot quietly make our numbers wrong.

Why this was needed

The most expensive incident in integration is not one where the connection drops. When the connection drops, an alert sounds and people come running. What is really expensive is an incident where wrong numbers come out with no error at all.

Suppose the partner upgrades its order API from 1.3 to 1.4. Three things changed. region is renamed to market, amount changes from the integer 12300 to the string "12300.00", and one more value is added to status, namely on_hold. What happens to our collector?

rec.get("region") raises no exception. It returns None. Every row of the regional sales report ends up gathered under "Uncategorized."
sum(r["amount"] for r in rows) raises a TypeError in Python — the lucky case. But if the code concatenates strings or it is JavaScript, it quietly becomes "1230012300…".
In code that branches only on if status == "paid", on_hold falls into no branch, and that order drops out of the aggregation entirely.

In all three, no alert sounds. By the time the customer says "the numbers look a bit off" a few weeks later, several wrong reports have already gone out.

How it works

There are three ways to deal with this problem.

First, know what breaks. If you classify changes from the consumer's (the side reading the response) point of view, the rule is simple. Adding a field is safe, while a field disappearing, being renamed, or changing type is not safe. Growth of an enumerated value is also not safe — because when a value not in our branch arrives, it goes into no branch. On the request side, where the direction is reversed, the rule is reversed too. Adding an optional field to the request is safe, but adding a required field breaks, because our request will be rejected.

The criterion here is "what do we do with fields we do not know." The JSON Schema 2020-12 core specification decides that handling with additionalProperties, and required is the list of keys that must be present. When writing a consumer contract, you usually allow unknown fields. That way we do not break every time the partner adds a field.

Second, write the contract on our side. There is a temptation to adopt the partner's OpenAPI specification as our contract as is, but then our tests turn red even for changes to fields we do not read. When the noise grows, people turn the tests off. So in the contract you write only the fields we actually read. This approach is commonly called a consumer-driven contract.

Third, keep the contract as a test, not a document. All it takes is a single script that takes the partner's response, compares it to the contract, and ends with a non-zero code if they differ. If you put it in the pipeline, even when the partner moves without notice, we know first. "I did not see the upgrade notice email" is the sentence most often written in incident reports.

파트너 응답 ──▶ 계약 대조기 ──▶ 위반 0 ? 통과
                    │
                    └─ 위반 n ? 파이프라인 실패 + 무엇이 어긋났는지 필드 단위로 출력

The version notation itself is a promise too. Semantic Versioning says to raise the major number for changes that break compatibility. But that is the publisher's promise, so the partner may not keep it. If it went from 1.3 to 1.4 and we broke, that is not because we read it wrongly but because they broke their promise — yet proving that requires the output of the contract test.

What it looks like in the field

First, an upgrade does not arrive all at once. The old and new versions run side by side for months. So the reading side must accept both. Absorb the places where names changed with an alias table and the places where types changed with a normalization function, and use only one shape internally. If you scatter the alias table around the code, three months later when you cut off the old version, nobody knows where to delete.

Second, "required but sometimes empty" is the most common. The specification says required, but 3% of the actual responses are empty strings. So a contract test must not read the specification but scan the actual responses, not a sample but all of them.

Third, amounts and times are always the problem. They often change from an integer minimum unit to a decimal string, or the reverse. The moment you receive them as floating point, rounding appears, so normalize to an integer or a string and do not go through a real number.

Fourth, where to run the contract test is the real issue. You cannot run it against the partner's production environment every minute. Usually you run it a few times a day against the sandbox they provide, and once right before deployment. Sandbox and production sometimes run different versions, so always leave in the output of the contract test which address it was measured against.

What you will do in the next lab

You start a server that serves both 1.3 and 1.4 of the partner order API and get both versions' responses in hand. You extract the field differences in a machine-readable form and build a small tool that decides, per type of change, what breaks us. Then you build a consumer contract listing only the fields we read and a comparator that checks against that contract, and attach a reading layer that accepts both the old and the new version. Finally, you create a situation where the partner has moved again without notice and confirm that the contract test catches it at the field level.