Fed It 32 Rows and Got Rejected: Reviving a Dead Batch Axis

Goal

Using the same weights, build /root/onnxq-shape/dyn.onnx, whose batch axis is a symbol, and /root/onnxq-shape/fixed.onnx, whose batch axis is fixed to 1, and build a tool /root/onnxq-shape/axes.py that reads the axes and actually feeds values in. At the end you find a model whose batch axis is dead because a constant shape is pinned, and bring it back to life.

Why it matters

An axis in ONNX is either fixed as an integer, open as a symbol name, or left entirely empty. The three mean completely different things, but they cannot be told apart if you look at the file carelessly. This is usually the cause when an experiment that raises the batch to measure throughput gets stuck on the first line. It is also important that the rejection happens not when opening the session but when feeding in values. If you conclude it is not a model problem because the session opened, you end up digging in the wrong place. When the same symbol name is used in two inputs, it is not "both dynamic" but the constraint "the two must have the same number of rows". Violating it gets you rejected, but the runtime does not point out the symbol name; it cites the name of a node after optimization. There is a layer between the cause and the symptom. The quietest incident is when a constant shape is pinned in a Reshape. The input is open as a symbol, but from that point the batch axis turns into an integer. There is no error or warning, and shape inference trusts that constant as it is and produces an answer that is consistent front to back. The grader does not trust the text you wrote out. It sets up files it built itself in a temporary directory, actually runs your tool, and checks the same file against the answers the grader gets by reading and running it. The axis names, shapes and pinned row counts change on every run.

Steps

Create and run /root/onnxq-shape/build_shapes.py to make /root/onnxq-shape/dyn.onnx and /root/onnxq-shape/fixed.onnx.
Create dims in /root/onnxq-shape/axes.py so that it reads out whether the input and output axes are integers or symbols.
Add feed so that it builds values with the given number of rows, actually feeds them in, and writes the result or the rejection.
Add infer so that it separates what onnx.shape_inference filled from what it could not.
Add symbols so that it collects where each same symbol name was used.
Add scan so that it finds a Reshape with a constant shape pinned.
Add repair so that it restores the batch position of the pinned shape and reattaches a symbol name to the output axis.
Make a report with /root/onnxq-shape/shape_report.json and /root/onnxq-shape/shape_report.md.

Notes

Python is /opt/onnx-lab/bin/python. The system python3 has neither onnx nor numpy. Example run: /opt/onnx-lab/bin/python /root/onnxq-shape/axes.py dims /root/onnxq-shape/dyn.onnx
This Pod has no network. You build the materials yourself.
Model contract: both files have the input name x and the output name y, axis 1 of the input is 5, and axis 1 of the output is 3. Axis 0 of dyn.onnx is a symbol name and axis 0 of fixed.onnx is the integer 1. The initializer names and values must be exactly the same in the two files — only the same model with just the axis declaration different makes a valid comparison.
Execution contract: axes.py <명령> ... (the placeholder is the command). The answer is output as one JSON blob on standard output. On success the exit code is 0, and for an unknown command it is 2. A rejection by the runtime is not a failure but an answer, so catch the exception, write it as JSON, and leave the exit code at 0.
dims <모델> response (the placeholder is the model): {"inputs": {이름: 축목록}, "outputs": {이름: 축목록}} (where the placeholders are the name and the list of axes). An axis is an integer if fixed, a string if a symbol, and null if there is none, and if there is no shape declaration at all, the list of axes is null. Names filled by initializers are not put in inputs.
feed <모델> <행수...> (the placeholders are the model and the number of rows) takes one number of rows at a time in the order of the runtime inputs. For each input it uses the declared axis 1 onward as it is and fills only axis 0 with the given number of rows (any values are fine). Response: {"status": "ok"|"error", "error_type": 예외 클래스 이름 또는 null, "message": 문자열, "shapes": 출력 모양 목록 또는 null} (where the placeholders are the exception class name or null, a string, and a list of output shapes or null).
infer <모델> response (the placeholder is the model): {"value_info": {중간 텐서 이름: 축목록}, "unknown": [이름...]} (where the placeholders are the intermediate tensor name, the list of axes, and names). unknown is a sorted list of those names, among the names that a node produces but that are not graph outputs, whose shapes inference could not fill.
symbols <모델> response (the placeholder is the model): {심볼 이름: ["텐서이름:축번호", ...]} (where the placeholders are the symbol name, the tensor name and the axis number). It collects from the runtime inputs and the graph outputs.
scan <모델> response (the placeholder is the model): {"frozen_reshape": [노드 이름 정렬], "shapes": {노드 이름: 상수 형}} (where the placeholders are the sorted node names, the node name and the constant shape). It includes only a Reshape whose target shape is an initializer and whose position 0 is positive. If position 0 is -1 or 0, the axis is alive, so it is not included. If the node name is empty, use that node's first output name.
repair <모델> <출력> (the placeholders are the model and the output) changes position 0 of the constant shape of each included Reshape to -1, redeclares axis 0 of the graph output with the symbol of axis 0 of the first runtime input, and saves to a new file. It does not touch the weights or nodes. The response is {"out", "fixed", "batch_symbol"}.
To see the phenomenon with your own eyes in steps 6 and 7, you need one file that has the incident in it. Build /root/onnxq-shape/frozen.onnx with /root/onnxq-shape/make_frozen.py, try scan and feed on it, and keep the fixed result as /root/onnxq-shape/healed.onnx. The grader does not read these files and runs your tool on files it builds separately.
In shape_report.json, write dyn_input and fixed_input (each the list of axes of the input) and trials (each item having model, rows and status, at least 4 of them). There must be both a record that dyn.onnx accepted and a record that fixed.onnx rejected.
Write shape_report.md in the four sections ## 어떤 축이 열려 있나 ## 굳은 축에 무엇을 넣었나 ## 추론이 못 채운 곳 ## 보내는 쪽에 요청할 것 (the Korean headings mean "Which axes are open", "What you fed into the fixed axis", "Where inference could not fill" and "What to request from the sending side"), and write the symbol name of the batch axis in the text.
Official documents: ONNX Concepts · onnx.shape_inference · ONNX IR · ORT Python API
Common mistakes: assuming the shape is right because the session opened, letting a rejection blow up as an exception and leaving no record, reading an empty value_info as inference having failed, and writing the actual number of rows in the batch position of a Reshape.

Build two versions that differ only in the axis declaration

Create and run /root/onnxq-shape/build_shapes.py to make /root/onnxq-shape/dyn.onnx and /root/onnxq-shape/fixed.onnx. The weights are exactly the same in the two files, and only the declaration of axis 0 differs.

In the shape list of make_tensor_value_info, if you put a string at position 0 it is a symbol, and if you put an integer it is fixed to that value. Make the weights only once and put the same object in both graphs — if the values differ, you cannot compare the two models later.

Read whether an axis is open or fixed

Create dims <모델> (the placeholder is the model) in /root/onnxq-shape/axes.py so that it reads out the axes of the runtime inputs and outputs. Write an integer as an integer, a symbol as a string, and nothing as null.

An axis has either dim_param (a symbol) or dim_value (an integer), or neither. Write the three distinctly. And names filled by initializers are not runtime inputs, so leave them out of inputs.

Actually feed values in

Add feed <모델> <행수...> (the placeholders are the model and the number of rows) so that for each runtime input it builds values with that number of rows, feeds them in, and outputs {"status", "error_type", "message", "shapes"}. Even if it is rejected, the exit code is 0.

For the later axes, use the declared integers as they are and change only axis 0 to the given number of rows. A rejection is not a failure but an answer — if you catch the exception and write the class name and the first line, that text later tells you which axis expected what. Wrap the step of opening the session and the step of feeding values separately. They fail in different places.

What inference filled and what it could not

Add infer <모델> (the placeholder is the model) so that it separates the intermediate tensors that onnx.shape_inference filled from the names it could not fill. The response is {"value_info", "unknown"}.

infer_shapes returns a new ModelProto and the filled results go into graph.value_info. unknown is the names that a node produces but that are not graph outputs, among those that were not filled. For a graph with no intermediate tensor at all, value_info comes back empty — that is not a failure but means there was nothing to fill.

The same name is the same value

Add symbols <모델> (the placeholder is the model) so that for each symbol name it collects the places it was used as "텐서이름:축번호" (the placeholders are the tensor name and the axis number). Then feed different numbers of rows into two inputs that use the same symbol and confirm the rejection with feed.

If the same name is written on axis 0 of two inputs, it means to the runtime that the two numbers of rows must be the same. Violating it gets you rejected, but the runtime does not point out the symbol name; it cites the name of a node after optimization. So if you build this table in advance, you can interpret that error.

Find a Reshape with a constant shape pinned

Add scan <모델> (the placeholder is the model) so that it finds a Reshape whose target shape is an initializer and whose position 0 is positive. The response is {"frozen_reshape", "shapes"}.

If position 0 is -1, it is computed from the other axes, so the axis is alive. 0 means to use that axis of the input as it is, so it is also alive. You must pick only positive values. If the target shape is not an initializer but the output of another node, it is decided at execution time and is not a target of this check.

Bring the dead batch axis back to life

Add repair <모델> <출력> (the placeholders are the model and the output) so that it changes position 0 of the pinned constant shape to -1, redeclares axis 0 of the graph output with the symbol name of the first runtime input, and saves to a new file.

To change an initializer in place, build a new tensor and overwrite with CopyFrom. For the output axis, clear dim_value and put the name in dim_param. Do not touch the weights or nodes — only if the fixed file gives the same values at the numbers of rows it originally accepted can you say you fixed it.

Put the two versions side by side and report

Write dyn_input, fixed_input and trials in /root/onnxq-shape/shape_report.json, and write /root/onnxq-shape/shape_report.md in the four sections ## 어떤 축이 열려 있나 ## 굳은 축에 무엇을 넣었나 ## 추론이 못 채운 곳 ## 보내는 쪽에 요청할 것 (the Korean headings mean "Which axes are open", "What you fed into the fixed axis", "Where inference could not fill" and "What to request from the sending side").

trials is a record of which model you fed how many rows and what came out. The grader feeds in the same combinations itself and checks, so measure and write the actual results. In the body of the report you must write the symbol name of the batch axis so that the receiving side can talk about it by that name.