The Hard Part of Parallelism Is Joining, Not Splitting

In one line

Adding branches takes just a few lines of edges, but if you do not decide the merge rule, the graph dies with an exception. And the order in which results merge is not chance but fixed.

Why this was needed

We made one question get searched at the same time in three places: the internal wiki, past tickets and product documents. It took three lines of edges.

for source in ("wiki", "ticket", "manual"):
    graph.add_node(source, search(source))
    graph.add_edge(START, source)
    graph.add_edge(source, END)

It died on the first run.

langgraph.errors.InvalidUpdateError: At key 'note':
Can receive only one value per step. Use an Annotated key to handle multiple values.

Three nodes wrote to the same key in the same superstep. In a graph that runs in order, the last value would have remained and that would have been it (that is a problem too, but it passes quietly), but when they write at the same time, LangGraph stops, saying "I do not know which of the two to keep".

This exception is a kind one. The bug that was silently overwriting in the graph that ran in order was simply exposed the moment it became parallel.

How it works

LangGraph executes in units of supersteps. If there are several nodes that can go in one superstep, those nodes run together. So nodes side by side, however many, are one superstep, and use up only one recursion limit.

Merging is done by each channel's reducer. When several values come into one channel in the same superstep,

• if there is a reducer, they are merged one by one with that function.
• if there is no reducer, it stops with InvalidUpdateError.

So the question to ask before making it parallel is not "how many should I run at once" but "do these branches write to the same slot?"

The order of merging is fixed — something I measured myself

This is a place people often misunderstand. It is easy to think "it's parallel, so whichever finishes first will come first". Measured directly in this lab image, that is not so.

노드를 z, a, m 순으로 더함 →  합쳐진 결과 ['a', 'm', 'z']
노드를 m, z, a 순으로 더함 →  합쳐진 결과 ['a', 'm', 'z']
'aaa' 를 일부러 느리게 만들고 'bbb' 를 즉시 끝나게 함 → ['aaa', 'bbb']

It is in node-name order. Not the order they were added, and not the order they finished. Even if a slow node gets the earlier name, it still comes first.

Why does this matter? Because it means the result is deterministic. The same input gives the same answer, so you can write tests and compare regressions. But let us not rely on names — if you want to use the order as meaning, sort explicitly in the merging node.

When the number is decided at run time — Send

If the sources are not fixed, you cannot draw the edges in advance. In such a case you use Send.

def fan_out(state):
    return [Send("probe", {"source": name, "topic": state["topic"]})
            for name in state["picked"]]

graph.add_conditional_edges("plan", fan_out, ["probe"])

Send is an instruction: "run this node with this input". Two things differ from the usual.

• The number is decided at run time. That node runs as many times as the length of the list.
• What that node receives is not the whole state but what Send passed. So the node looks only at the piece that came over, like state["source"].

The returned values are merged by the reducer as usual. Measured directly, the order of the results follows the order in which the Sends were created.

Limit the width

"Search every source" is a good idea only when there are three sources. At twenty, twenty calls go out at once, and each one is money and time. So set an upper limit on the width (fan-out width).

When you set the limit, what matters is that the rule for choosing must be deterministic. If you choose arbitrarily on ties, the same question gets different answers, and then you cannot explain "it worked yesterday but not today". Make a total order by sorting by score and breaking ties by name.

If one fails, the whole dies

This is the most painful spot in parallel. If one branch throws an exception, that whole superstep ends in an exception, and the results of the other branches that had already succeeded disappear too.

The fix is simple. Return failure as a value, not as an exception.

def node(state):
    if 못 찾겠다:
        return {"failures": ["ticket"]}       # 예외를 던지지 않는다
    return {"findings": [...]}

Then the merging node can know "two of the three found something and one failed", and can tell the user so as well. Leaving the name of the failed source is the key — if you do not, a partial result pretends to be the full result.

What it looks like in the field

First, an exception right after you make it parallel. It is the InvalidUpdateError above. The cause is not the parallelism but the overwriting that was there originally.

Second, the collecting node runs early. If you send each branch to END and set up a collecting node separately, you cannot know when the collecting node runs. If you send the branches to converge on the collecting node, that node runs once after all the branches have finished.

Third, a partial failure becomes a total failure. It happens often that one source dies and the system cannot give any answer at all.

Fourth, the width is not controlled. If you scatter the list as it is, on the day the list gets long the cost grows by that much.

What really matters in practice

• Before making it parallel, first ask "do they write to the same slot?" The reducer is the answer.
• The order of merging is fixed but do not use it as meaning. If the order carries meaning, sort explicitly in the merging node.
• Return failure as a value. An exception throws away even branches that had already succeeded.
• Put an upper limit on the width, and make the choosing rule deterministic.

What you will do in the next lab

You grow /root/work/agpara/fanout.py one step at a time. First you build a graph that splits into three sources and converges, and catch for yourself what exception arises when two write at the same time to a key with no reducer. You confirm what decides the order of merging by changing the names, and set up a separate collecting node to rank. Then you build a branch whose number is decided at run time with Send, put an upper limit on the width, and finally make it so that even if one branch fails, the remaining answers survive.