Slice a Big Computation, Then Move It to a Worker

Goal

You will handle a computation that holds the loop for a long time in two ways: splitting it into slices and handing the turn back between them, and moving the whole thing to a worker thread. And you will measure the value of each in numbers.

Why it matters

The advice "if it is heavy, send it to a worker" is only half the story. A worker does not share memory, so it cannot see the current request's objects as they are, and starting a thread also takes time. If that time is greater than the work itself, moving it is a loss.

Splitting is the opposite. It stays on the same thread, so it sees state as it is, and in exchange you pay for yielding between slices in total time. And the time of one slice becomes the floor of the latency, because no one can cut in while a slice is running.

If you measure both values, "where do we put this job?" becomes a calculation rather than a matter of taste. In the last step you harden that calculation into a rule.

Steps

1. In /root/work/worker/split.mjs, create chunks(total, size).
2. In the same file, create hashRange(from, to, seed) and runSliced(total, sliceSize).
3. Measure the all-at-once version and the sliced version, and write them to /root/work/worker/report.json, under runs.whole·runs.sliced.
4. From the two runs, calculate and record slicing.costRatio·tailRatio·sliceMs.
5. Create /root/work/worker/hash-worker.mjs and move the work with runInWorker(total).
6. Measure while the worker runs and record it in runs.worker, and also record worker.startupMs.
7. Turn "where do we put it?" into a rule with place(job).

Notes

• chunks(total, size) is an array of {from, to} objects. If size is 0 or less, throw an exception; if you leave it as is, it becomes an endless loop.
• hashRange(from, to, seed = 0) is the value of running h = (h * 31 + i) >>> 0 from from up to, but not including, to. It is a computation that depends on order, so you must pass the previous value as the seed between slices for the answer to be the same.
• runSliced(total, sliceSize) returns {result, slices}.
• runInWorker(total) returns the same value as hashRange(0, total) as a promise, and spawnCostMs() measures only the startup time with one empty job.
• The rule for place(job): if cpuMs is 1 or less, it is "loop"; if it is greater and sharedState is truthy, it is "slice"; otherwise, when cpuMs is 50 or more, it is "worker", and "slice" when it is less. If cpuMs is missing or is not a number, throw an exception.
• Do not run a program that starts a worker with node -e. The worker inherits the parent's execution arguments and dies with ERR_INPUT_TYPE_NOT_ALLOWED. Keep the measuring program in a file too and run it like node measure.mjs.

Split without dropping anything at the boundaries

In /root/work/worker/split.mjs, export chunks(total, size). It is an array of {from, to}, and the last slice may be shorter. If size is 0 or less, throw an exception.

to does not include that position, so that it connects exactly with the from of the next slice.

The number of slices is Math.ceil(total / size), and when joined together they must cover 0 to total with no gaps and no overlaps.

Split it and hand the turn back in between

In the same file, export hashRange(from, to, seed = 0) and runSliced(total, sliceSize). runSliced returns {result, slices} and yields to the loop once between slices.

Run h = (h * 31 + i) >>> 0, but pass the result of the previous slice as the seed of the next one. If you do not pass it, the answer changes.

The yield is the single line await new Promise(r => setImmediate(r)). If you only divide and do not have this line, from the loop's point of view it is the same as running it all at once. The grader checks how many times a timer fired in between.

The same answer, a different distribution

Measure a 40-million-iteration computation (a) all at once and (b) split into slices, and in /root/work/worker/report.json write node·total and runs.whole·runs.sliced. Both runs hold samples·p50·p99·max·workMs·result, and runs.sliced also holds slices.

You can put a ruler in this file with the same shape as the withLag from lab 2.

result must be the same in both runs. If it differs, you did not pass the seed between slices; splitting does not change the answer.

Calculate the value of splitting

In slicing, write costRatio (sliced.workMs / whole.workMs, to three decimal places), tailRatio (whole.max / sliced.max, to two decimal places), and sliceMs (sliced.workMs / sliced.slices, to three decimal places).

costRatio is slightly greater than 1. It is the price of going around the loop once per yield.

sliceMs matters. While a slice is running, no one can cut in, so once you set a target latency, the slice size follows from it. If you want to respond within 50ms, one slice must not exceed 50ms.

Move it to another thread

Create /root/work/worker/hash-worker.mjs (it sends the result back through parentPort), and in split.mjs export runInWorker(total). It returns the same value as hashRange(0, total) as a promise, and it must work when called twice at the same time.

A worker does not share memory. Pass the values you need with workerData and get the result back with postMessage.

So "modify the current request's object in the worker" is not possible. Things are simply copied in and copied back; this is where the jobs that can be moved and the ones that cannot part ways.

Calculate the value of the worker

Measure while the worker runs and record the following in runs.worker: samples·p50·p99·max·workMs·result. Then export spawnCostMs() and write its value to worker.startupMs.

While the worker runs, the latency of this thread stays near 0, because the computation is not here. Instead, look at startupMs.

If the startup cost is greater than the work itself, moving it is a loss. That is why real services start a few workers ahead of time and reuse them; this number is the reason.

Harden it into a rule

Export place(job). If job.cpuMs is 1 or less, it is "loop"; if it is greater and job.sharedState is truthy, it is "slice"; otherwise, when cpuMs is 50 or more, it is "worker", and "slice" when it is less. If cpuMs is missing or is not a number, throw an exception.

The most dangerous thing is to return some arbitrary answer for a job that has no numbers. "Not measured" and "light" are different.

The boundary of 50 came from the startup cost measured in the earlier step. On a different machine you get a different value, and then this constant has to change too. The rule is tied to measurement.