Keeping a Node service non-blocking
One synchronous call in a request handler stops the whole process serving anyone, not just the caller who triggered it.
One thread, many requests
JavaScript in this runtime executes on a single thread. Concurrency comes from the fact that waiting on a socket, a file or a database costs no thread time: the work is handed off and a callback is queued for when it finishes. Thousands of waiting requests are cheap because none of them occupies the thread.
The moment code computes rather than waits, that model breaks. Parsing a very large payload, hashing in a loop or serialising a huge structure holds the thread, and every other request queues behind it regardless of how simple it is.
Recognising the blocking calls
Anything whose name ends in a synchronous suffix belongs in start-up code and nowhere near a handler. Beyond those, the blocking work is usually a loop over a large array, a regular expression that backtracks badly, or a serialisation of a structure that grew larger than anyone expected.
Latency measured at the high percentiles rather than the average is what exposes this. An average stays comfortable while a small share of requests waits behind whatever held the thread.
Where the heavy work goes
Genuinely heavy computation belongs in a separate process, reached over a socket or a queue, so the handler goes back to waiting. Splitting a long loop into chunks that yield between them is a weaker alternative that keeps the process responsive without moving the work.
Auditing a handler
- List every call the handler makes and mark each one as waiting or computing.
- Move every computing step that grows with input size out of the handler.
- Replace any synchronous file or crypto call with its waiting equivalent.
- Measure latency at the ninety-ninth percentile, not the average.
- Watch the gap between event loop ticks; a growing gap is the thread being held.
// holds the thread for every other request too
const data = fs.readFileSync(path);
// hands off and lets the thread serve others meanwhile
fs.readFile(path, (err, data) => { /* ... */ });The same read, blocking and not