Node and MongoDB under a model
The service layer around a model has a narrower job than the model itself, and keeping it narrow keeps it debuggable.
What the layer owes the model
Four things and no more: accept a request, validate and shape its fields into exactly the structure the model expects, hand that structure to the model, and return the answer with enough identifying detail to trace it later. Anything else that creeps in becomes logic that exists in the service and not in training, which is where serving and training start to disagree.
The transformation applied at serving time has to be the same code, or at minimum the same stored parameters, as the transformation applied during training. Two separately maintained copies drift, and the drift shows up as a quality drop nobody can locate.
What the database is for
A document store is a good fit for the request and response records themselves: they arrive with varied shapes, they are written once and read for analysis, and they benefit from being kept whole rather than split across tables.
Store the input as received, the shaped input, the output, and an identifier for the parameter set that produced it. Without that last field there is no way to attribute a change in behaviour to a change in the model.
Keeping the event loop free
A single-threaded runtime stops serving everything the moment one request blocks it. Inference that takes real time belongs in a separate process reached over a socket or a queue, not inside the request handler. The handler's job is to wait on that call, not to perform the work.
The path of one request
- Validate the incoming fields against a declared schema and reject early on failure.
- Apply the stored transformation parameters to produce the shaped input.
- Hand the shaped input to the inference process and await its answer.
- Write the request, the shaped input, the answer and the parameter set identifier as one document.
- Return the answer along with the identifier that names which parameters produced it.
{
received: { /* fields exactly as sent */ },
shaped: { /* after the stored transformation */ },
answer: { /* what inference returned */ },
parameters: "identifier of the parameter set used",
at: "timestamp"
}One document per request keeps attribution possible