What is a message queue, and why do we use one?

A queue stores messages that describe work to be done. One part of the system adds a job, and another picks it up and processes it later. That way the user doesn't have to wait in an open request for the whole long job to finish.

For example, after a document is uploaded, the API creates a job, and a separate program later reads the text and runs classification. A queue also helps with short load spikes. But it doesn't run business code by itself and doesn't endlessly increase the consumers' capacity. You have to watch the backlog, errors and processing rate. The job's result should be stored somewhere so the user can come back to it.