Thread pool for a boss/worker model
This is a pretty simple idea - a boss thread assigns work to a pool of worker threads who do nothing until some work enters their queue. This way the boss can fill a queue very quickly and you have multiple back end processes that can consume that queue.
I'm using threading and not an async thing because some of the work I'll be assigning to threads are long-polling operations. The workers will hit some REST API route on some other application, and some of those routes take up to 30 seconds to complete or have dependencies or followup work. Rather than block and spin in an async call,…