I have a distributed design problem that I thought Monks might find interesting and perhaps be willing to offer suggestions on.
I'm writing real time bidding software. It works like this:
- My software receives a request to bid on an auction
- I return a bid on said request (I must respond in 85 to 100 milliseconds)
- Later, I receive a notification on whether or not I won the auction
Point number 3 above means I don't find out if I've won a request until shortly after I've made the request.
Currently, on a server farm of about 36 boxes, we're receiving roughly 400 bid requests a second, or about 11 requests/second per box. This is an extremely light load and will likely increase by one or two orders of magnitude when the system goes live.
We have various campaigns, each with individual daily and total budgets that cannot be overspent (a bit of wiggle room is allowed). The servers run separate databases and the only shared data source we have is a Redis server with two slaves, though we could implement others.
So that's the setup. Now on to the problem.