Millsap’s ACM Queue article on performance tuning introduced me to some basics of queuing theory. The article discusses some considerations for tuning the performance of a database. One aspect that Millsap addresses is queuing delay, or the amount of time it takes for a system to begin processing a request. This is the amount of time the request waits in the queue before the system acts on it. As utilization of the system increases, the delay increases. A request needs to wait until more things in front finish.
Millsap goes on to explain that to achieve the desired response time of a system you need enough resources to keep the utilization below the point where the wait time gets too high.
When we typically plan the number of ingest channels we need typically we ask our users what is the most number of things they want to record at a time. We stop there. We do not ask how long they are willing to wait to start the next record. In a system where we strictly schedule and manage the records, we can build the system to accommodate the exact maximum. This is usually only possible in a Master Control setting.
However, when the records start roughly randomly, like for newsgathering, we need to allow for the overhead of increased load. To record 13 simultaneous feeds without missing the next feed, we actually need 16 ingest channels. Except at peak times, we should not run the ingest system beyond 81% utilization.
When planning an ingest system, remember to ask your users the following questions.
- What is the maximum number of simultaneous records?
- Are the records rigidly scheduled or random?
- How long is the average ingest?
- How long are you willing to wait for the next record to start?
If the answer to the second question is random and the wait time is less than average record length, calculate the knee of your utilization curve and plan for the extra channels needed to keep the wait consistent.