Process Variability

Waiting line analysis examines how uncertainty—whether in arrival rates or processing

times—affects wait times in a system. Waiting is undesirable in both service and manufac-
turing contexts: in service settings (such as fast-food restaurants and call centers), long wait
times reduce customer satisfaction, while in manufacturing, waiting work-in-process (WIP)
ties up capital and increases storage and inventory costs. Given its significant impact, ex-
tensive research has been dedicated to mitigating wait times. Since waiting is immediately
felt in service environments, our focus will be on managing wait times in service systems.

1 Why do Customers Wait?

At a fundamental level, customers wait when the system’s capacity is lower than the arrival
rate. For example, if customers arrive at a fast-food restaurant at a rate of 10 customers
per hour, but employees can only serve 8 customers per hour, a backlog forms—resulting in
2 customers still waiting after the first hour.
However, waiting can occur even when system capacity exceeds the average arrival rate
due to variability in arrivals or service times. Suppose the arrival rate is on average 10
customers per hour, and the system has a capacity of 12 customers per hour. If, due to
variability, 15 customers arrive in a given hour, the system can only serve 12, leaving at least
3 customers waiting.
Our focus is on managing wait times caused by uncertainty in arrival and service pro-
cesses. To do so, we use the framework of a Queuing System to identify key levers for
improvement.

2 The Queuing Framework

Our primary focus is on single-phase queuing systems. In such systems, customers (or any
other units—without loss of generality, we will refer to them as customers) arrive, wait in

1
Process Variability Professor O. Mouchtaki

one or more lines for service, receive service at a single point, and then leave the system.
Many real-world service settings fit this model. For example:

• Fast-food restaurants where customers line up at a counter, place an order, receive

their food, and leave.

• Food trucks where customers queue up, order food, wait for it to be prepared, and
then depart.

• Cafés where customers order drinks, wait for baristas to prepare them, and exit once
served.

However, this model does not apply when customers must go through multiple stages of
service. Consider a cafeteria where customers must first wait in line to get their food, then
queue separately at the cashier to pay before leaving. In this case, a multi-phase queuing
system is more appropriate.

Key Drivers of Performance in a Queuing System

To analyze a queuing system, we introduce three fundamental quantities that determine its
performance:

• Arrival Rate: The average number of customers arriving at the system per unit time.

– Example: If an average of 20 customers arrive at a coffee shop per hour, the arrival
rate is 20 customers/hour.

• Service Rate: The average number of customers that each server can process per
unit time.

– Example: If a barista can make and serve 15 drinks per hour, the service rate is
15 customers/hour.

These two measures alone, however, do not provide a complete picture of the system’s
performance. They only capture averages and do not account for fluctuations in customer
arrivals or service times. To quantify variability, we introduce the coefficient of variation
(CV)—a normalized measure of randomness in a system. It is defined as:

Standard deviation of the random quantity

Coefficient of variation = .
Mean of the random quantity

2
Process Variability Professor O. Mouchtaki

• Example 1: Suppose a food truck receives an average of 10 customers per hour, but in
some hours, it gets as few as 5 and in others as many as 20. If the standard deviation
of arrivals is 4 customers/hour, the coefficient of variation of arrivals is:

4
CV = = 0.4.
10

A lower CV (closer to 0) means more predictable arrivals, while a higher CV (greater

than 1) indicates high variability.

• Example 2: Similarly, service times can also vary. If a barista takes exactly 2 minutes
to prepare each coffee, the service time has no variability (CV = 0). But if the
preparation time fluctuates widely—sometimes 1 minute, sometimes 5 minutes—the
CV will be higher, affecting wait times.

By understanding both average rates and their variability, we can better predict queue
lengths and customer wait times, enabling more effective system design and management.

Performance Metrics
The three key metrics used to quantify the performance of a queuing system are:

• Utilization of the Server: This represents the proportion of time the server is busy
serving customers, calculated as:

Arrival rate
Utilization = .
Service rate

– Example: If a cashier can serve 10 customers per hour and an average of 8 cus-
tomers arrive per hour, then the server utilization is 8
10
= 0.8, meaning the cashier
is busy 80% of the time.

In what follows, we will only consider systems in which the utilization is strictly less
than 100%.

• Average Waiting Time: This measures the average time a customer spends waiting
in the queue before receiving service. It is closely related to the flow time, which is the
sum of the average waiting time and the average service time.

– Example: In a bank, if customers wait an average of 5 minutes before reaching

the teller and spend 3 minutes being served, the flow time is 8 minutes.

3
Process Variability Professor O. Mouchtaki

• Average Queue Length: This measures the average number of customers waiting in
the queue at any given time.

– Example: In a grocery store checkout line, if there are typically 4 people waiting,
the average queue length is 4.

The average queue length is related to the average waiting time by Little’s Law. In this
context, the queue length corresponds to inventory, the waiting time corresponds to
flow time, and the arrival rate represents throughput (this follows from our assumption
that utilization is strictly less than 100%). Therefore, Little’s Law implies that

Average Queue Length = Arrival Rate × Average Waiting Time.

Measuring Performance Metrics

We have previously established how to derive the utilization of a server as a function of
the arrival and service rates, and how to express the average queue length as a function of
the average waiting time. The following formula provides an approximation for the average
waiting time.

Waiting time in a single-server queue

1 Utilization CV2a + CV2s

!
Waiting Time = × × ,
Service Rate 1 − Utilization 2
where CVa (resp. CVs ) is the coefficient-of-variation of arrivals (resp. services).

We illustrate this formula in Figure 1 and discuss its implications in the next section.

3 Insights
The waiting time formula provides key insights into the behavior of processes with variability:

1. A utilization of 1 is problematic: As utilization approaches 1, the waiting time

grows indefinitely. This underscores the need for extra capacity to prevent excessive
delays.

2. Small reductions in utilization have a big impact: The relationship between

waiting time and utilization is nonlinear. Even a slight decrease in utilization can

4
Process Variability Professor O. Mouchtaki

Figure 1: Waiting time as a function of utilization for different levels of variability

lead to a significant reduction in waiting time. For example, lowering utilization from
99% to 90% (a decrease of approximately 10%) reduces waiting time by a factor of 11!

3. Higher variability leads to longer waits: Keeping utilization, arrival rate, and
service rate constant, an increase in variability—whether in arrival times or service
times—results in longer waits. This is due to a higher coefficient of variation.