3.

6 Linear Quadratic Regulator (LQR)

The idea is to introduce and optimize a performance index as depicted in the following figures
Step response Initial value problem
x(t)
x0
’minimize’ area
ref. input w0

’minimize’ area
t
t
t0

For a good controller performance, one would demand for a fast response and little overshooting,
hence for minimizing the shaded areas. Meanwhile, one would also want to save the energy used
by the controller in implementing control actions. The performance index for the LQR controller
is introduced

• for discrete LTI systems:

N
X −1
x[k]T Qx[k] + u[k]T Ru[k] + x[k]T Qf x[k]


J(x, u) = (3.58)
k=0

• for continuous LTI systems:

ZT
x> (t)Qx(t) + u> (t)Ru(t) + x(T )T Qf x(T ) dt


J(x, u) = (3.59)
0

where Q, Qf are symmetric, positive semidefinite (n × n) matrices and R is a symmetric,

positive definite (p × p) matrix. The matrices Q, Qf and R can be regarded as tuning parameters
in order to meet design requirements. While Q penalizes slow responses and overshoots, R adds
a penalization to steering actuation, and Qf puts penalization on the end state of the horizon,
which consists of steps [0, N ] for discrete case or duration [0, T ] for the continuous case.
Typical LQR design employs infinite horizon, i.e. we replace N by ∞ and drop the terms
related to Qf in (3.58) and (3.59).

3.6.1 Solving discrete LQR problems

Solutions to discrete LQR problems are derived using the dynamic programming principle, the
optimal solution would be obtained recursively backward from the last time step. We present
hereby the results for the cases with finite horizon and infinite horizon.

Discrete LQR with finite horizon The cost-to-go Vt (z), which is the optimal cost depending
on the state value z, counted from the time-to-go t until the end of the horizon, is Vt (z) = zT Pt z
with Pt symmetric, positive definite, obtained using the following algorithm:

1. set PN = Qf
2. for t = N, . . . , 1, compute

−1 T
Pt−1 = Q + AT Pt A − AT Pt B R + BT Pt B B Pt A (3.60)

25

−1 T
3. for t = 0, . . . , N − 1, define Kt = R + BT Pt+1 B B Pt+1 A. The optimal input for time
step t is u[t] = −Kt x[t].

Note that in this linear state feedback controller, the gain Kt changes over time.

Discrete LQR with infinite horizon The performance function to optimize:

∞
X
x[k]T Qx[k] + u[k]T Ru[k]


J(x, u) = (3.61)
k=0

The cost-to-go V (z) does not depend on time, it is associated with current state as V (z) =
zT Pz, where P is a symmetric, positive definite solution of the following matrix equation:

−1 T
P = Q + AT PA − AT PB R + BT PB B PA (3.62)

and the optimal input is a constant state feedback controller u[k] = −Kx[k] with:

−1 T
K = R + BT PB B PA (3.63)

The equation (3.62) is called the (discrete) Algebraic Riccati Equation (ARE). In MATLAB,
we can use the command dlqr to solve the discrete ARE and obtain the linear state feedback
controller.

3.6.2 Solving continuous LQR problems

Continuous LQR with finite horizon Summary of the continous LQR solution:

1. set PT = Qf
2. for t ∈ [0, T ), solve the following Riccati differential equation backward in time to get Pt :

−Ṗt = AT Pt + Pt A − Pt BR−1 BT Pt + Q (3.64)

3. for t ∈ [0, T ), the optimal input is u(t) = −K(t)x(t), with K(t) = R−1 BT Pt .

Continuous LQR with infinite horizon The performance function to optimize:

Z∞
x> (t)Qx(t) + u> (t)Ru(t) dt


J(x, u) = (3.65)
0

Similar to discrete LQR with infinite horizon, for continuous LTI systems, we can design the
optimal controller u(t) = −K(t)x(t) with

K = R−1 BT P (3.66)
where P is the solution of the following (continuous) ARE:

AT P + PA − PBR−1 BT P + Q = 0 (3.67)

In MATLAB, we can use the command lqr to solve the continuous ARE and obtain the linear
state feedback controller.
It is known that if the system is controllable and the pair (A, C) is observable (concept to be
introduced in the next chapter) with C is defined as Q = CT C, then there exist a unique positive
definite matrix P that satisfies the ARE (3.67). Similar result holds for the discrete LQR case.

26
3.6.3 Choice of Q and R Matrices
In this section, some rough ideas on the choices of the Q and R matrices shall be given.
• As the choice of the Q and R matrices is crucial for the result, the LQR concept should be
regarded more as a mathematical recipe for carrying out the controller design rather than
as a self-contained procedure, which comes up with the ’optimal’ controller. In practice one
would choose certain matrices Q and R, then compute the controller based on these matrices
and compare simulations to given specifications. Eventually, the whole design process has
to be repeated with different Q and R matrices to end up at the desired controller behavior
after some iterations.

• The matrix Q can be chosen such that Q = CT C, where y = Cx, so that the objective
function includes quadratic terms of outputs and inputs. R could be a diagonal matrix.
• Or one can start with diagonal matrices for both Q and R, and choose
1
qi,i = i = 1, . . . , n
Maximum acceptable value for x2i
1
rj,j = j = 1, . . . , p
Maximum acceptable value for u2j

27