5.7.
QUADRATIC FORMS 69
Example 5.7.1 A quadratic form in two variables:
q = d11 u21 + d12 u1 u2 + d22 u22 .
The symmetric matrix is
d11 d12 /2
.
d12 /2 d22
Then we have q = u′ Du.
Positive and Negative Definiteness:
Definition 5.7.2 A quadratic form q(u1 , u2 , · · · , un ) = u′ Du is said to be
(a) positive definite (PD) if q(u) > 0 for all u ̸= 0;
(b) positive semidefinite (PSD) if q(u) ≥ 0 for all u ̸= 0;
(c) negative definite (ND) if q(u) < 0 for all u ̸= 0;
(d) negative semidefinite (NSD) if q(u) ≤ 0 for all u ̸= 0.
Otherwise q is called indefinite (ID).
Sometimes, we say that a matrix D is, for instance, positive definite if
the corresponding quadratic form q(u) = u′ Du is positive definite.
Example 5.7.2
q = u21 + u22
is positive definite (PD),
q = (u1 + u2 )2
is positive semidefinite (PSD), and
q = u21 − u22
is indefinite.
70CHAPTER 5. LINEAR MODELS AND MATRIX ALGEBRA (CONTINUED)
Determinantal Test for Sign Definiteness:
We state without proof that for the quadratic form q(u) = u′ Du, the
necessary and sufficient condition for positive definiteness is the principal
minors of |D|, namely,
|D1 | = d11 > 0,
d11 d12
|D2 | = > 0,
d21 d22
···
d11 d12 · · · d1n
d21 d22 · · · d2n
|Dn | = > 0.
··· ··· ··· ···
dn1 dn2 · · · dnn
The corresponding necessary and sufficient condition for negative def-
initeness is that the principal minors alternate in sign as follows:
|D1 | < 0, |D2 | > 0, |D3 | < 0, etc.
Two-Variable Quadratic Form
Example 5.7.3 Is q = 5u2 + 3uv + 2v 2 either positive or negative? The
symmetric matrix is
5 1.5
.
1.5 2
Since the principal minors of |D| is |D1 | = 5 and
5 1.5
|D2 | = = 10 − 2.25 = 7.75 > 0,
1.5 2
so q is positive definite.
5.7. QUADRATIC FORMS 71
Three-Variable Quadratic Form
Example 5.7.4 Determine whether
q = u21 + 6u22 + 3u23 − 2u1 u2 − 4u2 u3
is positive or negative definite. The matrix D corresponding this quadratic
form is
1 −1 0
D=
−1 6 −2
,
0 −2 3
and the principal minors of |D| are
|D1 | = 1 > 0,
1 −1
|D2 | = = 6 − 1 = 5,
−1 6
and
1 −1 0
|D3 | = −1 6 −2 = 11 > 0.
0 −2 3
Thus, the quadratic form is positive definite.
Example 5.7.5 Determine whether
q = −3u21 − 3u22 − 5u23 − 2u1 u2
is positive or negative definite. The matrix D corresponding this quadratic
form is
−3 −1 0
D=
−1 −3 0
.
0 0 −5
72CHAPTER 5. LINEAR MODELS AND MATRIX ALGEBRA (CONTINUED)
Leading principal minors of D are
|D1 | = −3 < 0,
|D2 | = 8 > 0,
|D3 | = −40 < 0.
Therefore, the quadratic form is negative definite.
5.8 Eigenvalues and Eigenvectors
Consider the matrix equation:
Dx = λx.
Any number λ such that the equation Dx = λx has a non-zero vector-
solution x is called the eigenvalue (or called the characteristic root) of
the above equation. Any non-zero vector x satisfying the above equation
is called the eigenvector (or called the characteristic vector) of D for the
eigenvalue λ.
Recipe - How to calculate eigenvalues:
From Dx = λx, we have the following homogeneous-equation system:
(D − λI)x = 0.
Since we require that x be non-zero, the determinant of (D−λI) should
vanish. Therefore all eigenvalues can be calculated as roots of the equa-
tion (which is often called the characteristic equation or the characteristic
polynomial of D)
|D − λI| = 0.
5.8. EIGENVALUES AND EIGENVECTORS 73
Example 5.8.1 Let
3 −1 0
D=
−1 3 0.
0 0 5
3−λ −1 0
|D − λI| = −1 3−λ 0 = (5 − λ)(λ − 2)(λ − 4) = 0,
0 0 5−λ
and therefore the eigenvalues are λ1 = 2, λ2 = 4, and λ3 = 5.
Properties of Eigenvalues:
Proposition 5.8.1 A quadratic form q(u1 , u2 , · · · , un ) = u′ Du is
positive definite if and only if eigenvalues λi > 0 for all i = 1, 2, · · · , n.
negative definite if and only if eigenvalues λi < 0 for all i = 1, 2, · · · , n.
positive semidefinite if and only if eigenvalues λi ≥ 0 for all i =
1, 2, · · · , n.
negative semidefinite if and only if eigenvalues λi ≤ 0 for all i =
1, 2, · · · , n
indefinite if at least one positive and one negative eigenvalues exist.
Definition 5.8.1 Matrix A is said to be diagonalizable if there exists a non-
singular matrix P and a diagonal matrix D such that
P −1 AP = D.
Matrix U is an orthogonal matrix if U ′ = U −1 .
74CHAPTER 5. LINEAR MODELS AND MATRIX ALGEBRA (CONTINUED)
Theorem 5.8.1 (The Spectral Theorem for Symmetric Matrices) Suppose that
A is a symmetric matrix of order n and λ1 , · · · , λn are its eigenvalues. Then there
exists an orthogonal matrix U such that
λ1 0
..
U −1 AU =
. .
0 λn
Usually, U is the normalized matrix formed by eigenvectors. It has the
property U ′ U = I. "Normalized" means that for any column u of the
matrix U , we have u′ u = 1.
Example 5.8.2 Diagonalize the matrix
1 2
A=
.
2 4
First, we need to find the eigenvalues:
1−λ 2
= λ(λ − 5) = 0,
2 4−λ
i.e., λ1 = 0 and λ2 = 5.
For λ1 = 0, we solve
1 − 0 2 x1 0
= .
2 4 − 0 x2 0
The eigenvector, corresponding to λ1 = 0, is v1 = C1 · (2, −1)′ , where C1
is an arbitrary real constant. Similarly, for λ2 = 5, we have v2 = C2 · (1, 2)′ .
Let us normalize the eigenvectors, i.e. let us pick constants Ci such that