Lecture 9
Functions of several variables:
Lagrange Multipliers
M. Rajesh Kannan and D. Sukumar
Department of Mathematics,
Indian Institute of Technology Hyderabad,
email: rajeshkannan@[Link], suku@[Link]
October 14, 2024
Recap
Directional derivative
Gradient
More about gradient
Lagrange Multipliers
Directional Derivative
Definition
The derivative of a function f : R2 → R at P(x0 , y0 ) in the
direction of the unit vector u = u1 i + u2 j is defined as follows:
df f (x0 + su1 , y0 + su2 ) − f (x0 , y0 )
= lim ,
ds u,P s→0 s
if the limit exists.
▶ Notation : (Du f )P = df
ds .
u,P
▶ Remark: The partial derivatives fx and fy are the directional
derivatives of f at P in the directions of x-axis and y -axis,
respectively.
Illustration
Formula for Directional derivative
▶ Let f : R2 → R. Consider the line x = x0 + su1 y = y0 + su2
through the point P = (x0 , y0 ) parameterized by the arc
length parameter s increasing in the direction of the unit
vector u = u1 i + u2 j.
▶ By the Chain rule,
df ∂f dx ∂f dy
= +
ds u,P ∂x P ds ∂y P ds
∂f ∂f
= u1 + u2
∂x P ∂y P
" # " #
∂f ∂f
= i+ j · u1 i + u2 j.
∂x P ∂y P
Gradient
The gradient vector of f (x, y ) at a point P(x0 , y0 ) is the vector
∂f ∂f
∇f = i+ j
∂x ∂y
obtained by evaluating the partial derivatives of f at P0 .
▶ ∇f is read ”grad f ” as well as ”gradient of f ” and ”del f ”.
The Directional Derivative is a Dot Product
Theorem
If f (x, y ) is differentiable in an open region containing P(x0 , y0 ),
then df
= (∇f )P · u
ds u,P
In brief, Du f = ∇f · u.
Problem
Find the directional derivative of f (x, y ) = xe y + cos(xy ) at the
point (2, 0) in the direction of v = 3i − 4j.
Two dimensional Cauchy-Schwarz
▶ Note that,
Du f = ∇f · u = |∇f ||u| cos θ,
where θ is the angle between the vectors u and ∇f .
Gradient Ascent/Descent
Theorem
▶ The function f increases most rapidly when cos θ = 1 or when
θ = 0 and u is the direction of the gradient ∆f . That is, at
each point P in its domain, f increases most rapidly in the
direction of the gradient vector ∇f at P. The derivative in
this direction is
Du f = |∇f | cos(0) = |∇f |.
▶ Similarly, f decreases most rapidly in the direction of −∇f .
The derivative in this direction is Du f = −|∇f |.
▶ Any direction u orthogonal to a non-zero gradient ∇f is a
direction of zero change in f , as Du f = 0.
Level curves
The set of points in the plane where a function f (x, y ) has a
constant value f (x, y ) = c is called a level curve of f . The set of
all points (x, y , f (x, y )) in space, for (x, y ) in the domain of f , is
called the graph of f . The graph of f is also called the surface
z = f (x, y ).
Example
Gradient and Tangent to Level curves
▶ If a differentiable function f (x, y ) has a constant value c along
a smooth curve r (t) = g (t)i + h(t)j, then f (g (t), h(t)) = c.
▶ Differentiating both sides, we get
d d
f (g (t), h(t)) = (c)
dt dt
∂f dg ∂f dh
+ =0
∂x dt ∂y dt
∂f ∂f dg dh
i+ j · i+ j =0
∂x ∂y dt dt
Gradient and Tangent to Level curves
▶ If a differentiable function f (x, y ) has a constant value c along
a smooth curve r (t) = g (t)i + h(t)j, then f (g (t), h(t)) = c.
▶ Differentiating both sides, we get
d d
f (g (t), h(t)) = (c)
dt dt
∂f dg ∂f dh
+ =0
∂x dt ∂y dt
∂f ∂f dg dh
i+ j · i+ j =0
∂x ∂y dt dt
▶ That is, ∇f is normal to the tangent vector dr dt , so it is normal
to the curve.
▶ At every point (x0 , y0 ) in the domain of a differentiable
function f (x, y ), the gradient of f is normal to the level curve
through (x0 , y0 ).
Gradient and Tangent to Level curves
▶ At every point (x0 , y0 ) in the domain of a differentiable
function f (x, y ), the gradient of f is normal to the level curve
through (x0 , y0 ).
▶ The equation of the tangent line to the level curve is given by
fx (x0 , y0 )(x − x0 ) + fy (x0 , y0 )(y − y0 ) = 0.
Problem
Find an equation for the tangent to the ellipse
x2
+ y2 = 2
4
at the point (−2, 1).
Gradient and Tangent to Level curves
▶ At every point (x0 , y0 ) in the domain of a differentiable
function f (x, y ), the gradient of f is normal to the level curve
through (x0 , y0 ).
▶ The equation of the tangent line to the level curve is given by
fx (x0 , y0 )(x − x0 ) + fy (x0 , y0 )(y − y0 ) = 0.
Problem
Find an equation for the tangent to the ellipse
x2
+ y2 = 2
4
at the point (−2, 1).
Algebra of Gradients
Theorem
▶ ∇(f + g ) = ∇f + ∇g
▶ ∇(f − g ) = ∇f − ∇g
▶ ∇(kf ) = k∇f for any real number k.
▶ ∇(fg ) = f ∇g + g ∇f
▶ ∇ gf = g ∇f −f ∇g
g2
Algebra of Gradients
Theorem
▶ ∇(f + g ) = ∇f + ∇g
▶ ∇(f − g ) = ∇f − ∇g
▶ ∇(kf ) = k∇f for any real number k.
▶ ∇(fg ) = f ∇g + g ∇f
▶ ∇ gf = g ∇f −f ∇g
g2
proof: Exercise!
Constrained Maxima and Minima
Goal: Find the extreme values of a function whose domain is
constrained to lie in a subset of the plane-for example, a disk, a
closed triangular region, or along a curve.
Problem
Find the point p(x, y , z) on the plane 2x + y − z − 5 = 0 that is
closest to the origin.
Constrained Maxima and Minima
Goal: Find the extreme values of a function whose domain is
constrained to lie in a subset of the plane-for example, a disk, a
closed triangular region, or along a curve.
Problem
Find the point p(x, y , z) on the plane 2x + y − z − 5 = 0 that is
closest to the origin.
Reformulation: p
min x2 + y2 + z2
subject to the constraint that
2x + y − z − 5 = 0
Solution
▶ Regard x and y as independent variables, and write
z = 2x + y − 5
▶ The problem reduces to finding the point (x, y ) at which the
function
h(x, y ) = f (x, y , 2x + y − 5) = x 2 + y 2 + (2x + y − 5)2 .
▶ hx = 0, hy = 0 gives x = 5/3 and y = 65 .
▶ Hence, z = −5 5 5 −5
6 , and the closet point is ( 3 , 6 , 6 ).
Problem
Find the points on the hyperbolic cylinder x 2 − z 2 − 1 = 0 that are
closest to the origin.
Approach -1
▶ We seek for the points on the cylinder closest to the origin.
▶
min x 2 + y 2 + z 2
subject to
x 2 − z 2 − 1 = 0.
▶ If we regard x and y as independent variables in the
constraint equation, then z 2 = x 2 − 1.
▶ The problem reduces to h(x, y ) = x 2 + y 2 + (x 2 − 1)
▶ The critical point is (0, 0), which is not part of the cylinder.
Caveat
▶ The first derivative test gives the points in the domain of
h(x, y ) = 2x 2 + y 2 − 1 where h has a minimum value. But we
want the points on the cylinder where h is minimum.
▶ The problem is the domain for the function is the entire
XY -plane, but the domain in which we want to the distance
does not include the band between the line x = −1 and x = 1.
Approach - 2
Treat y and z as independent variables.
▶ Substitute x 2 = z 2 + 1 in the objective function x 2 + y 2 + z 2 ,
we get k(y , z) = 1 + y 2 + 2z 2 .
▶ The critical point is y = 0, z = 0, and hence x = ±1.
▶ As k(y , z) = 1 + y 2 + 2z 2 ≥ 1, so the points (±1, 0, 0) give a
minimum value for k.
▶ Thus the minimum distance from the origin to a point on the
cylinder is 1 unit.
A geometric approach
▶ Another way to find the points on the cylinder closest to the
origin is to imagine a small sphere centered at the origin
expanding like a soap bubble until it just touches the cylinder.
▶ At each point of contact, the cylinder and sphere have the
same tangent plane and normal line.
▶ If the sphere and the cylinder are represented by the level
surfaces obtained by setting:
x 2 + y 2 + z 2 − a2 = 0
and
x 2 − z 2 − 1 = 0,
then the gradients ∇f and ∇g will be parallel where the
surfaces touch.
▶ That is, ∇f = λ∇g . Thus
2xi + 2y j + 2zk = λ(2xi − 2zk).
▶ So,
2x = 2λx 2y = 0 2z = −2λz.
▶ Note that, x ̸= 0 on the surface, so λ = 1.
▶ Then, it is to see that z = 0 and y = 0. Thus the required
points are of the from (x, 0, 0).
▶ The points on the surface x 2 − z 2 = 1 which are of the
(x, 0, 0) are (±1, 0, 0). Thus the closest points are (±1, 0, 0).
▶ This is a protype for the Lagrange method.
The orthogonal gradient theorem
Theorem
Suppose that f (x, y , z) is differentiable in a region whose interior
contains a smooth curve
C : r (t) = x(t)i + y (t)j + z(t)k.
If P is a point on C where f has local maximum or minimum
relative to its values on C , then ∇f is orthogonal to C at P.
Proof:
▶ We claim that ∇f is orthogonal to the tangent vector of the
curve C at P.
▶ The function f on C is given by f (x(t), y (t), z(t)), and the
derivative with respect to t is given by:
df ∂f dx ∂f dy ∂f dz ′
= + + = ∇f · r .
dt ∂x dt ∂y dt ∂z dt
▶ At any point P where f has a local maximum or minimum
relative to its values on the curve, df
dt = 0. So
′
∇f .r = 0.
Lagrange Multiplier method - Theory
▶ Let f (x, y , z) and g (x, y , z) are differentiable functions.
▶ Let P be a point on the surface g (x, y , z) = 0 such that the
function f has a local maximum/minimum value relative to its
other values.
▶ Assumption ∇g ̸= 0 at points on the surface g (x, y , z) = 0.
▶ Then f takes on a local maximum or minimum at P on every
differentiable curve passes through P contained in the surface
g (x, y , z) = 0.
▶ By the Orthogonal Gradient Theorem, we have ∇f is
orthogonal to the tangent vector of every such differentiable
curve passes through P. Also ∇g is orthogonal to all the
tangent vectors.
▶ Thus, at the point P0 , ∇f is some scalar multiple of ∇g .
Lagrange Multiplier method - Single constraint
Theorem
Suppose that f (x, y , z) and g (x, y , z) are differentiable and
∇g ̸= 0 when g (x, y , z) = 0. To find the local maximum and
minimum values of f subject to the constraint g (x, y , z) = 0 (if
these exist), find the values of x, y , z, and λ that simultaneously
satisfy the equations
∇f = λ∇g g (x, y , z) = 0.
For the functions of two variable, the condition is similar, but
without the variable z.
Problem
Find teh greatest and smallest values that the function
f (x, y ) = xy
takes on the ellipse
x2 y2
+ = 1.
8 2
Lagrange Multiplier method - Two constraints
Theorem
Suppose that f (x, y , z), g1 (x, y , z) = 0 and g2 (x, y , z) are
differentiable and ∇g1 is not parallel to ∇g2 . To find the local
maximum and minimum values of f subject to the constraint
g1 (x, y , z) = 0 and g2 (x, y , z) = 0 (if these exist), find the values
of x, y , z, µ, and λ that simultaneously satisfy the equations
∇f = λ∇g1 + µ∇g2 g1 (x, y , z) = 0 and g2 (x, y , z) = 0.
Problem
The plane x + y + z = 1 cuts the cylinder x 2 + y 2 = 1 in an
ellipse. Find the points on the ellipse that lie closest to and
farthest from the origin.