Lagrange Multipliers

1 Lagrange Multipliers
1.1 Introduction
The Lagrange Multiplier Method is a mathematical technique used to find the max-
imum or minimum values of a function subject to a constraint.
It is widely used in:

• Mathematics

• Physics

• Engineering

• Economics

• Optimization problems

1.2 Basic Idea

Suppose we want to optimize a function

f (x, y)

subject to a constraint
g(x, y) = 0.
Instead of solving directly, we introduce a new variable called the Lagrange multi-
plier λ.

1.3 Main Formula

The method is based on the condition:

∇f = λ∇g,

where:

• ∇f is the gradient of the objective function,

• ∇g is the gradient of the constraint,

• λ is the Lagrange multiplier.

1
1.4 Steps of Lagrange Multipliers
1.4.1 Step 1: Objective Function
Consider the function to be optimized:

f (x, y) = x2 + y 2 .

1.4.2 Step 2: Constraint

Given constraint:
x + y = 10.
Rewrite as:
g(x, y) = x + y − 10 = 0.

1.4.3 Step 3: Form Equations

Compute gradients:
∇f = (2x, 2y), ∇g = (1, 1).
Apply:
∇f = λ∇g.
So,
2x = λ, 2y = λ.

1.4.4 Step 4: Solve Equations

From above:
2x = 2y ⇒ x = y.
Using constraint:
x + y = 10 ⇒ 2x = 10 ⇒ x = 5,
so
y = 5.

1.4.5 Step 5: Function Value

f (5, 5) = 52 + 52 = 25 + 25 = 50.

1.5 Final Answer

The minimum value is:
50
at the point:
(5, 5).

2
1.6 Geometrical Meaning
The method works at the point where:
• the level curve of f (x, y)
• touches the constraint curve g(x, y) = 0
At this point, the gradients are parallel.

1.7 Applications
Mathematics
• Optimization problems
Physics
• Energy minimization
Economics
• Profit maximization
• Cost minimization
Engineering
• Design optimization
Machine Learning
• Constrained optimization

1.8 Simple Example

Maximize:
f (x, y) = xy
subject to:
x + y = 10.

1.8.1 Step 1: Constraint

g(x, y) = x + y − 10 = 0.

1.8.2 Step 2: Gradients

∇f = (y, x), ∇g = (1, 1).

1.8.3 Step 3: Apply Formula

y = λ, x = λ ⇒ x = y.
Using constraint:

x + y = 10 ⇒ 2x = 10 ⇒ x = 5, y = 5.

3
1.8.4 Step 4: Maximum Value
f (5, 5) = 25.

1.9 Important Notes

Without Constraint: Use ordinary derivatives.
With Constraint: Use Lagrange multipliers.

1.10 Key Terms

Term Meaning
Objective Function Function to optimize
Constraint Condition that must be satisfied
Gradient Vector of partial derivatives
Multiplier λ Extra variable used in optimization

1.11 Short Definition

Lagrange multipliers are used to find maxima or minima of a function subject to con-
straints.

2 What is Gradient (∇)?

The symbol

∇
is called the del or gradient operator.
The gradient tells us:

• the direction of fastest increase of a function,

• and how quickly the function changes.

It is very important in:

• Calculus

• Physics

• Optimization

• Machine Learning

• Lagrange Multipliers

4
3 Gradient of a Function
Suppose we have a function:

f (x, y)
Its gradient is written as:

∇f
and defined as:
 
∂f ∂f
∇f = ,
∂x ∂y
This means:

• take partial derivative with respect to x,

• take partial derivative with respect to y,

• and combine them into a vector.

4 Example 1
Find gradient of:

f (x, y) = x2 + y 2

Step 1: Partial Derivative with Respect to x

∂f
= 2x
∂x
because y 2 is treated as constant.

Step 2: Partial Derivative with Respect to y

∂f
= 2y
∂y

Final Gradient
∇f = (2x, 2y)
This is a vector.

5
5 Meaning of Gradient
At every point (x, y):

∇f = (2x, 2y)
tells us:

• which direction increases the function fastest,

• how steep the increase is.

6 Gradient in Lagrange Multipliers

In Lagrange multipliers we use:

∇f = λ∇g
where:

• f (x, y) = objective function

• g(x, y) = 0 = constraint

• λ = Lagrange multiplier

7 Why Are Gradients Equal?

At the maximum or minimum point:

• the function curve touches the constraint curve,

• both have the same direction.

So their gradients become parallel.

That is why:

∇f = λ∇g

8 Simple Visualization
Think of climbing a hill.

• Gradient points toward the steepest upward direction.

• Bigger gradient means steeper hill.

In optimization:

• gradient helps find maxima and minima.

6
9 Example 2
Given:

f (x, y) = xy
Find gradient.

Partial Derivative with Respect to x

∂f
=y
∂x

Partial Derivative with Respect to y

∂f
=x
∂y

Gradient
∇f = (y, x)

10 Gradient in 3 Variables
For:

f (x, y, z)
gradient is:
 
∂f ∂f ∂f
∇f = , ,
∂x ∂y ∂z
Example:

f = x2 + y 2 + z 2
then:

∇f = (2x, 2y, 2z)

11 Important Terms
Symbol Meaning
∇ Gradient operator
∇f Gradient of function
∂ Partial derivative
Vector Quantity with direction and magnitude

7
12 Short Definition
Gradient is a vector containing all partial derivatives of a function. It points in the
direction of maximum increase of the function.

8
13 Lagrange Multipliers with Two Constraints
When a function has two constraints, we use two Lagrange multipliers.

14 Problem Setup
Suppose we want to maximize or minimize:

f (x, y, z)
subject to two constraints:

g(x, y, z) = 0
and

h(x, y, z) = 0
Then we introduce two multipliers:

λ and µ

15 Main Formula
For two constraints:

∇f = λ∇g + µ∇h
where:

• ∇f = gradient of objective function

• ∇g = gradient of first constraint

• ∇h = gradient of second constraint

• λ, µ = Lagrange multipliers

16 Steps for Solving

Step 1: Write Objective Function
Example:

f (x, y, z) = x + y + z

9
Step 2: Write Constraints
x2 + y 2 + z 2 = 14

x+y−z =0
Rewrite as:

g(x, y, z) = x2 + y 2 + z 2 − 14 = 0

h(x, y, z) = x + y − z = 0

Step 3: Find Gradients

Gradient of f

∇f = (1, 1, 1)
Gradient of g

∇g = (2x, 2y, 2z)

Gradient of h

∇h = (1, 1, −1)

Step 4: Apply Formula

Using:

∇f = λ∇g + µ∇h
we get:

(1, 1, 1) = λ(2x, 2y, 2z) + µ(1, 1, −1)

Step 5: Compare Components

Comparing x, y, z components:

1 = 2λx + µ

1 = 2λy + µ

1 = 2λz − µ
along with constraints:

x2 + y 2 + z 2 = 14

10
 x+y−z =0
Now solve all equations together.

17 Geometrical Meaning
With one constraint:

• optimization occurs on a surface.

With two constraints:

• optimization occurs on the intersection of two surfaces.

At optimum point:

• gradients become linearly dependent.

18 General Formula for n Constraints

If there are many constraints:

g1 = 0, g2 = 0, g3 = 0
then:

∇f = λ1 ∇g1 + λ2 ∇g2 + λ3 ∇g3

19 Important Idea
Each constraint introduces:

• one extra multiplier.

Number of Constraints Number of Multipliers

1 1 multiplier
2 2 multipliers
3 3 multipliers

20 Why This Works

At the maximum or minimum point:

• movement is restricted by constraints,

• gradients of constraints determine allowed directions,

• gradient of objective function becomes combination of constraint gradients.

11
21 Simple Summary
For two constraints:

∇f = λ∇g + µ∇h
This method helps find maxima or minima when a function must satisfy two conditions
simultaneously.

21.1 Geometrical and Physical Meaning of Lagrange Multipli-

ers
The method of Lagrange multipliers has a clear geometrical interpretation.
Suppose we have two functions:
• An objective function f (x, y) whose maximum or minimum we want to find,

• A constraint function g(x, y) = 0 which restricts the motion of the point (x, y).
Geometrically, both f (x, y) and g(x, y) represent curves in the (x, y)-plane.

Geometrical Picture
The function f (x, y) represents a family of level curves

f (x, y) = constant,

while the constraint

g(x, y) = 0
represents a fixed curve on which the solution must lie.
The solution occurs at the point where a level curve of f (x, y) just touches (is tangent
to) the constraint curve.
At this point:
• The two curves do not cross each other,

• They only touch each other,

• Their tangent directions are the same.

This implies that their gradients are parallel:

∇f = λ∇g.

Physical Interpretation
In physics, the constraint g(x, y) = 0 restricts the motion of a system to a surface or
curve (for example, a particle moving on a wire or surface).
The function f (x, y) represents a physical quantity such as:
• Energy,

• Potential,

12
 • or any quantity to be optimized.

The condition ∇f = λ∇g means that:

• The system cannot move freely in all directions,

• It can only move along the constraint,

• At equilibrium, the force (or gradient of f ) is balanced by the constraint force (given
by λ∇g).

Key Idea
At the optimal point, the curve of f (x, y) just touches the constraint curve
g(x, y) = 0, and both curves share the same direction of steepest change.

Thus, Lagrange multipliers help us find points where a function is optimized under a
restriction by studying how two curves interact geometrically.

13