Ecole Nationale Polytechnique de Constantine Pr. K.
KAABECHE-DJERAFI
5ème année Génie des procédés
Cours : Optimisation
-------------------------------------------------------------------------------------------------------------------------------
Chapter I. General principles and study of functions
Introduction
Optimization is the use of specific methods to determine the most cost-effective and efficient
solution to a problem or to the design of a process. This technique is one of the main quantitative
tools in industrial decision-making. A wide variety of problems in the design, construction,
operation, and analysis of chemical product plants (as well as many other industrial processes) can
be solved by optimization. The aim is to find the values of the process variables that yield the best
value of the performance criterion.
A trade-off usually exists between investment and operating costs. The process factors describe
the model and performance criterion, forming the optimization problem.
• Why optimize?
• Why are engineers interested in optimization?
• What are the advantages of using this method rather than making decisions intuitively?
Engineers work to improve the initial design of equipment and to enhance its operation once
installed, in order to achieve maximum output, maximum profit, minimum cost, and the least
energy consumption.
Role in Process Engineering
It is to find the values of the variables (T, P, Q, V…) that lead to the best performance criteria, or
to try to find the best combination of variables to achieve the most favorable situation
Examples:
• A plant site location can be optimized ;
• Optimization of product and raw material distribution ;
• Optimization of maintenance and spare parts ;
Ecole Nationale Polytechnique de Constantine Pr. K. KAABECHE-DJERAFI
5ème année Génie des procédés
Cours : Optimisation
-------------------------------------------------------------------------------------------------------------------------------
• Optimization of operations and equipment : heat exchangers (optimal flow rate of the heat
transfer fluid, outlet cooling water temperature, etc.), distillation columns.
I. Characteristics of an Optimization Operation
To carry out an optimization, the following three steps must be followed:
1. Determine the function to be optimized, known as the objective function;
2. Identify the parameters with respect to which the optimization is performed, called
performance criteria;
3. Choose the appropriate technique for optimization.
II. Definitions
An n-dimensional optimization problem can generally be written as follows:
Minimize f(X) = f(x₁, x₂, ..., xₙ)
Subject to Gᵢ(X) = 0 or Gᵢ(X) ≤ 0,
with X = {x₁, x₂, ..., xₙ}
• The quantity f(X) is the criterion to be minimized, called the objective function.
• The vector X, composed of n variables, represents the parameters of the problem.
• The functions Gᵢ(X) represent the equality and inequality constraints.
III. Reminders:
III.1. Local and Global Extremum
The goal of optimization is to determine the vector 𝑋𝑚𝑖𝑛 that yields the minimum value of
the objective function.
• If 𝐸𝑚𝑖𝑛 is the smallest value of E across all possible combinations of variable values, the
minimum is called a global minimum.
• If the extremum found is not global, it is only an extremum within a specific search
region, and in this case, it is called a local minimum.
Ecole Nationale Polytechnique de Constantine Pr. K. KAABECHE-DJERAFI
5ème année Génie des procédés
Cours : Optimisation
-------------------------------------------------------------------------------------------------------------------------------
If the extremum obtained is not global, it can only be considered an extremum within a specific
search region. In this case, it is referred to as a local extremum.
III.2. Jacobian Vector
It is an n-dimensional vector whose elements are the first-order partial derivatives of the
function:
𝜕𝑓
𝜕𝑥1
𝜕𝑓
𝐽(𝑋) = 𝜕𝑥2
⋮
𝜕𝑓
[𝜕𝑥𝑛 ]
III.3. Hessian Matrix
The Hessian of f is an 𝑛𝑥𝑛 square matrix whose element in row 𝑖 and column 𝑗 is
𝜕2 𝑓
given by
𝜕𝑥𝑖𝜕𝑥𝑗
𝜕 2𝑓 𝜕 2𝑓
⋯
𝜕𝑥12 𝜕𝑥1 𝜕𝑥𝑛
𝐻= ⋮ ⋱ ⋮
𝜕 2𝑓 𝜕 2𝑓
⋯
[𝜕𝑥𝑛 𝜕𝑥1 𝜕𝑥𝑛2 ]
III.4. Constraints
For certain problems, the search region for the optimum is limited by conditions known as
constraints.
Example of a constraint: 𝑥𝑖−1 ≤ 𝑥 ≤ 𝑥𝑖+1 ∶ 𝑔(𝑋) ≤ 0
Note: The region where the constraints are satisfied is called the feasible region. The region that
is excluded is known as the infeasible region.
Ecole Nationale Polytechnique de Constantine Pr. K. KAABECHE-DJERAFI
5ème année Génie des procédés
Cours : Optimisation
-------------------------------------------------------------------------------------------------------------------------------
IV. Properties of Objective Functions
IV.1 Unimodal and Multimodal Functions
• A function is said to be unimodal within a given interval of variables if it has only one
extremum in that interval. In this case, the extremum is considered global within the
interval.
• A multimodal function has multiple extrema within the interval.
IV.2. Concave Function
IV.2.1. Concavity Criterion for Single-Variable Functions:
Figure I.1. Graph of a Concave Function
The function is concave if:
𝑓(𝑥) ≥ 𝑓1 (𝑥) (1)
According to the figure:
𝑥𝐵 − 𝑥 = 𝜃 (𝑥𝐵 − 𝑥𝐴 ) ⟹ 𝑥 = 𝑥𝐵 − 𝜃(𝑥𝐵 − 𝑥𝐴 )
𝑥 = (1 − 𝜃)𝑥𝐵 + 𝜃𝑥𝐴 (2)
Ecole Nationale Polytechnique de Constantine Pr. K. KAABECHE-DJERAFI
5ème année Génie des procédés
Cours : Optimisation
-------------------------------------------------------------------------------------------------------------------------------
And on the other hand:
𝑓(𝑥𝐵 − 𝑓(𝑥𝐴 ) 𝑓(𝑥𝐵 ) − 𝑓1 (𝑥)
=
𝑥𝐵 − 𝑥𝐴 𝑥𝐵 − 𝑥
𝑓(𝑥𝐵 − 𝑓(𝑥𝐴 ) 𝑥𝐵 − 𝑥𝐴 1
= =
𝑓 (𝑥𝐵) − 𝑓1 (𝑥) 𝑥𝐵 − 𝑥 𝜃
𝑓1 (𝑥 ) = 𝑓(𝑥𝐵 ) − 𝜃(𝑓(𝑥𝐵 ) − 𝑓(𝑥𝐴 ))
𝑓1 (𝑥) = (1 − 𝜃)𝑓(𝑥𝐵 ) + 𝜃𝑓(𝑥𝐴 ) (3)
By substituting (2) and (3) into (1):
𝑓((1 − 𝜃 )𝑥𝐵 + 𝜃 × 𝑥𝐴 ) ≥ (1 − 𝜃 )𝑥𝐵 + 𝜃 × 𝑥𝐴
Remarks:
• If the inequality is strict, the function is said to be strictly concave.
• For a convex function, the convexity criterion is:
𝑓 (𝑥 ) ≤ 𝑓1 (𝑥 )
• A linear function (straight line) is both concave and convex at the same time.
• For single-variable functions, calculating the second derivative allows us to determine
the nature of the function.
IF : 𝑓 ′′ (𝑥) < 0 : fonction concave ;
𝑓 ′′ (𝑥 ) > 0 : fonction convexe ;
𝑓 ′′ (𝑥 ) = 0 : fonction concave et convexe.
IV.2.2. Criteria for Functions of Several Variables
The concept of convexity and concavity also applies to functions of several variables, as the
Hessian matrix must be evaluated to determine the nature of the function:
o 𝐻(𝑋) Is positive definite if:
𝑋 𝑇 . 𝐻(𝑋). 𝑋 > 0, ∀𝑋 ≠ 0: 𝑓 (𝑋) : Is convex
o 𝐻(𝑋) Is negative definite if:
Ecole Nationale Polytechnique de Constantine Pr. K. KAABECHE-DJERAFI
5ème année Génie des procédés
Cours : Optimisation
-------------------------------------------------------------------------------------------------------------------------------
𝑋 𝑇 . 𝐻(𝑋). 𝑋 < 0, ∀𝑋 ≠ 0: 𝑓 (𝑋) : Is concave
o 𝐻(𝑋) Is indefinite if:
𝑋 𝑇 . 𝐻 (𝑋). 𝑋 > 0, 𝐅𝐨𝐫 𝐯𝐚𝐥𝐮𝐞𝐬 𝐨𝐟. 𝑋
𝑋 𝑇 . 𝐻 (𝑋). 𝑋 < 0, For other values.
Note:
If, for the first two cases, the inequalities are taken in the broad sense (≥ or ≤), we say that 𝐻 (𝑋)
is semi-definite.
IV.2.3 Other Criteria for Evaluating the Nature of 𝐻(𝑋)
o 𝐻(𝑋) is positive definite :
First Test:
All the diagonal elements of 𝐻(𝑋) are positive, and the determinants of the principal
minors are positive
Second Test:
All the eigenvalues must be positive.
o 𝐻(𝑋) is negative definite:
First Test:
All the diagonal elements of 𝐻(𝑋) are negative, and the determinants of the even-order
minors are positive, while those of the odd-order minors are negative.
Second Test:
All the eigenvalues must be negative.
Reminders:
• Minor of order 𝒌 : It is the square submatrix of size k×k, obtained by removing the last
𝑛 − 𝑘 rows and columns from a square n ×n matrix.
• Eigenvalue of 𝑨 :
An eigenvalue of 𝑯(𝑿) is a scalar 𝛼 such that there exists a non-zero vector 𝑉 satisfying:
Ecole Nationale Polytechnique de Constantine Pr. K. KAABECHE-DJERAFI
5ème année Génie des procédés
Cours : Optimisation
-------------------------------------------------------------------------------------------------------------------------------
𝐴. 𝑉 = 𝛼𝑉 ⟹ (𝐴 − 𝛼𝐼 )𝑉 = 0
The vector 𝑉 is an eigenvector of the matrix 𝐴
Eigenvalues are found by solving the characteristic equation:
𝑑é𝑡 (𝐴 − 𝛼𝐼 ) = 0
IV.2.3 Conditions for the existence of an extremum (local) for a
differentiable function:
Let 𝑓 be a function of several variables.
1. Necessary condition (first order):
A point 𝑋0 is a candidate for a local extremum if the gradient vanishes:
∇𝑓(𝑋0 ) = 0
2. Sufficient condition (second order):
Analyze the Hessian matrix 𝐻(𝑋0 ), i.e., the matrix of second partial derivatives.
o If 𝐻(𝑋0 ) is positive definite, then 𝑋0 is a local minimum.
o If 𝐻(𝑋0 ) is negative definite, then 𝑋0 is a local maximum.
o If 𝐻(𝑋0 ) is indefinite, then 𝑋0 is a saddle point.
o If 𝐻(𝑋0 ) is semi-definite, further analysis is required to conclude.