AL-BALQA APPLIED UNIVERSITY

AL-HUSON UNIVERSITY COLLEGE

Electrical Engineering Department

ELE5453 Artificial Intelligence and

Machine Learning

February 24, 2026

Contents

1 Introduction to Optimization 6
1.1 Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 Engineering Applications of Optimization . . . . . . . . . . . . . . . . . . . . . . . 7
1.3 Convexity and Concavity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.4 Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.5 Statement of an Optimization Problem . . . . . . . . . . . . . . . . . . . . . . . . 10
1.5.1 Examples on real optimization problems: . . . . . . . . . . . . . . . . . . . 10
1.5.2 The general form of optimization problems: . . . . . . . . . . . . . . . . . 11
1.5.3 Convex optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.6 Unconstrained and constrained optimization . . . . . . . . . . . . . . . . . . . . . 12
1.7 Gradient Descent Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.7.1 What is the gradient of a function? . . . . . . . . . . . . . . . . . . . . . . 12
1.7.2 What is Gradient Descent? . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.7.3 What is the idea of Gradient Descent? . . . . . . . . . . . . . . . . . . . . 13
1.8 Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2 Artificial Intelligence and Machine Learning 26

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.2 What is Artificial Intelligence? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.3 What is Machine Learning? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.4 How AI and ML Work Together . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.5 What is the difference between Artificial Intelligence and Machine Learning? . . . 28
2.6 Real-World Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3 Machine Learning 30
3.1 Introduction to Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.2 How does Machine Learning work . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.3 Need for Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.4 Classification of Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.4.1 Supervised Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.4.2 Unsupervised Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.4.3 Reinforcement Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

4 Core Concepts and Terminology in Machine Learning 33

4.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.1.1 Structured and Unstructured Data . . . . . . . . . . . . . . . . . . . . . . 33
4.1.2 Key Components of a Dataset . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.1.3 Types of Datasets Based on the Type of ML Task . . . . . . . . . . . . . . 34
4.2 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.2.1 Examples of Machine Learning Models: . . . . . . . . . . . . . . . . . . . . 34

1
 4.2.2 Steps in Using a Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.3 Training and Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.4 Overfitting versus Underfitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.4.1 Overfitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.4.2 Underfitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

5 Supervised Learning - Data Preprocessing 39

5.1 Why do we need Data Preprocessing? . . . . . . . . . . . . . . . . . . . . . . . . . 39
5.1.1 Get the Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
5.1.2 Importing Libraries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
5.1.3 Importing the Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
5.1.4 Handling Missing data: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
5.1.5 Encoding Categorical data: . . . . . . . . . . . . . . . . . . . . . . . . . . 43
5.1.6 Splitting the Dataset into the Training set and Test set . . . . . . . . . . . 46
5.1.7 Feature Scaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
5.1.8 Combining all the steps: . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

6 Regression Analysis in Machine learning 50

6.1 Terminologies in Regression Analysis . . . . . . . . . . . . . . . . . . . . . . . . . 51
6.2 Types of Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

7 Simple Linear Regression in Machine Learning 52

7.1 Simple Linear Regression Model: . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
7.2 Implementation of Simple Linear Regression Algorithm using Python . . . . . . . 52

8 ML Polynomial Regression 59
8.1 Need for Polynomial Regression: . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
8.2 Implementation of Polynomial Regression using Python . . . . . . . . . . . . . . . 60
8.3 Steps for Polynomial Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
8.3.1 Data Pre-processing Step . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

9 Classification Algorithm in Machine Learning 67

9.1 What is the Classification Algorithm? . . . . . . . . . . . . . . . . . . . . . . . . . 67
9.2 Learners in Classification Problems: . . . . . . . . . . . . . . . . . . . . . . . . . . 68
9.3 Types of ML Classification Algorithms: . . . . . . . . . . . . . . . . . . . . . . . . 68
9.4 Evaluating a Classification model: . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
9.4.1 Confusion Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
9.5 Use cases of Classification Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . 69

10 K-Nearest Neighbor(KNN) Algorithm for Machine Learning 70

10.1 Why do we need a K-NN Algorithm? . . . . . . . . . . . . . . . . . . . . . . . . . 70
10.2 How does K-NN work? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
10.3 Distance Metrics in KNN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
10.4 How to select the value of K in the K-NN Algorithm? . . . . . . . . . . . . . . . . 72
10.5 Advantages and Disadvantages of KNN Algorithm: . . . . . . . . . . . . . . . . . 72
10.6 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
10.7 Python implementation of the KNN algorithm . . . . . . . . . . . . . . . . . . . . 74
10.7.1 Data Pre-Processing Step: . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
10.7.2 Fitting K-NN classifier to the Training data: . . . . . . . . . . . . . . . . . 76
10.7.3 Predicting the Test Result . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

2
 10.7.4 Creating the Confusion Matrix . . . . . . . . . . . . . . . . . . . . . . . . 77
10.8 Visualizing the Training set result . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
10.9 Visualizing the Test set result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
10.10Combining all the steps: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

11 Support Vector Machine Algorithm 81

11.1 Types of SVM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
11.2 How does SVM works? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
11.2.1 Linear SVM: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
11.2.2 Non-Linear SVM: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
11.3 Python Implementation of Support Vector Machine . . . . . . . . . . . . . . . . . 85
11.3.1 Data Pre-processing step . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
11.3.2 Fitting the SVM classifier to the training set: . . . . . . . . . . . . . . . . 86
11.3.3 Predicting the test set result: . . . . . . . . . . . . . . . . . . . . . . . . . 87
11.3.4 Creating the confusion matrix: . . . . . . . . . . . . . . . . . . . . . . . . . 87
11.3.5 Visualizing the training set result: . . . . . . . . . . . . . . . . . . . . . . . 88
11.3.6 Visualizing the test set result: . . . . . . . . . . . . . . . . . . . . . . . . . 88
11.4 Combining all the steps: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

12 Markov Decision Processes 90

12.1 Markov Chains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
12.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
12.1.2 Markov Property . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
12.1.3 Components of a Markov Chain . . . . . . . . . . . . . . . . . . . . . . . . 91
12.2 Markov Reward Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
12.2.1 Introduction to Markov Reward Process . . . . . . . . . . . . . . . . . . . 92
12.2.2 Components of a Markov Reward Process . . . . . . . . . . . . . . . . . . 92
12.2.3 Example: Simple Markov Reward Process . . . . . . . . . . . . . . . . . . 93
12.2.4 Computing the Return . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
12.2.5 Value Function for MRPs . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
12.2.6 Bellman Equation for Value Function . . . . . . . . . . . . . . . . . . . . . 93
12.2.7 Example: Computing Value Function . . . . . . . . . . . . . . . . . . . . . 94
12.3 Markov Decision Process (MDP) . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
12.3.1 Introduction to Markov Decision Process . . . . . . . . . . . . . . . . . . . 94
12.3.2 Components of MDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
12.3.3 Bellman Equation for MDP . . . . . . . . . . . . . . . . . . . . . . . . . . 95
12.3.4 Policy and Optimal Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
12.3.5 Solving an MDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
12.3.6 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

13 Reinforcement Learning 97
13.1 What is Reinforcement Learning? . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
13.2 Reinforcement Learning Loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
13.3 Markov Decision Process (MDP) . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
13.4 Types of Reinforcement Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
13.5 Q-Learning Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
13.6 Applications of Reinforcement Learning . . . . . . . . . . . . . . . . . . . . . . . . 99

3
14 Introduction to Neural Networks 100
14.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
14.2 Biological Inspiration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
14.3 Neuron Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
14.3.1 Single-Input Neuron . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
14.3.2 Transfer (Activation) Functions . . . . . . . . . . . . . . . . . . . . . . . . 103
14.3.3 Multiple-Input Neuron . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
14.4 Network Architectures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
14.4.1 A Layer of Neurons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
14.4.2 Multiple Layers of Neurons . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
14.5 Implementation of neural network . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
14.5.1 Implementation of neural network from scratch using NumPy . . . . . . . 106
14.5.2 Activation Functions: Adding Non-Linearity . . . . . . . . . . . . . . . . . 106
14.6 Neural Networks using NumPy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
14.6.1 Make a Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
14.6.2 Characterize Labels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
14.6.3 Import Libraries and Characterize Activation Function . . . . . . . . . . . 108
14.6.4 Initialize Weights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
14.6.5 Characterize the Feedforward Neural Network . . . . . . . . . . . . . . . . 109
14.6.6 Characterize the Loss Function . . . . . . . . . . . . . . . . . . . . . . . . 109
14.6.7 Backpropagation of Error . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
14.6.8 Training the Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . 110
14.6.9 Predicting with a Trained Model . . . . . . . . . . . . . . . . . . . . . . . 111
14.6.10 Initialize Weights for Training . . . . . . . . . . . . . . . . . . . . . . . . . 112
14.6.11 Train the Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
14.6.12 Print Trained Weights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
14.6.13 Make Predictions with a Trained Model . . . . . . . . . . . . . . . . . . . 113

4
List of Figures

1.1 Example of a concave function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.2 Example of a convex function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.3 Determining concavity or convexity of functions example 1. . . . . . . . . . . . . . 9
1.4 Determining concavity or convexity of functions example 2. . . . . . . . . . . . . . 9
1.5 Determining concavity or convexity of functions example 3. . . . . . . . . . . . . . 9
1.6 Minimum of f (x) is same as maximum of −f (x). . . . . . . . . . . . . . . . . . . 10

3.1 The process of a machine learning algorithm . . . . . . . . . . . . . . . . . . . . . 30

9.1 Example of classification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

10.1 KNN. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

14.1 Schematic Drawing of Biological Neurons . . . . . . . . . . . . . . . . . . . . . . . 101

14.2 Schematic Drawing of Biological Neurons . . . . . . . . . . . . . . . . . . . . . . . 102
14.3 Multiple-Input Neuron . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
14.4 Neuron with R Inputs, Abbreviated Notation . . . . . . . . . . . . . . . . . . . . 104
14.5 Layer of S Neurons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
14.6 Three-Layer Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
14.7 Three-Layer Network, Abbreviated Notation . . . . . . . . . . . . . . . . . . . . . 106

5
Chapter 1

Introduction to Optimization

1.1 Optimization
ˆ Definition of Optimization: Optimization is the process of obtaining the best result
under given circumstances by minimizing effort or maximizing benefit.
ˆ Role in Engineering: Engineers make various technological and managerial decisions
at different stages of system design, construction, and maintenance, aiming for optimal
outcomes.
ˆ Mathematical Formulation: Optimization involves expressing effort or benefit as a func-
tion of decision variables and finding conditions that yield the maximum or minimum value.
ˆ Optimization Methods: No single method can solve all optimization problems efficiently,
leading to the development of different mathematical programming techniques.
ˆ Operations Research: A branch of mathematics that applies scientific methods to decision-
making problems, seeking optimal solutions.
ˆ Historical Background: Operations research originated during World War II when the
British military sought systematic methods for resource allocation, leading to techniques
such as linear programming.

Table 1.1: Some of Mathematical programming or optimization techniques [1]

optimization techniques
Calculus methods
Nonlinear programming
Geometric programming
Linear programming
Dynamic programming
Integer programming
Stochastic programming
Separable programming
Multiobjective programming
Modern or nontraditional optimization techniques
Genetic algorithms
Simulated annealing
Ant colony optimization

6
1.2 Engineering Applications of Optimization
Optimization, in its broadest sense, can be applied to solve any engineering problem. Some typical
applications from different engineering disciplines indicate the wide scope of the subject [1]:

1. Design of aircraft and aerospace structures for minimum weight.

2. Design of civil engineering structures such as frames, foundations, bridges, towers, chimneys,
and dams for minimum cost.

3. Optimum design of electrical machinery such as motors, generators, and transformers.

4. Shortest route taken by a salesperson visiting various cities during one tour.

5. Optimal production planning, controlling, and scheduling.

1.3 Convexity and Concavity

When we talk about convexity and concavity, we are referring to the shape of the curve or function.
Now, convexity and concavity do sound like complicated terms. However, they are simply a way
of describing what a curve or function looks like [2].

What is a Concave Function?

A concave function is a function where a straight segment between any two points on the graph
does not lie above the curve of the graph. In other words, the straight line is always below or on
the curve.
Fig. 1.1 shows an example of a concave function. You can see that if we pick any two points
on the curve, and draw a line segment between them, the line segment will always lie below the
curve.

Fig. 1.1: Example of a concave function.

Although this example only uses two points, for a function to be concave, the rule must be
true for all combinations of points on that function, in the given range.

7
What is a Convex Function?
A convex function is a function where a straight segment between any two points on the graph
does not lie below the curve of the graph, In other words, the straight line is always above or at
the same place as the function’s curve. It is the opposite of a concave function.
Fig. 1.2 shows an example of a convex function. You can see that if we pick any two points on
the curve, and draw a line segment between them, the line segment will always above or at the
same level as the function itself.

Fig. 1.2: Example of a convex function.

Although this example only uses two points, for a function to be concave the rule must be true
for all combinations of points on that function, in the given range.

Can a Function be both Concave and Convex?

Yes, this is possible. This is because both functions have an equal sign in the inequality. The
most common example of this is any straight line, as the function for a point between any two
points will match the equivalent function between both functions.

The convexity or concavity of univariate (single-variable) functionS using its second

derivative

ˆ Convex Function: A function f (x) defined over an interval I is convex if its second derivative
f ′′ (x) is non-negative for all x in I. Mathematically, if f ′′ (x) ≥ 0 for all x in I, then f (x) is
convex.

ˆ Concave Function: A function f (x) defined over an interval I is concave if its second deriva-
tive f ′′ (x) is negative for all x in I. Mathematically, if f ′′ (x) ≤ 0 for all x in I, then f (x) is
concave.

8
Example 1.1 For the following functions, determine whether they are concave, convex or both.

Fig. 1.3: Determining concavity or convexity of functions example 1.

The cubic is neither concave nor convex, but it is convex when x > 0 and concave when x < 0.

Fig. 1.4: Determining concavity or convexity of functions example 2.

Now, above we have a quadratic function. We can see that any line segment drawn will lie
below the curve. Thus, the function is concave.

Fig. 1.5: Determining concavity or convexity of functions example 3.

Finally, we have a straight line. Any line segment will lie on the line and hence it is both
concave and convex.

9
1.4 Duality
It can be seen from Fig. 1.6 that if a point x∗ corresponds to the minimum value of function
f (x), the same point also corresponds to the maximum value of the negative of the function,
−f (x). Thus without loss of generality, optimization can be taken to mean minimization since
the maximum of a function can be found by seeking the minimum of the negative of the same
function.

Fig. 1.6: Minimum of f (x) is same as maximum of −f (x).

1.5 Statement of an Optimization Problem

In optimization problems, there are several key components that define the problem and guide
the search for the optimal solution. Here are the main components of an optimization problem,
along with details about each component:

1. Objective Function: The objective function is a mathematical function that represents

the quantity to be optimized or minimized. It defines the goal of the optimization problem.
Example: Area of a square that needs to be maximized.

2. Decision Variables: Decision variables are the variables that can be adjusted or controlled
to optimize the objective function. They represent the choices or decisions to be made in
the optimization process. Example: The lengths of the square.

3. Constraints: Constraints are conditions or limitations that must be satisfied during the
optimization process. They represent the restrictions on the values of the decision variables.
Example: The perimeter of the square must be less than a certain value.

1.5.1 Examples on real optimization problems:

1. Device sizing in electronic circuits

ˆ Decision Variables: Device widths and lengths.

ˆ Objective Function: Power consumption.
ˆ Constraints: Manufacturing limits, timing requirements, maximum area.

10
 2. Optimal Design of a Steel Bridge

ˆ Decision Variables: Dimensions of the bridge components, such as the cross-sectional

area of the steel members, and material properties, such as the type and grade of steel
used.
ˆ Objective Function: Minimize the total cost of the steel bridge, including material
costs, construction costs, and maintenance costs over its lifetime.
ˆ Constraints: Structural Constraints that ensure that the bridge is structurally can
withstand the loads it will be subjected to. Economic Constraints, which limit the total
cost of the bridge to within a specified budget.

1.5.2 The general form of optimization problems:

All optimization problems can be presented by some standard form. Each and every optimization
problem contains objective function(s) f (⃗x) which we need to optimize. The general form of
optimization problem is:

min f (⃗x) (1.1)

⃗
x
subject to gi (⃗x) ≤ 0, i = 1, . . . , m (1.2)
hj (⃗x) = 0, j = 1, . . . , p (1.3)

where

ˆ ⃗x = [x1 x2 · · · xn ]T is the vector of decision variables.

ˆ f (⃗x) : Rn → R is the objective function that we need to optimize.

ˆ gi (⃗x) : Rn → R, i = 1, 2, · · ·, m are inequality constraints.

ˆ hj (⃗x) : Rn → R, j = 1, 2, · · ·, p are equality constraints.

Goal: Find the optimal value of ⃗x that minimizes f (⃗x) while satisfying all the constraints.

1.5.3 Convex optimization

The convex problem has three additional requirements:

1. the objective function must be convex,

2. the inequality constraint functions must be convex,

3. the equality constraint functions hj (⃗x) = aTj ⃗x − bj must be affine.

Note: An affine function is a type of mathematical function that combines a linear transfor-
mation with a translation. It is a function of the form: f (x) = A⃗x + b.

11
1.6 Unconstrained and constrained optimization
If we need to optimize the objective function without any additional constraints, then it is called
unconstrained optimization. For example, find the minimum of the function f (x) = x2 − 2x − 3.

min f (x) = x2 − 2x − 3 (1.4)

x

Whenever, the objective function is accomplished with another correlation (constrained func-
tion), these optimization problems are called constrained optimization problems. For example,
find the minimum of the function f (x) = x2 − 2x − 3 where x ≥ 2.

min f (x) = x2 − 2x − 3 (1.5)

x
s.t x ≥ 2 (1.6)

For the unconstrained problem, the solution is fmin = −4 whereas for the constrained problem
the solution is fmin = −3.

Example 1.2 For the given simple objective function f (x) = x2 −2x−3 where x is a real number.

1. Check the convexity of the function.

Solution:

f (x) =x2 − 2x − 3 (1.7)

f ′ (x) =2x − 2 (1.8)
f ′′ (x) =2 (1.9)

f ′′ (x) > x for all x, so it is always convex.

2. If the function is convex, find the value of x at which we get the minimum value of the
function.
Solution:
f ′ (x) = 2x − 2 = 0 ⇒ x∗ = 1

3. If the function is convex, find its minimum value.

Solution:
f (1) = (1)2 − 2(1) − 3 = −4

1.7 Gradient Descent Algorithm

1.7.1 What is the gradient of a function?
ˆ Definition: The gradient of a scalar-valued function f (⃗x) is a vector containing the partial
derivatives of f with respect to each variable.

ˆ Mathematical Representation: The gradient is given by:

 
∂f ∂f ∂f
∇f (⃗x) = , ,··· ,
∂x1 ∂x2 ∂xn

12
 ˆ Geometric Interpretation: The gradient points in the direction of the steepest ascent of
the function at a given point.

ˆ Magnitude Significance: The magnitude of the gradient represents the rate of change of
the function in the steepest direction.

ˆ Key Insight: The gradient provides both the direction and the steepness of the slope at a
specific point in the input space.

1.7.2 What is Gradient Descent?

Gradient Descent is an iterative optimization algorithm used to minimize a function by iteratively
moving in the direction of steepest descent as determined by the negative of the gradient. It is
commonly used to find the minimum of a differentiable function, such as a loss function in machine
learning models.

1.7.3 What is the idea of Gradient Descent?

The main intuition behind Gradient Descent is that by iteratively moving in the direction of steep-
est descent, we can eventually converge to a local minimum (or saddle point) of the function. The
learning rate controls the size of the steps we take, ensuring that we don’t overshoot the minimum
or oscillate around it.

Here’s a brief overview of how Gradient Descent works:

1. Initialization: Choose a starting point x0 randomly or based on prior knowledge.

2. Iterative Update:

ˆ At each iteration t, compute the gradient of the objective function f (x) at the current
point xt . The gradient represents the direction of steepest ascent.
ˆ Update the current point xt by taking a small step in the opposite direction of the
gradient, scaled by a learning rate α. This step is crucial for converging towards the
minimum:

xt+1 = xt − α∇f (xt ) (1.10)

ˆ Repeat this process until a stopping criterion is met, such as reaching a maximum
number of iterations, achieving a sufficiently small gradient magnitude, or converging
to a predefined minimum threshold.

3. Convergence Criteria:
Common convergence criteria include reaching a maximum number of iterations, achieving
a sufficiently small gradient magnitude, or reaching a minimum threshold for the change in
objective function value between iterations.

4. Learning Rate:
The learning rate α controls the step size in each iteration. It is a hyperparameter that
needs to be carefully chosen:

ˆ If α is too small, the algorithm may converge slowly.

ˆ If α is too large, the algorithm may oscillate or diverge.

13
Example 1.3 Let’s consider the function f (x) = x2 , and we want to find the minimum value of
this function using gradient descent.

Solution:

1. Objective Function:

f (x) = x2 (1.11)

2. Gradient:
The gradient of f (x) with respect to x is:

∇x f (x) = 2x (1.12)

3. Gradient Descent Update Rule:

We’ll use the update rule of gradient descent:

xt+1 = xt − α∇f (xt ) (1.13)

4. Initialization:
We choose an initial value for x, let’s say x0 = 3.

5. Iterative Updates:
We’ll perform iterative updates using the gradient descent update rule until convergence or
a predefined number of iterations:

xt+1 = xt − α2xt (1.14)

6. Convergence Criterion:
We’ll stop the iterations when either the absolute difference between consecutive values of
x becomes smaller than a predefined threshold or when we reach a maximum number of
iterations.
Let’s assume we choose a learning rate α = 0.1 and set a maximum of 10 iterations.
Now, let’s perform the iterative updates:

ˆ Iteration 1: x1 = x0 − α(2x0 ) = 3 − 0.1 × (2 × 3) = 2.4

ˆ Iteration 2: x2 = x1 − α(2x1 ) = 2.4 − 0.1 × (2 × 2.4) = 1.92
ˆ Iteration 3: x3 = x2 − α(2x2 ) = 1.92 − 0.1 × (2 × 1.92) = 1.536
ˆ Iteration 4: x4 = x3 − α(2x3 ) = 1.536 − 0.1 × (2 × 1.536) = 1.2288
ˆ Iteration 5: x5 = x4 − α(2x4 ) = 1.2288 − 0.1 × (2 × 1.2288) = 0.98304
ˆ Iteration 6: x6 = x5 − α(2x5 ) = 0.98304 − 0.1 × (2 × 0.98304) = 0.786432
ˆ Iteration 7: x7 = x6 − α(2x6 ) = 0.786432 − 0.1 × (2 × 0.786432) = 0.6291456
ˆ Iteration 8: x8 = x7 − α(2x7 ) = 0.6291456 − 0.1 × (2 × 0.6291456) = 0.50331648
ˆ Iteration 9: x9 = x8 − α(2x8 ) = 0.50331648 − 0.1 × (2 × 0.50331648) = 0.402653184
ˆ Iteration 10: x1 0 = x9 −α(2x9 ) = 0.402653184−0.1×(2×0.402653184) = 0.3221225472

14
1.8 Coding
”fmincon” is a MATLAB function for constrained optimization. Below are examples for codes
demonstrating how to use ”fmincon” to solve an optimization problem with constraints:

Example 1.4 A network operator allocates bandwidth to a premium user. The objective is to
minimize the allocated bandwidth while satisfying multiple linear constraints related to quality of
service (QoS) and system capacity.

Given Parameters:

ˆ Spectral efficiency: η = 4 bits/s/Hz

ˆ Minimum required data rate: Rmin = 20 Mbps

ˆ Maximum available bandwidth: Bmax = 10 MHz

ˆ Minimum guaranteed bandwidth: Bmin = 3 MHz

Decision Variable:

ˆ x (allocated bandwidth in MHz)

Rate Relationship:

R = ηx

Thus, to satisfy the QoS requirement:

4x ≥ 20

1. Formulate the problem in the standard form.

min x (1.15)
x
s.t. 4x ≥ 20 (1.16)
x ≤ 10 (1.17)
x≥3 (1.18)

Rewrite the constraints in standard inequality form Ax ≤ b:

min x (1.19)
x
s.t. − 4x + 20 ≤ 0 (1.20)
x − 10 ≤ 0 (1.21)
−x+3≤0 (1.22)

Note: The objective function is linear, and all constraints are linear. Therefore, this is a
convex optimization problem.

15
 2. Solve the problem using “fmincon”.

% Objective function
objective = @(x) x;

% Initial guess
x0 = 1;

% Linear inequality constraints A*x <= b

A = [-4; 1; -1];
b = [-20; 10; -3];

% No equality constraints
Aeq = [];
beq = [];

% No additional bounds
lb = [];
ub = [];

% Solve
[x_opt, fval, exitflag] = fmincon(objective, x0, A, b, Aeq, beq, lb, ub);

% Display results
fprintf(’Optimal bandwidth: %.4f MHz\n’, x_opt);
fprintf(’Minimum objective value: %.4f\n’, fval);

The results:

Optimal bandwidth: 5.0000 MHz

Minimum objective value: 5.0000

16
Example 1.5 For the given simple objective function f (x, y) = (x − 2)2 + (y − 3)2 subject to
2x + 3y ≤ 0 and x − 2y ≤ −3, where x and y are real numbers.

1. Formulate the following problem in the standard form.

min (x − 2)2 + (y − 3)2 (1.23)

x,y

s.t. 2x + 3y ≤ 0 (1.24)
x − 2y + 3 ≤ 0 (1.25)

2. Solve the problem using ”fmincon”.

% Define the objective function

fun = @(x) (x(1) - 2)^2 + (x(2) - 3)^2;

% Initial guess
x0 = [0, 0];

% Define the linear inequality constraints: A*x <= b

A = [2 3; 1 -2];
b = [0 -3];
% Define the linear equality constraints: Aeq*x = beq
Aeq = [];
beq = [];
% Define the lower and upper bounds: lb <= x <= ub
lb = [ ];
ub = [ ];

% Solve the optimization problem

[x, fval, exitflag] = fmincon(fun, x0, A, b, Aeq, beq, lb, ub);

% Display the results

fprintf(’Optimal solution: x = (%.4f, %.4f), f(x) = %.4f\n’, x(1), x(2), fval);

The results:

Optimal solution: x = (-1.2857, 0.8571), f(x) = 15.3878

17
Example 1.6 For the given simple objective function f (x, y) = (x)2 +(y)2 −3 subject to x+y ≥ 2,
where x and y are real numbers.

1. Formulate the following problem in the standard form.

min x2 + y 2 − 3 (1.26)
x,y

s.t. −x−y+2≤0 (1.27)

2. Solve the problem using ”fmincon”.

% Define the objective function

fun = @(x) (x(1))^2 + (x(2))^2 - 3;

% Initial guess
x0 = [0, 0];

% Define the linear inequality constraints: A*x <= b

A = [-1 -1];
b = [-2];
% Define the linear equality constraints: Aeq*x = beq
Aeq = [];
beq = [];
% Define the lower and upper bounds: lb <= x <= ub
lb = [ ];
ub = [ ];

% Solve the optimization problem

[x, fval, exitflag] = fmincon(fun, x0, A, b, Aeq, beq, lb, ub);

% Display the results

fprintf(’Optimal solution: x = (%.4f, %.4f), f(x) = %.4f\n’, x(1), x(2), fval);

The results:

Optimal solution: x = (1.0000, 1.0000), f(x) = -1.0000

18
Example 1.7 For the given simple objective function f (⃗x) = x21 + x22 + x23 subject to x1 + x2 = 7
and x1 − x3 = 2, where ⃗x is a real vector.

1. Formulate the following problem in the standard form. Solution:

min x21 + x22 + x23 (1.28)

x1 ,x2 ,x3

s.t. x1 + x2 − 7 = 0 (1.29)
x1 − x3 − 2 = 0 (1.30)

2. Solve the problem using ”fmincon”. Solution:

% Define the objective function

fun = @(x) (x(1))^2 + (x(2))^2 + (x(3))^2;

% Initial guess
x0 = [4, 5, -2];

% Define the linear inequality constraints: A*x <= b

A = [];
b = [];
% Define the linear equality constraints: Aeq*x = beq
Aeq = [1 1 0; 1 0 -1];
beq = [7;2];
% Define the lower and upper bounds: lb <= x <= ub
lb = [ ];
ub = [ ];

% Solve the optimization problem

[x, fval, exitflag] = fmincon(fun, x0, A, b, Aeq, beq, lb, ub);

% Display the results

fprintf(’Optimal soln: x=(%.4f,%.4f,%.4f), f(x)=%.4f\n’,x(1),x(2),x(3),fval);

The results:

Optimal solution: x = (3.0000, 4.0000, 1.0000), f(x) = 26.0000

19
Example 1.8 Consider a firm that makes two kinds of chocolate: Type A and Type B. Only Milk
and Choco are required for both types. One unit of Milk and three units of Choco are required to
make one piece of Type A. To make a piece of Type B, however, we need one unit of Milk and
two units of Choco. The firm only has 5 Milk units and 12 Choco units. The factory’s profit is as
follows. Each piece of Type A is sold by $6, while each piece of Type B is sold by $5.

1. Formulate the following problem in the standard form.

ˆ Let the total number of produced pieces of Type A be x

ˆ Let the total number of produced pieces of Type B be y

ˆ Let the total profit be z = 6x + 5y

ˆ The firm tries to produce as many pieces of Type A and Type B to maximize its profit, but
the resources (i.e., Milk and Choco) are limited.

ˆ Each piece of Type A and Type B needs 1 unit of Milk. The total amount of Milk available
is 5 units. Thus, this constraint can be represented as x + y ≤ 5.

ˆ Each piece of Type A needs 3 units of Choco while each piece of Type B needs 2 units
of Choco. The total amount of Choco available is 12 units. Thus, this constraint can be
represented mathematically as 3x + 2y ≤ 12.

ˆ We have two more constraints, which are x ≥ 0 and y ≥ 0

min −(6x + 5y)
x,y
s.t. x+y−5≤0
3x + 2y − 12 ≤ 0 (1.31)
−x ≤ 0
−y ≤ 0

2. The factory’s goal is to increase earnings. How many units of Type A and
Type B should be produced in order to maximize profits?

% Define the objective function

fun = @(x) (-6*x(1)) - (5*x(2));

% Initial guess
x0 = [0, 0];

% Define the linear inequality constraints: A*x <= b

A = [1 1; 3 2; -1 0; 0 -1];
b = [5 12 0 0];

% Define the linear equality constraints: Aeq*x = beq

Aeq = [];
beq = [];

% Define the lower and upper bounds: lb <= x <= ub

lb = [ ];
ub = [ ];

20
% Solve the optimization problem
[x, fval, exitflag] = fmincon(fun, x0, A, b, Aeq, beq, lb, ub);

% Display the results

fprintf(’Optimal solution: x = (%.4f, %.4f), f(x) = %.4f\n’, x(1), x(2), -fval);

The results:

Optimal solution: x = (2.0000, 3.0000), f(x) = 27.0000

21
Example 1.9 A wireless transmitter communicates with a receiver over a single channel. The
achievable data rate follows Shannon’s formula. The objective is to determine the optimal trans-
mission power that maximizes the data rate subject to a maximum power constraint.

Given Parameters:
ˆ Channel coefficient: h = 2

ˆ Noise power: σ 2 = 1

ˆ Maximum transmission power: Pmax = 5 Watts

ˆ Bandwidth: B = 1 MHz

Decision Variable:
ˆ p (transmission power in Watts)
General Shannon Capacity Formula:
ˆ C = B log2 (1 + SNR)

ˆ If the baseband model is y = hx + n, then

 2

C = B log2 1 + |h|σ2P

where h is the channel coefficient, x is the transmitted signal, n is the additive noise, and
σ 2 represents the noise power.
1. Formulate the following problem in the standard form.

|2|2 p
 
6
max 1 × 10 × log2 1+ (1.32)
p 1
s.t. 0≤p≤5 (1.33)
(1.34)

|2|2 p
 
6
max 1 × 10 × log2 1+ (1.35)
p 1
s.t. p≤5 (1.36)
p≥0 (1.37)
(1.38)

|2|2 p
 
6
min − 1 × 10 × log2 1 + (1.39)
p 1
s.t. p − 5 ≤ 0 (1.40)
−p≥0 (1.41)
(1.42)
Note: The objective function log2 (1 + 2p) is convex in p, and the constraints are convex.
Therefore, this is a convex optimization problem.

22
 2. Solve the problem using ”fmincon”.

% Parameters
h = 2;
sigma2 = 1;
Pmax = 5;

% Objective function (negative because fmincon minimizes)

objective = @(p) -1*10^6*log2(1 + (abs(h)^2*p)/sigma2);

% Initial point
p0 = 1;

% Define the linear inequality constraints: A*x <= b

A = [1; -1];
b = [5 0];

% Define the linear equality constraints: Aeq*x = beq

Aeq = [];
beq = [];

% Define the lower and upper bounds: lb <= x <= ub

lb = [ ];
ub = [ ];

% Solve using fmincon

[p_opt, fval, exitflag] = fmincon(objective, p0, A, b, Aeq, beq, lb, ub);

% Display results
fprintf(’Optimal power: %.4f\n’, p_opt);
fprintf(’Maximum rate: %.4f bits/s\n’, -fval);

The results:

Optimal power: 5.0000

Maximum rate: 4392317.4221 bits/s

23
Example 1.10 A wireless transmitter communicates with a legitimate receiver in the presence of
an eavesdropper.
The transmitter wants to minimize its transmission power while guaranteeing a minimum se-
crecy rate. The secrecy rate is defined as the difference between the legitimate channel capacity
and the eavesdropper channel capacity.

Given Parameters:

ˆ Legitimate channel coefficient: hb = 2

ˆ Eavesdropper channel coefficient: he = 0.5

ˆ Noise power: σ 2 = 1

ˆ Minimum required secrecy rate: Rmin = 1 bits/s/Hz

ˆ Bandwidth: B = 1 Hz

Decision Variable:

ˆ p (transmission power in Watts)

Secrecy Rate Formula:

|hb |2 p |he |2 p
   
Rs (p) = log2 1+ − log2 1+
σ2 σ2

1. Formulate the following problem in the standard form.

min p (1.43)
p

|2|2 p |0.5|2 p
   
s.t. log2 1 + − log2 1 + ≥1 (1.44)
1 1
p≥0 (1.45)

Rewriting the constraint in standard form:

min p (1.46)
p

s.t. 1 − log2 (1 + 4p) + log2 (1 + 0.25 ∗ p) ≤ 0 (1.47)

−p≤0 (1.48)

Note: This is a convex optimization problem.

24
 2. Solve the problem using “fmincon”. Solution:

%Parameters
hb = 2;
he = 0.5;
sigma2 = 1;
Rmin = 1;

%Define the objective function

fun = @(p) p;

%Initial guess
p0 = [1];

%Nonlinear inequality constraint

nonlcon = @(p) deal(Rmin - (log2(1 + (abs(hb)^2*p)/sigma2)-log2(1 + (abs(he)^2*p)/sigma2

%Define the linear inequality constraints: A*x <= b

A = [-1];
b = [0];

%Solve using fmincon

[p_opt, fval] = fmincon(fun, p0, A, b, [], [], [], [], nonlcon);

%Display results
fprintf(’Optimal power: %.4f\n’, p_opt);

The output of the code:

Optimal power: 0.2857

25