REPUBLIQUE ALGERIENNE DEMOCRATIQUE ET POPULAIRE

MINISTERE DE L’ENSEIGNEMENT SUPERIEUR ET DE LA RECHERCHE SCIENTIFIQUE

UNIVERSITÉ MOHAMED BOUDIAF - M’SILA

FACULTÉ : TECHNOLOGIE DOMAINE: Sciences et Technologies

DEPARTEMENT : ELECTRONIQUE FILIERE : Électronique
N° : OPTION : Electronique des Systèmes
Embarqués

Mémoire présenté pour l’obtention du diplôme de Master Académique

Par : Bouguerra Oussama

Benslimane Oussama

Intitulé

SOLAR RADIATION PREDICTION

USING MACHINE LEARNING

Le jury composé de :

Mr. ATTALLAH Bilal Université de M'sila Président

Mr. DJERIOUI Mohamed Université de M'sila Encadreur
Mr. BRIK Youcef Université de M'sila Co-Encadreur
Mr. BOURAS Mounir Université de M'sila Examinateur

Année universitaire : 2019 /2020

 THANKS
First of all, we thank our coach Dr. DJERIOUI Mohamed

who agreed to mentor us to make this thesis. We are

grateful to him. Throughout this work, we have benefited

from his great competence, his intellectual rigor, his

efficiency, and his invaluable advice.

We would also like to thank Dr. BRIK Youcef for

reserving some of his time to follow our work. We would

also like to thank the jury members who honoured us

with the honour of judging this thesis and proclaiming its

scientific value. We would also like to thank all those who

contributed directly or indirectly to the realization of

this thesis.
 Dedication
I dedicate this modest work to my dear father for his

sacrifices as a testimony of all my affection throughout

my studies. To my dear mother who has not stopped

encouraging me since my first school year. To my

brother Omar Elfarouk ,Bilal. To my sisters Sanna And

all the family , To all my friends especially: Ahmed,

Mohammed,Salah,Houcine ,Abd Halim,Adel.

To all the teachers in the Department of Electronics.

Bouguerra Oussama
 Dedication
I dedicate this modest work to my dear father for his

sacrifices as a testimony of all my affection throughout

my studies. To my dear mother who has not stopped

encouraging me since my first school year.

Benslimane Oussama
 Table of Contents
LISTE OF FIGUERS ...............................................................................................................V
LIST OF TABLES...................................................................................................................VI
NOTATION AND ABBREVIATED TERMS …………………...….…………………....VII
INTRODUCTION..................................................................................................................... 1

CHAPTER I State of the Art

I. INTRODUCTION..................................................................................................................4
[Link] RADIATION ...........................................................................................................4
II.1. Definition ........................................................................................................................4
II.2. Type of Solar Radiation.................................................................................................4
[Link] Normal Irradiance (DNI) .............................................................................4
[Link] Horizontal Irradiance (DHI)......................................................................4
[Link] Radiation ..................................................................................................5
[Link] Horizontal Irradiance (GHI) ......................................................................5
II.3. Why Predict Solar Radiation ........................................................................................6
[Link] Energy Issues ...............................................................................................6
[Link] Radiation..............................................................................................7
[Link] Radiation In The Earth’s Atmosphere ...............................................................7
III. LOCATION OF THE SUN IN THE SKY........................................................................8
[Link] Time System ...........................................................................................................8
[Link] Solar Declination (𝜹).......................................................................................9
[Link] Time Angle (𝝎 ) .............................................................................................10
III.2. System of Horizontal Coordinates ............................................................................11
[Link] (𝑨𝒛) .........................................................................................................11
[Link] Height......................................................................................................12
IV. SOLAR PHOTOVOLTAIC ENERGY...........................................................................12
[Link] Effect.......................................................................................................12
[Link] System.....................................................................................................12
[Link] Cell....................................................................................................13
[Link] Battery......................................................................................................18
 [Link] ..............................................................................................................18
[Link].................................................................................................................18
[Link](Users)........................................................................................................18
V. Conclusion ...........................................................................................................................18

CHAPTER II Machine learning and deep learning techniques

[Link].................................................................................................................20
[Link] LEARNING .....................................................................................................20
[Link] Principle ...............................................................................................................21
[Link] of Learning in Machine Learning..............................................................21
II.1.2. Regression And Classification..............................................................................22
III. ALGORITHMS OF MACHINE LEARNING...............................................................23
III .1. Random Forest ..........................................................................................................23
III .1.1. Definition Of The Model ....................................................................................23
III .1.2. How It’s Work.....................................................................................................24
III .[Link] The Random Forest .............................................................................26
III .2. Gradient Boosting Machine......................................................................................28
III .2.1. Definition of The Model .....................................................................................28
III .2.2. The Cart Decision Tree ......................................................................................29
III .2.3. Gradient Boosting Machine Algorithm ............................................................30
III .2.4. Hyperparameter Tuning Gradient Boosting Machine....................................31
III .[Link] Learning versus Deep Learning ................................................................33
[Link] LEARNING.............................................................................................................34
IV.2. History of Deep Learning...........................................................................................35
IV.3. Why Deep Learning?..................................................................................................36
IV.4. Deep Learning Application Areas.............................................................................37
V- ALGORITHMS OF DEEP LEARNING .........................................................................37
[Link] Neural Networks ..................................................................................................37
[Link] ...............................................................................................................39
[Link]-layer Perceptron ..........................................................................................39
 [Link] Functions ..............................................................................................40
[Link] functions...........................................................................................42
[Link] Regularization .........................................................................................44
[Link] Neural Networks .........................................................................................44
[Link] Short-Term Memory (LSTM)............................................................................47
[Link] ............................................................................................................47
[Link] Principle ......................................................................................................48
[Link] Algorithm ....................................................................................................51
[Link] of LSTM .............................................................................................52
V.4. Bidirectional Long Short-Term Memory (BLSTM) ................................................52
[Link]...................................................................................................................54

CHAPTER III Results and Discussion

[Link].................................................................................................................55
II. Proposed System .................................................................................................................55
III. DESCRIPTION OF THE DATASET .............................................................................56
II .[Link]..............................................................................................................57
[Link] Matrix ......................................................................................................57
[Link] Charts....................................................................................................................58
IV. Preprocessing Dataset .......................................................................................................58
IV.1. Steps involved in data preprocessing ........................................................................58
IV.2. Data Standardization.....................................................................................................59
[Link] EVALUTION...............................................................................................59
[Link] ENVIRONMENT..........................................................................59
[Link] Colab ...........................................................................................................59
[Link] ......................................................................................................................60
[Link] Notebook ...................................................................................................60
V.1.4. Visual Studio Code ................................................................................................61
[Link] Presentation ..................................................................................................61
 [Link] Software .............................................................................................................61
V.2. EVALUATION CRITERIA........................................................................................63
[Link] Of Pre section Error.............................................................................63
V.3. SIMULATION RESULTS ..........................................................................................65
[Link] Forest........................................................................................................65
[Link] Boosting Machine (GBM)......................................................................67
[Link] LSTM (BI-LSTM) ...........................................................................69
[Link] Neural Network (DNN) ................................................................................71
[Link] Short Term Memory (LSTM) .....................................................................73
[Link] OF RESULTS.............................................................................75
[Link]...................................................................................................................76
 LISTE OF FIGURES

CHAPTER I State of the Art

Fig.I. [Link] of Solar Radiation. ...............................................................................................5

Fig.I. [Link] distribution of solar radiation ...........................................................................8
Fig.I. [Link]’s motion of revolution around the Sun .................................................................9
Fig.I. [Link] Coordinate System ..............................................................................................11
Fig.I. [Link] Coordinate System......................................................................................12
Fig.I. [Link] diagram of an autonomous photovoltaic system. ..........................................13
Fig.I. [Link] of a photovoltaic cell.....................................................................................13
Fig.I. [Link]-voltage characteristic of a photovoltaic solar cell. ..........................................14
Fig.I. [Link] of light on current-voltage characteristics of a photovoltaic cell. ................15
Fig.I. [Link] of temperature on current-voltage characteristics of a photovoltaic cell....15
Fig.I. [Link] of temperature on the power-voltage characteristic of a cell photovoltaic.16

CHAPTER II Machine learning and deep learning techniques

[Link]. [Link] of Observation....................................................................................23

[Link]. [Link] diagram of the random forest method ........................................................25
[Link]. [Link] error depending on the number of trees in the random forest model. ...............27
[Link]. [Link] error depending on mtry value...........................................................................28
[Link]. [Link] of the CART tree impurity reduction concept. .........................................30
[Link]. [Link] learning and Deep learning ..........................................................................34
[Link]. [Link] relationship between artificial intelligence, ML, and deep learning....................34
[Link]. 8. The performance difference between Deep Learning and most ML algorithms
depends on the amount of data. .................................................................................................37
[Link]. 9. Deep Neural Network architecture. ...........................................................................38
[Link]. [Link] Perceptron. .........................................................................................................39
[Link]. [Link] Perceptron (MLP) structure.....................................................................40
[Link]. [Link] Function Graph. ..........................................................................................41
[Link]. [Link] Function Graph. ...............................................................................................42
[Link]. [Link] Graph...............................................................................................................42
[Link]. 15. Recurrent neural networks have loops. ....................................................................45
[Link]. 16. A recurring neural network unfolded.......................................................................45
[Link]. [Link] Problem of Long-Term Dependencies-1. ..........................................................46
[Link]. [Link] Problem of Long-Term Dependencies-2. ..........................................................47
[Link]. 19. Diagram represents an LSTM Input Gate. ...............................................................49
[Link]. [Link] represents a LSTM Forget Gate. ................................................................49
[Link]. 21. Diagram represents an LSTM Forget Gate. .............................................................50
[Link]. 22. Diagram represents an LSTM Output Gate. ............................................................50
[Link]. [Link] represents a chain of three LSTM cells......................................................51
[Link]. [Link] BLSTM RNN structure with three consecutive time steps.......................53

CHAPTER III Results and Discussion

Fig. III. [Link] matrix to identify the essential characteristics of the whole. ................55
Fig. III. [Link] matrix to identify the essential characteristics of the whole. ................57
Fig. III. 3. Bar charts to better understand the data. .................................................................58
Fig. III. [Link] Colab Environment. .....................................................................................60
Fig. III. [Link] Python Logo ......................................................................................................60
Fig. III. [Link] Jupyter Notebook logo. .....................................................................................61
Fig. III. [Link] Studio Code Software Interface. ...................................................................61
Fig. III. [Link] and calculated outputs for the random forest test. ........................................66
Fig. III. [Link] and calculated outputs for learning with random forest. ..............................67
Fig. III. [Link] and calculated outputs for learning and testing with random forest. ..........67
Fig. III. [Link] and calculated outputs for the test with Gradient boosting regression. ......68
Fig. III. [Link] and calculated outputs for learning with Gradient boosting regression......69
Fig. III. [Link] and calculated outputs for learning and testing with GBR. ........................69
Fig. III. [Link] and calculated outputs for the test with BI-LSTM. ....................................70
Fig. III. [Link] and calculated outputs for learning with BI-LSTM....................................71
Fig. III. [Link] and calculated outputs for learning and testing with BI-LSTM. ................71
Fig. III. [Link] and calculated outputs for the test with Deep Neural Network. .................72
Fig. III. [Link] and calculated outputs for learning with Deep Neural Network. ...............73
Fig. III. [Link] and calculated outputs for learning and testing with DNN.........................73
Fig. III. [Link] and calculated outputs for the LSTM test. ..................................................74
Fig. III. [Link] and calculated outputs for learning with LSTM. ........................................75
Fig. III. [Link] and calculated outputs for learning and testing with LSTM.......................75
 LISTE OF TABLES
CHAPTER II Machine learning and deep learning techniques
Table. II. [Link] Gradient Boosting Machine.................................................................31
Table. II. [Link] of Deep Learning......................................................................................36
Table. II. [Link] Algorithm...................................................................................................51

CHAPTER III Results and Discussion

Table. III. [Link] units of each dataset. .....................................................................................57
Table. III. [Link] characteristics. .......................................................................................63
Table. III. [Link] hyperparameters of the random forest technical dataset. ..............................65
Table. III. 4. Results were obtained from the random forest technical basis. ..........................66
Table. III. [Link] hyperparameters of the technical basis Gradient boosting regression. .........67
Table. III. 6. Results were obtained from the technical basis of the GBR. ..............................68
Table. III. [Link] of the BI-LSTM technical dataset............................................69
Table. III. [Link] were obtained from the Bi-LSTM technical basis...................................70
Table. III. [Link] hyperparameters of the Deep Neural Network technical dataset. .................71
Table. III. 10. Results were obtained from the Deep Neural Network technical basis. ...........72
Table. III. [Link] hyperparameters of the Long Short Term Memory technical dataset.........73
Table. III. [Link] obtained from the Long Short Term Memory technical basis...............74
Table. III. [Link] evaluation for different techniques. ..............................................75
 NOTATION AND ABBREVIATED TERMS
RE : Renewable Energy
DNI : Direct Normal Irradiance
DHI : Diffuse Horizontal Irradiance
GHI : Global Horizontal Irradiance

𝑮𝒅 : Diffuse Horizontal Irradiance

𝑮𝒃 : beam irradiance

𝜽𝒛 : zenith angle

Φ : latitude of the place

𝜹 : declination

𝑮𝒔𝒄 : solar constant

r: distance

𝝎 : Time Angle
TSV : dedicated solar time

𝑨𝒛 : Azimuth

𝑰𝒄𝒄 : short circuit current

𝑽𝒄𝒐 : circuit voltage

DC : direct current
AC : alternating current
ML : Machine Learning
DL : Deep learning
AI : artificial intelligence
CART : Classification And Regression Trees

𝑩𝒐𝒑𝒕𝒊 : the number of samples bootstrap

𝒎𝒕𝒓𝒚𝒐𝒑𝒕𝒊 : the number of variables selected at each separation

RF : Random forest
GBM : Gradient Boosting Machine

𝑭𝐚𝐩𝐩𝐫𝐨𝐱(𝒙) : find an approximation

ANNs : Artificial Neural Networks

NNs : Neural Networks
MLP : Multilayer Perceptron
FNN : Feedforward Neural Networks
SGD : Stochastic Gradient Descent
DNN : Deep neural network
TNNs : Traditional neural networks
RNNs : Recurrent neural networks
LSTM : Long Short-Term Memory
MTSA : Multiple Travelling Salesman Assignment
NRNS : Nested RNS
𝑋𝑡 : current input
BILSTM : Bidirectional Long Short-Term Memory
BRNNs : Bidirectional Recurrent neural networks

𝒙𝒊𝒏 : The normalized value.

MAE : Mean Absolute Error

MSE : Mean Square Error
RMSE : Root Mean Square Error
RRMSE : Relative Root Mean Square Error
R : Correlation Coefficient
R ² : Coefficient Of Determination
GPU: graphics processing unit
API : Application Programming Interface
CPU : Central Processing Unit
RAM : Random Access Memory
INTRODUCTION
INTRODUCTION

Renewable Energy is becoming a technology and an ever more viable alternative to

traditional non-renewable energy sources. Therefore, the relentless danger of climate
change forces humanity to strive for new and more effective ways to produce energy.
Particularly when we base all our basic needs on this form of energy.

According to the International Energy Agency (IEA) [1], global renewable electricity
capacity is projected to rise by over 1 TW, a 46 percent increase over the period 2018 to
2023.

Solar photovoltaic (PV) represents more than half of this expansion and dominates the
growth of renewable ability.

However, because the energy output of PV panels depends on weather conditions such
as cloud cover and solar irradiance, the PV panels' energy output is unstable. To
understand and manage the output variability is of interest for several actors in the energy
market.

Nowadays, the use of Artificial Intelligence Technologies is becoming increasingly

widespread due to their ability to solve highly complex problems. The increase in the
efficiency of computers and algorithms has helped to solve problems, not only in
engineering but also in many fields such as medicine[2], finance[3], and the
environment[4]. That is why these instruments are becoming more popular and can still
make some progress and can be implemented in more areas or issues.

Machine learning is a computer science subfield, and it is categorized as a form of

artificial intelligence. It can be used in many domains and the advantage of this approach is
that a model can solve problems that clear algorithms can not represent.

In the case of Renewable Energy Source (RES), these machine learning models are
used to forecast the energy produced in a power station or even to forecast the actions of
weather conditions. In the energy industry, such predictions are of great significance. The
problem of Solar Irradiance variability and unpredictability which reaches the surface of
the Earth is well known. A precise forecast of this variable will therefore facilitate better
planning and operation of power delivery at the economic level or at the level of energy

1
output, either by making alternative arrangements for traditional power and overall
timetables or by investing the correct amount of energy resources and reserves to minimize
the operating costs of the power system.

Additionally, related literature on the prediction of solar surface irradiation, estimation

of PV capacity and current state-of-the-art studies is reviewed [5]. Machine learning
algorithms such as support vector machine (SVM) [6,7], k-nearest neighbor [8,9], and
Random Forest [10] have been used in studies; others compared or combined the
prediction results with multiple machine learning models.

Deep learning is a particular subfield of machine learning designed to allow machines

to simulate the way the human brain thinks, and its operational model is based on
neuroscience [11]. Deep learning is designed for the representation of input and target data
using a neural network structure. Actually, several types of research have addressed in
solar irradiance prediction based on deep learning models. In [12], the authors have
proposed a deep learning hybrid model to predict solar radiation in 2 stages, a
convolutional network is used to extract features, and a Long-Short-Term Memory
(LSTM) network is used for the prediction stage. [13] proposed a recurrent neural network
model to investigate how emerging deep learning algorithms, especially compared to
swallowing artificial neural networks, contribute to accurate solar radiation prediction.
Guijo-Rubio et al. [14] have presented how evolutionary artificial neural network
algorithms can obtain extremely accurate solar radiation prediction, exclusively working
with satellite-based data. To predict solar irradiation from weather measurements, an
advanced machine learning-based model was used in [15]. This model is based on LSTM
networks which, in time series data, have an extended capacity to explain time
dependencies. A deep learning approach based on traditional neural networks has been
used to predict solar irradiance at thirty stations in Turkey [16].

The purpose of this memory is to research the viability of machine learning algorithms
to forecast day-ahead solar irradiance of the next hour and hourly. The studied machine
learning algorithms are the networks Random Forest, Gradient Boosting Machine, Deep
Neural Network ,LSTM, and Bidirectional LSTM. The research uses historical weather
data from the HI-SEAS weather station (Dataset).

⠀The rest of the memory is organized as follows:

2
 Chapter I introduces a few of the features and behaviors of this RES. In this chapter,
we will see the value of this form of technology increasing and understand why we should
invest in it.

Chapter II offers an overview of how deep neural networked machine learning works.
It begins with a general machine learning summary and then continues with a more
thorough take on the core components of machine learning. The structure of a neural
network and the algorithm for gradient descent optimization are discussed in some detail.

Chapter III focuses on the empirical results, analysis, and discussion of the results are
presented.

Finally, the last section describes the key conclusions taken from the analysis along
with implications for further field studies.

3
 CHAPTER I

State of the Art

Chapter I State of the Art

I. INTRODUCTION

Renewable energies are inexhaustibly provided by the Sun, the wind, the heat of the
Earth, waterfalls, tides, or the growth of plants. These are the energies of the future. Today,
they are underexploited about their potential. For example, renewable energy accounts for
only 20% of global electricity consumption. Control of the random nature of renewable energy
sources such as solar radiation on the ground could allow the proper sizing of solar systems of
all kinds and allow power grid operators to integrate them better. Unfortunately, radiation data
are not widely available. With this in mind, we have endeavored during this study to
contribute to the research of modeling methodologies for the prediction of solar radiation. This
type of prediction is essential because it could, in the long term, make it possible to exploit
better the solar Renewable Energy (RE), whose intermittence heavily penalizes its use. To
make this prediction, it is necessary to have statistical and mathematical tools dedicated to this
type of analysis[17].

[Link] RADIATION

II.1. Definition

Solar radiation is the energy per unit vicinity obtained from the Sun in the shape of
electromagnetic radiation. The SI unit of photovoltaic irradiance is watt per rectangular meter
W/m2. The find out about and size of photovoltaic irradiance is fascinating for the prediction
of the power era of photovoltaic energy plants[18].

II.2. Type of Solar Radiation

[Link] Normal Irradiance (DNI)

Additionally acknowledged as beam irradiance is the photovoltaic radiation measured

at a floor of the earth perpendicular to the Sun. It solely measures the direct radiation from the
solar disk and excludes the diffuse radiation[18].

[Link] Horizontal Irradiance (DHI)

Is the radiation measured on a horizontal floor on Earth, coming from mild scattered
by way of the atmosphere. It measures radiation from all points in the sky except for radiation

4
Chapter I State of the Art

from the solar disk. In the absence of atmosphere, there has to be nearly no diffuse sky
radiation[18].

[Link] Radiation

Is the radiation mirrored by using non-atmospherical factors such as the ground.

However, photovoltaic panels tend to be tilted away from the mirrored radiation trajectory, so
it hardly ever has relevance in the whole radiation acquired by using their surface. An
exception is in stipulations the place the floor is surrounded with the aid of snow, which can
make more significant extensively the mirrored radiation received [18].

[Link] Horizontal Irradiance (GHI)

Is the total irradiance from the solar on a horizontal floor on Earth. It is the sum of
DHI, DNI (after accounting for the photovoltaic zenith perspective of the solar 𝜃𝑧 ) and
mirrored radiation. However, due to the fact reflected radiation is commonly insignificant in
contrast to direct and diffuse radiation for all realistic functions, international horizontal
radiation is stated to be the sum of direct and diffuse radiation solely :

𝑮 = 𝑮𝒅 + 𝑮𝒃 𝒄𝒐𝒔(𝜃𝑧 ) (I.1)

the location G denotes the Global Horizontal Irradiance,𝑮𝒅 the Diffuse Horizontal
Irradiance, 𝑮𝒃 Direct Normal Irradiance or beam irradiance and 𝜃𝑧 the zenith angle. The
referred to types of photograph voltaic radiation are established in Figure I.1

Fig.I. [Link] of Solar Radiation.

The energy, 𝐸𝑝ℎ , of each photon is directly related to the wavelength λ by the relationship

5
Chapter I State of the Art

ℎ𝐶
𝐸𝑝ℎ = (I.2)
λ

Where h is the Planck constant, h = 6,62.10−34 J.s [18].

II.3. Why Predict Solar Radiation

The installation of any solar energy system in a given site requires preliminary studies.
Indeed, sizing and simulation are essential to ensure optimal operation. To carry out such
tasks, reliable measurements over relatively long periods of certain meteorological variables,
and especially those of solar radiation, are essential. The lack of a long series of data or data
series of low quality (discontinuity and unreliability) can combine errors in the design, sizing,
and prediction of solar system performance, which hurts investment. Unfortunately,
measurements of solar radiation are generally inaccurate and rare worldwide; Especially in
Algeria, due to the high price of measuring devices. There are only a small number of solar
radiation stations, which is why there is a lack of solar radiation measurements in large areas
on the one hand. On the other hand, where these data exist, there are generally periods of
failure due to failures or low monitoring, since the majority of these stations belong to
establishments that do not benefit economically from these data.

However, other meteorological parameters such as temperature ambient, humidity,

wind speed are relatively easily measured in a larger number of weather stations with a
relatively low cost compared to that of radiation. On the other hand, the optimization of a solar
system or the simulation of its performances require at least, daily data, even, schedule of solar
irradiation. Therefore, it is desirable to develop relationships between available meteorological
data and solar irradiation; and to develop precise techniques (models) to predict solar
irradiation. These models are tools that allow the generation of long series of solar irradiation
data at different time steps; this could provide useful information to decision-makers in terms
of selection, design, and planning of new solar power plants [19].

[Link] Energy Issues

The world has developed, and energy needs are growing to support both economic
development and the requirements in terms of comfort and consumption of populations. At the
moment, we are coming to a critical moment in energy exploitation: we realize the fragility

6
Chapter I State of the Art

and inconsistency of our functioning. Indeed, the planet’s resources in fossil sediment are
depleting, and oil is rarefied, and, in addition to the economic consequences, it is clear that we
must either find alternatives to current energy sources or find an alternative to our mode of
civilization itself. Without energy, our daily life disappears. Also, the exploitation of fossil
fuels poses another problem: the impact on the environment is massive. If this has long been
ignored, the preservation of the environment becomes a global issue, again with significant
economic stakes. The environmental community has long been aware of this policy, and
recently public opinion. The ecological and industrial disasters plunge people into a grip of
awareness of the dangers generated by the impact of humanity on our planet. We shall see in
this section the evolution of mentalities and policies over the past thirty years in the face of
these difficulties and this ever-increasing demand for energy. We will then focus on the
implications of these various directives at the international and national levels [19].

[Link] Radiation

Extraterrestrial solar radiation covers an extensive range of wavelengths. It does not

depend on any meteorological parameters. Still, it depends on some parameters astronomical
and geographic such as the latitude of the place (φ), the solar declination (δ), and the hour
angle at sunset (ω0) [20].

On a horizontal surface, and for day n, extraterrestrial radiation 𝐺0 (MJ.𝑚−2 .𝐷𝑎𝑦 −1 ) is

obtained from the following equation:

24 × 60 2𝜋𝑛
𝐺0 = 𝐺𝑠𝑐 [1 + 0.034 cos ( 365 )] [𝜔0 . sin(𝜑) . sin(𝛿 ) + cos(𝜑) . cos(𝛿 ) . sin(𝜔0 )] (I.3)
𝜋

𝐺𝑠𝑐 : Is the solar constant equal to 0.0082 MJ.𝑚−2 .𝑚𝑛−1 .

n: is the number of the day from the first of January.

[Link] Radiation In The Earth’s Atmosphere

The radiation received on the Earth’s atmosphere occupies only a small portion of the
spectrum of solar electromagnetic waves. Wavelengths between 0.2 and 2.5 µm characterize
it; it includes the domain of the visible (light waves from 0.4 to 0.8 µm).

7
Chapter I State of the Art

Solar energy collectors, which correspond to solar cells, will therefore have to be
compatible with these wavelengths to be able to trap the photons and return them in the form
of electrons [20].

Fig.I. [Link] distribution of solar radiation

III. LOCATION OF THE SUN IN THE SKY

There are two major movements of the earth: the revolution of the world around them
The Sun and the earth’s rotation around its polar axis. These two movements are
Important in solar energy applications. Generally, it is more convenient to study the apparent
motion of the Sun in the celestial vault.

The position of the Sun in the celestial vault is detected by two systems of classical
coordinates: the time system and the horizontal system.

[Link] Time System

The Earth rotates around the Sun in an eccentric elliptical trajectory 𝐸0 = 0.0167, as
shown in Figure (I.3). The annual periodicity of this motion makes it possible to understand
the phenomenon of the seasons. The distance earth-sun, therefore, varies during the year. On
average, the earth-sun distance is used as the basis for the “astronomical unit,” 1 ua
corresponding to 150.106 km (𝑟0 ). It reaches its maximum at the summer solstice (Aphélie;
1,017 ua or 152,106 km) and its minimum at the winter solstice (Perihelia; 0,983 ua or

8
Chapter I State of the Art

147,106 km). It depends on the day of the year number j, which varies from 1 to 365 (or 366
for leap years). The ground distance (r) is given by the equation (I-4) proposed by and which
gives a good precision [19].

𝑟
𝐸0 = ( 𝑟0 ) = 1.00110 + 0.034221𝑐𝑜𝑠𝛽 + 0.001280𝑠𝑖𝑛𝛽 + 0.000719𝑐𝑜𝑠𝛽

(I-4)

+0.000077𝑠𝑖𝑛2𝛽

2π(j−1)
With: 𝛽 = in radians
365

It is, of course, possible to determine the apparent position, in hourly coordinates,

using two angular coordinates: the solar declination (𝛿) and the hourly angle w.

Fig.I. [Link]’s motion of revolution around the Sun

[Link] Solar Declination (𝜹)

The solar declination represents the angle between the Earth-Sun direction and the
plane of the equator at noon dedicated solar time. This is a magnitude that often intervenes in
the various calculations related to solar radiation. It varies sinusoidally during the year
between -23.45 (winter solstice) and 23.45 (summer solstice) and cancels out at equinoxes.
Several mathematical formulae have been proposed to calculate approximate values of (𝜹).
Equation (I.5) is the one we adopted for our calculations; it gives a (𝜹) with great precision :

𝛿 (𝑟𝑎𝑑 ) = 0.006918 − 0.399912𝑐𝑜𝑠𝛽 + 0.070257𝑠𝑖𝑛𝛽 − 0.006758𝑐𝑜𝑠2𝛽

9
Chapter I State of the Art

+0.000907𝑠𝑖𝑛2𝛽 + 0.002697𝑐𝑜𝑠3𝛽 + 0.00148𝑠𝑖𝑛3𝛽 (I.5)

2π(j−1)
Wit: 𝛽 = in radians
365

[Link] Time Angle (𝝎 )

𝝎 is the angle between the meridian and the hour circle that contains the Sun. It is
positively counted westward from the meridian. It measures the Sun’s course in the sky. The
hourly angle varies from 15° per hour canceling at noon, dedicated solar time (TSV), it is
calculated from the real solar time by the relationship:

𝝅
𝝎= (𝑻𝑺𝑽 − 𝟏𝟐 ) (I.6)
𝟏𝟐

TSV represents dedicated solar time in hours based on earth rotation around its polar axis
and its revolution around the Sun. The duration of the solar day varies during the year
because:

 Earth sweeps different surfaces in its rotation around the Sun.

 The earth axis is tilted about the ecliptic plane.

The real solar time differs from the legal time (TL) of the site considered for three main
reasons:

 The difference in longitude between the site under consideration and the longitude
used as the legal time reference (TL) is 4 minutes per degree of longitude.
 Correction due to legal time changes between summer and winter.
 The difference in solar time from one day to the next. Moreover, it is the correction of
the equation times. This correction varies during the year from -14.3mn to +16.4mn.
The approximate formula can calculate it:

𝐸𝑡 = 9.87𝑠𝑖𝑛2𝛽 − 7.53𝑐𝑜𝑠𝛽 − 1.5𝑠𝑖𝑛𝛽 (I.7)

The relationship between TSV and TL taking these corrections into account is:

24𝐿
𝑇𝑆𝑉 = 𝑇𝐿 + 𝐸𝑡 + +𝐶 (I.8)
2𝜋

10
Chapter I State of the Art

L is the longitude of the site considered about the meridian of Greenwich, affected by the sign
(+) for the longitudes East and the sign (-) for the longitudes West.

C: correction due to time change.

The time angle is linked to legal time (TL) by the relationship :

2𝜋 24𝐿
𝜔= (𝑇𝐿 + 𝐸𝑡 + + 𝐶 − 24 ) (I.9)
24 2𝜋

𝜔 is expressed in degrees, the TL and Et parameters are expressed in hours. Figure(I.4)

illustrates the entire time coordinate system.

We also calculate the hourly angle of sunset and sunrise by the relationship following :

𝜔𝑠 = 𝑐𝑜𝑠 −1 (− tan(𝜑) . tan(𝛿 )) where 𝜔 is the latitude and 𝛿 the angle of declination [22].

Fig.I. [Link] Coordinate System

III.2. System of Horizontal Coordinates

The horizontal system is most convenient for common applications. Two angular sizes
determine the position of the Sun: The height of the Sun (h) and azimuth ( 𝑨𝒛 )(Figure I. 5).

[Link] (𝑨𝒛 )

It is the horizontal angle of the direction of the Sun with the direction of the south. The
knowledge of the azimuth.’ 𝑨𝒛 ’ makes it possible to calculate the angle of incidence of the
rays on a non-horizontal surface [19].

11
Chapter I State of the Art

[Link] Height

It is the vertical angle of the direction of the Sun with the horizontal plane. Sometimes
one talks about the zenith angle, which is the complement to h, such as Az = 90°-h. The height
angular of the sun h measures the angular distance of the Sun from the horizon[19].

Fig.I. [Link] Coordinate System.

IV. SOLAR PHOTOVOLTAIC ENERGY

Solar photovoltaic energy has been of great interest in recent years. It is non-polluting
energy and provides real solutions to the various problems that The European Union and its
Member States are currently discussing climate change and the energy crisis [20].

[Link] Effect

The photovoltaic effect is a physical phenomenon specific to certain materials called

semiconductors (the best known is silicon). It converts energy directly light from solar rays
into electricity through production and transport in a semiconductor material of positive or
negative electrical loads as a result of the light. The electrical energy obtained is called
photovoltaic energy [21].

[Link] System

The photovoltaic system consists of a field of modules and a set of components that
adapt the electricity produced by the modules to the specifications of receptors[21].

The following figure represents the synoptic diagram of an autonomous photovoltaic system

12
Chapter I State of the Art

Fig.I. [Link] diagram of an autonomous photovoltaic system.

[Link] Cell

[Link]

The photovoltaic cell is a thin layer (two-tenths of a millimeter) of a semiconductor

metal, processed to be able to convert light into direct current.

It consists of the stacking of two layers of silicon previously exposed to ion beams, one
to phosphorus(-) ions and the other to boron(+) ions. The first layer has an excess of electron
and the other a deficit. They are said respectively doped N and doped P. This process is called
the «doping» and serves to create an electric field between the two zones where is created a
junction called PN, and directed of the zone (P) to the zone (N).

The area (N) is covered by a metal grid that acts as a cathode K while a metal plate A
covers the other side of the crystal and acts as an anode. A light beam that strikes the device
can penetrate the crystal through the grid and cause a voltage to appear between the cathode
and anode.

Fig.I. [Link] of a photovoltaic cell.

13
Chapter I State of the Art

[Link] Principle

When the two doped layers are introduced into contact, the extra electrons in the doped
N cloth diffuse into the doped P material. The in the beginning doped N vicinity will become
positively charged, and the at first doped P region is negatively charged. An electric field is
thus created between them, which tends to push the electrons back into the N zone and the
holes towards the P zone; a junction known as PN has been formed. By adding metal contacts
to the N and P zones, a diode is obtained.

When this diode is illuminated, photons having an energy (ℎ𝑣 ) greater than or equal to
the bandwidth of the forbidden band 𝐸𝑔 Excite the silicon atoms and create positive and
negative charges; thus, the electrons and holes created in the P and N regions respectively
diffuse and reach the space charge zone, accelerated by the internal electric field, they cross
the transition zone. The N region receives electrons and charges negatively. The P region
accepts holes and charges positively.

If a charge is placed at the cell terminals, the electrons in the N-zone join the holes in
the P-zone via the external connection, creating an electrical current [21].

[Link] of A Photovoltaic Cell

The function of a cell can be represented by the curve I=f (V), which indicates the
evolution of the current generated by the photovoltaic cell according to the voltage at these
terminals from the short circuit to the open circuit.

Fig.I. [Link]-voltage characteristic of a photovoltaic solar cell.

14
Chapter I State of the Art

According to the current-voltage characteristic, it is possible to deduce the electrical

parameters of the cell:

 the short circuit current (𝐼𝑐𝑐 ) corresponding to the current supplied by the cell
when the voltage at its terminals is zero.
 the circuit voltage (𝑉𝑐𝑜 ) corresponding to the voltage that appears at the
terminals of the cell when the current flow is zero.

Between these two values, there is an optimum, at a voltage known as the maximum
voltage 𝑉𝑚 and a maximum current I'm, giving the greatest power (𝑃𝑚𝑝𝑝 ) or peak power[21].

[Link] of Temperature And Illumination

The pattern of the current-voltage characteristic (Figure (I.9) , Figure (I.10)) varies
according to the environmental conditions (illumination and temperature).

Fig.I. [Link] of light on current-voltage characteristics of a photovoltaic cell.

Fig.I. [Link] of temperature on current-voltage characteristics of a photovoltaic cell.

The influence of temperature is significant on the current/voltage characteristic. The open-
circuit voltage decreases with the temperature increase, but the current varies very little with

15
Chapter I State of the Art

temperature. Therefore the maximum power (Figure (I.11)) delivered by the photovoltaic cell
decreases.

Fig.I. [Link] of temperature on the power-voltage characteristic of a cell photovoltaic.

Unlike the variation in temperature, the variation in illumination affects the short
circuit current that decreases as the illuminance decreases. The open-circuit voltage is not
sensitive to this variation. This implies that:

 The top-quality energy of the telephone is proportional to the illumination.

 The most electricity factors are at about the identical voltage (Figure (I.11)).

[Link]

Apart from the specific efficiency of each cell type (depending on the properties of the
material used), the final efficiency depends on the energy captured on the cell surface. This
depends on the solar irradiation arriving on the surface of the cell, which, in addition to the

16
Chapter I State of the Art

factors mentioned above (latitude, declination, solar angle, etc.), depends on the angle of
incidence.

The maximum efficiency, in the same place, is obtained when the solar radiation is
perpendicular to the catchment area, that is to say, the angle of incidence of the radiation on
the cell is 90° [22].

 Photovoltaic Cell Manufacturing Technology

Several technologies are currently being developed. They are classified into three
categories (generations) whose detailed description is widely discussed in the literature
specialized

 The first generation consists of monocrystalline silicon cells and Poly-

crystalline. Monocrystalline cells have a small yield superior to poly-crystalline
cells (15-16%). Nevertheless, they remain relatively expensive due to their high
purity requirements. Poly-crystalline cells are cheaper than the previous ones
but
Lower yields. Poly-crystalline silicon is the most widespread technology in the
global market due to its good result (11-14%) for controlled manufacturing
costs.
 The second generation: thin-film technology enables the use of fragile layers of
silicon that are applied to glass, plastic flexible, or metal. It is cheaper than
others and works with low light (in overcast conditions or inside a building),
but there
Performance is less useful than others.
 The third generation: The third-generation concepts are
 The photo pile with one or more intermediate bands,
 Conversion of photons not used directly in the PV cell,
 Quantum Well Cells

Developments around the third generation target yields ranging from 30-70% and significant
cost reductions[22].

17
Chapter I State of the Art

[Link] Battery

The fact that solar energy is not available for operation of the powered system requires
the use of batteries in the installations autonomous to store energy.

In autonomous solar systems:

 Lead-acid batteries: They constitute the overwhelming majority of the accumulators.

Its good technological mastery, its low cost, its good load/discharge efficiency. Its
operating conditions are not difficult to satisfy militate in favor of its wide use.
 Nickel Cadmium Batteries: Elles sont les plus chers, mais aussi très résistant aux
surcharges et aux décharges, et résistent bien aux basses températures.

[Link]

In any autonomous photovoltaic system, a so-called regulation system, which is used

to control the current going through the accumulators, protecting them against overloads and
deep discharges, to maximize its service life. The regulator also allows an optimal transfer of
energy from the field photovoltaic to use.

[Link]

To supply AC equipment, a device Static conversion electronics or DC/AC converter

is used for the transformation AC direct current.

[Link](Users)

There are two types of devices powered by the system, the direct current such as
telecommunications equipment, water pumping, and AC in the case of domestic use. This case
requires an inverter. The use of photovoltaic energy must be thought in terms of power. It is,
therefore, more advantageous to look for consumers running continuously rather than adding
an inverter and a 220 𝑉𝑎𝑐 consumer.

V. Conclusion

In this chapter, we have defined the majority of meteorological parameters, which are
included in the calculations of solar radiation, whether out of the atmosphere or soil. We have

18
Chapter I State of the Art

recalled the most important empirical models of solar irradiation. Finally, we presented the
experimental data, which are necessary for forming a database for our prediction models
throughout this study. There are several ways to process solar radiation data. The next chapter
is devoted to the state of the art of solar radiation prediction as that a time series.

19
 CHAPTER II

Machine learning and deep learning

techniques
Chapter II Machine learning and deep learning techniques

[Link]

Artificial intelligence (AI) has over the past decade become a popular topic both within
and outside the scientific community; a wealth of articles in technology and non-technological
journals have the subject of machine learning (ML), Deep Learning (DL) and AI. Still there is
confusion about AI, ML and DL. The terms are strongly interconnected, but not
interchangeable. In this review we do not (try) use technical jargon to better explain these
concepts to a clinical audience.
In 1956, a group of computer scientists suggested that computers could be
programmed to think and argue, "that every aspect of learning or any other property of
intelligence [could] be described in such a precise way that a machine [could] be induced to
simulate it." They described this principle as "artificial intelligence." Simply put, AI is an area
focused on automating the intellectual tasks normally performed by humans, and ML and DL
are specific methods to achieve this goal. This means that they are in the field of AI, but AI
includes approaches that do not include any form of "learning." Thus, the sub-field known as
symbolic AI focuses on hardcoding rules (i.e., explicit writing) for each possible scenario in a
particular area of interest. These rules, written by people, come from a priori knowledge of the
particular topic and task to be completed. For example, if you would program an algorithm to
modulate the room temperature of an office, he or she probably already know what
temperatures are convenient for people to work at and would cool the room, when
temperatures rise above a certain threshold and heat when they fall below a lower threshold to
program. Although symbolic AI can solve clearly defined logical problems, it often fails in
tasks that require higher pattern recognition, such as speech recognition or image
classification. These more complicated tasks are where ML and DL methods perform well.
This report summarizes machine learning and deep learning methodology for the audience
without extensive technical computer programming background [23].

[Link] LEARNING

Machine learning is one of the domains of artificial intelligence. A machine learning

algorithm has to aims to analyze the information available from a large number of statistical
data to learn to carry out a study without having explicitly been programmed for this upstream.

20
Chapter II Machine learning and deep learning techniques

Machine learning has been a real success in recent years. With the growth exponential number
of digital data available, we need to use new analysis methods, and so-called machine learning
methods correspond to this need. In this paper, we will study several types of machine
learning algorithms, and in this part, we will explain different generalities common to all.
We’ll start with a quick introduction of the other learning families, and then the general
operation of a machine learning algorithm. Then we will list the main advantages and
disadvantages, with a closer look at over-learning. Finally, we will explain the simplest
machine learning model, as well as its use in this memory [24].

[Link] Principle

[Link] of Learning in Machine Learning

[Link] Learning

Supervised learning is intended to create a predictive function for one of the variables
Of our database as a function of others. The variable we want to predict is called
"variable to be explained," the other variables used to guide the prediction are the "variables
Explanatory". The algorithm tries to learn, by browsing the database available,
The different causal links between the explanatory variables and the variable to be
explained. Once our model is created, it matches each possible combination of explanatory
variables, a prediction of the corresponding variable to be explained. For this, he must group
the individuals of the base into subgroups by maximizing the homogeneity of the variable to
be defined. Still, the groups must be separated only according to their explanatory variables.
There are two categories of supervised learning:
 Regression algorithms, when the variable to be explained, is quantitative. In this case,
the prediction is a value.
 Classification algorithms, when the variable to be explained, is qualitative. In this case,
the prediction is often a probability of belonging to the different modalities of the
variable in question [25].

21
Chapter II Machine learning and deep learning techniques

[Link] Learning
Non-served learning occurs when there is no response variable to predict. These
algorithms are also used on a database, but in this case, its purpose is to determine the
structure present in that database.
To do this, it must, like the supervised algorithms, group individuals into the most
homogeneous subgroups possible, however here we no longer have variables to explain, so
homogeneity must be done on all variables.
What interests us here is not the prediction of new data, but instead, how groups are
determined and what commonalities exist between the individuals of each subgroup.
Of the two learning families discussed above, we will only use supervised learning
algorithms in this brief. We aim to estimate a variable to be explained, the S/P ratio of
contracts, based on the information available about these contracts, which are the explanatory
variables. Supervised learning methods, therefore, appear to be adapted to our problems, such
as the CART algorithm (Classification And Regression Trees), bagging, or random forest. The
purpose of this section is to present the operation and the main characteristics common to
these methods [25].

[Link] Learning

In this type of study, each marked and unlabeled statistic may want to be utilized to
shape the indispensable knowledge. The framework receives a reward for each good or
incorrect forecast. Depending on the reward, the subsequent forecast should be generated. At
the factor when new information is given to the framework, the framework will pastime to
stumble on the excellent execution way or be part of a couple of execution pathways for
forecasting and pause for the reward. When the received reward takes place to be greatest with
recognition to the previous rewards for equal input, at that point, this pathway flips out to be
agreeable Reinforcement gaining knowledge of is utilized in net-primarily based games, for
example, Chess [25].

II.1.2. Regression And Classification

II. 1.2.1. Classification

22
Chapter II Machine learning and deep learning techniques

The classification consists of determining the instructions of belonging of new objects

from recognized previous examples, the variable to consider can therefore take discrete values
called classes. We will see that our problem is multi-class, that is, the exchange of opinions
can be represented using several courses, as an antagonist to the binary classification
represented using a two-class output variable [26].

[Link]. [Link] of Observation

II. [Link]

Regression is used when predicting a continuous variable, which can therefore take
any value. The classes representing the note change have therefore were considered constant
and the regression algorithm allowed to obtain decimal values that we finally rounded to get a
vector of predicted levels [26].

III. ALGORITHMS OF MACHINE LEARNING

III .1. Random Forest

III .1.1. Definition Of The Model

The method of machine learning that we will see in this part is the "random forest,"
better known as random forest, introduced in 2001 by L. Breiman in his publication bearing
the name of the method in question. The random forest (RF) method is part of the family of set
methods, as is the bagging is seen earlier. As a reminder, the purpose of these so-called
established methods is to create several slightly different trees from a single initial base to
group these trees and thus reduce the variance of our model. To make a large number of other
trees, we can either modify the learning base of each tree or change the algorithm used to
create it [24].

23
Chapter II Machine learning and deep learning techniques

III .1.2. How It’s Work

In the bagging section, the trees were created from the CART algorithm. It consists in
choosing at each node the separation maximizing the intergroup variance among all possible, a
separation representing a test on one of the p explanatory variables of the model. However, in
this part, this algorithm will be modified so that before each separation, which is called a node,
we randomly pull m explanatory variables among the p’s of our model. The separation chosen
for the node in question will therefore be the optimal one only among the tests involving the
selected m variables. Thus, at each node, the information carried per variable p-m drawn
randomly is ignored. Apart from the choice of separation, the rest of the algorithm takes place
Then, for each bootstrap sample, we create the maximum tree using our new modified
algorithm. As for the bagging method, we do not prune the trees because we want to keep the
bias as low as possible. Finally, we aggregate the maximum trees together, so that the
prediction of our random forest model is the average of the predictions of the maximum trees.

It is interesting to note that bagging could be considered as a method of

Random forest, in the particular case where m = p. Indeed, if m = p, then at each node, we
choose optimal separation among the tests for one of the selected p variables, that is, all
Variables. We find the CART algorithm used in the bagging method.

To remain in line with the ratings used previously:

 B number of bootstrap samples created

 𝑛 𝑙𝑒𝑎𝑟𝑛𝑖𝑛𝑔 The size of the sample of data used for learning
 Ŷ𝑖 estimation of the output variable of individual I by our random forest model
 ŶRF(m)i,k the estimation of the output variable of individual I buy the maximum tree
created from of the bootstrap k sample with a random forest algorithm with the
parameter mtry = m

it is achieved that

𝐵
1
Ŷ𝑖 = ∑ ŶRF(m)i,k (II.1)
𝐵 𝑘=1

24
Chapter II Machine learning and deep learning techniques

The formula is, therefore, the same as for a bootstrap, the difference in how to build
the tree after creating the bootstrap sample [24].

As in the previous part, this method is explained in a prominent and concise manner
with a diagram:

initial database

creation of B bootstarp samples

Sample Sample Sample

...
01 02 B

creation of modified CART trees by selecting single m variable at each

node

tree tree ... tree

01 02 B

Ŷ1 Ŷ2 Ŷ𝐵

𝐵
1
the final result is the average : ∑ Ŷ𝑗
𝐵 𝑗=1

[Link]. [Link] diagram of the random forest method

25
Chapter II Machine learning and deep learning techniques

III .[Link] The Random Forest

As with the bagging model, the first step in creating our random model forest is the
determination of the parameters. There are two of them: B, the number of samples bootstrap,
and mtry the number of variables selected at each separation. To stay consistent with the
previous part, we will note a 𝐵𝑜𝑝𝑡𝑖 and 𝑚𝑡𝑟𝑦𝑜𝑝𝑡𝑖 the chosen parameters.

For parameter B, we will operate in the same way as for the bagging model. We are
therefore going to test a random forest model with a large number of trees, here 1000. Then
we will find from what number of trees the OOB error, defined in the previous part, is
stabilized.

To create our random forest model with 1000 trees to determine 𝐵𝑜𝑝𝑡𝑖 We had to set a
value for mtry. As a reminder, a good indication of this parameter is mtry ≈ √𝑝 , with p the
number of variables that can potentially be selected for each draw.

Here, p = 29, this number is more significant than that mentioned in the section on
elimination variables. Indeed, in a random forest, for the category variable, for example, the
model can draw separately each of the six variables created during the aggregation of the
database. So if we seek to determine the number of variables, the category of the vehicle
counts for six here. In contrast, it counted for only one in the selection of variables, which
explains this difference.

For now, we set the mtry parameter as follows: mtry ≈ √29 ≈ 5.4, we will take so for
this test the rounding to the nearest integer, that is to say, mtry = 5. Of course, we will then
determine the value of 𝑚𝑡𝑟𝑦𝑜𝑝𝑡𝑖 using a more sophisticated method. We set it to 5 only the
time of the tests to determine 𝐵𝑜𝑝𝑡𝑖 [26].

III .1.3.1. Hyperparameter tuning random forest

Although random forests perform well out-of-the-box, there are several tunable
hyperparameters that we should consider when training a model. The main hyperparameters to
consider include:

26
Chapter II Machine learning and deep learning techniques

n_estimators = number of trees in the forest

max_features(mtry) = max number of features considered for splitting a node

max_depth = max number of levels in each decision tree

min_samples_split = min number of data points placed in a node before the node is split

min_samples_leaf = min number of data points allowed in a leaf node

bootstrap = method for sampling data points (with or without replacement)

III .1.3.2. Selection Of The Number Of Trees

We can see on this graph the evolution of the OOB error as a function of the number of
bootstrap samples, and therefore of the number of trees. We can see that the error stabilizes
later as for the bagging model. Here we will now fix 𝐵𝑜𝑝𝑡𝑖 = 400 [26].

[Link]. [Link] error depending on the number of trees in the random forest model.

III .1.3.3. Selecting The Mtry Parameter

Now that we have determined the first parameter, Bopti, let's move on to the choice of
the second, 𝑚𝑡𝑟𝑦𝑜𝑝𝑡𝑖 .

For the mtry parameter, we will create models with different values for mtry, and then
we will test these models on our validation basis to calculate the MSE error. We will choose
the mtry that gave the model with the lowest MSE.

27
Chapter II Machine learning and deep learning techniques

We first had to choose a range of mtry values that we are going to test. As the
indication mtry ≈ √𝑝 gave us a result between 5 and 6, we took a range around these values,
so we have arbitrarily set it between 2 and 12. If we subsequently find that this range is not
sufficient to make our choice, we give ourselves the option to change it later [26].

Finally, we specify that all models tested on this range are tested with 400 bootstrap
samples, as we have chosen previously.

We can see that the indication mtry ≈ √𝑝 looks good because models with mtry
values of 5 and 6 are better than others. We now fix 𝑚𝑡𝑟𝑦𝑜𝑝𝑡𝑖 = 6, whose model got a slightly
smaller error than the one with mtry = 5.

Thus, the estimate made in this part will be composed of 12 random forest models,
each having 400 bootstrap samples and a value of 6 for the parameter mtry. The models thus
created obtained an MSE on the test basis of 0.87 by predicting the S / P and 0.74 by the
number of claims. These are the best scores so far [24].

[Link]. [Link] error depending on mtry value.

III .2. Gradient Boosting Machine

III .2.1. Definition of The Model

Gradient Boosting Machine (GBM) is a machine learning technique used for

regression or classification problems.

28
Chapter II Machine learning and deep learning techniques

The most usual approach of boosting is likely the one developed via Freund and
Schapire in 1995: the AdaBoost (for Adaptive Boosting). However, it will now not be precise
in this brief. However, the reader is referred to Article A Decision-Theoretic Generalization of
On-Line Learning and an Application to Boosting for more critical data about this method.
Like different boosting methods, the Gradient Boosting optimizes the overall performance of a
set of so-called “low” prediction fashions by means of assembling them into a remaining
model. This is known as a “low” prediction model, a classification or regression approach that
is simply barely greater nice than a random draw. The low prediction mannequin normally
used with a Gradient Boosting is a CART choice tree. More concretely, this approach of the
Gradient Tree Boosting then consists in making a succession of choice bushes the place every
mannequin is constructed on the residual error of the preceding one [25].

III .2.2. The Cart Decision Tree

The Classification and Regression Tree (CART) is a supervised statistical learning

method designed to construct a prediction model. The main idea of the CART tree is to
recursively and binary partition the space of the explanatory variables to obtain all possible
values of the variable to be predicted. Suppose we have an n-sample of a variable Y with p
predictors [𝑥1 ,..., 𝑥𝑝 ]. The goal is to be able to predict Y values for new [𝑥1 ,..., 𝑥𝑝 ] values.
Let’s insert 𝑡𝑝 , the parent node and 𝑡𝑔 , 𝑡𝑑 the child nodes, respectively, left and right. From the
set of observations on the variable Y, the CART tree selects at each iteration and, therefore, at
each node𝑡𝑝 , the variable 𝑥𝑗 and its value 𝑥𝑗 which best segment the space of the variables in
two sub-most homogeneous areas possible in the tg and td nodes. This notion of homogeneity
results in a function of impurity noted I (.). Several functions define the impurity i(t) of a node
t. Still, in the case of the study of a continuous random variable Y, as is the case here, the node
variance may represent a good indicator of the heterogeneity present in the node. The impurity
of a parent node remaining constant regardless of the cutting performed to create the yarn
nodes, maximizing the homogeneity in the yarn nodes is then equivalent to optimizing the
variation of impurity ∆𝑖 defined as:

∆𝑖 = 𝑖(𝑡𝑝 ) − 𝐸(𝑖(𝑡𝑓𝑖𝑙𝑠 )) (II.4)

29
Chapter II Machine learning and deep learning techniques

This concept of impurity reduction can be seen in (Figure II.5), where the removal of
impurity is more critical in the illustration on the left than in the illustration on the right.

[Link]. [Link] of the CART tree impurity reduction concept.

The problem of maximizing impurity ∆𝑖 defined in equation (II.4) can also be written:

𝑚𝑎𝑥𝑥𝑖<𝑥𝑗,𝑗=1,…,𝑝 [𝑖(𝑡𝑝 ) − 𝑃𝑔 ∗ 𝑖(𝑡𝑔 ) − 𝑃𝑑 ∗ 𝑖(𝑡𝑑 )] (II.5)

where 𝑃𝑔 and 𝑃𝑑 are the probabilities of the left and right nodes, respectively. This
problem translates the way the algorithm works: it scans all possible values for all variables 𝑥𝑖
for j = 1,...,p to find the best cutout that will maximize the variation of impurity ∆𝑖 . If the node
variance represents the impurity function, then the maximization problem 3.2 can be rewritten:

𝑚𝑎𝑥𝑥𝑖<𝑥𝑗,𝑗=1,…,𝑝 [𝑃𝑔 ∗ 𝑉𝑎𝑟(𝑡𝑔 ) + 𝑃𝑑 ∗ 𝑉𝑎𝑟(𝑡𝑑 )] (II.6)

For more details on the machine learning method CART, the reader can refer to
Roman Timofeev, Classification, And Regression Trees[27].

III .2.3. Gradient Boosting Machine Algorithm

The Gradient Boosting is an iterative algorithm that initially distributes weights equal
to all predictions and then adapts them to each step, so that bad predictions are over-weighted
to the next step so that the “low” prediction model Pay more attention to it. Let’s go back to
Friedman’s original article by using his notes to develop the idea of Gradient Boosting in more
detail. We have a sample size (n) consisting of (p) explanatory variables. Note 𝑥𝑖 = (𝑥𝑖,1 ,...
𝑥𝑖,𝑝 ) ∈𝑅𝑝 , The vector of the explanatory (p) variables corresponding to the observation i. The
objective is to find an approximation 𝐹approx (𝑥) of a function F(x) linking the explanatory
variables x to the response variable y and minimizing the expectation of a certain loss function

30
Chapter II Machine learning and deep learning techniques

L(y, F(x)) on the attached distribution of x and y [28]. Mathematically, the function F(x) is
then defined by:

𝐹 ∗ (𝑥 ) ∈ 𝑎𝑟𝑔𝑚𝑖𝑛𝐹 𝐸𝑦,𝑥 [L(y,F(x))] (II.7)

Algorithm Gradient Boosting Machine

 Initialize 𝐹0 (𝑥) = 𝑎𝑟𝑔𝑚𝑖𝑛𝑝 ∑𝑁
𝑖−1 𝐿(𝑦𝑖 , 𝑝 )

 For 𝑚 = 1 to M do :
o Step 1. Compute the negative gradient
𝜕𝐿(𝑦𝑖 , 𝐹 (𝑥𝑖 ))
ȳ𝑖 = −
𝜕𝐹𝑥𝑖
o Step 2. Fit a model
𝑁

𝛼𝑚 = 𝑎𝑟𝑔𝑚𝑖𝑛𝛼,𝛽 ∑[ ȳ − 𝛽ℎ(𝑥𝑖 ; 𝛼𝑚 )]²

𝑖=1

o Step 3. Choose a gradient descent step size as

𝑁

𝑝𝑚 = 𝑎𝑟𝑔𝑚𝑖𝑛𝑝 ∑ 𝐿(𝑦𝑖 , 𝐹𝑚 − 1(𝑥𝑖 ) + 𝑝ℎ(𝑥𝑖 ; 𝛼))

𝑖−1

o Step 4. Update the estimation of F(x)

𝐹𝑚 (𝑥) = 𝐹𝑚−1 (𝑥) + 𝑝𝑚 ℎ(𝑥; 𝛼𝑚 )
 End for
 Output the final regression function 𝐹𝑚 (𝑥)

[Link] Gradient Boosting Machine

III .2.4. Hyperparameter Tuning Gradient Boosting Machine

To avoid over-learning of the data and to degrade future predictions, the Gradient
Boosting has several hyper-parameters that allow optimizing its use by limiting this over-
learning phenomenon [29]:

A-Min_Samples_Split

 Defines the minimal range of samples (or observations) which are required in a node to
be viewed for splitting.

31
Chapter II Machine learning and deep learning techniques

 It is used to manage over-fitting. Higher values stop a model from mastering members
of the family, which would possibly be noticeably unique to the specific pattern chosen
for a tree.
 Too extreme values can lead to under-fitting; hence, it ought to be tuned to the use of
CV.

B- Min_Samples_Leaf

 Defines the minimal samples (or observations) required in a terminal node or leaf.
 It is used to manage over-fitting comparable to min_samples_split.
 Generally, decrease values ought to be chosen for imbalanced type troubles due to the
fact that the areas in which the minority classification will be in the majority will be
minimal.

C- Max_Depth

 The most depth of a tree.

 Used to manipulate over-fitting as greater depth will permit mannequin to analyze
family members very particular to a specific sample.
 Should be tuned the usage of CV.

D-Max_Features

 The range of facets to reflect on consideration while looking for an exceptional split.
These will be randomly selected.
 As a thumb-rule, the rectangular root of the complete quantity of facets works
excellent. However, we ought to take a look at up to 30-40% of the total variety of
features.
 Higher values can lead to over-fitting; however, relies upon case to case.

E- Learning_Rate

 He determines the have impact of every tree on the closing effect. GBM works by way
of beginning with a preliminary estimate, which is up to date the usage of the output of

32
Chapter II Machine learning and deep learning techniques

every tree. The gaining knowledge of parameter controls the magnitude of this
alternate in the forecast.
 Lower values are commonly favored as they make the mannequin strong to the unique
traits of the tree and, as a consequence permitting it to generalize well.
 Lower values would require a greater wide variety of timber to mannequin all the
family members and will be computationally expensive.

F- N_estimators

 The number of sequential trees to be modeled.

 Though GBM is relatively robust at a higher number of trees, it can still overfit at a
point. Hence, this should be tuned using a CV for a particular learning rate.

G- Loss

 It refers to the loss function to be minimized in each split.

 It can have various values for classification and regression case. Generally, the default
values work fine. Other values should be chosen only if you understand their impact on
the model.

III .[Link] Learning versus Deep Learning

Machine learning (ML) is extensively utilized in identifying several types of attacks.

An ML methodology could assist the network admin in performing required actions for
averting intrusions. Nevertheless, most traditional ML methods reside within shallow learning
and usually emphasize to feature engineering and selection. With enormous intrusion data that
rises in the real-time network environment, external learning is unable to resolve the
classification issue efficiently [30]. A deep learning approach possesses the potential to extract
improved representations from the dynamic datasets and is capable of creating many efficient
prototypes. [Link] et al. [31] introduced the hypothesis of DL in the year 2006, and over the
years, the theory underwent a spectacular rise in the area of ML. Most recent ML
methodologies work well because of human-designed representations and input features. In
ML, ‘representation learning’ or ‘feature learning’ represents a set of methods that makes a
system capable of automatically learning the representations required for detecting features

33
Chapter II Machine learning and deep learning techniques

from the training dataset. DL, on the contrary, could be considered as establishing both
representation learning and machine learning together. DL pursuits to together learn essential
features along with multiple levels of cumulative intricacy and abstraction and the concluding
prediction. Figure (II.6) illustrates the fundamental difference between ML and DL, where
traditional ML involves manual feature selection, and on the contrary, DL employs automated
feature selection.

[Link]. [Link] learning and Deep learning

[Link] LEARNING

Deep Learning is a new area of ML research, which has been introduced to bring the
ML closer to its primary objective: artificial intelligence. It concerns algorithms inspired by
the structure and functioning of the brain. They can learn multiple levels of representation to
model complex relationships between data.

[Link]. [Link] relationship between artificial intelligence, ML, and deep learning.
Deep Learning is based on the idea of artificial neural networks and is designed to
manage large amounts of data by adding layers to the network. A deep learning model can
extract characteristics from raw data through multiple layers of processing consisting of
multiple linear and non-linear transformations and learn about these characteristics step by

34
Chapter II Machine learning and deep learning techniques

step across each layer with minimal human intervention. Over the past five years, deep
learning has moved from a niche market where only a handful of researchers were interested
in the area most sought by researchers. Research related to deep learning now appears in top
journals like Science, Nature, and Nature Methods, to name a few. Deep learning has
conquered the GO, learned to drive a car, diagnosed cancer and autism, and even become an
artist. The term "Deep Learning" The time period "Deep Learning" used to be first added to
ML via Dechter (1986) and artificial neural networks by using Aizenberg et al. (2000) [32].

IV.2. History of Deep Learning

This paper presents a history of deep learning from Aristotle to the present. The
various milestones are summarized in this table

Year contributor Contribution

300 AC Aristotle introduction of associationism, beginning of the history of
humans trying to understand the brain
1873 Alexander Bain introduction of Neural Groupings as the first models of neural
networks
1943 McCulloch and Pitts Introduction of the McCulloch–Pitts (MCP) model considered as
the ancestor of artificial neural networks
1949 Donald Hebb consider as the father of neural networks, and it introduces the
Hebb learning rule that will serve as the foundation for modern
neural networks
1958 Frank Rosenblatt introduction of the first perceptron
1974 Paul Werbos introduction of retro propagation
1980 Teuvo Kohonen introduction of self-organizing cards
1980 Kunihiko Fukushima introduction of Neocognitron, which inspired convolutive neural
networks
1982 John Hopfield introduction of Hopfield networks
1985 Hilton and Sejnowski introduction of Boltzmann machines
1986 Paul Smolensky introduction of Harmonium, which will later be known as
restricted Boltzmann machines

35
Chapter II Machine learning and deep learning techniques

1986 Michael. Jordan definition and introduction of recurrent neural networks

1990 Yann LeCun Introduction of LeNet and demonstrated the capabilities of deep
neural networks
1997 Schuster and Paliwal introduction of bi-directional recurrent neural networks
1997 Hochreiter and introduction of LSTM, which have solved the problem of
Schmidhuber vanishing gradient in recurrent neural networks
2006 Geoffrey Hinton introduction of the Deep Belief Network
2009 Salakhutdinov and introduction of Deep Boltzmann Machines
Hinton
2012 Alex Krizhevsky Introduction of AlexNet, which won the ImageNet challenge
[Link] of Deep Learning [33].

IV.3. Why Deep Learning?

ML algorithms work well for a wide variety of problems. However, they have failed to
solve some significant issues of AI, such as voice recognition and object recognition. The
development of deep learning was motivated in part by the failure of traditional algorithms in
such a task of AI. But it wasn’t until more data was made available thanks to Big Data and
connected objects and computing machines became more powerful that the real potential of
Deep Learning was understood.

One of the significant differences between Deep Learning and traditional ML

algorithms is that it adapts well. The greater the amount of data provided, the better the
performance of a Deep Learning algorithm. Unlike several ML classics algorithms that have a
terminal higher than the amount of data they may receive, sometimes called "performance
plateau," Deep Learning models do not have such limitations (theoretically), and they even
went so far as to surpass human performance in areas such as image processing [34].

36
Chapter II Machine learning and deep learning techniques

[Link]. 8. The performance difference between Deep Learning and most ML algorithms
depends on the amount of data.

IV.4. Deep Learning Application Areas

In-depth learning is gradually investing in our daily lives:

 Voice recognition
 Automatic tagging of music pieces
 Advanced speech synthesis
 The design of new pharmaceutical molecules

All these applications now use Deep Learning techniques [34].

V- ALGORITHMS OF DEEP LEARNING

There are a large number of variations of deep architectures. Most of them are derived
from some original parenting architectures. It is not always possible to compare the
performance of all architectures, as they are not all evaluated on the same datasets. Deep
Learning is a rapidly growing field, and new architectures, variants, or algorithms appear
every week.

[Link] Neural Networks

Deep Neural Networks (DNNs), additionally acquainted as Neural Networks (NNs),

are computing device mastering fashions which are stimulated from the human brain. The first
fashions of neural community developed by way of a neurophysiologist Warren McCulloch

37
Chapter II Machine learning and deep learning techniques

and a mathematician Walter Pitts in 1943. They mentioned how neurons would possibly work.
Neural networks consist of the layer of entering neurons or alerts, which can be different
characteristic values, an output layer the place the result of the community obtained, and a
quantity of a range of hidden layers between the enter and output layers. Also, each layer has a
few or various neurons.

Input alerts are surpassed thru the network, layer by way of layer, by way of the usage
of the weighted connections to, in the end attain the output layer. At some neurons, a nonlinear
feature can be triggered. The purpose of gaining knowledge of the method is discovering
weights that would make the neural community display favored behavior. This is the instance
of Multilayer Perceptron (MLP), which is additionally referred to as Feedforward Neural
Networks (FNN). An accepted structure of artificial neural networks is proven in Figure
(II.9).

Despite the reality that the feedforward neural networks have been efficaciously
applied and bought higher overall performance in many tasks, it does now not think about the
transient prospect that characterizes sequential data. That is, they are no longer moderately
proper when it comes to the statistics that rely on the preceding data. To remedy such a variety
of troubles, the neural networks have advanced to so-referred to as recurrent neural networks.
In the subsequent subsections [43].

[Link]. 9. Deep Neural Network architecture[35].

38
Chapter II Machine learning and deep learning techniques

[Link]

The fundamental unit of neural networks is one of the types of artificial neurons, called
a perceptron. A perceptron takes a fixed number of inputs and produces a single output. The
way of computing the result essentially has several steps, such as taking an input, calculating
the weighted sum by introducing weights, bias term, and applying activation function. The
process described above can be formalized as follows:

The perceptron has n inputs represented as an input vector x = (𝑥1 , 𝑥2 , . . . , 𝑥𝑛 ). Each input
has an assigned weight that defined by a vector of weights w = (𝑤1 , 𝑤2 , . . . , 𝑤𝑛 ).
Consequently, the weighted input values are combined, which gives the weighted sum:

ε = 𝑤 · 𝑥 = ∑𝑛𝑖=1 𝑤𝑖 · 𝑥𝑖 (II.8)

At this step, the activation function is applied to the weighted sum to calculate the output, and
the weighted sum is compared with a threshold θ to produce an output y that is either 0 or 1,
depending on whether or not it exceeds the threshold. Thus,

1, ε ≥ 0
𝑦 = σ(ε) = { (II.9)
0, ε < 0

[Link]. [Link] Perceptron.

[Link]-layer Perceptron

A multilayer perceptron (or neural network) is a shape composed of countless hidden

layers of neurons the place the output of a neuron of a layer will become the enter of a neuron
of the subsequent layer. Moreover, the end result of a neuron can additionally be the enter of a

39
Chapter II Machine learning and deep learning techniques

neuron of the identical layer or of the neuron of preceding layers (this is the case for recurrent
neural networks)

nodes that are no goal of any connection are referred to as entering neurons. An MLP
that must be utilized to enter patterns of dimension n needs to have n enter neurons, one for
every dimension. Input neurons are usually enumerated as neuron 1, neuron 2, neuron three

 nodes that are no supply of any connection are referred to as output neurons. An MLP
can have extra than one output neuron. The range of output neurons relies upon on the
way the goal values (desired values) of the education patterns are described.
 all nodes that neither enter neurons nor output neurons are referred to as hidden
neurons.

[Link]. [Link] Perceptron (MLP) structure.

[Link] Functions

An activation function is a mathematical function applied to a combination of 𝑥𝑖

inputs. It thus introduces the principle of non-linearity, which makes it possible to adapt and
differentiate the results. The choice of the activation function depends on the application,
whether it is necessary to have binary outputs or not. In this work, the functions Sigmoid,
Softmax, Tanh as well as ReLu are used [36].

40
Chapter II Machine learning and deep learning techniques

 Sigmoid

The sigmoid function is defined by:

1
𝑓 (𝑥) = 1+𝑒 −𝑥 𝑂𝑟 𝑥 ∈ R (II.9)

The goal is to convert the input value into a probability of 1 if it is a very large
positive number, and conversely, to 0 if the input is a very large negative number

[Link]. [Link] Function Graph.

 Softmax

The Softmax function, or normalized exponential function, is defined by:

𝑧
𝑒 𝑗
𝑓(𝑥)𝑗 = 𝐾 for all j ∈ {1, … , K}, x = {x1, … , xk} ou k ∈ R (II.10)
∑𝐾=1 𝑒 𝑧𝑘

The function gives an output vector of K strictly positive real numbers and sum 1.

 Tanh

The hyperbolic tangent function, noted Tanh, is defined by:

𝑒 𝑥 − 𝑒 −𝑥
𝑓 (𝑥 ) = tanh(𝑥) = (II.11)
𝑒 𝑥 + 𝑒 −𝑥

The output in Tanh will be stored in the field [-1,1].

41
Chapter II Machine learning and deep learning techniques

[Link]. [Link] Function Graph.

 ReLu

The ReLu linear grinding unit function is the simplest function, defined by:

0 𝑓𝑜𝑟 𝑥 < 0
𝑓 (𝑥 ) = { (II.12)
𝑥 𝑓𝑜𝑟 𝑥 ≥ 0

[Link]. [Link] Graph.

The output is therefore 0 if the input is less than 0. Otherwise, it will be equal to the input.

[Link] functions

Generally, the functions of optimization in neural networks have the objective of

calculating the gradient to find optimized values (weight), and this by modifying the weights
in the opposite direction, this cycle is repeated until the objective function is minimized. In
this work, different optimizations functions are tested: Adam, Adamax, Adadelta, and SGD
[37].

 SGD

The word “stochastic” refers to a random probability system or process. Therefore, in

stochastic gradient descent, some samples are randomly selected instead of the dataset of each

42
Chapter II Machine learning and deep learning techniques

iteration SGD (Stochastic Gradient Descent), which updates the parameters for each example
of the 𝑥𝑖 dataset and 𝑦𝑖 label (label).

 Initialize with 𝑥0 (at random)

 Repeat: 𝑥𝑡 + 1 = 𝑥𝑡 − 𝜂 × 𝛻(𝑥𝑡) Until convergence

 η: modulates the correction (η too low, slow convergence; η too high, oscillation)

 𝛻(𝑥𝑡): The gradient at point 𝑥𝑡 shows the direction and importance of the slope in the
vicinity of 𝑥𝑡.

This method is usually faster. However, due to frequent updates, convergence becomes more
difficult (find the minimum objective function).
 Adam

“Adam Adaptive Moment optimization” is one of the latest and most effective
algorithms for gradient descent optimization. Adam calculates the exponential mean of the
gradient as well as the squares of the gradient for each parameter. The learning rate is then
multiplied by the mean of the gradient and divided by the square root of the exponential mean
of the gradients. Then the update is added.

𝑣𝑡 = 𝛽1 × 𝑣𝑡−1 − (1 − 𝛽1 ) × 𝑔𝑡 (II.13)

𝑠𝑡 = β2 × 𝑠𝑡−1 − (1 − β2 ) × 𝑔²𝑡 (II.14)

𝑣𝑡
∆w𝑡 = −ƞ × 𝑔𝑡 (II.15)
√𝑠𝑡 +ϵ

𝑤𝑡+1 = 𝑤𝑡 + ∆w𝑡 (II.16)

o Ƞ: initial learning rate.

o 𝑔𝑡: the gradient in a time t.
o 𝑣𝑡: the exponential average of gradients.
o 𝑠 : the exponential average of the squares of the gradients.
o 𝛽1 , 𝛽2, 𝜖 : hyperparameters usually initialized at 0.90, 0.99, and 1e-10.

43
Chapter II Machine learning and deep learning techniques

 Adamax

Adamax is a generalization of Adam to get a more stable convergence, or instead of taking:

𝑠𝑡 = β2 × 𝑠𝑡−1 − (1 − β2 ) × 𝑔²𝑡 (II.17)

𝑠𝑡 become :

𝑝 𝑝 𝑝
𝑠𝑡 = β2 × 𝑠𝑡−1 − (1 − β2 ) × 𝑔𝑡 (II.18)

Ou p = ∞.

 Adadelta

Adadelta proposes an adaptation of the gradient depending on its second-order moment

and the state of the corrected network layer and uses a fixed size w to take into account the
sum of the squares of gradients accumulated in the past.

[Link] Regularization

Deep neural network (DNN) fashions have several parameters and can model
relatively composite functions. This capability is a boon and a bane. Such prototypes would
often overfit on the training-set and would drop accuracy and generalizability over the test-set.
Regularization in ANN terminology speaks of the technique of regulating neural network
layers for stopping the over-fitting. Dropout (also recognized as dropout chance or dropout
rate) is the most extensively utilized regularization approach in DL. During the learning
manner, the hidden layer(s) neurons are chosen randomly and are discarded, relying on the
dropout rate. Precisely, randomly chosen neurons are dropped-out, i.e., dropped out neurons
could not replace weights anymore, accordingly supporting the learning manner to avert the
problem of overfitting [38].

[Link] Neural Networks

Humans don’t start thinking from scratch every second. By reading this essay, you
understand each word according to your understanding of the preceding words. You don’t
throw everything away and start thinking from scratch again. Your thoughts have
perseverance.

44
Chapter II Machine learning and deep learning techniques

Traditional neural networks (TNNs) can’t do that, and that seems to be a significant
gap. For example, imagine that you want to classify the type of event that occurs at each stage
of a movie. It is not known how a traditional neural network could use its reasoning on
previous events in the film to inform them later.

[Link]. 15. Recurrent neural networks have loops.

Recurrent neural networks (RNNs) solve this problem. They are networks with loops
that allow information to persist. In figure II.15 above, a neural network segment, "A," looks
at an Xt input and provides a ht value. A loop allows information to be passed from one stage
of the network to another.

These loops make recurring neural networks a bit mysterious. However, if you think a
little more, it turns out that they are not all different from a standard neural network. A
network of recurrent neurons can be considered as multiple copies of the same network, each
transmitting a message to a successor, as shown in Figure II.16. Consider what happens if we
unravel the loop:

[Link]. 16. A recurring neural network unfolded.

45
Chapter II Machine learning and deep learning techniques

This chain nature reveals that recurring neural networks are intimately linked to
sequences and lists. They are the natural architecture of the neural network to be used for such
data.

In recent years, there has been an incredible success in applying NRNs to a variety of
issues: speech recognition, language modeling, translation, captioning of images, etc [39].

[Link] Problem of Long-Term Dependencies

One of the appeals of RNNs is the concept that they may be capable of joining
preceding records to the current task, such as the use of remaining video frames may inform
the appreciation of the contemporary structure. If RNNs ought to do this, they’d be
instrumental. But can they? It depends [39].

Sometimes, we solely want to seem at current statistics to operate the current task. For
example, reflect on consideration on a language mannequin is making an attempt to predict the
subsequent phrase-based totally on the preceding ones. If we are making an attempt to
envision the ultimate phrase in “the clouds are in the sky,” we don’t want any in addition
context – it’s rather apparent the subsequent expression is going to be the sky. In such cases,
the place the hole between the applicable records and the region that it’s wanted is small,
RNNs can examine to use the previous data.

[Link]. [Link] Problem of Long-Term Dependencies-1.

But there are additionally instances the place we want extra context. Consider making
an attempt to predict the remaining phrase in the textual content “I grew up in France… I
communicate fluent French.” Recent data suggests that the next phase is likely the identity of a
language. However, if we favor to slender down which language, we want the context of

46
Chapter II Machine learning and deep learning techniques

France, from in addition back. It’s absolutely feasible for the hole between the applicable
statistics and the factor the place it is required to emerge as very large.

Unfortunately, as that gap grows, RNNs become unable to learn to connect the information.

[Link]. [Link] Problem of Long-Term Dependencies-2.

In theory, RNNs are succesful in coping with such “long-term dependencies.” A

human ought to cautiously pick out parameters for them to resolve the toy troubles of this
form. Sadly, in practice, RNNs don’t show up to be in a function to find out about them. The
hassle used to be explored in depth by using way of Hochreiter (1991) [German] and Bengio
et al. (1994), who decided some pretty essential causes why it may additionally be
problematic. Thankfully, LSTMs don’t have this problem!

[Link] Short-Term Memory (LSTM)

[Link]

Short-term, long-term memory networks (LSTM) is a synthetic structure of recurrent

neuronal community (RNN) used in the discipline of deep learning. ... LSTM networks are
well suited to classify, process, and make predictions based entirely on time collection data, as
there may be delays of unknown time between vital opportunities in a time [Link] is a
kind of recurrent neural community that addresses the disappearing and exploding gradient
problems that can occur with NRNs, preventing them from having long time
sequences. Therefore, the MTSA may have a “short-term memory,” as the name suggests. The
LSTM incorporates extraordinary devices called reminiscence blocks into the recurrent hidden
layer. Reminiscence blocks include reminiscence cells with auto-connections storing the

47
Chapter II Machine learning and deep learning techniques

community’s temporal nation in addition to specific multiplicative tools called gates to

manage movement with the flow of information. Each reminiscence block in the single
structure contained an entrance door and an exit door. The entrance door controls the floating
of the entrance activations in the reminiscence cell. The output door controls the output float
of telephone activations in the relaxation of the network. Later, the overhanging door was once
brought to the reminiscent block. The negligence door scales the inner nation of the mobile
phone before including it as an entry into the phone via the cell’s auto-recurring connection,
therefore by adapting or resetting the cell’s reminiscence [39].

[Link] Principle

LSTMs have internal mechanisms called gates, which are used to regulate the flow of
information by learning what information is useful to keep or forget in time. In doing so, the
network will be able to transmit relevant data, thanks to the state of the cell, acting as a
transport route that transfers related information throughout the sequence chain. As the status
continues, information is added or deleted through three main doors. Initially, the input is the
output predicted previously (ℎ𝑡−1) as well as the current input 𝑋𝑡). This data will be broken
down into three streams. The objective is to update the status of the cell from (𝐶𝑡−1 to 𝐶𝑡.) [40].

A- Input Gate

The Input Gate updates the status of the cell. The activation function is Sigmoid. For each
input, the gate will provide an output value between 0 and 1, and decide which value to update
(0 means not necessary and 1 means important). This result is then multiplied by the current
state

𝑓𝑡 = (𝑊𝑡 [ℎ𝑡−1, 𝑋𝑡] + 𝑏𝑡) (II.19)

𝐶𝑡 = 𝑓𝑡 × 𝐶𝑡−1 (II.20)

With:

 ℎ𝑡−1: The exit at the moment t-1

 𝑋𝑡 : Current input at time t
 𝑏𝑡 : Biases

48
Chapter II Machine learning and deep learning techniques

 𝑊𝑡 : Weight
 σ: Sigmoid function

[Link]. 19. Diagram represents an LSTM Input Gate.

B- Forget Gate

The Forget Gate forgets, decides what information should be discarded or kept. The
information of the previous state is passed to the Tanh function giving values between -1 and
1. Then, to select the important characteristics, a sigmoid layer will decide which value will be
updated

𝑓𝑐= tanh(𝑊𝑐[ℎ𝑡−1] + 𝑏𝑐) (II.21)

𝐼𝑡 = (𝑊𝑖[ℎ𝑡−1, 𝑋𝑡] + 𝑏𝑖 ) (II.22)

[Link]. [Link] represents a LSTM Forget Gate.

With:

 tanh : Hyperbolic Tangent activation function

 𝐶 : A candidate value

These two results will be multiplied and added to the state. At this stage, the state of the cell
is:

49
Chapter II Machine learning and deep learning techniques

𝐶𝑡 = 𝑓𝑡 × 𝐶𝑡−1 + 𝑓𝑐 × 𝐼𝑡 (II.23)

[Link]. 21. Diagram represents an LSTM Forget Gate.

C- Output Gate

The output Gate will pass the previous hidden state and the current entry in a sigmoid
function

𝑜𝑡 = (𝑊𝑜[ℎ𝑡−1, 𝑋𝑡] + 𝑏𝑜) (II.24)

The new hidden state is, therefore:

ℎ𝑡 = 𝑜𝑡 × tanh(𝐶𝑡) (II.25)

[Link]. 22. Diagram represents an LSTM Output Gate.

50
Chapter II Machine learning and deep learning techniques

[Link]. [Link] represents a chain of three LSTM cells.

[Link] Algorithm

Algorithm: Operating algorithm of the doors according to three algorithms "if-then otherwise."
If Value Input " 0, then
Input Gate: Transmission of external information inside the block
If Value is Forgotten " 0, then
The previous state of the CEC affects the calculation of the present state
If Value Output Gate " 0, then
Input Gate: Transmission of Inside Information to the Outside of the Block
otherwise
Output Gate: No information provided on the exit
End if
otherwise
the cell forgets its past
end if
otherwise
Output Gate: Information blocked
End if
Table. II. [Link] Algorithm [41].

51
Chapter II Machine learning and deep learning techniques

[Link] of LSTM

 The constant error backpropagation within memory cells results in LSTM's ability to
bridge very long time lags in case of problems similar to those discussed above
 For long time lag problems such as those discussed in this paper, LSTM can handle
noise, distributed representations, and continuous values. In contrast to nite state
automata or hidden Markov models, LSTM does not require an a priori choice of a nite
number of states. In principle, it can deal with unlimited state numbers.
 For troubles mentioned in this paper, LSTM generalizes nicely even if the positions of
broadly separated, applicable inputs in the enter sequence do now not matter. Unlike
preceding approaches, ours shortly learns to distinguish between two or greater
extensively isolated occurrences of a specific component in an input sequence, besides
relying on excellent quick time lag training exemplars. There appears to be no need for
parameter no tuning. LSTM works well over a broad range of parameters such as
learning rate, input gate bias, and output gate bias. For instance, to some readers, the
learning rates used in our experiments may seem extensive. However, a large learning
rate pushes the output gates towards zero, thus automatically countermanding its
adverse effects [42].

V.4. Bidirectional Long Short-Term Memory (BLSTM)

The thought of Bi-directional Long Short-Term Memory (BLSTM) originates from

bidirectional RNN that approaches sequence facts in each frontward and backward
instructions the use of two excellent hidden layers. BLSTMs be a part of these hidden layers
with the equal output layer. One inadequacy of traditional RNNs is that they are only
successful in the use of the preceding context. BRNNs [42] repair this by way of doling out
records in both directions. A BLSTM network computes the ahead hidden layer sequence
output, the output sequence of the backward hidden layer, and the output layer y by reiterating
the ahead layer beginning t =1 to T, backward hidden layer because t = T to 1, and then the
final output is upgraded using the following equations:

ℎ𝑡→ = 𝐻(𝑊𝑥ℎ→ 𝑋𝑡 + 𝑊ℎ→ ℎ→ ℎ𝑡+1

→
+ 𝑏ℎ → ) (II.26)

52
Chapter II Machine learning and deep learning techniques

ℎ𝑡← = 𝐻(𝑊𝑥ℎ← 𝑋𝑡 + 𝑊ℎ← ℎ← ℎ𝑡+1

←
+ 𝑏ℎ ← ) (II.27)

𝑦𝑡 = 𝑊ℎ𝑦→ ℎ𝑡→ + 𝑊ℎ𝑦← ℎ𝑡← + 𝑏𝑦 (II.28)

Both the output of the ahead and backward layers are calculated by way of potential of
the preferred LSTM equations, Equations (II.27) - (II.28). The BLSTM layer produces an
output vector, 𝑦𝑡 , which is calculated via the equation:

𝑦𝑡 = 𝜎(ℎ𝑡→ , ℎ𝑡← ) (II.29)

Where 𝜎 function combines each the output sequences, the 𝜎 function could be of 4
kinds: concatenating, summation, common and multiplication function, and incorporating
BRNNs with LSTM neurons outcomes a bidirectional LSTM recurrent neural network
(BLSTM RNN). The BLSTM RNN is successful in gaining access to long term context
statistics in each backward and ahead direction. The aggregate of both the forward and
backward LSTM layers is regarded as a single BLSTM layer. It has been proven that the
bidirectional fashions are appreciably higher than regular unidirectional models in some
domains, like phoneme classification and speech recognition [42]. (Figure II.24) illustrates a
bidirectional LSTM shape with three consecutive time steps.

[Link]. [Link] BLSTM RNN structure with three consecutive time steps[42].

53
Chapter II Machine learning and deep learning techniques

[Link]

In this chapter we have presented the basic concepts of Machine learning and deep
learning including its operating principle, its main components and also its limitations. Then,
regarding machine learning, we talked about Random forest (RF) and Gradient boosting
Machine (GBM), about these methods and their use, and we mentioned the pros and cons of
both types.

Then we described a new variant of neural network called Deep Learning (DL). This
technique is characterized by its ability to solve the problem of the complexity of training
(NN) as well as its power to represent the forms (inputs) in a powerful, automatic and
discriminating way.

Finally, we discussed the different models of Deep Learning namely the CNN, the
LSTM, and BILSTM. The latter was very detailed because it will be the subject of several
experiments in the next chapter.

54
 CHAPTER III

Results and Discussion

Chapter III Results and Discussion

[Link]

This last chapter is devoted to a simulation study. It is mainly aimed at assessment of

the performance of the proposed techniques as a solution in the design of a solar radiation
prediction system in the future. So it is to validate and evaluate the performance of each of the
methods presented (RF-GBM - BILSTM-LSTM -DNN) in terms of learning time, and other
regression parameters are then presented. The main efficiency requirements are formulated on
the two essential points, namely, specification tests that verify that the program is performing
the task for which it was designed, and performance tests that will be used to measure the
effectiveness with which this task is completed. To carry out a comparative study.

II. Proposed System

All algorithms based on machine learning follow a predictive model that estimates a
certain type of data with high accuracy. A large data set is essential for the learning algorithm
to understand the behavior of the system. The first step for machine learning is data
acquisition. The collected data were shared by various interested parties and summarized in
useful information. The steps included in this process are data purification and data
delimitation. The data were separated into two disjoint sets, training, testing . The training
dataset was used for model training and testing. The dataset was used for model optimization
and evaluation.

Fig. III. [Link] matrix to identify the essential characteristics of the whole.

55
Chapter III Results and Discussion

Training Dataset: The sample of data used to fit the model.

Testing Dataset: The sample of data used to provide an unbiased evaluation of a final model
fit on the training dataset.

Dataset: A dataset consists of about two components, the two components are rows and
columns. In addition, a main feature of a record is that it is organized in such a way that each
row contains an observation.

Machine learning Algorithm : An “algorithm” in machine learning is a procedure that is run

on data to create a machine learning “model”.

Machine Learning Model : A “model” in machine learning is the output of a machine

learning algorithm run on data.

III. DESCRIPTION OF THE DATASET

NASA HI-SEAS missions serve as a testbed and training ground for humans as we
develop the capability to explore Mars. A recent NASA Space Apps Challenge hackathon
asked participants to use the data collected on the HI-SEAS site to predict solar radiation
given a set of measurable weather conditions. Knowing when the conditions are most
favorable to solar radiation incident is crucial in deciding when and where to deploy solar
energy recovery equipment, especially for settlers or astronauts on the surface of Mars.
These datasets are four-month weather data from the HI-SEAS weather station (September
to December 2016). For each dataset, the fields are:

A-line number (1-n) is useful to sort the results of this export The UNIX date time_t
(seconds since January 1, 1970). Useful for sorting the results of this export with other export
results Date in yyyy-mm-dd format Local time in hh: mm: ss Format 24 hours Digital data, if
applicable (maybe an empty string) Text data, if functional (can be an empty string).[44]

56
Chapter III Results and Discussion

The units of each dataset are:

Solar radiation: watts per meter^2 (W/m2) Temperature: degrees Fahrenheit (°F)
Humidity: percentage (%) Barometric pressure: (Hg)
Wind direction: degrees (°) Wind speed: miles per hour (mph)
Time at sunrise: Hawaii time Time at sunset: Hawaii time
Table. III. [Link] units of each dataset.

II .[Link]

At each timestamp of each day, there are values for all other variables. No other
variables affect time or date values. Therefore, the date and time of day are independent
variables.

For each date, there is a value for "Time at Sunrise" and "Time at Sunset." The
difference in these values gives the length of a given day, which is directly related to the
date. Further exploration of the dataset is required to determine whether the size of a given
date outweighs the amount of useful information it provides.

Temperature, pressure, and humidity do not directly affect each other significantly, but
since they are all properties that describe the local atmosphere, they do not vary independently
of each other. Similarly, these three variables have a strong relationship with the time of day.

[Link] Matrix

Fig. III. [Link] matrix to identify the essential characteristics of the whole.

57
Chapter III Results and Discussion

First, a fundamental correlation matrix is generated to remove irrelevant data and

identify the essential characteristics of the set.

[Link] Charts

Then, to better understand the data, hourly, and monthly averages of several variables
were viewed using bar charts.

Fig. III. 3. Bar charts to better understand the data.

IV. Preprocessing Dataset

Data preprocessing is a data mining technique in which raw data is converted into an
understandable format. The real data is often incomplete: missing attribute values, missing
specific attributes of interest, data preprocessing is a proven method of solving such
problems[45].

IV.1. Steps involved in data preprocessing

 Importing the required Libraries

 Importing the data set
 Handling the Missing Data.
 Encoding Categorical Data.

58
Chapter III Results and Discussion

 Splitting the data set into test set and training set.
 Feature Scaling.

IV.2. Data Standardization

The standardization of data helps to overcome the differences in the “standards” of

variables. Indeed, variables with large values may have a more significant Influence than
variables with small values, but may not be more significant. Normalization is a data pre-
processing tool used in the data mining system. Such as 0 to 1. Standardization is particularly
useful for classification and regression algorithms. To optimize the prediction models, we
introduce the normalization of the data by the min-max method. This technique provides a
linear transformation over the original data range. Rescheduling is often accomplished using a
linear interpretation formula, such as:

𝑥𝑖 −𝑚𝑖𝑛(𝑥)
𝑥𝑖𝑛 = (III.1)
𝑚𝑎𝑥(𝑥) −𝑚𝑖𝑛(𝑥)

Avec :

 𝑥𝑖𝑛 : The normalized value.

 𝑋 = [𝑥1 , 𝑥2 , … 𝑥𝑖 ]: The vector to normalized.
 𝑚𝑖𝑛(X): The minimum value of X.
 𝑚𝑎𝑥(X): The maximum value of X.

[Link] EVALUTION

[Link] ENVIRONMENT

[Link] Colab

Google Colab is a free cloud service that now supports free GPUs, improving coding
skills in the Python programming language. Develop in-depth learning applications using
popular libraries such as Keras, TensorFlow, PyTorch. The most important feature that
distinguishes Colab from other free cloud computing services is: Colab provides a GPU and is
entirely free [46].

59
Chapter III Results and Discussion

Fig. III. [Link] Colab Environment.

[Link]

Python is a high-level programming language used for general programming. Created by

Guido van Rossum and released in 1991, Python has a design philosophy that emphasizes the
readability of the code, including using virtual spaces. It provides constructions that allow
explicit programming at a small and large scale. Python has a dynamic gadget and automated
reminiscence management. It helps more than one programming paradigms, inclusive of
object-oriented, imperative, functional, and procedural, and has a complete general library.
Python interpreters are reachable for many running structures [47].

Fig. III. [Link] Python Logo

[Link] Notebook

The Jupyter Notebook is an open-source net utility that lets you create and share files
that comprise stay code, equations, visualizations, and narrative textual content. Uses include
data cleaning and transformation, numerical simulation, statistical modeling, data
visualization, machine learning, and much more [48].

60
Chapter III Results and Discussion

Fig. III. [Link] Jupyter Notebook logo.

V.1.4. Visual Studio Code

Is a source code editor developed by Microsoft for Windows, Linux and macOS. It
includes built-in Git and support for debugging, syntax highlighting, smart code completion,
snippets, and code refactoring. It is exceptionally customizable, permitting customers to
exchange the theme, keyboard shortcuts, preferences, and set up extensions that add extra
features. The supply code is free and open-source, launched beneath the permissive MIT
license. Compiled binaries are freeware for any purpose [49].

Fig. III. [Link] Studio Code Software Interface.

[Link] Presentation

[Link] Software

61
Chapter III Results and Discussion

 Tensorflow
TensorFlow was created by the Google Brain team to research ML and Deep Learning. It
is considered a modern version of Theano.

• Advantages

 supported by Google
 A very large community
 Multi-GPU support

• Disadvantages

 Slower than other frameworks in many benchmarks, although Tensorflow is

catching up.
 Theano still outperforms RNN support

 Keras

The highest level, the most user-friendly framework on the list. It allows users to choose
whether the models they build are running on Theano or TensorFlow.

• Advantages

 Python
 The perfect backend for Theano or TensorFlow
 High-level, intuitive interface

• Disadvantages

 Less flexible than other APIs

[Link]
Deep Learning and machine learning is an area with intense computational
requirements, and the availability of resources (especially in GPU) dedicated to this task will
fundamentally influence the user experience because, without its resources, it will take too
long to learn from one’s mistakes what can be discouraging. The experiments were all carried

62
Chapter III Results and Discussion

out on a machine that offers acceptable performance, the characteristics of which are as
follows:

CPU Intel Core i5-5200U @ 2.20GHz

GPU NVIDIA GeForce 820M GPU
RAM 8GB
Table. III. [Link] characteristics.

V.2. EVALUATION CRITERIA

[Link] Of Pre section Error

To make the analysis as rigorous as possible, the tests must be performed under the
same conditions whenever possible. All experiments were taken on the same computer in
order not to compromise the performance analysis. Each test was repeated four times. This is
because some results were not equal with each run of a simulation. In both methods, the
simulation time presents small related variations to obtain the performance error of each
prediction and to analyze each parameter and better, and it was necessary to use some
mathematical tools, pervasive in this case type of studies:

A - Mean Absolute Error (MAE)

The absolute mean error is a quantity often used to measure deviation from predictions
and measurements. Its formula is given by:

𝟏
𝐌𝐀𝐄 = 𝒏 ∑𝐧𝐣=𝟏|𝐲 − 𝐲𝐢 | (III.2)

 n = The number of errors

 Σ = Symbol of summation
 y = Measured solar irradiation
 yi = Solar irradiation estimated by a neural network.

B- Mean Square Error (MSE)

The mean square error is handy for comparing several estimators, especially when one
of them is biased; it is given by :

63
Chapter III Results and Discussion

𝟏
𝑴𝑺𝑬 = 𝒏 ∑𝒏𝒋=𝟏(𝒚 −𝒚𝒊 ) ² (III.3)

 n = The number of errors

 Σ = Symbol of summation
 y = Measured solar irradiation
 yi = Solar irradiation estimated by a neural network.

C- Root Mean Square Error (RMSE)

The RMSE is a measure of the variation of the predicted values around the measured
values. The smaller the value, the better the model, it is given by
𝟏
RMSE=√𝑴𝑺𝑬=√ ∑𝒏𝒋=𝟏(𝒚 −𝒚𝒊 ) ² (III.4)
𝒏

 n = The number of errors

 Σ = Symbol of summation
 y = Measured solar irradiation
 yi = Solar irradiation estimated by a neural network.

D- Relative Root Mean Square Error (RRMSE)

The percentage of RRMSE is obtained by dividing RMSE by the measured mean
values, which are defined by:

𝟏
√ ∑𝒏
𝒋=𝟏(𝒚−𝒚𝒊 )²
𝒏
RRMSE = 𝟏 𝒏 × 𝟏𝟎𝟎 (III.5)
∑ 𝒚
𝒏 𝒋=𝟏

Different ranges of RRMSE can be defined to show the capability of the models, so that
model accuracy is:

 Excellent for RRMSE 10%;

 Good for 10% RRMSE 20%;
 Fair for 20% RRMSE 30%;
 Low for RRMSE > 30%.

64
Chapter III Results and Discussion

E- Correlation Coefficient (R)

The correlation coefficient measures how close the predicted values are to the actual values.
Clearly, the value of the correlation coefficient more comparable to the unit implies a better
prediction.

𝒄𝒐𝒗(𝒚𝒊, 𝒚)
R= (III.6)
𝝈𝒚𝒊 𝝈𝒚

F- Coefficient Of Determination (R ²)

The coefficient of determination is a statistical measure, which indicates how the

regression line adjusts the actual data. A value of R² close to 1 means that the regression line
changes the data well.

This indicator varies between 0 and 1. A value of 1 indicates a perfect agreement

between measure and model, while the value 0 indicates a total disagreement.

𝟏
∑𝒏
𝒋=𝟏(𝒚−𝒚𝒊 )²
𝑹𝟐 = 𝟏 − 𝒏 𝟏 𝒏 (III.7)
∑ 𝒚
𝒏 𝒋=𝟏

V.3. SIMULATION RESULTS

[Link] Forest
 The hyperparameters of the random forest technique in 4 learning and testing time

Parameter Number Max Max Min MinSamples Test

cases estimators depth features samples Leaf size
split
𝟏𝒆𝒓 CAS 1640 None Auto 2 1 0.20
2é𝑚𝑒 CAS 840 None Auto 15 10 0.20
3é𝑚𝑒 CAS 200 64 Sqrt 2 1 0.33
4é𝑚𝑒 CAS 4000 None 0.5 2 1 0.33
Table. III. [Link] hyperparameters of the random forest technical dataset.

65
 Chapter III Results and Discussion

Random forest

ERROR LEARNING TEST

CAS MSE MAE R² RRMSE 𝑇𝑎𝑝𝑝 MSE MAE R² RRMSE 𝑇𝑡𝑒𝑠𝑡

𝟏𝒆𝒓 CAS 13.99 1.99 0.999 1.603 336 268.90 9.64 0.995 12.23 6.12

2é𝑚𝑒 CAS 88.69 4.92 0.999 4.177 149 364.81 11.22 0.993 14.25 1.98

3é𝑚𝑒 CAS 19.75 2.45 0.999 1.956 118 527.83 15.23 0.993 13.80 7.70

4é𝑚𝑒 CAS 13.69 2.08 0.999 1.62 351 315.35 11.34 0.995 10.66 15.64

Table. III. 4. Results were obtained from the random forest technical basis.

Next, we compared the actual outputs with the RF regression outputs in the learning and
testing base. We obtained the best results in the 1𝑠𝑡 the case according to the evaluation
criteria (R² =0.995, RRMSE=12.23, MAE = 9.64), Figure III.10-11 reports the correspondence
between the two test outputs in the 1𝑠𝑡 case

Fig. III. [Link] and calculated outputs for the random forest test.

66
Chapter III Results and Discussion

Fig. III. [Link] and calculated outputs for learning with random forest.

Fig. III. [Link] and calculated outputs for learning and testing with random forest.

[Link] Boosting Machine (GBM)

Gradient boosting regression hyperparameters in 4 learning and testing time

Parameter Number Learning Max Max Loss Test Samples Samples

estimators rate depth features size split leaf
cases

1𝑒𝑟 CAS 3000 0.01 None sqrt ls 0.20 2 1

2é𝑚𝑒 CAS 2000 0.1 64 0.5 ls 0.20 15 10
𝟑é𝒎𝒆 CAS 5000 0.1 128 0.5 huber 0.20 15 10
4é𝑚𝑒 CAS 4000 0.1 32 auto huber 0.33 15 10
Table. III. [Link] hyperparameters of the technical basis Gradient boosting regression.

67
 Chapter III Results and Discussion

Technique: Gradient boosting regression

ERROR LEARNING TEST

Cases MSE MAE R² RRMSE 𝑇𝑎𝑝𝑝 MSE MAE R² RRMSE 𝑇𝑡𝑒𝑠𝑡

1𝑒𝑟 CAS 0.79 0.71 0.999 0.39 1.34 368.98 12.27 0.993 14.33 45.36

2é𝑚𝑒 CAS 0.78 0.48 0.999 0.39 0.30 195.69 8.45 0.996 10.44 18.00

𝟑é𝒎𝒆 CAS 12.09 0.97 0.999 1.54 0.40 183.41 8.04 0.996 10.10 49.19

4é𝑚𝑒 CAS 15.14 1.04 0.999 1.71 0.33 269.61 9.83 0.996 9.86 41.69

Table. III. 6. Results were obtained from the technical basis of the gradient boosting
regression.

Then we compared the actual outputs with the GBR regression outputs in the learning and
testing base. We obtained the best results in the 3𝑟𝑑 the case according to the evaluation
criteria (R²=0.996, RRMSE=10.10, MAE= 8.04) Figure III.13-14 reports the correspondence
between the two test outputs in the 3𝑟𝑑 case

Fig. III. [Link] and calculated outputs for the test with Gradient boosting regression.

68
Chapter III Results and Discussion

Fig. III. [Link] and calculated outputs for learning with Gradient boosting regression.

Fig. III. [Link] and calculated outputs for learning and testing with GBR.

[Link] LSTM (BI-LSTM)

The hyperparameters of the BI-LSTM technique in 4 learning time and test

parameter Number Batch Number Batch optimizer Validation activation

cases Epoch size neurons size
1𝑒𝑟 CAS 500 32 150 0.15 Nadam True linear
𝟐é𝒎𝒆 CAS 430 64 100 0.33 Nadam True linear
3é𝑚𝑒 CAS 179 32 100 0.33 Nadam True linear
4é𝑚𝑒 CAS 235 128 200 0.33 Nadam True Relu
Table. III. [Link] of the BI-LSTM technical dataset.

69
 Chapter III Results and Discussion

BI-LSTM

Error LEARNING TEST

CASES MSE MAE R² RRMSE 𝑇𝑎𝑝𝑝 MSE MAE R² RRMSE 𝑇𝑡𝑒𝑠𝑡

1𝑒𝑟 CAS 2.88 1.53 0.999 0.757 3h.45 1.53 0.98 0.999 1.13 5.03
m

𝟐é𝒎𝒆 CAS 0.61 0.59 0.999 0.34 1h.24 0.85 0.74 0.999 0.69 3.13
m

3é𝑚𝑒 CAS 3.05 1.38 0.999 0.763 37m.6 2.39 1.17 0.999 0.92 3.05
s

4é𝑚𝑒 CAS 8.89 1.93 0.999 1.31 1h.35 17.93 2.88 0.999 2.54 11.96
m

Table. III. [Link] were obtained from the Bi-LSTM technical basis.

Then we compared the actual outputs with the Bi-Directional Long Short Term Memory
regression outputs in the learning and testing base. We obtained the best results in the 2𝑛𝑑 the
case according to the evaluation criteria (R²=0.999, RRMSE=0.67, MAE = 1.24) Figure
III.19-20 reports the correspondence between the two test outputs in the 2𝑛𝑑 case.

Fig. III. [Link] and calculated outputs for the test with BI-LSTM.

70
Chapter III Results and Discussion

Fig. III. [Link] and calculated outputs for learning with BI-LSTM.

Fig. III. [Link] and calculated outputs for learning and testing with BI-LSTM.

[Link] Neural Network (DNN)

Parameter Numbe Batc Number test Optim Lean Activa Dropou

cases r h neurons siz izer rates -tion t
Epoch size e rates
1er CAS 144 64 (64,128, 0.2 Adam 0.00 Relu 0.30
256,512,1) 5
2éme CAS 96 64 (64,128,256, 0.2 Adam 0.01 Relu 0.20
512,512,1024,1
)
3éme CAS 318 64 (64,128,256, 0.2 NAdam 0.00 Relu 0.30
512,1024,1) 5
𝟒é𝐦𝐞 CAS 281 128 (64,128,256, 0.2 NAda 0.00 linear 0.30
512,512,1) m 5

Table. III. [Link] hyperparameters of the Deep Neural Network technical dataset.

71
 Chapter III Results and Discussion

Technique: Deep Neural Network

Error LEARNING TEST

CASES MSE MAE R² RRMSE 𝑇𝑎𝑝𝑝 MSE MAE R² RRMSE 𝑇𝑡𝑒𝑠𝑡

1𝑒𝑟 CAS 20.326 3.48 0.999 1.99 175.2 16.13 3.05 0.999 2.99 1.88

2é𝑚𝑒 CAS 241.01 10.00 0.997 6.88 126.3 104.76 6.37 0.998 7.63 1.36

3é𝑚𝑒 CAS 24.635 3.852 0.999 2.20 256.3 16.10 3.62 0.999 2.99 2.35

𝟒é𝒎𝒆 CAS 8.00 2.24 0.999 1.25 235.3 6.01 1.99 0.999 1.83 2.23

Table. III. 10. Results were obtained from the Deep Neural Network technical basis.

Then we compared the actual outputs with the Deep Neural Network regression outputs in
the learning and testing base. We obtained the best results in the 4𝑡ℎ the case according to the
evaluation criteria (R²=0.999, RRMSE=2.21, MAE = 8.77) Figure III.22-23 reports the
correspondence between the two test outputs in the 4𝑡ℎ case.

Fig. III. [Link] and calculated outputs for the test with Deep Neural Network.

72
Chapter III Results and Discussion

Fig. III. [Link] and calculated outputs for learning with Deep Neural Network.

Fig. III. [Link] and calculated outputs for learning and testing with Deep Neural Network.

[Link] Short Term Memory (LSTM)

Parameter Number Batch Number test Optim- Valida- Activa- Dropout

cases Epoch size neurons size izer tion tion rates

1𝑒𝑟 CAS 74 32 [Link] 0.15 adam True Relu 0.2

2é𝑚𝑒 CAS 171 64 [Link] 0.20 adam True Relu 0.2

𝟑é𝒎𝒆 CAS 300 64 [Link] 0.20 Nadam True linear 0.2

4é𝑚𝑒 CAS 283 64 [Link] 0.20 Nadam True linear 0.3

Table. III. [Link] hyperparameters of the Long Short Term Memory technical dataset.

73
 Chapter III Results and Discussion

Technique: LSTM

Error LEARNING TEST

CASES MSE MAE R² RRMSE 𝑇𝑎𝑝𝑝 MSE MAE R² RRMSE 𝑇𝑡𝑒𝑠𝑡

1𝑒𝑟 CAS 187.47 7.94 0.998 6.07 49m.6 213.61 8.68 0.995 12.19 5.23
s

2é𝑚𝑒 CAS 144.67 7.72 0.998 5.33 1h.49 165.51 7.95 0.997 12.86 7.98
m

𝟑é𝒎𝒆 CAS 6.40 1.66 0.999 1.12 3h.22 9.11 2.17 0.999 2.25 7.82
m

4é𝑚𝑒 CAS 33.99 4.15 0.999 2.58 2h.02 37.65 4.42 0.999 4.57 5.78
m

Table. III. [Link] obtained from the Long Short Term Memory technical basis.

Next, we compared the actual outputs with the Long Short Term Memory regression outputs
in the learning and testing base. We obtained the best results in the 3𝑟𝑑 the case according to
the evaluation criteria (R²=0.999, RRMSE=2.65, MAE = 12.62) Figure III.25-26 reports the
correspondence between the two test outputs in the 3𝑟𝑑 case

Fig. III. [Link] and calculated outputs for the LSTM test.

74
 Chapter III Results and Discussion

Fig. III. [Link] and calculated outputs for learning with LSTM.

Fig. III. [Link] and calculated outputs for learning and testing with LSTM.

[Link] OF RESULTS

The table below makes a comparison of in terms of Training accuracy and Validation
accuracy with the same NASA HI-SEAS dataset.

comparison of the results

Error LEARNING TEST

Tech MSE MAE R² RRMSE 𝑇𝑎𝑝𝑝 MSE MAE R² RRMSE 𝑇𝑡𝑒𝑠𝑡

𝐿𝑆𝑇𝑀 6.40 1.66 0.999 1.12 3h.2m 9.11 2.17 0.999 2.25 7.82

𝐷𝑁𝑁 8.00 2.24 0.999 1.25 235.3 6.01 2.24 0.999 1.25 2.23

𝑩𝑰. 𝑳𝑺𝑻𝑴 0.61 0.59 0.999 0.34 1h.2m 0.85 0.74 0.999 0.69 3.13

𝐺𝐵𝑅 12.09 0.97 0.999 1.54 0.40 183.41 8.04 0.996 10.10 49.19

RF 13.99 1.99 0.999 1.603 336 268.90 9.64 0.995 12.23 6.12

Table. III. [Link] evaluation for different techniques.

75
Chapter III Results and Discussion

Based on the results obtained from Tables III.14, the following can be noted.

One of these results is that the BI-LSTM method One of the best ways used to predict solar
radiation using artificial intelligence

[Link]

Throughout this work, it was possible to learn more about artificial intelligence techniques,
in particular about the ML and DL models and how these models could be applied to solar
irradiance Prediction.

After this, two different algorithms were developed using the studied LSTM and DNN
methods to perform solar irradiance predictions. Before the final tests, the models were
adjusted to perform the best predictions.

76
Conclusion
 Conclusion
As we have seen in the introduction to this thesis , the Solar system sizing requires reliable,
comprehensive and long-term solar radiation data. To overcome the lack of long series of
reliable solar data, necessary for the optimization and optimal sizing of solar systems, an
approach to predict hourly global solar radiation from cheaper meteorological data was presented
in this manuscript. Ten different associations of five meteorological variables were used for
develop two types of ANN models. The model that gives the best performance is a neuronal
autoregressive with external BI-LSTM inputs with eight inputs. It was used to predict hourly
solar radiation from more available and cheaper weather data (Solar radiation, Humidity, Wind
direction, Time at sunrise, Temperature ,Barometric pressure ,Wind speed ,Time at sunset). The
prediction accuracy of the proposed model is about 0.64% with a R² value of 0.999. In addition,
different learning samples were used, the results showed that the proposed model required
concrete training beforehand in order to give an acceptable predictive accuracy.

Future research could include a model that re-evaluates the times for each region and updates the
input masks daily; such a model could improve the accuracy of the morning and dusk time
region, as changes in season and obstacles affect dawn and dusk.

In conclusion, our approach provides radiation series synthetic solar to be used in the optimal
sizing and planning of solar energy systems.

68
BIBLIOGRAPHY
 BIBLIOGRAPHY

[1] Renewables 2018. Power. Oct. 8, 2018. URL: [Link]

(visited on 08/08/2019).

[2] Xiangyang Yea, Qing T. Zenga,b , Julio C. Facellia , Diana I. Brixnerc , Mike Conwaya ,
Bruce E. Bray Predicting Optimal Hypertension Treatment Pathways Using Recurrent Neural
Networks. International Journal of Medical Informatics 2020.

[3] Chunchun Chena , Pu Zhang b , Yuan Liuc , Jun Liud, Financial quantitative investment
using convolutional neural network and deep learning technology, Neurocomputing ,2019

[4] Mohammad Mehedi Hassan , Abdu Gumaei , Ahmed Alsanad , Majed Alrubaian , Giancarlo
Fortino , A Hybrid Deep Learning Model for Efficient Intrusion Detection in Big Data
Environment , Information Sciences,2019

[5] A. Khosravi* , R.N.N. Koury, L. Machado, J.J.G. Pabon , Prediction of hourly solar radiation
in Abu Musa Island using machine learning algorithms , Journal of Cleaner Production ,2018

[6] Junho Lee, Wu Wang, Fouzi Harrou⁎ , Ying Sun , Reliable solar irradiance prediction using
ensemble learning-based models: A comparative study, Energy Conversion and Management,
2020

[7] Jianwu Zeng, Wei Qiao , Short-term solar power prediction using a support vector machine ,
Renewable Energy,2013

[8] Nor Azuana Ramli, Mohd Fairuz Abdul Hamid, Nurul Hanis Azhan, and Muhammad Alif
As-Siddiq Ishak , Solar power generation prediction by using k-nearest neighbor method , AIP
Conference Proceedings 2019

[9] Chao-Rong Chen and Unit Three Kartini , k-Nearest Neighbor Neural Network Models for
Very Short-Term Global Solar Irradiance Forecasting Based on Meteorological Data , 2017

[10] J Liu1 , M Y Cao2 , D Bai1 and R Zhang, Solar radiation prediction based on random forest
of feature-extraction ,2019
[11] JürgenSchmidhuber, Deep learning in neural networks: An overview, Neural Network ,2015

[12] S. Ghimire, R. C. Deo, N. Raj, J. Mi. Deep solar radiation forecasting with convolutional
neural network and long short-term memory network algorithms. Appl. Energy
2019;253:113541.

[13] Z. Pang, F. Niu, Z. O'Neill. Solar radiation prediction using recurrent neural network and
arti_cial neural network: A case study with comparisons. Renew. Energy 2020;156:279-289.

[14] D. Guijo-Rubio et al. Evolutionary artificial neural networks for accurate solar radiation
prediction. Energy 2020

[15] Qing X, Niu Y. Hourly day-ahead solar irradiance prediction using weather forecasts by
LSTM. Energy 2018;148:461–8.

[16] Kaba K, Sar‫ي‬gül M, Avc‫ ي‬M, Kand‫ي‬rmaz HM. Estimation of daily global solar radiation
using deep learning model. Energy 2018;162:126–35.

[17] Kira M. Sargent « Supporting Renewable Energy: Lessons from the Deer Island Treatment
Plant » Doctoral thesis, Washington University in St. Louis Environmental Studies Program,
Spring 2010 St. Louis, Missouri

[18] Alberto E. Gabás Royo « SOLAR IRRADIANCE FORECASTING USING NEURAL

NETWORKS », Faculty of Information Technology and Communication Sciences (ITC) Master
of Science Thesis September 2019

[19] M.A. Atwater, J.T. Ball, “A numerical solar radiation model based on standard
meteorological observations”, Sol. Energy 21, pp. 163–170, 1978

[20] TRAHI Fatiha « Prédiction de l’irradiation solaire globale pour la région de Tizi-Ouzou
par les réseaux de neurones artificiels. Application pour le dimensionnement d’une installation
photovoltaïque pour l’alimentation du laboratoire de recherche LAMPA. » MEMOIRE DE
MAGISTER EN ELECTRONIQUE , Université Mouloud Mammeri de Tizi-Ouzou
[21] Lila Croci « Gestion de l’énergie dans un système multi-sources photovoltaïque et éolien
avec stockage hybride batteries/supercondensateurs » Doctoral thesis, L’UNIVERSITE DE
POITIERS, Submitted on 7 Feb 2014

[22] Saad Motahhir, Abdelaziz El Ghzizal, Aziz Derouich « Modélisation et commande d’un
panneau photovoltaïque dans l’environnement PSIM » Doctoral thesis, Submitted on 19 Apr
2018

[23] Rene Y. Choi; Aaron S. Coyner; Jayashree Kalpathy-Cramer; Michael F. Chiang; J. Peter
Campbell, Introduction to Machine Learning, Neural Networks, and Deep Learning,
Translational Vision Science & Technology , February 2020

[24] Adrien BELLEVILLE « Prédiction des S/P individuels sur le produit flottes automobiles »
Mémoire présenté le : pour l’obtention du Diplôme Universitaire d’actuariat de l’ISFA et
l’admission à l’Institut des Actuaires

[25] Shalev-Shwartz, S. & Ben-David, S., “Understanding Machine Learning: From Theory to
Algorithms”, Cambridge University Press, 2014

[26] Morgane LAUR « Anticipation des changements de notes des obligations du portefeuille
d’un assureur par méthode de machine learning » Mémoire présenté devant l’Université Paris
Dauphine pour l’obtention du diplôme du Master Actuariat et l’admission à l’Institut des
Actuaires

[27] Classification And Regression Trees for Machine Learning. Jason Brownlee. April 8, 2016
URL:[Link]
learning/

[28] Alexey Natekin, Alois Knoll, Gradient boosting machines, a tutorial, fortiss GmbH,
Munich, Germany, NEUROROBOTICS, Gradient boosting machines, a tutorial,2013

[29] Complete Machine Learning Guide to Parameter Tuning in Gradient Boosting (GBM) in
Python, AARSHAY JAIN, FEBRUARY 21, 2016, URL :
[Link]
boosting-gbm-python/
[30] Sepp Hochreiter Fakultat “LONG SHORT-TERM MEMORY” Technische Universitat
Munchen 1997

[31] Mike Schuster and Kuldip K Paliwal, “Bidirectional recurrent neural networks,” Signal
Processing, IEEE Transactions, vol. 45, no. 11, pp. 2673–2681, 1997

[32] Moualek Djaloul Youcef « Deep Learning pour la classification des images » Mémoire de
fin d’études pour l’obtention du diplôme de Master en Informatique 2016

[33] Haohan Wang ,Bhiksha Raj “On the Origin of Deep Learning” Article · February 2017

[34] HAWKINS Jeff, BLAKESLEE Sandra. On intelligence : how a new understanding of the
brain will lead to the creation of truly intelligent machines. 2007

[35] Meriem Bahi , Mohamed Batouche , Deep Learning for Ligand-Based Virtual Screening in
Drug Discovery , University Constantine-2 Abdelhamid Mehri Constantine, Algeria ,2018

[36] Chigozie Enyinna Nwankpa, Winifred Ijomah, Anthony Gachagan, and Stephen Marshall ,
Activation Functions: Comparison of Trends in Practice and Research for Deep Learning ,2018

[37] Adam — latest trends in deep learning optimization. Vitaly Bushaev. Oct,22,2018. URL :
[Link]

[38] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep

convolutional neural networks,” in Advances in neural information processing systems, pp.
1097–1105, 2012.

[39] “Understanding LSTM Networks -- colah’s blog.” [Online]. Available:

[Link] [Accessed: 08-Jun-2018].

[40] Understanding LSTM Networks. Oinkina . Posted on August 27, 2015 . URL :
[Link]

[41] Huimei Han , Xingquan Zhu ,Ying Li , Generalizing Long Short-Term Memory Network
for Deep Learning from Generic Data , 2020
[42] Z. Cui, S. Member, R. Ke, S. Member, and Y. Wang, “Deep Stacked Bidirectional and
Unidirectional LSTM Recurrent Neural Network for Network-wide Traffic Speed Prediction,”
pp. 1–12, 2018.

[43] Daniel Durstewitz , Georgia Koppe, Andreas Meyer, Lindenberg Deep neural networks in
psychiatry,2019

[44] Solar Radiation Prediction .Task from NASA Hackathon .

URL:[Link]

[45] Data Preprocessing in Machine Learning Model . Jack Daniel .URL :

[Link]
3af34d0f3ceb

[46] Google Colab Free GPU Tutorial .Fuat . Jan 26, 2018 .URL : [Link]
learning-turkey/google-colab-free-gpu-tutorial-e113627b9f5d

[47] [Link]: [Link]

[48] Jupyter .URL : [Link]

[49] Visual Studio Code . URL : [Link]

 Abstract

This thesis presents a prediction study of the different components of the solar radiation using
artificial neural networks (ANN). The results of this study are crucial for the design and sizing of
any solar energy system. A series of experimental hourly measurements of year variables were
available for this study. Models (ANN) with different structures, in particular, different
combinations of inputs as well as other numbers of hidden neurons, have been set up. To
evaluate these models, the regression coefficient (R2) and the error estimators Relative Root
Mean Square Error (RRMSE) and Mean Square Error (MSE) were used. Random Forest (RF)
and Gradient Boosting Machine (GBM) and Deep Neural Network (DNN), Long Short Term
Memory (LSTM) were compared with Bidirectional LSTM to generate horizontal hourly global
solar radiation from less expensive exogenous variables. The results show BI-LSTM superiority
with 8 entries. The test of this model to produce accurate forecasts offers good accuracy
(R²=0.999, MSE = 0.85 and RRMSE = 0.69%). Using different sizes of the learning sample we
showed that from one year of data, our model gives satisfactory results. A comparison of our
results with the literature confirmed that our models (ANN) exceed other estimation methods and
that the proposed models ensure an authentic prediction of the different components of hourly
solar irradiation from endogenous and exogenous variables that are more available and less
expensive.

Keywords: ANN, solar power, forecasting, renewable energy, machine learning, deep learning.

Résumé
Cette mémoire présente une étude prédictive des différentes composantes de la rayonnement
solaire utilisant des réseaux neuronaux artificiels (RNN). Les résultats de cette étude sont
cruciaux pour la conception et le dimensionnement de tout système d’énergie solaire. Une série
de mesures horaires expérimentales de variables de huit ans était disponible pour cette étude. Des
modèles (RNN) avec différentes structures, en particulier, différentes combinaisons d’entrées
ainsi que différents nombres de neurones cachés ont été mis en place. Pour évaluer ces modèles,
on a utilisé le coefficient de régression (R2) et les estimateurs d’erreur Erreur quadratique
moyenne relative (RRMSE) et Erreur quadratique moyenne (MSE). On a comparé Random
Forest (RF) et Gradient Boosting Machine (GBM) et Deep neural Network (DNN), Long Short
Term Memory (LSTM) avec Bidirectional LSTM pour générer un rayonnement solaire mondial
horaire horizontal à partir de variables exogènes moins coûteuses. Les résultats montrent une
supériorité BI-LSTM avec 8 entrées. Le test de ce modèle pour produire des prévisions
authentiques montre une bonne précision (R²=0,999, MSE = 0,85 et RRMSE = 0,69 %). En
utilisant différentes tailles de l’échantillon d’apprentissage, nous avons montré qu’à partir d’une
année de données, notre modèle donne des résultats satisfaisants. Une comparaison de nos
résultats avec la littérature a confirmé que nos modèles (RNN) dépasser les autres méthodes
‫‪d’estimation et que les modèles proposés assurent une prédiction authentique des différentes‬‬
‫‪composantes de l’irradiation solaire horaire à partir de variables endogènes et exogènes plus‬‬
‫‪disponibles et moins coûteuses.‬‬

‫‪Mots-clés : ANN, énergie solaire, prévision, énergies renouvelables, apprentissage automatique,‬‬

‫‪apprentissage profond.‬‬

‫ملخص‬
‫تقدم هذه الرسالة دراسة تنبؤ للمكونات المختلفة لإلشعاع الشمسي باستخدام شبكات عصبية اصطناعية ‪ (RNA).‬إن نتائج هذه‬
‫الدراسة تشكل أهمية حاسمة في تصميم أي نظام للطاقة الشمسية وتحجيم حجمه ‪ .‬وقد توفرت لهذه الدراسة سلسلة من القياسات‬
‫التجريبية بالساعة لمتغيرات ثماني سنوات ‪.‬وتم إعداد نماذج )‪ (RNA‬ذات هياكل مختلفة‪ ،‬وخاصة تركيبات مختلفة من‬
‫المدخالت فضالً عن أعداد مختلفة من الخاليا العصبية المخفية ‪.‬لتقييم هذه النماذج‪ ،‬تم استخدام معامل االنحدار )‪ (R2‬ومقدرات‬
‫األخطاء النسبة للخطأ المتوسط الحسابي للجذر المتوسط )‪ (RMSE‬وخطأ مربع المتوسط ‪ (MSE).‬تمت مقارنة الغابة‬
‫العشوائية )‪ (RF‬وماكينة تعزيز التدرج )‪ (GBM‬والشبكة العصبية العميقة)‪ ، (DNN‬الذاكرة طويلة األجل )‪ (LSTM‬مع‬
‫تقنية ‪ LSTM‬ثنائية االتجاه لتوليد إشعاع شمسي عالمي بالساعة أفقي من المتغيرات الخارجية األقل تكلفة ‪.‬وتظهر النتائج تفوق‬
‫بي‪-‬إل تي سي مع ‪ 8‬مداخل ‪.‬إن اختبار هذا النموذج إلنتاج توقعات حقيقية يبين دقة جيدة‪،MSE = 0.85 ، (R²=0.999‬‬
‫‪RMSE = 0.69%).‬باستخدام أحجام مختلفة من عينة التعلم التي أظهرنا أنه من عام واحد من البيانات يقدم نموذجنا نتائج‬
‫مرضية ‪ .‬وقد أكدت مقارنة النتائج التي تحققناها بالمؤلفات أن نماذجنا تتجاوز أساليب التقدير األخرى وأن النماذج المقترحة‬
‫تضمن التنبؤ الحقيقي بالمكونات المختلفة لإلشعاع الشمسي بالساعة من المتغيرات الداخلية والخارجية األكثر توافرا ً وأقل تكلفة ‪.‬‬

‫الكلمات المفتاحية ‪ :‬شبكة العصبية اسطناعية ‪ ,‬الطاقة الشمسية ‪ ,‬الطاقات المتجددة ‪ ,‬تعلم اآللة ‪ ,‬تعلم العميق‬