Convergence Algorithm
Uploaded by
suganthiConvergence Algorithm
Uploaded by
suganthiConvergence Algorithm
Dr V Suganthi
Associate Professor
Department of Computer Science
Sri Ramakrishna College of Arts and
Science
For the perceptron to function properly, the two classes c1 and c2 must be
linearly separable.
This, in turn, means that the patterns to be classified must be sufficiently
separated from each other to ensure that the decision surface consists of a
hyperplane.
This requirement is illustrated in Fig. 1.4 for the case of a two-
dimensional perceptron.
In Fig. 1.4a, the two classes c1 and c2 are sufficiently separated from each
other for us to draw a hyperplane (in this case, a striaght line) as the
decision boundary.
If, however, the two classes c1 and c2 are allowed to move too close to
each other, as in Fig. 1.4b, they be-come nonlinearly separable, a situation
that is beyond the computing capability of the perceptron.
Suppose, for example, we are given a set of random variables and the as-
signed task is to find the relationships that may exist between them, if any. In regression,
which is a special form of function approximation, we typically find the following
scenario:
• One of the random variables is considered to be of particular interest; that random
variable is referred to as a dependent variable, or response.
• The remaining random variables are called independent variables, or regressors;
their role is to explain or predict the statistical behavior of the response.
• The dependence of the response on the regressors includes an additive error term,
to account for uncertainties in the manner in which this dependence is formulated;
the error term is called the expectational error, or explanational error, both of which
are used interchangeably.
Such a model is called the regression model.
There are two classes of regression models: linear and
nonlinear. In linear regression models, the dependence of the
response on the regressors is defined by a linear function,
which makes their statistical analysis mathematically tractable.
On the other hand,
in nonlinear regression models, this dependence is defined by
a nonlinear function, hence the mathematical difficulty in their
analysis.
LINEAR REGRSSION MODEL
PRELIMINARY CONDITIONS
• Consider the situation depicted in Fig. 2.1a, where an unknown
stochastic environment is the focus of attention. The environment is
probed by applying a set of inputs, constituting the regressor
where the superscript T denotes matrix transposition.
The resulting output of the environment, denoted by d, constitutes the corresponding
response, which is assumed to be scalar merely for the convenience of presentation.
Ordinarily, we do not know the functional dependence of the response d on the
regressor x, so we propose a linear regression model, parameterized as:
(2.2)
where w1, w2, ..., wM denote a set of fixed, but unknown, parameters, meaning that the
environment is stationary. The additive term ε, representing the expectational error of
the model, accounts for our ignorance about the environment. A signal-flow graph
depiction of the input–output behavior of the model described in Eq. (2.2) is presented
in Fig. 2.1b.
Using matrix notation, we may rewrite Eq. (2.2) in the compact form
(2.3)
where the regressor x is defined in terms of its elements in Eq. (2.1).
Correspondingly,