Neural Network
Training Using
MATLAB
Phuong Ngo
School of Mechanical Engineering
Purdue University
Neural Network Toolbox
Available Models in MATLAB:
• Feedforward Neural Networks
• Adaptive Neural Network Filters
• Perceptron Neural Networks
• Radial Basis Neural Networks
• Probabilistic Neural Networks
• Generalized Regression Neural Networks
• Learning Vector Quantization (LVQ) Neural Networks
• Linear Neural Networks
• Hopfield Neural Network
ME697Y 2
Feedforward Neural Network
trainlm Levenberg-Marquardt
trainbr Bayesian Regularization
trainbfg BFGS Quasi-Newton
trainrp Resilient Backpropagation
trainscg Scaled Conjugate Gradient
traincgb Conjugate Gradient with
Powell/Beale Restarts
traincgf Fletcher-Powell Conjugate
Gradient
traincgp Polak-Ribiére Conjugate Gradient
trainoss One Step Secant
traingdx Variable Learning Rate Gradient
Descent
traingdm Gradient Descent with
Momentum
traingd Gradient Descent
ME697Y 3
Radial Basis Function Network
• Exact Design (newrbe)
This function can produce a network with zero error on
training vectors. It is called in the following way:
net = newrbe(P,T,SPREAD)
• More Efficient Design (newrb)
The function newrb iteratively creates a radial basis
network one neuron at a time. Neurons are added to the
network until the sum-squared error falls beneath an error goal
or a maximum number of neurons has been reached. The call
for this function is
net = newrb(P,T,GOAL,SPREAD)
ME697Y 4
Fuzzy Basis Function Network
(Not included in Neural
Network Toolbox)
• Backpropagation Algorithm
• Adaptive Least Square with Genetic Algorithm
ME697Y 5
Training Steps
• Generate training and checking data
• Select the structure of the neural network
• Perform the training
• Verify the error with checking data
ME697Y 6
Examples
2 x12 ( x2 1)2 x1 3 5 x12 x22 1 ( x11) 2 x22
F ( x1 , x2 ) 3(1 x1 ) e 10( x1 x2 )e e
5 3
3 x1 3, 3 x2 3
x1 x2
F ( x1 , x2 ) sin( x1 x2 )e , 3 x1 3, 3 x2 3
ME697Y 7
MATLAB Code (GenerateTrainingData)
function [x,Cx,d,Cd]=GenerateTrainingData(n,m,range)
% n - number of training samples
% m - number of checking samples
% range - zx2 range of input (z is the number of inputs)
% x - zxn matrix of training inputs
% Cx - zxm matrix of checking inputs
% d - 1xn matrix of training outputs
% Cd - 1xm matrix of checking outputs
if nargin < 3, error('Not enough input arguments'),end
[z,~] = size(range); % Obtain the number of system input
x = zeros(z,n);
Cx = zeros(z,m);
for i = 1:z
x(i,:) = (range(i,2)-range(i,1))*rand(1,n)+range(i,1)*ones(1,n); % Generate random
training inputs
Cx(i,:) = (range(i,2)-range(i,1))*rand(1,m)+range(i,1)*ones(1,m); % Generate random
checking inputs
end
d = zeros(1,n); % Define matrix d as an array of training outputs
for i = 1:n
d(i) = NonlinearFunction(x(:,i)); % Calculate d matrix
end
Cd = zeros(1,m); % Define matrix Cd as an array of checking outputs
for i = 1:m
Cd(i) = NonlinearFunction(Cx(:,i)); % Calculate Cd matrix
end
save('[Link]') % Save training data into file
ME697Y 8
MATLAB Code (Main Program)
addpath('./FBFN'); % add FBFN library
n = 900; % Define n as the number of training samples
m = 841; % Define m as the number of checking samples
InputRange = [-3 3; -3 3]; % Range of Input Signal
[x,Cx,d,Cd]=GenerateTrainingData(n,m,InputRange);
DP = [25,0,0]; % Specify the maxnimum number of fuzzy rules
warning('off');
[m_matrix,sigma_matrix,temp_w,NR,NDEI,CR] = adnfbf2(x,d,Cx,Cd,DP);
save('[Link]')
plot(1:length(NDEI),NDEI)
xlabel('Number of Fuzzy Rules');
ylabel('NDEI');
ME697Y 9
adnfbf2.m
% [fismat,NR,TR,CR] = ADNEWFBF(x,d,Cx,Cd,DP)
% x - nxN matrix of N input vectors.
% d - 1xN vector of N target outputs
% Cx - nxCN matrix of CN input vectors for checking.
% Cd - 1xCN vector of CN target outputs
% DP - Design parameters (optional).
% Returns:
% m_matrix,sigma_matrix,temp_w - parameters of fbfn found
% NR - the number of fuzzy basis functions used.
% training_error: NDEI
% TR - training record: [row of errors]
% CR - checking recored: [row of errors]
%
% Design parameters are:
% DP(1) - Maximum number of FBF(Ms), default = N.
% DP(2) - Root-sum-squared error goal, default = 0.0.
% DP(3) - Spread of pseudo-FBF(sigma), default = del_x/Ms
% Missing parameters and NaN's are replaced with defaults.
ME697Y 10
DEMO
ME697Y 11