понедельник, 27 февраля 2017 г.
Multistep Neural Network Prediction example
Multistep Neural Network Prediction
https://www.mathworks.com/help/nnet/ug/multistep-neural-network-prediction.html
% Multistep Neural Network Prediction
% Set Up in Open-Loop Mode
[X,T] = maglev_dataset;
net = narxnet(1:2,1:2,10);
[x,xi,ai,t] = preparets(net,X,{},T);
net = train(net,x,t,xi,ai);
y = net(x,xi,ai);
view(net)
% Multistep Closed-Loop Prediction From Initial Conditions
% Close loop
netc = closeloop(net);
view(netc);
[x,xi,ai,t] = preparets(netc,X,{},T);
yc = netc(x,xi,ai);
plot(1:3999, cell2mat(t), 'b', 1: 3999, cell2mat(yc),'r');
% Multistep Closed-Loop Prediction Following Known Sequence
x1 = x(1:20);
t1 = t(1:20);
x2 = x(21:40);
% The open-loop neural network is then simulated on this data.
[x,xi,ai,t] = preparets(net,x1,{},t1);
[y1,xf,af] = net(x,xi,ai);
%Now the final input and layer states returned by the network are converted to closed-loop form along with the network.
% The final input states xf and layer states af of the open-loop network become the initial input states xi and layer states ai of the closed-loop network.
[netc,xi,ai] = closeloop(net,xf,af);
%Typically use preparets to define initial input and layer states.
% Since these have already been obtained from the end of the open-loop simulation,
% you do not need preparets to continue with the 20 step predictions of the closed-loop network
[y2,xf,af] = netc(x2,xi,ai);
% Note that you can set x2 to different sequences of inputs to test different scenarios for however many time steps you would like to make predictions.
% For example, to predict the magnetic levitation system's behavior if 10 random inputs are used:
x2 = num2cell(rand(1,10));
[y2,xf,af] = netc(x2,xi,ai);
% Following Closed-Loop Simulation with Open-Loop Simulation
[~,xi,ai] = openloop(netc,xf,af);
% Now you can define continuations of the external input and open-loop
% feedback, and simulate the open-loop network
x3 = num2cell(rand(2,10));
y3 = net(x3,xi,ai);
% Set Up in Open-Loop Mode
[X,T] = maglev_dataset;
net = narxnet(1:2,1:2,10);
[x,xi,ai,t] = preparets(net,X,{},T);
net = train(net,x,t,xi,ai);
y = net(x,xi,ai);
view(net)
% Multistep Closed-Loop Prediction From Initial Conditions
% Close loop
netc = closeloop(net);
view(netc);
[x,xi,ai,t] = preparets(netc,X,{},T);
yc = netc(x,xi,ai);
plot(1:3999, cell2mat(t), 'b', 1: 3999, cell2mat(yc),'r');
% Multistep Closed-Loop Prediction Following Known Sequence
x1 = x(1:20);
t1 = t(1:20);
x2 = x(21:40);
% The open-loop neural network is then simulated on this data.
[x,xi,ai,t] = preparets(net,x1,{},t1);
[y1,xf,af] = net(x,xi,ai);
%Now the final input and layer states returned by the network are converted to closed-loop form along with the network.
% The final input states xf and layer states af of the open-loop network become the initial input states xi and layer states ai of the closed-loop network.
[netc,xi,ai] = closeloop(net,xf,af);
%Typically use preparets to define initial input and layer states.
% Since these have already been obtained from the end of the open-loop simulation,
% you do not need preparets to continue with the 20 step predictions of the closed-loop network
[y2,xf,af] = netc(x2,xi,ai);
% Note that you can set x2 to different sequences of inputs to test different scenarios for however many time steps you would like to make predictions.
% For example, to predict the magnetic levitation system's behavior if 10 random inputs are used:
x2 = num2cell(rand(1,10));
[y2,xf,af] = netc(x2,xi,ai);
% Following Closed-Loop Simulation with Open-Loop Simulation
[~,xi,ai] = openloop(netc,xf,af);
% Now you can define continuations of the external input and open-loop
% feedback, and simulate the open-loop network
x3 = num2cell(rand(2,10));
y3 = net(x3,xi,ai);
Data Generator function
%% Data generator function
function [X,Xtrain,Ytrain,fig] = data_generator()
% data generator
X = 0.01:.01:10;
f = abs(besselj(2,X*7).*asind(X/2) + (X.^1.95)) + 2;
fig = figure;
plot(X,f,'b-')
hold on
grid on
% available data points
Ytrain = f + 5*(rand(1,length(f))-.5);
Xtrain = X([181:450 601:830]);
Ytrain = Ytrain([181:450 601:830]);
plot(Xtrain,Ytrain,'kx')
xlabel('x')
ylabel('y')
ylim([0 100])
legend('original function','available data','location','northwest')
function [X,Xtrain,Ytrain,fig] = data_generator()
% data generator
X = 0.01:.01:10;
f = abs(besselj(2,X*7).*asind(X/2) + (X.^1.95)) + 2;
fig = figure;
plot(X,f,'b-')
hold on
grid on
% available data points
Ytrain = f + 5*(rand(1,length(f))-.5);
Xtrain = X([181:450 601:830]);
Ytrain = Ytrain([181:450 601:830]);
plot(Xtrain,Ytrain,'kx')
xlabel('x')
ylabel('y')
ylim([0 100])
legend('original function','available data','location','northwest')
Simple Narx Example
1.
% Simple First Narx Net Example
load magdata
y = con2seq(y);
u = con2seq(u);
% y = Feedback Signal and Taget signal
sizey = size(y);
sizeu = size(u);
d1 = [1:2];
d2 = [1:2];
narx_net = narxnet(d1,d2,10);
narx_net.divideFcn = '';
narx_net.trainParam.min_grad = 1e-10;
[p,Pi,Ai,t] = preparets(narx_net, u, {}, y);
narx_net.view;
narx_net = train(narx_net,p,t,Pi);
% Simulation
yp = sim(narx_net, p, Pi);
e = cell2mat(yp)-cell2mat(t);
plot(e);
% closed loop
narx_net_closed = closeloop(narx_net);
view(narx_net_closed)
y1 = y(1700:2600);
u1 = u(1700:2600);
[p1,Pi1,Ai1,t1] = preparets(narx_net_closed,u1,{},y1);
yp1 = narx_net_closed(p1,Pi1,Ai1);
TS = size(t1,2);
plot(1:TS,cell2mat(t1),'b',1:TS,cell2mat(yp1),'r')
2. % Multiple External Variables in X cell two paramers
[X, T] = ph_dataset;
net = narxnet(10);
[x, xi,ai,t] = preparets(net,X,{},T);
net.view
net = train(net,x,t,xi,ai);
y=net(x,xi,ai);
e = gsubtract(t,y);
sz = size(t);
%plot(cell2mat(e));
plot(1:1991,cell2mat(t),'b',1:1991,cell2mat(y),'r');
воскресенье, 26 февраля 2017 г.
NN Generalization Overfitting, Normalization, Preparets
Improve Neural Network Generalization and Avoid Overfitting
https://www.mathworks.com/help/nnet/ug/improve-neural-network-generalization-and-avoid-overfitting.html
Preparets
https://www.mathworks.com/help/nnet/ref/preparets.html
NARX Reffs
Design Time Series NARX Feedback Neural Networks
https://www.mathworks.com/help/nnet/ug/design-time-series-narx-feedback-neural-networks.html?s_tid=gn_loc_drop
Modeling and Prediction with NARX and Time-Delay Networks
https://www.mathworks.com/help/nnet/modeling-and-prediction-with-narx-and-time-delay-networks.html
Multistep Neural Network Prediction
https://www.mathworks.com/help/nnet/ug/multistep-neural-network-prediction.html
https://en.wikipedia.org/wiki/Nonlinear_autoregressive_exogenous_model
https://www.researchgate.net/post/Can_anyone_help_regarding_NARX_network_in_network_timeseries_analysis_tool
http://www.wseas.us/e-library/transactions/research/2008/27-464.pdf
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1032.9079&rep=rep1&type=pdf
Improve Neural Network Generalization and Avoid Overfitting
https://www.mathworks.com/help/nnet/ug/improve-neural-network-generalization-and-avoid-overfitting.html
NARX
Design Time Series NARX Feedback Neural Networks
All the specific dynamic networks discussed so far have either been focused networks,
with the dynamics only at the input layer, or feedforward networks. The nonlinear
autoregressive network with exogenous inputs (NARX) is a recurrent dynamic network,
with feedback connections enclosing several layers of the network. The NARX model is
based on the linear ARX model, which is commonly used in time-series modeling.
The defining equation for the NARX model is
y(t) = f (y(t −1), y(t − 2), , y(t − ny ),u(t −1),u(t − 2), ,u(t − nu)) … …
where the next value of the dependent output signal y(t) is regressed on previous values
of the output signal and previous values of an independent (exogenous) input signal. You
can implement the NARX model by using a feedforward neural network to approximate
the function f. A diagram of the resulting network is shown below, where a two-layer
feedforward network is used for the approximation. This implementation also allows for a
vector ARX model, where the input and output can be multidimensional.
There are many applications for the NARX network. It can be used as a predictor, to
predict the next value of the input signal. It can also be used for nonlinear filtering, in
which the target output is a noise-free version of the input signal. The use of the NARX
network is shown in another important application, the modeling of nonlinear dynamic
systems.
Design Time Series NARX Feedback Neural Networks
3-23
Before showing the training of the NARX network, an important configuration that is
useful in training needs explanation. You can consider the output of the NARX network
to be an estimate of the output of some nonlinear dynamic system that you are trying
to model. The output is fed back to the input of the feedforward neural network as part
of the standard NARX architecture, as shown in the left figure below. Because the true
output is available during the training of the network, you could create a series-parallel
architecture (see [NaPa91]), in which the true output is used instead of feeding back the
estimated output, as shown in the right figure below. This has two advantages. The first
is that the input to the feedforward network is more accurate. The second is that the
resulting network has a purely feedforward architecture, and static backpropagation can
be used for training.
The following shows the use of the series-parallel architecture for training a NARX
network to model a dynamic system.
The example of the NARX network is the magnetic levitation system described beginning
in “Use the NARMA-L2 Controller Block” on page 4-18. The bottom graph in the
following figure shows the voltage applied to the electromagnet, and the top graph shows
the position of the permanent magnet. The data was collected at a sampling interval of
0.01 seconds to form two time series.
The goal is to develop a NARX model for this magnetic levitation system.
3 Dynamic Neural Networks
3-24
First, load the training data. Use tapped delay lines with two delays for both the input
and the output, so training begins with the third data point. There are two inputs to the
series-parallel network, the u(t) sequence and the y(t) sequence.
load magdata
y = con2seq(y);
u = con2seq(u);
Create the series-parallel NARX network using the function narxnet. Use 10 neurons in
the hidden layer and use trainlm for the training function, and then prepare the data
with preparets:
d1 = [1:2];
d2 = [1:2];
narx_net = narxnet(d1,d2,10);
narx_net.divideFcn = '';
narx_net.trainParam.min_grad = 1e-10;
[p,Pi,Ai,t] = preparets(narx_net,u,{},y);
(Notice that the y sequence is considered a feedback signal, which is an input that is
also an output (target). Later, when you close the loop, the appropriate output will be
connected to the appropriate input.) Now you are ready to train the network.
narx_net = train(narx_net,p,t,Pi);
Design Time Series NARX Feedback Neural Networks
3-25
You can now simulate the network and plot the resulting errors for the series-parallel
implementation.
yp = sim(narx_net,p,Pi);
e = cell2mat(yp)-cell2mat(t);
plot(e)
You can see that the errors are very small. However, because of the series-parallel
configuration, these are errors for only a one-step-ahead prediction. A more stringent test
would be to rearrange the network into the original parallel form (closed loop) and then
to perform an iterated prediction over many time steps. Now the parallel operation is
shown.
3 Dynamic Neural Networks
3-26
There is a toolbox function (closeloop) for converting NARX (and other) networks from
the series-parallel configuration (open loop), which is useful for training, to the parallel
configuration (closed loop), which is useful for multi-step-ahead prediction. The following
command illustrates how to convert the network that you just trained to parallel form:
narx_net_closed = closeloop(narx_net);
To see the differences between the two networks, you can use the view command:
view(narx_net)
view(narx_net_closed)
Design Time Series NARX Feedback Neural Networks
3-27
All of the training is done in open loop (also called series-parallel architecture), including
the validation and testing steps. The typical workflow is to fully create the network in
open loop, and only when it has been trained (which includes validation and testing
steps) is it transformed to closed loop for multistep-ahead prediction. Likewise, the R
values in the GUI are computed based on the open-loop training results.
You can now use the closed-loop (parallel) configuration to perform an iterated prediction
of 900 time steps. In this network you need to load the two initial inputs and the two
initial outputs as initial conditions. You can use the preparets function to prepare the
data. It will use the network structure to determine how to divide and shift the data
appropriately.
y1 = y(1700:2600);
u1 = u(1700:2600);
[p1,Pi1,Ai1,t1] = preparets(narx_net_closed,u1,{},y1);
yp1 = narx_net_closed(p1,Pi1,Ai1);
TS = size(t1,2);
plot(1:TS,cell2mat(t1),'b',1:TS,cell2mat(yp1),'r')
3 Dynamic Neural Networks
3-28
The figure illustrates the iterated prediction. The blue line is the actual position of the
magnet, and the red line is the position predicted by the NARX neural network. Even
though the network is predicting 900 time steps ahead, the prediction is very accurate.
In order for the parallel response (iterated prediction) to be accurate, it is important that
the network be trained so that the errors in the series-parallel configuration (one-stepahead
prediction) are very small.
You can also create a parallel (closed loop) NARX network, using the narxnet command
with the fourth input argument set to 'closed', and train that network directly.
Generally, the training takes longer, and the resulting performance is not as good as that
obtained with series-parallel training.
Design Time Series NARX Feedback Neural Networks
3-29
Each time a neural network is trained, can result in a different solution due to different
initial weight and bias values and different divisions of data into training, validation,
and test sets. As a result, different neural networks trained on the same problem can
give different outputs for the same input. To ensure that a neural network of good
accuracy has been found, retrain several times.
There are several other techniques for improving upon initial solutions if higher accuracy
is desired. For more information, see “Improve Neural Network Generalization and Avoid
Overfitting” on page 8-31.
Multiple External Variables
The maglev example showed how to model a time series with a single external input
value over time. But the NARX network will work for problems with multiple external
input elements and predict series with multiple elements. In these cases, the input and
target consist of row cell arrays representing time, but with each cell element being an Nby-
1 vector for the N elements of the input or target signal.
For example, here is a dataset which consists of 2-element external variables predicting a
1-element series.
[X,T] = ph_dataset;
The external inputs X are formatted as a row cell array of 2-element vectors, with each
vector representing acid and base solution flow. The targets represent the resulting pH of
the solution over time.
You can reformat your own multi-element series data from matrix form to neural
network time-series form with the function con2seq.
The process for training a network proceeds as it did above for the maglev problem.
net = narxnet(10);
[x,xi,ai,t] = preparets(net,X,{},T);
net = train(net,x,t,xi,ai);
y = net(x,xi,ai);
e = gsubtract(t,y);
To see examples of using NARX networks being applied in open-loop form, closedloop
form and open/closed-loop multistep prediction see “Multistep Neural Network
Prediction” on page 3-51.
All the specific dynamic networks discussed so far have either been focused networks,
with the dynamics only at the input layer, or feedforward networks. The nonlinear
autoregressive network with exogenous inputs (NARX) is a recurrent dynamic network,
with feedback connections enclosing several layers of the network. The NARX model is
based on the linear ARX model, which is commonly used in time-series modeling.
The defining equation for the NARX model is
y(t) = f (y(t −1), y(t − 2), , y(t − ny ),u(t −1),u(t − 2), ,u(t − nu)) … …
where the next value of the dependent output signal y(t) is regressed on previous values
of the output signal and previous values of an independent (exogenous) input signal. You
can implement the NARX model by using a feedforward neural network to approximate
the function f. A diagram of the resulting network is shown below, where a two-layer
feedforward network is used for the approximation. This implementation also allows for a
vector ARX model, where the input and output can be multidimensional.
There are many applications for the NARX network. It can be used as a predictor, to
predict the next value of the input signal. It can also be used for nonlinear filtering, in
which the target output is a noise-free version of the input signal. The use of the NARX
network is shown in another important application, the modeling of nonlinear dynamic
systems.
Design Time Series NARX Feedback Neural Networks
3-23
Before showing the training of the NARX network, an important configuration that is
useful in training needs explanation. You can consider the output of the NARX network
to be an estimate of the output of some nonlinear dynamic system that you are trying
to model. The output is fed back to the input of the feedforward neural network as part
of the standard NARX architecture, as shown in the left figure below. Because the true
output is available during the training of the network, you could create a series-parallel
architecture (see [NaPa91]), in which the true output is used instead of feeding back the
estimated output, as shown in the right figure below. This has two advantages. The first
is that the input to the feedforward network is more accurate. The second is that the
resulting network has a purely feedforward architecture, and static backpropagation can
be used for training.
The following shows the use of the series-parallel architecture for training a NARX
network to model a dynamic system.
The example of the NARX network is the magnetic levitation system described beginning
in “Use the NARMA-L2 Controller Block” on page 4-18. The bottom graph in the
following figure shows the voltage applied to the electromagnet, and the top graph shows
the position of the permanent magnet. The data was collected at a sampling interval of
0.01 seconds to form two time series.
The goal is to develop a NARX model for this magnetic levitation system.
3 Dynamic Neural Networks
3-24
First, load the training data. Use tapped delay lines with two delays for both the input
and the output, so training begins with the third data point. There are two inputs to the
series-parallel network, the u(t) sequence and the y(t) sequence.
load magdata
y = con2seq(y);
u = con2seq(u);
Create the series-parallel NARX network using the function narxnet. Use 10 neurons in
the hidden layer and use trainlm for the training function, and then prepare the data
with preparets:
d1 = [1:2];
d2 = [1:2];
narx_net = narxnet(d1,d2,10);
narx_net.divideFcn = '';
narx_net.trainParam.min_grad = 1e-10;
[p,Pi,Ai,t] = preparets(narx_net,u,{},y);
(Notice that the y sequence is considered a feedback signal, which is an input that is
also an output (target). Later, when you close the loop, the appropriate output will be
connected to the appropriate input.) Now you are ready to train the network.
narx_net = train(narx_net,p,t,Pi);
Design Time Series NARX Feedback Neural Networks
3-25
You can now simulate the network and plot the resulting errors for the series-parallel
implementation.
yp = sim(narx_net,p,Pi);
e = cell2mat(yp)-cell2mat(t);
plot(e)
You can see that the errors are very small. However, because of the series-parallel
configuration, these are errors for only a one-step-ahead prediction. A more stringent test
would be to rearrange the network into the original parallel form (closed loop) and then
to perform an iterated prediction over many time steps. Now the parallel operation is
shown.
3 Dynamic Neural Networks
3-26
There is a toolbox function (closeloop) for converting NARX (and other) networks from
the series-parallel configuration (open loop), which is useful for training, to the parallel
configuration (closed loop), which is useful for multi-step-ahead prediction. The following
command illustrates how to convert the network that you just trained to parallel form:
narx_net_closed = closeloop(narx_net);
To see the differences between the two networks, you can use the view command:
view(narx_net)
view(narx_net_closed)
Design Time Series NARX Feedback Neural Networks
3-27
All of the training is done in open loop (also called series-parallel architecture), including
the validation and testing steps. The typical workflow is to fully create the network in
open loop, and only when it has been trained (which includes validation and testing
steps) is it transformed to closed loop for multistep-ahead prediction. Likewise, the R
values in the GUI are computed based on the open-loop training results.
You can now use the closed-loop (parallel) configuration to perform an iterated prediction
of 900 time steps. In this network you need to load the two initial inputs and the two
initial outputs as initial conditions. You can use the preparets function to prepare the
data. It will use the network structure to determine how to divide and shift the data
appropriately.
y1 = y(1700:2600);
u1 = u(1700:2600);
[p1,Pi1,Ai1,t1] = preparets(narx_net_closed,u1,{},y1);
yp1 = narx_net_closed(p1,Pi1,Ai1);
TS = size(t1,2);
plot(1:TS,cell2mat(t1),'b',1:TS,cell2mat(yp1),'r')
3 Dynamic Neural Networks
3-28
The figure illustrates the iterated prediction. The blue line is the actual position of the
magnet, and the red line is the position predicted by the NARX neural network. Even
though the network is predicting 900 time steps ahead, the prediction is very accurate.
In order for the parallel response (iterated prediction) to be accurate, it is important that
the network be trained so that the errors in the series-parallel configuration (one-stepahead
prediction) are very small.
You can also create a parallel (closed loop) NARX network, using the narxnet command
with the fourth input argument set to 'closed', and train that network directly.
Generally, the training takes longer, and the resulting performance is not as good as that
obtained with series-parallel training.
Design Time Series NARX Feedback Neural Networks
3-29
Each time a neural network is trained, can result in a different solution due to different
initial weight and bias values and different divisions of data into training, validation,
and test sets. As a result, different neural networks trained on the same problem can
give different outputs for the same input. To ensure that a neural network of good
accuracy has been found, retrain several times.
There are several other techniques for improving upon initial solutions if higher accuracy
is desired. For more information, see “Improve Neural Network Generalization and Avoid
Overfitting” on page 8-31.
Multiple External Variables
The maglev example showed how to model a time series with a single external input
value over time. But the NARX network will work for problems with multiple external
input elements and predict series with multiple elements. In these cases, the input and
target consist of row cell arrays representing time, but with each cell element being an Nby-
1 vector for the N elements of the input or target signal.
For example, here is a dataset which consists of 2-element external variables predicting a
1-element series.
[X,T] = ph_dataset;
The external inputs X are formatted as a row cell array of 2-element vectors, with each
vector representing acid and base solution flow. The targets represent the resulting pH of
the solution over time.
You can reformat your own multi-element series data from matrix form to neural
network time-series form with the function con2seq.
The process for training a network proceeds as it did above for the maglev problem.
net = narxnet(10);
[x,xi,ai,t] = preparets(net,X,{},T);
net = train(net,x,t,xi,ai);
y = net(x,xi,ai);
e = gsubtract(t,y);
To see examples of using NARX networks being applied in open-loop form, closedloop
form and open/closed-loop multistep prediction see “Multistep Neural Network
Prediction” on page 3-51.
ML ntstool narx
1. NARX =
Nonlinear autoregressive with exogenous (external) input
y(t) = f(y(t – 1), ..., y(t – d), x(t – 1), ..., (t – d))
2. NAR =
Nonlinear autoregressive
y(t) = f(y(t – 1), ..., y(t – d))
3. NARX without Feedback as in p 1.
---------------------------------------------------------------------------------
Static, feedforward-dynamic, and recurrent-dynamic networks
1.Static:
p = {0 0 1 1 1 1 0 0 0 0 0 0};
stem(cell2mat(p))
net = linearlayer;
net.inputs{1}.size = 1;
net.layers{1}.dimensions = 1;
net.biasConnect = 0;
net.IW{1,1} = 2;
view(net)
Simulate:
a = net(p);
stem(cell2mat(a))
2. Dynamic
net = linearlayer([0 1]);
net.inputs{1}.size = 1;
net.layers{1}.dimensions = 1;
net.biasConnect = 0;
net.IW{1,1} = [1 1];
view(net)
a = net(p);
stem(cell2mat(a))
3. Recurrent-dynamic network
net = narxnet(0,1,[],'closed');
net.inputs{1}.size = 1;
net.layers{1}.dimensions = 1;
net.biasConnect = 0;
net.LW{1} = .5;
net.IW{1} = 1;
view(net)
a = net(p);
stem(cell2mat(a))
Notice that recurrent-dynamic networks typically have a longer response than
feedforward-dynamic networks. For linear networks, feedforward-dynamic networks
are called finite impulse response (FIR), because the response to an impulse input will
become zero after a finite amount of time. Linear recurrent-dynamic networks are called
infinite impulse response (IIR), because the response to an impulse can decay to zero (for
a stable network), but it will never become exactly equal to zero. An impulse response for
a nonlinear network cannot be defined, but the ideas of finite and infinite responses do
carry over.
Focused time-delay
neural network (FTDNN)
Notice that the input to the network is the same as the target.
ftdnn_net = timedelaynet([1:8],10);
ftdnn_net.trainParam.epochs = 1000;
ftdnn_net.divideFcn = '';
p = y(9:end);
t = y(9:end);
Pi=y(1:8);
ftdnn_net = train(ftdnn_net,p,t,Pi);
yp = ftdnn_net(p,Pi);
e = gsubtract(yp,t);
rmse = sqrt(mse(e))
linear: to Compare
lin_net = linearlayer([1:8]);
lin_net.trainFcn='trainlm';
[lin_net,tr] = train(lin_net,p,t,Pi);
lin_yp = lin_net(p,Pi);
lin_e = gsubtract(lin_yp,t);
lin_rmse = sqrt(mse(lin_e))
[X,Xi,Ai,T,EW,shift] = preparets(net,inputs,targets,feedback,EW)
y(t) = f (y(t −1), y(t − 2), , y(t − ny ),u(t −1),u(t − 2), ,u(t − nu))
Nonlinear autoregressive with exogenous (external) input
y(t) = f(y(t – 1), ..., y(t – d), x(t – 1), ..., (t – d))
2. NAR =
Nonlinear autoregressive
y(t) = f(y(t – 1), ..., y(t – d))
3. NARX without Feedback as in p 1.
---------------------------------------------------------------------------------
Static, feedforward-dynamic, and recurrent-dynamic networks
1.Static:
p = {0 0 1 1 1 1 0 0 0 0 0 0};
stem(cell2mat(p))
net = linearlayer;
net.inputs{1}.size = 1;
net.layers{1}.dimensions = 1;
net.biasConnect = 0;
net.IW{1,1} = 2;
view(net)
Simulate:
a = net(p);
stem(cell2mat(a))
2. Dynamic
net = linearlayer([0 1]);
net.inputs{1}.size = 1;
net.layers{1}.dimensions = 1;
net.biasConnect = 0;
net.IW{1,1} = [1 1];
view(net)
a = net(p);
stem(cell2mat(a))
3. Recurrent-dynamic network
net = narxnet(0,1,[],'closed');
net.inputs{1}.size = 1;
net.layers{1}.dimensions = 1;
net.biasConnect = 0;
net.LW{1} = .5;
net.IW{1} = 1;
view(net)
a = net(p);
stem(cell2mat(a))
Notice that recurrent-dynamic networks typically have a longer response than
feedforward-dynamic networks. For linear networks, feedforward-dynamic networks
are called finite impulse response (FIR), because the response to an impulse input will
become zero after a finite amount of time. Linear recurrent-dynamic networks are called
infinite impulse response (IIR), because the response to an impulse can decay to zero (for
a stable network), but it will never become exactly equal to zero. An impulse response for
a nonlinear network cannot be defined, but the ideas of finite and infinite responses do
carry over.
Focused time-delay
neural network (FTDNN)
Notice that the input to the network is the same as the target.
ftdnn_net = timedelaynet([1:8],10);
ftdnn_net.trainParam.epochs = 1000;
ftdnn_net.divideFcn = '';
p = y(9:end);
t = y(9:end);
Pi=y(1:8);
ftdnn_net = train(ftdnn_net,p,t,Pi);
yp = ftdnn_net(p,Pi);
e = gsubtract(yp,t);
rmse = sqrt(mse(e))
linear: to Compare
lin_net = linearlayer([1:8]);
lin_net.trainFcn='trainlm';
[lin_net,tr] = train(lin_net,p,t,Pi);
lin_yp = lin_net(p,Pi);
lin_e = gsubtract(lin_yp,t);
lin_rmse = sqrt(mse(lin_e))
If you have an application for a dynamic network, try the linear network first(linearlayer) and then the FTDNN (timedelaynet). If neither network issatisfactory, try one of the more complex dynamic networks discussed in the remainder ofthis topic.
Prepare
p = y(9:end);
t = y(9:end);
Pi = y(1:8);
Replace with
[p,Pi,Ai,t] = preparets(ftdnn_net,y,y);
[X,Xi,Ai,T,EW,shift] = preparets(net,inputs,targets,feedback,EW)
The input arguments for preparets are the network object (net), the external (nonfeedback)
input to the network (inputs), the non-feedback target (targets), the
feedback target (feedback), and the error weights (EW) (see “Train Neural Networks
with Error Weights” on page 3-43). The difference between external and feedback
signals will become clearer when the NARX network is described in “Design Time Series
NARX Feedback Neural Networks” on page 3-22. For the FTDNN network, there is
no feedback signal.
The return arguments for preparets are the time shift between network inputs and
outputs (shift), the network input for training and simulation (X), the initial inputs
(Xi) for loading the tapped delay lines for input weights, the initial layer outputs (Ai) for
loading the tapped delay lines for layer weights, the training targets (T), and the error
weights (EW).
Time Series Distributed Delay Neural Networks TDNN
time = 0:99;
y1 = sin(2*pi*time/10);
y2 = sin(2*pi*time/5);
y = [y1 y2 y1 y2];
t1 = ones(1,100);
t2 = -ones(1,100);
t = [t1 t2 t1 t2];
d1 = 0:4;
d2 = 0:3;
p = con2seq(y);
t = con2seq(t);
dtdnn_net = distdelaynet({d1,d2},5);
dtdnn_net.trainFcn = 'trainbr';
dtdnn_net.divideFcn = '';
dtdnn_net.trainParam.epochs = 100;
dtdnn_net = train(dtdnn_net,p,t);
yp = sim(dtdnn_net,p);
plotresponse(t,yp)
NARX is
nonlinear
autoregressive network with exogenous inputs (NARX) is a recurrent dynamic network,
with feedback connections enclosing several layers of the network. The NARX model is
based on the linear ARX model, which is commonly used in time-series modeling.
ML + C Sharp
https://www.codeproject.com/Articles/594636/Using-Matlab-from-a-Csharp-application
https://www.codeproject.com/Articles/4216/MATLAB-Engine-API
http://stackoverflow.com/questions/5901664/calling-a-matlab-function-from-c-sharp
MathWorks
https://www.mathworks.com/help/matlab/matlab_external/call-matlab-function-from-a-c-client.html
https://www.mathworks.com/help/matlab/matlab_external/call-matlab-function-from-c-client.html
https://www.mathworks.com/help/matlab/call-matlab-com-automation-server.html
пятница, 24 февраля 2017 г.
понедельник, 13 февраля 2017 г.
воскресенье, 12 февраля 2017 г.
Подписаться на:
Комментарии (Atom)