daily streamflow forecasting using artificial neural networks

DAILY STREAMFLOW FORECASTING USING
ARTIFICIAL NEURAL NETWORKS
Emrah DOĞAN
Research Assist., Sakarya University, Civil Engineering Department,
[email protected]
Sabahattin IŞIK
Assist. Prof. Sakarya University, Civil Engineering Department,
[email protected]
Tarık TOLUK
MSc. Civil Eng., University, Civil Engineering Department,
[email protected]
Mehmet SANDALCI
Assist. Prof. Sakarya University Civil Engineering Department,
[email protected]
ABSTRACT
Forecasting of streamflows is required for proper water resources planning and
management. This study presents the application and comparison of artificial neural
network (ANN) approaches and autoregressive (AR) method. ANN and AR(4)
methods are employed to predict daily streamflows at Çifteler station in the Sakarya
River. Three different ANN methods such as feed-forward backpropagation neural
networks (FFNN), radial basis neural networks (RBNN), and recurrent neural networks (RNN) are selected in modeling hydrological time-series and generating synthetic streamflows. Daily streamflows of Çifteler between 1989-1991 (1091 variables)
and between 1992-1993 (486 variables) were used for traning and test periods, respectively. Determination coefficients of AR(4), FFNN, RBNN, and RNN models were
found as 0.7547, 0.9495, 0.9479, and 0.9991, respectively. Finally, RNN model yields
the best result with a determination coefficient of 0.9991.
Keywords: Streamflow modelling, Autoregressive model, Artifical neural network
INTRODUCTION
Forecasting of streamflows are vital important for flood caution, operation of
flood-control-purposed reservoir, determination of river water potential, production
of hydroelectric energy, allocation of domestic and irrigation water in drought seasons, and navigation planning in rivers [Bayazıt, 1988].
449
RIVER BASIN FLOOD MANAGEMENT
Stochastic streamflow models are commonly used in hydrology. Recently, artifical neural network (ANN) models are also employed to water resources and hydrology problems [Gavin et. al., 2005]. A number of studies have been reported in literature. Some of them are given the below.
Oğuz [1983] developed a mathematical model that simulates movements of
yearly flows. Karabörk and Kahya [1998] obtained mathematical expressions of multivariate periodic autoregressive (PAR) and periodic autoregressive moving average
(PARMA) models for monthly streamflow observations of 12 stations located in the
Sakarya Basin. Jain and Srivastava [1999] used ANN methods to predict reservoir
inflows in resevoir operation. They compared ANN and ARIMA models, and concluded that ANN yielded better result. Zealand et. al. [1999] investigated the utility
of artificial neural networks (ANNs) for short term forecasting of streamflow. Birikundavyi et. al. (2002) investigated the performance of ANN methods in prediction
of daily streamflows. It is shown that ANN method yielded better results than
ARMA models. Cigizoglu [2003] incorporated ARMA models into flow forecasting
by artificial neural networks to overcome the limitation of the data.
Kumar et. al. [2004] employed RNN model in streamflows forecasting. Kişi
[2004] investigated the application of artificial neural networks (ANNs) in predicting
mean monthly streamflow and compared with AR models. Huang et. al. [2004] compared ANN and ARIMA models in streamflow forecasting.
In this study, a stochastic model, autoregressive AR(4), ANN methods, feedforward backpropagation neural networks (FFNN), radial basis neural networks
(RBNN), and recurrent neural networks (RNN) were used to forecast streamflows
and compared with each other. The models were applied to daily streamflows between 1989-1993 at Çifteler station in the Sakarya River. The models were trained for
1989-1991 daily streamflows (1091 variables) and tested for 1992-1993 daily streamflows (486 variables).
STREAMFLOW FORECASTING MODELS
AR(p) Model
Time series models are used to forecast streamflows in hydrology. General equation of AR (autoregressive) model can be written as below [Bayazıt,1998].
p
yi = Σφ j y j −1 + ε i
j =1
(1)
450
INTERNATIONAL CONGRESS ON RIVER BASIN MANAGEMENT
φ: regression coefficients of model,
ε: independent variable.
AR(p) model can be given as the following matrix.
 ρ1   1
ρ   ρ
 2  1
 .   .
 =
 .   .
 .   .
  
 ρ p   ρ p −1
[ρ ] = [
ρ1
1
.
.
.
1
.
. ρ p −1   φ1 
. ρ p − 2   φ2 

.
.  
x

.
.  


1 ρ1  
 

. .
1  φ p −1 
.
.
.
1
P
(2)
] x [φ ]
φ = P −1ρ
(3)
φ values are obtained from eq. (3) and by subsituting the following equation, general
equation of model can be obtained [Haan, 2002].
p
yi = Σ φ j yi − j + ε i = φ1 yi −1 + φ2 yi − 2 + ........ + φ p yi − p + ε i
(4)
j =1
Basic Principles of the Neural Networks
Artificial Neural Networks (ANNs) consist of large number of processing elements with their interconnections. ANNs are basically parallel computing systems
similar to biological neural networks. They can be characterized by three components:
♦ Nodes
♦ weights (connection strength)
♦ An activation (transfer) function
ANN modeling is a nonlinear statistical technique; it can be used to solve problems that are not amenable to conventional statistical and mathematical methods. In
the past few years there has been constantly increasing interest in neural networks
modeling in different fields of hydrology engineering [ASCE, 2000].
451
RIVER BASIN FLOOD MANAGEMENT
The basic unit in the artificial neural network is the node. Nodes are connected
to each other by links known as synapses, associated with each synapse there is a
weight factor. Usually neural networks are trained so that a particular set of inputs
produces, as nearly as possible, a specific set of target outputs.
Feed-Forward Backpropagation Neural Networks (FFNN)
The most commonly used ANN is the three-layer feed-forward ANN. In feedforward neural networks architecture, there are layers and nodes at each layer. Each
node at input and inner layers receives input values, processes and passes to the next
layer. This process is conducted by weights. Weight is the connection strength between two nodes. The numbers of neurons in the input layer and the output layer are
determined by the numbers of input and output parameters, respectively. In the
present feed-forward artificial neural networks are used. The model is shown in
Figure 1.
In the Figure 1, i, j, k denote nodes input layer, hidden layer and output layer,
respectively. w is the weight of the nodes. Subscripts specify the connections between the nodes. For example, wij is the weight between nodes i and j. The term
ʺfeed-forwardʺ means that a node connection only exists from a node in the input
layer to other nodes in the hidden layer or from a node in the hidden layer to nodes
in the output layer; and the nodes within a layer are not interconnected to each other.
i
wij
j
wjk
Input Layer
Hidden Layer
k
Output Layer
Fig. 1 A typical three-layer feed forward ANN
Radial Basis Neural Networks (RBNN)
RBNN were introduced into the neural network literature by Broomhead and
Lowe [1988]. Radial basis functions (RBF) are powerful techniques for interpolation
in multidimensional space. An RBF is a function which has built into it a distance
452
INTERNATIONAL CONGRESS ON RIVER BASIN MANAGEMENT
criterion with respect to a center. Such functions can be used very efficiently for interpolation and for smoothing of data. Radial basis functions have been applied in
the area of neural networks where they are used as a replacement for the sigmoidal
transfer function. Such networks have 3 layers, the input layer, the hidden layer with
the RBF non-linearity and a linear output layer. The most popular choice for the nonlinearity is the Gaussian. RBF networks have the advantage of not being locked into
local minima as do feed-forward networks. The basis functions in the hidden layer
produce a significant non-zero response to input stimulus only when the input falls
within a small localized region of the input space. Hence, this paradigm is also
known as a localized receptive field network [Lee and Chang, 2003]. The type of
input transformation of the RBNN is the local nonlinear projection using a radial
fixed shape basis function. After nonlinearly squashing the multi-dimensional inputs
without considering the output space, the radial basis functions play a role of regressors. Since the output layer implements a linear regressor the only adjustable parameters are the weights of this regressor. These parameters can therefore be determined using the linear least square method, which gives an important advantage for
convergence. In this study, different numbers of iterations and spread constants are
examined for the RBNN models with a simple trial-error method adding some loops
to the program codes.
Recurrent neural networks (RNN)
Forecasting of hydrologic time series is based on the previous values of the series depending on the number of persistence components. Recurrent neural networks
(RNN) are networks that include feedback connections in addition to the feedforward connections commonly used in artificial neural networks. In general, an
RNN includes an input layer, an output layer, and hidden layers. Several types of
RNN architectures have been proposed for modelling complex time-dependent phenomena [Williams and Zipser, 1989; Haykin, 1998]. The RNN used in this study is
the Elman RNN [Elman, 1990], which has feedback connections from its hidden layer
neurons back to its inputs. This is a discrete-time recurrent two-layer network with
feedback loops that allow for adaptability and non-linearity. The temporal representation capabilities of the RNN are better than those of purely feed-forward networks,
even those with tapped-delay lines [Saad et al., 1998]. An important step in designing
models driven by neural networks is the selection of the number of hidden neurons.
Because the target function is unknown, it is difficult to predict in advance what the
optimal network size should be. The appropriate network should neither overfit nor
underfit the data. In order to develop the optimum network model, many networks
are trained.
453
RIVER BASIN FLOOD MANAGEMENT
APPLICATION OF MODELS
Definition of Study Area
Daily streamflows between 1989-1993 at Çifteler river gauging station in the Sakarya River are used in this study. Daily streamflow data are obtained from Electrical
Power Resources Survey and Development Administration (EIE) [Toluk, 2006]. Çifteler river gauging station is close to Aktaş village which is locacted on 25 km southwest of Çifteler County in Eskişehir Province. Statistical variables of Çifteler stations
are given in Table 1.
Table 1. Statistical variables of Çifteler stations (m3/s)
Çifteler
Average
Standart Deviation
Skewness
Maximum
General Data
5.22
2.0008
-0.2914
10.90
1.39
Training Data
5.12
2.0721
-0.1679
10.90
1.39
Test Data
5.44
1.8433
-0.6170
8.92
1.56
Minimum
Average and standart deviation of general data are 5.22 and 2.000; streamflows
vary from 10.90 to 1.39. Average and standart deviation of traning data are 5.12 and
2.0721; streamflows vary from 10.90 to 1.39. Average and standart deviation of test
data are 5.44 and 1.8433; streamflows vary from 8.92 to 1.56.
Application of AR(p) Model
AR(p) model was applied to daily streamflows of Çifteler station. The application of AR model was performed by using Microsoft EXCEL. Since time lag was
taken as 4 days in ANN models and data were delayed 4 days, p was also taken as 4
days AR(p) application. Correlation coefficients, ρ1, ρ2, ρ3, ρ4, are given in Table 2.
Table 2. Correlation coefficients
Çifteler
ρ1
0.973129
ρ2
0.934881
ρ3
0.891842
ρ4
0.8500755
If ρ coefficients are subsituted in eq.(2), φ coefficients can be calculated from
eq.(3) as in Table 3.
454
INTERNATIONAL CONGRESS ON RIVER BASIN MANAGEMENT
Tablo 3. φ Coefficients
Çifteler
φ
0.85027
φ2
0.195
φ3
0.19751
φ4
-0.28206
If obtained φ coefficents are subsituted in eq.(4), AR(4) equation of Çifteler stations can be determined as:
yi=0.85027yi-1 + 0.195yi-2 + 0.19751yi-3 - 0.28206yi-4 + εi
Synthetic time series are generated by using this equation. Generated values and
test values were compared and given in Table 4. Some iterations of trials for AR(4)
model are given in Table 4. The best results were obtained in 4. trial with a determination coefficient (R2) of 0.759 and in 5. trial with mean square error (MSE) and average absolute error (AARE) of 1.091 and 17.71. Finally, values obtained in 5. trial were
concluded as the best result for AR(4) model.
Table 4. AR(4) Results
Iteration No :
1
2
3
4
5
R2
0.700
0.743
0.756
0.759
0.754
MSE
1.28161
1.16942
1.15801
1.10927
1.09096
AARE
18.042
19.037
18.804
18.235
17.715
Application of Feed-Forward Backpropagation Neural Networks
In this study, before the training of the network both input and output variables
were normalized within the range 0.1 to 0.9 as follows:
xi = 0.8
( x − xmin )
( xmax − xmin )
+ 0.1
(5)
where xi is the normalized value of a certain parameter, x is the measured value for
this parameter, xmin and xmax are the minimum and maximum values in the database
for this parameter, respectively.
Networks are sensitive to the number of nodes in their hidden layers. Too few
nodes can lead to underfitting and too many nodes can result in overfitting. In order
to reach an optimum amount of hidden layer nodes, 2, 3, 5, 10 nodes are tested.
Within this range, an FFNN model, having 4 inputs and two hidden layers with 2
nodes and 5000 iteration number, gives the best choice.
455
RIVER BASIN FLOOD MANAGEMENT
Application of Radial Basis Neural Networks
In this study, supervised learning algorithm was used. This algorithm has ability
to produce processor components. To develop RBNN model nerb function was used
via “MATLAB” software [MATLAB, 2004]. Different numbers of hidden layer neurons and spread constants are examined for obtaining an appropriate RBNN model.
After trial and error processes, RBNN model, having 4 inputs and 6 spread constants
200 iteration number, gives the best choice. The determination coefficient (R2), average absolute error (AARE) and mean square error (MSE) values of each RBNN in test
period are given in Table 5.
Table 5. RBNN results for the test period
200
Iteration Number
0.1
200
200
0.2
0.3
200
200
200
200
0.4
1
2
5
R2
0.5662
0.14831 0.02476
0.02135
0.04283
0.4549
0.9324
AARE(%)
13.4358 19.0767 32.4529
44.3932
18.3271
8.6812
5.3026
MSE
1.9277
12.5741 108.422 412.72356 41.8262 3.56819
0.23855
Spread Constant
200
Iteration Number
200
200
200
200
200
10
15
200
6
7
8
9
R2
0.9479*
0.9447
0.9464
0.9472
0.94768 0.94608
AARE(%)
4.97414 4.94873 4.95237
4.93795
4.94347 4.88147* 5.04109
MSE
0.19198* 0.19952 0.19411
0.19212
0.19241 0.194418 0.198954
Spread Constant
20
0.94561
Application of Recurrent Neural Networks
After the trial and error processes an RNN model, having 4 inputs and one hidden layers with 1 nodes and 10000 iteration number, gives the best choice. The results are shown in Table 6.
Table 6. RNN results for the test period
Trial
number
Input Number
1
2
3
4
4
4
4
4
Hidden
Layer
Nodes
1
1
1
1
Output
Number
Iteration
Number
1
1
1
1
10000
12000
15000
8000
R2
AARE
MSE
0.9996* 0.003142* 0.03484*
0.9707 0.101512 0.20159
0.9641 0.122542 0.22653
0.9784 0.101807 0.20214
It appears that while assessing the performance of any model for its applicability
in forecasting flow discharges, it is not only important to evaluate the average prediction error but also the distribution of prediction errors. The statistical performance
evaluation criteria employed so far in this study are global statistics (R2 and MSE)
456
INTERNATIONAL CONGRESS ON RIVER BASIN MANAGEMENT
and do not provide any information on the distribution of errors [Dogan et al., 2005].
Therefore, in order to test the robustness of the model developed, it is important to
test the model using some other performance evaluation criteria such as average
absolute relative error (AARE) and threshold statistics (the error percentage which is
less than 10 %) (TS10). The AARE and TS10 not only give the performance index in
terms of predicting flow discharges but also show the distribution of the prediction
errors.
After training the all of the neural network models, test performances were
checked. The performance of neural network models for prediction of flow discharge
is demonstrated in Figure 2 in the form of hydrograph and scatterplot. Figure 2 also
shows an analysis between the network outputs (estimations) and the corresponding
targets (observed data) for the test dataset. It is obvious that the predicted values
trained by the RNN catch the targets very well.
It is seen from the hydrographs that the RBNN and FFNN also estimate closely
follow the observed values. The underestimations and overestimations are obviously
seen for the AR model. This is also confirmed by the scatterplots. As seen from the fit
line equations and R2 values in scatterplots, the estimates of all the neural network
models are closer to the exact fit line (y=x line) than those of the AR.
RIVER BASIN FLOOD MANAGEMENT
457
Fig.2 Comparison Neural Network Models Results with AR
The comparison of models is shown in Table 7 in terms of the R2, MSE, AARE
and TS10 statistics in test period. Table 7 indicates that the RNN model has the lowest
MSE and AARE values while has the highest R2 and TS10 values.
458
INTERNATIONAL CONGRESS ON RIVER BASIN MANAGEMENT
Table 7. The comparison of models in test period
Models
AR(4)
FFNN
RBNN
RNN
R2
0.7547
0.9495
0.9479
0.9996
Performance of the Models
MSE
AARE
1.09096
17.715
0.1829
4.8088
0.19198
4.97414
0.0033142
0.03484
TS10
17.71
38.06
38.27
43.62
RESULTS
In this study, AR(4), feed-forward backpropagation neural networks (FFNN),
radial basis neural networks (RBNN), and recurrent neural networks (RNN) were
used to forecast streamflows and compared with each other. Models were applied to
daily streamflows at Çifteler river gauging stations in the Sakarya River. It is found
that the performances of ANN models are better than AR(4) model. Determination
coefficients of AR(4), FFNN, RBNN, and RNN models were found as 0.7547, 0.9495,
0.9479, and 0.9991, respectively. Finally, RNN model yields the best result with a
determination coefficient and a mean square error of 0.9991 and 0.0033142.
REFERENCES
ASCE Task Committee (2000). Artificial Neural Networks in Hydrology. I: Preliminary
concepts.’’ J. of Hydrologic Engineering, ASCE, 5(2), 115–123.
Bayazıt M. (1988). Hidrolojik Modeller, İ.T.Ü. rektörlüğü, İstanbul.
Birikundavyi S., Labib R., Trung H., and Rousselle J. (2002). Performance of Neural
Networks in Daily Streamflow Forecasting, J. of Hydrologic Engineering, 7, 5, 392398.
Broomhead, D. and Lowe, D. (1988). Multivariable functional interpolation and adaptive
networks. Complex Syst. 2, 321–355.
Cigizoğlu H., (2003). Incorporation of ARMA Models Into Flow Forecasting by Artificial
Neural Networks, Environmetrics, 14, 4, 417-427.
Dogan E, Sasal M, and Isik S. (2005). Suspended Sediment Load Estimation in Lower
Sakarya River by Using Soft Computational Methods. Proceeding of the International Conference on Computational and Mathematical Methods in Science and
Engineering, CMMSE 2005, Alicante, Spain, 395-406.
Elman, J.L. (1990). Finding structure in time. Cognitive Science 14: 179–211.
Gavin, B., Graeme, D., Holger, M. (2005). Input Determination For Neural Network
Models in Water Resources Applications, Part1-background and methodology,
Journal of Hydrology, 301,1, 75-92.
Haan T. (2002). Statistical Methods in Hydrology, Lowa State Pres.
Haykin, S. (1998). Neural Networks - A Comprehensive Foundation (2nd. ed.). PrenticeHall, Upper Saddle River, NJ.
RIVER BASIN FLOOD MANAGEMENT
459
Huang W., Bing Xu B., and Hilton A., (2004). Forecasting Flows in Apalachicola River
Using Neural Networks, Hydrological Processes, 18, 2545-2564.
Jain S., Das A., and Srivastava D. (1999). Applicationof ANN For Reservoir In Flow
Prediction and Operation, Journal of WaterResources Planning and Management,
125, 5, 263-271.
Karabörk M., Kahya E. (1998). Sakarya Havzasındaki Aylık Akımların Çok Değişkenli
Stokastik Modellenmesi, Tr Journal of Engineering and Environmental Science, 23,
133-147.
Kişi Ö. (2004). River Flow Modelling Using Artificial Neural Networks, J. of Hydrologic
Engineering, 9, 1, 60-63.
Kumar D., Raju K., and Sathish T. (2004). River Flow Forecasting Using Recurrent Neural Networks, Water Resources Management, Kluwer Academic Publishers, 18, 143161.
Lee, G.C. and Chang, S.H. (2003). Radial basis function networks applied to DNBR
calculation in digital core protection systems. Annals of Nuclear Energy, 30, 15611572.
MATLAB, (2004). Documentation Neural Network Toolbox Help, Version 7.0, Release
14, The MathWorks, Inc.
Oğuz B., (1983). Yıllık Akımların Gidiş Özelliklerini Benzeştiren Bir Matemetik Modelin
Araştırılması, Doktora Tez, İ.T.Ü. İnşaat Fakültesi Matbası, İstanbul.
Saad E.W., Prokhorov, D.V., and Wunsch, D.C. (1998). Comparative study of stock
trend prediction using time delay, recurrent and probabilistic neural networks, IEEE
Transactions on Neural Networks, 9(6): 1456–1470.
Toluk, T. (2006). Akarsu Akımlarının Yapay Sinir Ağı Metotları Kullanılarak Modellenmesi, SAÜ Fen Bilimleri Enst., Yüksek Lisans Tezi, Sakarya.
Williams R, and Zipser D. (1989). A learning algorithm for continually running fully recurrent neural networks. Neural Computation 1: 270–280.
Zealand C., Burn D., Simonovic S. (1999). Shortterm Streamflow Forecasting Using
Artificial Neural Networks, Journal of Hydrology, 214, 32-48.