Comparison of T-Norms and S-Norms for Interval Type-2 Fuzzy Numbers in Weight Adjustment for Neural Networks

Abstract

A comparison of different T-norms and S-norms for interval type-2 fuzzy number weights is proposed in this work. The interval type-2 fuzzy number weights are used in a neural network with an interval backpropagation learning enhanced method for weight adjustment. Results of experiments and a comparative research between traditional neural networks and the neural network with interval type-2 fuzzy number weights with different T-norms and S-norms are presented to demonstrate the benefits of the proposed approach. In this research, the definitions of the lower and upper interval type-2 fuzzy numbers with random initial values are presented; this interval represents the footprint of uncertainty (FOU). The proposed work is based on recent works that have considered the adaptation of weights using type-2 fuzzy numbers. To confirm the efficiency of the proposed method, a case of data prediction is applied, in particular for the Mackey-Glass time series (for τ = 17). Noise of Gaussian type was applied to the testing data of the Mackey-Glass time series to demonstrate that the neural network using a interval type-2 fuzzy numbers method achieves a lower susceptibility to noise than other methods.

1. Introduction

In the literature there exists research based on a similar idea to this paper, but with different approaches and implementations, such as the adjustment of fuzzy number weights in the input and output layer in the training process for the neural network [1], or the proposal of fuzzy number operations in a fuzzy neural network [2], and also because the proposed work operates with interval type-2 fuzzy numbers weights using different T-norm and S-norm in the adaptation of the weights, which represent the contribution and main difference with respect to the methods in the literature [3,4,5,6].

The proposed method in the present research is different to other papers, such as in Gaxiola et al. [7,8], where the fuzzy weights are obtained using interval type-2 fuzzy inference systems in [7] and generalized type-2 fuzzy inference systems in [8] for the connections between the layers, and without any changes for obtaining the change of the weights for each epoch of the backpropagation algorithm.

In the present approach, the use of interval type-2 fuzzy number weights is proposed. The lower and upper fuzzy numbers are obtained with the Nguyen-Widrow algorithm for the initial weights, modifying the internal calculations of the neurons by performing the multiplications of the inputs for the lower and upper type-2 fuzzy numbers weights separately, and then applying the T-norm for the lower outputs and the S-norm for the upper outputs; furthermore, we modified the backpropagation algorithm to achieve the lower change of the weights and the upper change of the weights in each epoch, respectively.

The proposed approach has the goal in the data of time series of achieving the best prediction error, which is the minimal error. In this case, the prediction for the Mackey-Glass time series is utilized to verify the efficiency of the proposed approach.

This paper is focused on analyzing fuzzy neural networks with the interval type-2 fuzzy number weights and providing a comparison with respect to the traditional neural networks. A same learning algorithm is used for the two neural models. On the other hand, with the purpose of further analyzing the performance of the model, we also applied noise in the real test data.

A comparison of the performance for the traditional neural network against the proposed fuzzy neural network with interval type-2 fuzzy numbers weights is performed in this paper. This comparison is based on the use of fuzzy numbers for the weights instead of the real numbers utilized in the traditional neural network; this modification is of great importance, due to the fact that the learning process of a neural network is directly affected by obtaining the optimal weights and consequently this has a critical impact on obtaining better results [9,10,11].

In the fuzzy neural network with interval type-2 fuzzy numbers weights, different T-norms and S-norms are applied for obtaining prediction error results, like the sum-product, Dombi [12], Hamacher [13,14] and Frank [15].

The adjustment of the weights in the backpropagation learning using interval type-2 fuzzy numbers is the main contribution of the proposed work in this paper for neural networks. This contribution provides to the neural network the robustness to support real data with uncertainty [16,17,18].

The main contribution of the proposed work is to improve backpropagation learning with the use of lower and upper type-2 fuzzy numbers for the adaptation for the weights. The use of the T-Norm and S-Norm to obtain the outputs of the neurons with the approaches of the sum-product, Dombi, Hamacher and Frank, enables the achievement of a better support for the uncertainty in the training process. With this, better results can be accomplished [19,20].

The next section presents a background of research on fuzzy numbers, and other methods of adaptation of weights and previous work of modifications to the backpropagation learning in neural networks. Section 3 explains the proposed methodology and the description of the problem to solve in the paper. Section 4 presents the simulation results for the proposed approach and the statistical tests. Finally, in Section 5, conclusions are offered and future work outlined.

2. Related Work

In the neural networks area, the backpropagation algorithm and variations of it is the training method that researchers use in literature [21,22,23]. In the bibliography, several papers have proposed different methods to improve the convergence of the backpropagation training algorithm [3,4,6]. In this paper, the most significant and essential works about the representation or managing of fuzzy numbers will be reviewed [9,10,11].

Dunyak et al. [1] presented a new algorithm for obtaining new weights (inputs and outputs) in the phase of training with any type of fuzzy numbers for a fuzzy neural network. Fard et al. [24] presented the sum and the product of two interval type-2 triangular fuzzy numbers and, basing it on the Stone-Weierstrass theorem, a fuzzy neural network working with interval type-2 fuzzy logic is developed. Li Z. et al. [2] described a fuzzified neural network computing the results of operations in two fuzzy numbers such as addition, subtraction, multiplication and division.

Asady B. [25] outlined a method for approximating trapezoidal fuzzy numbers in comparison with other methods of approximation. Coroianu L. et al. [26] described the inverse F-transform to accomplish optimal fuzzy numbers, maintaining the support and the convergence of the core. Yang D. et al. [27] presented an interval and modified interval neuron perceptron with interval weights and biases, and modified the learning algorithm for this approach.

Requena et al. [28] presented trapezoidal fuzzy numbers with the conventional parameters (a, b, c, d) in an artificial neural network, and also proposed a decision personal index (DPI) to obtain the distance between the numbers. Kuo et al. [29] described a fuzzy neural network using the real-coded genetic algorithm to generate the initial fuzzy weights, and using the extension principle, the fuzzy operations are determined. Molinari [30] presented generalized triangular fuzzy numbers and a comparison with others fuzzy numbers. Chai et al. [31] described a representation of fuzzy numbers, establishing the theorem “that for each fuzzy number there exists a unique skew fuzzy number and a unique symmetric fuzzy number”.

Figueroa-García et al. [32] made a comparison between interval type-2 fuzzy numbers using distance measures. Ishibuchi et al. [33] presented a comparison between real numbers, and different fuzzy numbers such as symmetric triangular, asymmetric triangular and symmetric trapezoidal as weights in the connections between layers in a neural network. Karnik et al. [34] presented mathematical operations of type-2 fuzzy sets for obtaining the join and meet under t-norm.

Raj et al. [35] described fuzzy ranking alternatives for fuzzy numbers as linguistic variables for fuzzy weights. Chu et al. [36] proposed a ranking of fuzzy numbers with a zone between the original point and the centroid point and making numerical examples with triangular fuzzy numbers.

Ishibuchi et al. [37,38] proposed to use the weights for a fuzzy neural network like triangular or trapezoidal fuzzy numbers. Feuring [39] presented a new backpropagation algorithm for learning in the neural network, in which the new lower and upper limits of weights are computed. Castro et al. [40] proposed a type-2 fuzzy neurons model, in which the rules used interval type-2 fuzzy neurons in the antecedents and an interval of type-1 fuzzy neuron in the consequents.

In the area of time series prediction there exists in the literature many recent works developed by and based on the use of type-2 fuzzy logic, like Castro et al. [41], and other researchers [42,43,44].

3. Proposed Methodology

The proposed method in this work has the goal of generalizing the backpropagation learning algorithm by using interval type-2 fuzzy numbers in the calculations, and this approach gives the neural network less susceptibility to data with uncertainty. In interval type-2 fuzzy numbers, it will be necessary to obtain the interval of the fuzzy numbers, which consists in the footprint of uncertainty (FOU), and the calculations in the neurons was obtained with the T-norms and S-norms of sum-product, Dombi, Hamacher and Frank for the corresponding applications [45,46,47,48].

The method of adjustment of weights for each neuron in the connections between the layers in the backpropagation learning algorithm is modified from the original adjustment (Figure 1).

The method in this paper consists of utilizing interval type-2 fuzzy number weights in neurons. This development modifies the internal calculation for the neurons and the adjustment of the weights to allow handle fuzzy numbers (Figure 2) [49].

We modified the operations in the neurons and the backpropagation learning to adjust the fuzzy numbers weights and accomplish the desired result, working to find the optimal process to operate with interval type-2 fuzzy number weights [50,51].

To determine the appropriate activation function f(-) to utilize, the linear and secant hyperbolic functions were considered in this approach.

3.1. Architecture of the Traditional Neural Network

The architecture of the neural network used in this work (see Figure 3) consists of a hidden layer with 16 neurons and of an output layer with 1 neuron, and a training data of the Mackey-Glass time series to the input data in the input layer.

3.2. Architecture of the Fuzzy Neural Network with Interval Type-2 Fuzzy Numbers Weights

In Figure 4a scheme of the proposed methodology of the fuzzy neural network with interval type-2 fuzzy numbers weights (FNNIT2FNW) is presented.

The architecture of fuzzy neural network with interval type-2 fuzzy number weights (see Figure 5) is explained as follow:

Phase 0: equation of the inputs data.

$$x=[x_1,x_2,\dots ,x_n]$$

(1)

Phase 1: Representation of the Interval type-2 fuzzy number weights [36].

$$\tilde{w}=[\underset{\_}{w},\overline{w}]$$

(2)

where $\overline{w}$ and $\underset{\_}{w}$ are generated with the Nguyen-Widrow algorithm [52] for the initial weights.

Phase 2: Calculation of the output of the hidden neurons with interval type-2 fuzzy number weights.

$$\underset{\_}{Net}=f\left({\displaystyle \sum }_{i=1}^n{x}_i\underset{\_}{w_{ij}}\right)$$

(3)

$$\overline{Net}=f\left({\displaystyle \sum }_{i=1}^n{x}_i\overline{w_{ij}}\right)$$

(4)

We used the secant hyperbolic as the activation function for the hidden neurons. Subsequently, we applied the S-Norm and T-Norm for calculating the lower and upper outputs of the hidden neurons, respectively.

$$\underset{\_}{y}=TNorm\left(\underset{\_}{Net},\overline{Net}\right)$$

(5)

$$\overline{y}=SNorm\left(\underset{\_}{Net},\overline{Net}\right)$$

(6)

where: we used the T-norms and S-norms.

Sum-Product:

$$TNorm\left(\underset{\_}{Net},\overline{Net}\right)=\underset{\_}{Net}.\ast \overline{Net}$$

(7)

$$SNorm\left(\underset{\_}{Net},\overline{Net}\right)=\underset{\_}{Net}+\overline{Net}-TNorm\left(\underset{\_}{Net},\overline{Net}\right)$$

(8)

Dombi: for $\gamma >0$.

$$TNormD\left(\underset{\_}{Net},\overline{Net},\gamma \right)=\frac{1}{1+{\left[{\left({\underset{\_}{Net}}^{-1}-1\right)}^{\gamma }+{\left({\overline{Net}}^{-1}-1\right)}^{\gamma }\right]}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\gamma $}\right.}}$$

(9)

$$SNormD\left(\underset{\_}{Net},\overline{Net},\gamma \right)=\frac{1}{1+{\left[{\left({\underset{\_}{Net}}^{-1}-1\right)}^{-\gamma }+{\left({\overline{Net}}^{-1}-1\right)}^{-\gamma }\right]}^{\raisebox{1ex}{$-1$}\!\left/ \!\raisebox{-1ex}{$\gamma $}\right.}}$$

(10)

Hamacher: for $\gamma >0$.

$$TNormH\left(\underset{\_}{Net},\overline{Net},\gamma \right)=\frac{\underset{\_}{Net}.\ast \overline{Net}}{\gamma +\left(1-\gamma \right)\left(\underset{\_}{Net}+\overline{Net}-\underset{\_}{Net}.\ast \overline{Net}\right)}$$

(11)

$$SNormH\left(\underset{\_}{Net},\overline{Net},\gamma \right)=\frac{\underset{\_}{Net}+\overline{Net}+\left(\gamma -2\right)\left(\underset{\_}{Net}.\ast \overline{Net}\right)}{1+\left(\gamma -1\right)\left(\underset{\_}{Net}.\ast \overline{Net}\right)}$$

(12)

Frank: for $s>0$.

$$TNormF\left(\underset{\_}{Net},\overline{Net},s\right)=lo{g}_s\left[\frac{1+\left(s^{\underset{\_}{Net}}-1\right)\left(s^{\overline{Net}}-1\right)}{s-1}\right]$$

(13)

$$SNormF\left(\underset{\_}{Net},\overline{Net},s\right)=1-lo{g}_s\left[\frac{1+\left(s^{1-\underset{\_}{Net}}-1\right)\left(s^{1-\overline{Net}}-1\right)}{s-1}\right]$$

(14)

Phase 3: Calculation of the output for the output neuron with interval type-2 fuzzy number weights.

$$\underset{\_}{Out}=f\left({\displaystyle \sum }_{i=1}^n\underset{\_}{y_i}\underset{\_}{w_{ij}}\right)$$

(15)

$$\overline{Out}=f\left({\displaystyle \sum }_{i=1}^n\overline{y_i}\overline{w_{ij}}\right)$$

(16)

In the output neuron the linear activation function is utilized. Subsequently, we applied the S-Norm and T-Norm for the lower and upper outputs of the output neurons, respectively.

$$\underset{\_}{y}=SNorm\left(\underset{\_}{Out},\overline{Out}\right)$$

(17)

$$\overline{y}=TNorm\left(\underset{\_}{Out},\overline{Out}\right)$$

(18)

where: we used the T-norms and S-norms.

Sum-Product:

$$TNorm\left(\underset{\_}{Out},\overline{Out}\right)=\underset{\_}{Out}.\ast \overline{Out}$$

(19)

$$SNorm\left(\underset{\_}{Out},\overline{Out}\right)=\underset{\_}{Out}+\overline{Out}-TNorm\left(\underset{\_}{Out},\overline{Out}\right)$$

(20)

Dombi: for $\gamma >0$.

$$TNormD\left(\underset{\_}{Out},\overline{Out},\gamma \right)=\frac{1}{1+{\left[{\left({\underset{\_}{Out}}^{-1}-1\right)}^{\gamma }+{\left({\overline{Out}}^{-1}-1\right)}^{\gamma }\right]}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\gamma $}\right.}}$$

(21)

$$SNormD\left(\underset{\_}{Out},\overline{Out},\gamma \right)=\frac{1}{1+{\left[{\left({\underset{\_}{Out}}^{-1}-1\right)}^{-\gamma }+{\left({\overline{Out}}^{-1}-1\right)}^{-\gamma }\right]}^{\raisebox{1ex}{$-1$}\!\left/ \!\raisebox{-1ex}{$\gamma $}\right.}}$$

(22)

Hamacher: for $\gamma >0$.

$$TNormH\left(\underset{\_}{Out},\overline{Out},\gamma \right)=\frac{\underset{\_}{Out}.\ast \overline{Out}}{\gamma +\left(1-\gamma \right)\left(\underset{\_}{Out}+\overline{Out}-\underset{\_}{Out}.\ast \overline{Out}\right)}$$

(23)

$$SNormH\left(\underset{\_}{Out},\overline{Out},\gamma \right)=\frac{\underset{\_}{Out}+\overline{Out}+\left(\gamma -2\right)\left(\underset{\_}{Out}.\ast \overline{Out}\right)}{1+\left(\gamma -1\right)\left(\underset{\_}{Out}.\ast \overline{Out}\right)}$$

(24)

Frank: for $s>0$.

$$TNormF\left(\underset{\_}{Out},\overline{Out},s\right)=lo{g}_s\left[\frac{1+\left(s^{\underset{\_}{Out}}-1\right)\left(s^{\overline{Out}}-1\right)}{s-1}\right]$$

(25)

$$SNormF\left(\underset{\_}{Out},\overline{Out},s\right)=1-lo{g}_s\left[\frac{1+\left(s^{1-\underset{\_}{Out}}-1\right)\left(s^{1-\overline{Out}}-1\right)}{s-1}\right]$$

(26)

Phase 4: The single output of the neural network is obtained:

$$y=\frac{\underset{\_}{y}+\overline{y}}{2}$$

(27)

3.3. Proposed Adjustment for Interval Type-2 Fuzzy Numbers with Backpropagation Learning

The backpropagation learning algorithm is performed by the adjustment of the interval type-2 fuzzy number weights, described as follows:

Stage 1:

The Nguyen-Widrow algorithm is utilized to initialize the lower and upper values of the interval type-2 fuzzy numbers weights for the neural network.

Stage 2:

The input pattern and the wanted output for the neural network is established.

Stage 3:

The output of the neural network is calculated. In the first instance, the inputs for the network are introduced and the output of the network is obtained performing the calculations of the outputs from the input layer until the output layer.

Stage 4:

Determine the error terms for the neurons of the layers. In the output layer, the calculation of lower ($\underset{\_}{{\delta }_{pk}^O}$) and upper ($\overline{{\delta }_{pk}^O}$) delta for each neuron “k” is performed with the follow equations:

$$\underset{\_}{{\delta }_{pk}^O}=\left(d_{pk}-y_{pk}\right)f_k^{O\prime }(\underset{\_}{y})$$

(28)

$$\overline{{\delta }_{pk}^O}=\left(d_{pk}-y_{pk}\right)f_k^{O\prime }\left(\overline{y}\right)$$

(29)

In the hidden layer, the calculation of lower ($\underset{\_}{{\delta }_{pj}^h}$) and upper ($\underset{\_}{\overline{{\delta }_{pj}^h}}$) delta for each neuron “j” is perform with the follow equations:

$$\underset{\_}{{\delta }_{pj}^{\mathrm{h}}}=f_j^{h\prime }\left(\underset{\_}{Net}\right){\displaystyle \sum }_k\underset{\_}{{\delta }_{pk}^O}\underset{\_}{w_{kj}}$$

(30)

$$\overline{{\delta }_{pj}^{\mathrm{h}}}=f_j^{h\prime }\left(\overline{Net}\right){\displaystyle \sum }_k\overline{{\delta }_{pk}^O}\overline{w_{kj}}$$

(31)

Stage 5:

The utilization of a recursive algorithm allows the actualization of the interval type-2 fuzzy number weights, beginning from the output neurons and updating backwards until the neurons in the input layer. The adjustment is described as follows:

The calculation of the change of interval type-2 fuzzy number weights is achieved with the equations described as follows:

Calculations of the output neurons:

$$\underset{\_}{∆w_{kj}\left(t+1\right)}=\underset{\_}{{\delta }_{pk}^O}\underset{\_}{y_{pj}}$$

(32)

$$\overline{∆w_{kj}\left(t+1\right)}=\overline{{\delta }_{pk}^O}\overline{y_{pj}}$$

(33)

Calculations of the hidden neurons:

$$\underset{\_}{∆w_{ji}\left(t+1\right)}=\underset{\_}{{\delta }_{pj}^h}{x}_{pi}$$

(34)

$$\overline{∆w_{ji}\left(t+1\right)}=\overline{{\delta }_{pj}^h}{x}_{pi}$$

(35)

Stage 6:

The method is recurrent until for each of the learned patterns the error terms are small enough.

$$E_p=\frac{1}{2}{\displaystyle \sum }_{k=1}^M{\delta }_{pk}^2$$

(36)

Alternatively, we have the option of working with fuzzy inputs and fuzzy targets. In this case, the modification of the proposed neural network must be in phase 2, multiplying by the lower input in Equation (2) and the upper input in Equation (3); besides, in phase 4, we can maintain the lower and upper final outputs of the output neuron because there is no need for performing the average.

The neural network used in this research consists of 16 neurons for the hidden layer, based on a study of the performance of the neural networks modifying the numbers of neurons in the hidden layer, starting with 5 neurons and increasing one by one, until reaching 120 neurons; we are presenting this study in the following section. To test the proposed method, experiments in the time series prediction are performed. A benchmark and chaotic time series such as the Mackey-Glass time series (for τ = 17) is used in this study.

Based on previous work, in the experiments, the backpropagation algorithm applying gradient descent and adaptive learning rate is utilized.

The neural networks manage interval type-2 fuzzy number weights in the hidden and output layer [53,54]. In the hidden and output layer of the network we used the backpropagation algorithm modified for working with interval type-2 fuzzy numbers to achieve new weights for the next epochs of the network [55,56,57].

4. Simulation Results

We achieved the experiments for the Mackey-Glass time series, and for this we used 1000 data points. In this case, 500 data points are considered for the training stage and 500 data points for the testing stage.

4.1. Neural Network with Interval Type-2 Fuzzy Numbers Weights (NNIT2FNW) for T-Norm and S-Norm of Sum-Product

We performed an experiment for determining manually the optimal number of neurons in the hidden layer of the fuzzy neural network with the interval type-2 fuzzy numbers; we increase the number of neurons by one unit at a time in the interval of 5 to 120 neurons. The acquired results from the experiments are presented in

Table 1. Results for the fuzzy neural network with interval type-2 fuzzy numbers with T-norm of sum-product in time series prediction using Mackey-Glass time series.

No. Neurons	Best Prediction Error MAE	Average MAE
5	0.0187	0.0240
6	0.0197	0.0245
7	0.0188	0.0250
8	0.0172	0.0231
9	0.0198	0.0259
10	0.0170	0.0246
11	0.0190	0.0252
12	0.0192	0.0248
13	0.0198	0.0255
14	0.0191	0.0251
15	0.0185	0.0227
16	0.0149	0.0180
17	0.0180	0.0238
18	0.0202	0.0242
19	0.0205	0.0239
20	0.0164	0.0247
21	0.0201	0.0243
22	0.0189	0.0241
23	0.0178	0.0249
24	0.0195	0.0250
25	0.0195	0.0259
26	0.0189	0.0233
27	0.0195	0.0246
28	0.0191	0.0248
29	0.0175	0.0245
30	0.0149	0.0233
31	0.0193	0.0245
32	0.0182	0.0259
33	0.0195	0.0252
34	0.0170	0.0243
35	0.0195	0.0241
36	0.0188	0.0251
37	0.0209	0.0248
38	0.0187	0.0243
39	0.0195	0.0254
40	0.0190	0.0246
41	0.0188	0.0263
42	0.0172	0.0233
43	0.0188	0.0249
44	0.0192	0.0237
45	0.0192	0.0247
46	0.0157	0.0247
47	0.0188	0.0252
48	0.0189	0.0246
49	0.0204	0.0247
50	0.0151	0.0246
51	0.0190	0.0250
52	0.0179	0.0239
53	0.0191	0.0242
54	0.0177	0.0240
55	0.0168	0.0240
56	0.0202	0.0251
57	0.0196	0.0255
58	0.0181	0.0250
59	0.0192	0.0248
60	0.0173	0.0239
61	0.0168	0.0236
62	0.0188	0.0239
63	0.0168	0.0240
64	0.0183	0.0238
65	0.0169	0.0252
66	0.0185	0.0250
67	0.0174	0.0253
68	0.0171	0.0230
69	0.0185	0.0244
70	0.0186	0.0248
71	0.0210	0.0251
72	0.0182	0.0249
73	0.0206	0.0247
74	0.0169	0.0249
75	0.0170	0.0240
76	0.0174	0.0233
77	0.0206	0.0245
78	0.0185	0.0244
79	0.0190	0.0247
80	0.0178	0.0246
81	0.0179	0.0247
82	0.0185	0.0243
83	0.0192	0.0254
84	0.0170	0.0237
85	0.0178	0.0242
86	0.0186	0.0260
87	0.0197	0.0233
88	0.0197	0.0256
89	0.0178	0.0252
90	0.0191	0.0257
91	0.0183	0.0265
92	0.0193	0.0240
93	0.0199	0.0240
94	0.0166	0.0242
95	0.0206	0.0248
96	0.0181	0.0236
97	0.0191	0.0252
98	0.0199	0.0248
99	0.0173	0.0249
100	0.0181	0.0248
101	0.0168	0.0237
102	0.0173	0.0250
103	0.0198	0.0245
104	0.0191	0.0237
105	0.0205	0.0245
106	0.0197	0.0246
107	0.0179	0.0256
108	0.0185	0.0244
109	0.0189	0.0241
110	0.0164	0.0242
111	0.0190	0.0254
112	0.0198	0.0250
113	0.0173	0.0245
114	0.0203	0.0244
115	0.0168	0.0248
116	0.0170	0.0233
117	0.0199	0.0254
118	0.0188	0.0252
119	0.0196	0.0247
120	0.0189	0.0250

. The fuzzy neural network with 16 neurons in the hidden layer obtained the best result with 0.0149 for the best prediction error, and 0.0180 for the average error (MAE). This experiment was realized with the T-norm and S-norm of sum-product.

The mean absolute error (MAE) is considered in obtaining the results of the experiments. The average error was obtained by taking 30 experiments with the equal parameters and conditions for all experiments.

The parameters for the fuzzy neural network with interval type-2 fuzzy numbers are of 500 epochs and 0.0000001 of error for the training phase.

We observe from Table 1 that the fuzzy neural network with interval type-2 fuzzy numbers and T-norm and S-norm of sum-product with 16 neurons in the hidden layer (FNNIT2FNSp) shows better results than the others; so, based on this fact, in the following experiments we work with this architecture for the neural network.

In Figure 6, we are presenting the plot of the real data of the Mackey-Glass time series against the predicted data of the interval type-2 fuzzy neural network (FNNIT2FNSp) with 16 neurons in the hidden layer. In Figure 7, an illustration of the convergence curves in the training process is presented.

We performed the same experiment that we presented before with the T-norm and S-norm of Dombi, Hamacher and Frank.

4.2. NNIT2FNW for T-Norm and S-Norm of Dombi

The architecture for the fuzzy neural network with T-norm of Dombi (FNNIT2FND) has 4 neurons in the hidden layer and $\gamma =0.8$, the best results with 0.0457 for the best result for the prediction error, and 0.0622 for the average error. We show some results for this architecture in

Table 2. Results for the fuzzy neural network with interval type-2 fuzzy numbers with T-norm of Dombi in time series prediction using Mackey-Glass time series.

Experiment	Prediction Error
1	0.0457
2	0.0466
3	0.0549
4	0.0581
5	0.0599
6	0.0636
7	0.0656
8	0.0671
9	0.0675
10	0.0694
Average	0.0622

, Figure 8 and Figure 9.

4.3. NNIT2FNW for T-Norm and S-Norm of Hamacher

The architecture for the fuzzy neural network with the T-norm of Hamacher (FNNIT2FNH) has 39 neurons in the hidden layer and $\gamma =1$, the best result with 0.0130 for the best prediction error, and 0.0164 for the average error. We show some results for this architecture in

Table 3. Results for the fuzzy neural network with interval type-2 fuzzy numbers with T-norm of Hamacher in time series prediction using Mackey-Glass time series.

Experiment	Prediction Error
1	0.0130
2	0.0138
3	0.0149
4	0.0154
5	0.0163
6	0.0165
7	0.0170
8	0.0175
9	0.0177
10	0.0183
Average	0.0164

, Figure 10 and Figure 11.

4.4. NNIT2FNW for T-Norm and S-Norm of Frank

The architecture for the fuzzy neural network with T-norm of Frank (FNNIT2FNF) has 19 neurons in the hidden layer and $\gamma =2.8$, the best results with 0.0117 for the best prediction error, and 0.0167 for the average error. We show some results for this architecture in

Table 4. Results for the fuzzy neural network with interval type-2 fuzzy numbers with T-norm of Frank in time series prediction using Mackey-Glass time series.

Experiment	Prediction Error
1	0.0117
2	0.0140
3	0.0153
4	0.0156
5	0.0158
6	0.0163
7	0.0170
8	0.0175
9	0.0177
10	0.0179
Average	0.0167

, Figure 12 and Figure 13.

4.5. Comparison of Traditional Neural Network Against NNIT2FNW for T-Norm and S-Norm

The acquired results in the experiments with the traditional neural network (TNN) are present on

Table 5. Results for the traditional neural network (TNN) in the Mackey-Glass time series and the comparison against the FNNIT2FNSp, FNNIT2FND, FNNIT2FNH, and FNNIT2FNF.

	Best Prediction Error	Average
TNN	0.0169	0.0203
FNNIT2FNSp	0.0149	0.0180
FNNIT2FND	0.0457	0.0622
FNNIT2FNH	0.0130	0.0164
FNNIT2FNF	0.0117	0.0167

and Figure 14 and Figure 15, and the neural network parameters are obtained are based on empirical testing. The best result of the prediction errors is of 0.0169, and the average error is of 0.0203 (MAE). In Table 2, we present the comparison of these results against the results of the fuzzy neural network with interval type-2 fuzzy numbers for all T-norms (FNNIT2FNSp, FNNIT2FND, FNNIT2FNH, FNNIT2FNF).

4.6. Comparison of the Proposed Methods for Mackey-Glass Data with Noise

We also implemented an experiment utilizing noisiness in the interval 0.1 to 1 in the test data to analyze the robustness of the traditional neural network (TNN) and the fuzzy neural network with interval type-2 fuzzy numbers for all T-norms (FNNIT2FNSp, FNNIT2FND, FNNIT2FNH, FNNIT2FNF). The obtained results for these experiments are presented in

Table 6. Results for the traditional neural network and fuzzy neural networks with all T-norms in the Mackey-Glass time series under different noise levels (n).

Noise Level	TNN	FNNIT2FNSp	FNNIT2FND	FNNIT2FNH	FNNIT2FNF
n = 0	0.0169	0.0149	0.0457	0.0130	0.0117
n = 0.1	0.0564	0.0617	0.0704	0.0556	0.0594
n = 0.2	0.1115	0.1135	0.0981	0.0960	0.0954
n = 0.3	0.1749	0.1275	0.1168	0.1171	0.1175
n = 0.4	0.2311	0.1554	0.1362	0.1360	0.1419
n = 0.5	0.3124	0.1661	0.1502	0.1536	0.1571
n = 0.6	0.3676	0.1897	0.1485	0.1576	0.1589
n = 0.7	0.4250	0.1866	0.1684	0.1770	0.1736
n = 0.8	0.4941	0.2018	0.1744	0.1811	0.1808
n = 0.9	0.5411	0.2077	0.1775	0.1887	0.1858
n = 1	0.5684	0.2075	0.1858	0.1920	0.1935

. We applied the noise by using the following equations:

$$DataNoise=Data+\left(NoiseLevel\times Noise\right)$$

(37)

$$Noise=2\times \left[rand\left(1,nData\right)-0.5\right]$$

(38)

where: “Data” are the test data points of the Mackey-Glass Time series, “NoiseLevel” is the level of noise in the range (0.1–1), “rand” is a uniformly distributed function for random numbers used for obtained the values for the noise.

We observe in Table 6 that the better performance was accomplished with the fuzzy neural network with interval type-2 fuzzy numbers with all T-norms for almost all the levels of noise in the data test. The prediction error for the traditional neural network was increasing considerably with the higher noise levels as a difference to the fuzzy neural network, which maintains the prediction error below 0.21.

We observe in Figure 16 that when at the test data is applied noise, the fuzzy neural network with interval type-2 fuzzy numbers with the Hamacher and Frank T-norms achieve minor prediction errors compared with the traditional neural network in the different levels of noise. An important fact is that the fuzzy neural network with T-norm of Dombi presents better performance than the others in the test with noise but without noise has a higher prediction error.

A statistical test, the t-student test, was applied to perform a comparison of the performance of TNN against FNNIT2FNH, and FNNIT2FNF in the prediction error; we selected these two fuzzy neural networks because they presented a better performance than FNNIT2FNSp, and FNNIT2FND. In the statistical tests we consider 30 experiments and a 95% of reliability in the tests.

We present in

Table 7. Parameters used in the t-student statistical test for the TNN against FNNIT2FNH and FNNIT2FNF.

	TNN	FNNIT2FNH	FNNIT2FNF
No. Experiments	30	30	30
Mean Data	0.02028	0.01638	0.01665
Standard Deviation	0.00158	0.00133	0.00123
Standard error of the mean	0.00029	0.00024	0.00023

the parameters for the statistical test of the TNN and FNNIT2FNHmodel. We used a Hypothesis testing of H0: TNN = FNNIT2FNHand for the alternative hypothesis H1: TNN > FNNIT2FNH by the comparison of these models; H0: TNN = FNNIT2FNF and H1: TNN > FNNIT2FNF.

The results obtained with the statistical test for the prediction errors for TNN against FNNIT2FNH are of 0.003907 in the estimated mean difference, 0.003153 in the lower limit of the difference, a t value of 1037, p value of 0.0001 and 56 degrees of freedom.

The results obtained with the statistical test for prediction errors for TNN against FNNIT2FNFare of 0.003631 in the estimated mean difference, 0.002897 in the lower limit of the difference, a t value of 9.93, p value of 0.0001 and 54 degrees of freedom.

The results obtained with the statistical test for prediction errors for FNNIT2FNHagainst FNNIT2FNF are of −0.00027 in the estimated mean difference, −0.000940 in the lower limit of the difference, a t value of −0.84, p value of 0.407 and 57 degrees of freedom.

The results demonstrate that there exists significant statistical evidence to affirm that the FNNIT2FNH and FNNIT2FNHF are better than the TNN, and that the FNNIT2FNH is equal to the FNNIT2FNF.

5. Conclusions

Based on the experiments, we have reached the conclusion that the fuzzy neural network with interval type-2 fuzzy number weights with T-norm of Sum-product (FNNIT2FNSp), Hamacher (FNNIT2FNH) and Frank (FNNIT2FNF) achieved better results than the traditional neural network for the benchmark time series used in this work, Mackey-Glass. This affirmation is based on the prediction errors of 0.0169 for TNN, and 0.0149, 0.0130 and 0.0117 for FNNIT2FNSp, FNNIT2FNH and FNNIT2FNF, respectively; and the average errors obtained of 30 experiments are of 0.0203, and 0.0180, 0.0164 and 0.0167, respectively.

The fuzzy neural network with interval type-2 fuzzy number weights with the T-norm presented a better tolerance and behavior than the traditional neural network when Gaussian noise is applied at the testing data. This inference was reached by analyzing that the fuzzy neural network with interval type-2 fuzzy number weights with T-norms show only minor prediction errors compared to the traditional neural network at increasing the levels of noise. Besides, analyzing Table 5 and Table 6, and Figure 10, from which the FNNIT2FNH and FNNIT2FNF show only minor prediction errors compared to the other paradigms in this work for the Mackey-Glass time series, and furthermore observing the results for the statistical tests performed to these paradigms.

The method proposed in this work, a fuzzy neural network with interval type-2 fuzzy number weights with T-norms, presents better performance, robustness and achieves lower results of prediction errors than the traditional neural network. Furthermore, the interval type-2 fuzzy number weights allow the neural network to have a minor susceptibility to increment the results of the prediction error when the real data is affected with noise.