On the Use of the Neural Networks

It is important to mention that since the 60's, NNs have been used to solve problems in communication networks, mostly those involving ambiguity, and an environment that varies over time. There have been very successful applications, always addressing problems difficult to tackle with traditional methods [149], which go from using the NNs as adaptive echo cancellers and adaptive line equalizers on telephone channels, to the development of complex control schemes for traffic control in ATM networks (e.g., call admission control, traffic parameters prediction, traffic flow control, quality of service parameters estimation, traffic prediction, and dynamic allocation of network resources). Regarding RNN, they are used in video compression with compression ratio that goes from 500:1 to 1000:1 for moving gray-scale images and full-color video sequences respectively [31]. They are also used as decoders for error correcting codes in noisy communication channels, which reduce the error probability to zero [2]. Furthermore, they are used in a variety of image processing techniques, which go from image enlargement and fusion to image segmentation [50]. A survey of RNN applications is given in [12]. After performing the suitable subjective quality test and calculating the MOS scores (see Sections 4.3, 4.4), these scores should be normalized. The goal of the normalization is to achieve better performance from the NN. The normalization formulae is as follows:

$$F_n=\frac{F-F_{min}}{F_{max}-F_{min}},$$ (42)

where $F_n$ and $F$ are the normalized and original values of the MOS scores respectively. $F_{max}$ is the maximum score value available on the used quality scale for the best quality. $F_{min}$ are the minimum score value available of the used quality scale for the worst quality. In addition, all the quality-affecting parameters' values should be normalized between 0 and 1. The quality database, the quality-affecting parameters and the corresponding quality scores (which represent the learning patterns or examples), will be used to train and test the NN. This database should be divided into two parts. The first part is used to train the NN, while the second one is used to test its performance. For the NN, there are many architectures (feedforward, recurrent, etc.) and many training algorithms (backpropagation, Levenberg-Marquardt, etc.). The two types of NN considered here are the Artificial Neural Networks (ANN) and the Random Neural Networks (RNN). We recommend the use of RNN as it offers some advantages over ANN, at least in our context. (See Section 4.5.1 for a comparison between ANN and RNN and Chapter 10 for new training algorithms for RNN.) For the NN architecture, the number of input neurons is equal to the number of parameters, while there is only one output neuron, which represents the corresponding, estimated quality score (all normalized). There are some known problems for using NN. One of these problems is overtraining. The network is said to be overtrained, when the performance of network is very good for the training database, while it gives very poor performance for the testing database (the inputs-output examples that have not been seen during the training process). Another problem is the choice of the number of neurons in the hidden layer. With too few hidden neurons, the NN will not learn correctly the problem. On the other hand, with too many hidden neurons, there is a risk of overtraining, and the system will be more complex. It is stated in [118] that whenever several networks can solve the problem, it is preferable to select the most simple one to make the network generalize well. Indeed, the most difficult problem (yet not efficiently solved) is how to find the optimal number of hidden neurons. There exist some heuristic methods aimed to find a rough approximation, however they work for some problems and give bad results for others. This is because these methods cannot take into consideration the complexity of the given problem. Instead, they try to map the problem complexity as a function of the number of inputs and outputs as well as the number of the training examples. We recommend to use the following procedure to identify the best NN architecture, provided that a three-layer feedforward architecture is used.

1. Start by one hidden neuron.
2. Train the NN by the training database until a minimum error goal is achieved, and measure its performance to predict the outputs.
3. Test it by the testing database, and measure its performance.
4. Increment the hidden neurons by one and if the number of hidden neurons is less than a maximum value, go to step 2.
5. Select the architecture that gives the best performance for both the training and testing databases.

The weights of the NN are initialized randomly, hence care should be taken to initialize to the same value to correctly identify the optimal number of hidden neurons by the above procedure. The minimum error goal to stop the training may be made variable to precisely identify the hidden neurons and hence get the best performance of the neural networks.