Consider any application where a multimedia flow arrives at a terminal after traveling through an IP network (for instance, a CDN - Content Delivery Network). A critical problem for virtually every such an application is to be able to quantitatively evaluate the (instantaneous) quality of the flow as perceived by the end user. This task can be done by a panel of humans (the corresponding area is called subjective testing), at least for small pieces of audio or video signals. To the best of our knowledge, the only method that is able to perform it automatically, efficiently (for instance, in real time), and accurately, is the technique we have been developing that we call PSQA (Pseudo-Subjective Quality Assessment). Accurately here means precisely the following statement: PSQA gives to the flow a value very close to the value that an average human observer would do (as oberved in a subjective testing experiment). The reason is that we use a specific learning tool to capture how humans behave face to these flows. Once trained, the tool behaves similarly to real human observers. The mathematical object allowing this accuracy is the Random Neural Network model, that is, an open queuing network with positive and negative customers used as a statistical learning tool. The two main papers explaining how this task is performed are
which describes the methodology and its use in video analysis, together with an analysis of video quality when an IP network is involved in the transmission process, and
in the context if audio applications.
It must be underlined that once trained, our tool is a blackbox (basically software) where specific inputs coming from the source of the signal and from the network, are mapped into a numerical evaluation of perceived quality. One of the interest of using a RNN is that this mapping is a particularly simple mathematical function (a rational one). This function can be analytically handled, and this is at the origin of the idea presented in
where our PSQA method is coupled to traditional performance evaluation techniques. The idea is that instead of analyzing a model to derive, say, the loss packet rate as a function of, say, the network load, for some dimensioning purposes, we propose to analyze directly perceived quality. In
a related work is performed to study the performance of a well known and widely used Forward Error Correction (FEC) technique for different network conditions. More recently, in
we present a connection bewteen PSQA and dependability metrics, in the context of our project of a P2P CDN (Content Delivery Network).
Other publications
For supplementary comparisons between learning tools see
and the paper
Evaluating users' satisfaction in packet networks using Random Neural Networks,
Artificial Neural Networks (ICANN 2006), LNCS 4131, Springer Berlin / Heidelberg, 2006.
See also
Quantifying the Quality of Audio and Video Transmissions over the Internet: the PSQA Approach,
Design and Operations of Communication Networks: A Review of Wired and Wireless Modelling and Management Challenges,
Imperial College Press, Edited by J. Barria, 2005.
for an overview on the PSQA method, and
Controlling Multimedia QoS in the Future Home Network Using the PSQA Metric,
The Computer Journal, 2(49), 2006.
for a recent application to wireless networks.
Some videos NEW
Look here for a video of a small tool we have developped which is able to show the evolution of some well known video sequences when network conditions change, and the impact on quality.
An application of PSQA in control: a P2P CDN for TV/IP NEW (2007)
We work on the application of our PSQA technique to the design of a P2P network for distributing real-time video flows (TV over IP). We made the choice of a structured system, that is, a P2P network with a central control manager. The main reason is that we have the PSQA method allowing to measure the perceived quality in real time, which suggests using its output as feedback information in order to optimally controlling the network. We also chose a multisource approach, to address the main drawback in this type of system, the dynamics of the peers that enter and leave the system continuously. The name multisource refers to the fact that, in our system, a node receives the TV stream from many different peers (sources in this case). Our method allows to split the flow in an arbitrary manner, possibly distributing the load in a way that depends on the types of the frames (with MPEG-2 or MPEG-4 coding), with an arbitrary amount of redundancy, and reducing the signaling to a negligible volume. For this purpose, our distribution system is based on the properties of pseudo-random generators, which constitutes an original application of these tools (see this paper in IPOM 2007 for details). In this paper in LANC07 (best paper award) and this other paper in Globecom07 we discuss about the performance of this multisource technique. In this paper in Lagos07 some of the optimization aspects of the design of such a P2P network are discussed. As a byproduct of this research effort, we started to work on a monitoring platform for measuring the quality level delivered by a CDN (Content Delivery Network); the first results of this effort (main design decisions) were presented in this other paper at IPOM 2007.