User-Perceived Quality Assessment for VoIP Applications

We designed and implemented a system that permits the measurement of network Quality of Service (QoS) parameters. This system allows us to objectively evaluate the requirements of network applications for delivering user-acceptable quality. To do this we compute accurately the network QoS parameters: one-way delay, jitter, packet loss and throughput. The measurement system makes use of a global clock to synchronise the time measurements in different points of the network. To study the behaviour of real network applications specific metrics must be defined in order to assess the user-perceived quality (UPQ) for each application. Since we measure simultaneously network QoS and application UPQ, we are able to correlate them. Determining application requirements has two main uses: (i) to predict the expected UPQ for an application running over a given network (based on the corresponding measured QoS parameters) and understand the causes of application failure; (ii) to design/configure networks that provide the necessary conditions so that an application will run with a desired UPQ level. This document presents the work we carried out in order to determine the relationship between network QoS and UPQ for voice over IP (VoIP) applications. For VoIP we measured the UPQ using the PESQ score (Perceptual Evaluation of Speech Quality). We performed uni-directional tests, which focus on the perceived quality of the speech itself depending on packet loss and jitter. Voice data is transmitted encoded; the codecs we tested are: G.711, G.726, GSM and G.729. We concluded that G.729 is to be the most robust codec in the range of network conditions under study. It provides almost the same acceptable perceived quality throughout the studied loss-jitter space. The codec G.711 performs better than the other codecs as long as network conditions are good (loss rate smaller than 3% and jitter below 20 ms). G.726 gives results that are better at most 7% than those for GSM; however this is achieved at the expense of a transmission rate which is 2.5 times larger.