Apart from video rate (or requested bitrate), Mean Opinion Score (MOS) has increasingly become a primary term representing Quality of Experience (QoE) in HTTP adaptive streaming (HAS). By monitoring this metric, QoE management can effectively maximize QoE for the users. However, due to various behaviors of particular commercial HAS players, deciding an appropriate monitoring interval has not been fully investigated yet. In this paper, an optimal interval is proposed to be equal to duration of a video chunk in order to aid service managers in early detecting QoE deterioration and limiting the probability of video rate deterioration. The optimal monitoring interval is evaluated by comparing with other values of interval in terms of ratio of video rate deterioration. Furthermore, MOS-based QoE monitoring method which takes into account the proposed interval is thus compared with video rate based monitoring method. The results show that with optimal interval, MOS monitoring guarantees a low ratio of video rate deterioration (around 10% for buffering state and 40% for steady state) and small average CPU Load (about 11.45%).
Video has become the most dominant application on the Internet. According to [
The rest of the paper is organized as follows: Section 2 will provide an overview of background knowledge; Section 3 will state related works; The proposals will be described in the Section 4; Meanwhile, Section 5 will show the evaluation results on the proposed method; Section 6 will conclude the paper and future work will also be stated.
HTTP adaptive streaming marked its initial appearance in 2006 by Move Networks [
With HAS technology, video content is stored at the server. The video content is thus divided into small chunks that are available at multiple video rate. HAS client (or HAS player) sends requests for suitable video rate to the server based on its current network condition status. Adaptively selecting video rate based on calculation of available resources at client side could be called as an Adaptive Bitrate Selection (ABR) [
re-activated.
Resources estimation plays an important role in ABR for selecting an appropriate video rate. During HAS session, resources estimation could be done based on either available bandwidth or playback buffer or power level estimation. Available bandwidth-based estimation is often performed throughout a moving average of available bandwidth. In order to maximize video rate, HAS player always attempts to stabilize a gap between available bandwidth and video rate, translated into conservatism margin ratio [
C o n s _ r a t i o = C t − R t C t (1)
where C t , R t are available bandwidth and video rate at time point t, respectively.
Service providers apply different values of C o n s _ r a t i o (e.g. Microsoft Smooth Streaming and Apple HTTP Live Streaming apply a conservatism value of 20% and 40%, respectively).
Buffer-based estimation is done by capturing the playback buffer size of chunk by chunk. According to [
B t k = B t k − 1 − Δ t + ( t ′ k − t ′ k − 1 ) = B t k − 1 − Δ t + V (2)
where V is equal to video chunk size (in second). The commercial HAS players have different values of V.
In order to maximize video rate, the perceived video quality defined by QoE has to be frequently monitored. According to [
In [
In this paper, the optimal monitoring interval is proposed to be equal to the size of video chunk. With the optimal monitoring interval, MOS monitoring method is expected to early detect video rate reduction with low computation cost. As the result, the control action is performed in early fashion. The effectiveness of the optimal interval is evaluated through evaluation criteria: ratio of video rate deterioration, average CPU load, detection time, and recovery time.
Mean Opinion Score (MOS) is defined as the “value on a predefined scale that a subject assign to his opinion of the performance of a system” and it is understood as the average of evaluation scores across subjects [
The aim of proposing an optimal interval is to maximize QoE by avoiding video rate deterioration when stimulus occurs. According to [
lim T → ∞ 1 T ∫ 0 T C t d t ≥ threshold (3)
Moreover, in order to guarantee that an expected encoding rate will be requested by HAS player, C t should meet the condition in Equation (4) (This equation actually is transformed from Equation (1)):
C t ≥ R ( t ) 1 − c o n s _ r a t i o (4)
Once those conditions in Equation (3) and Equation (4) are met during a streaming session, video rate will be maximized. However, the threshold has not been clearly identified by various HAS proprieties HAS. Instead of determining that threshold, other condition related to playback buffer is considered. A simple experiment was performed to determine the behavior of playback buffer and video rate when stimulus occurs.
The experiment’s scenario is as follow: a HAS player is playing a movie with high video rate under good network condition in which available bandwidth is high (around 5120 Kbps), packet loss, delay and jitter are negligible. Observing behaviors of both buffer occupancy level and video rate when:
1) The available bandwidth is dramatically decreased to 1024 Kbps at t = 20 s (before playback buffer reaches B max ).
2) The available bandwidth is dramatically decreased to 1024 Kbps at t = 60 s (after playback buffer reaches B max ).
There are two metrics were considered in this experiment t_delay_buffer, and t_delay_bitrate, that is, the duration time until the first adaption (or first change) of both playback buffer and video rate, respectively. The details of experimental setup are described as follows: there were three major entities including a client, a streaming server and a router. Microsoft smooth streaming player and a packet sniffer (Wireshark) were deployed at client, whereas Wireshark allows us to capture and analyze the traffic come from and to HTTP server offline. The router namely WAN Emulator is capable of controlling the available bandwidth of the client. During the experiment, the video rate was derived from HTTP GET packet header, whereas, the playback buffer was calculated through the Equation (2).
It is interesting to find that playback buffer always quickly reacts to the change of network condition. Thus, it can be used to predict video rate’s deterioration. The Equation (3) now could be replaced by other playback buffer related equation. Let B t k denote buffer occupancy of HAS player at time t k , whereas, let B t k − 1 denote the playback buffer size at time t k − 1 . Therefore, in order to keep video rate stable at expected level, buffer occupancy should not significantly decrease. So that:
B t k − B t k − 1 ≥ 0 (5)
Based on the Equation (2) in the background knowledge section, then we have:
Δ t ≤ V (6)
Thus, (4) and (6) now become the condition to prevent video rate from deteriorating within streaming session. It means that the stimulus has to be captured within Δ t ( s ) or before HAS player sends the next request. Therefore, the optimal monitoring interval is proposed to be t mon = V .
The purpose of this evaluation is to verify how elaborately the proposed interval facilitates maintaining the video rate level when the network condition is getting worse. More concretely, since the optimal interval of MOS monitoring is applied, the following metrics has been evaluated:
・ Ratio of video rate deterioration.
・ Average CPU load.
・ Detection time t d which represents how quickly video rate deterioration can be detected if compared with method which uses video rate as monitoring indicator.
t_delay_buffer | t_delay_bitrate |
---|---|
5.37 | 14.95 |
4.01 | 5.37 |
3.52 | 17.31 |
5.73 | 11.62 |
4.81 | 11.56 |
7.41 | 12.54 |
5.80 | 13.6 |
Recovery time t r of video rate which represents the duration time from when control action is generated until video rate is recovered to expected level.
In order to evaluate those criteria, two experiments were performed with environment setup as follow: A TestBed consisted of a router, a streaming server, and a client. Beyond routing and Nat function, the router played a role as a Controller which was installed on a VMware workstation of a desktop computer with Intel Core i5 3.10 GHz processor and 8 GB RAM. The Controller with QoE management algorithm (written in Python) [
For evaluating two first metrics, the experimental scenario was performed as follow: the estimated MOS was monitored with respect to interval t mon ∈ { 1 , 1.2 , 1.5 , 1.8 , 2 , 2.2 , 2.5 , 2.8 , 3 , 3.2 , 3.5 } . Meanwhile, the experimental procedure was:
1) A client starts watching a streaming video content.
2) Stimulus is generated in buffering state and steady state by decreasing
available bandwidth on purpose to make the network quality deteriorated (from 5120 Kbps to 1024 Kbps).
3) The packet loss, delay and jitter in the network and average CPU load in Controller (where QoE monitoring and QoE control are performed) are observed.
4) The deterioration is detected by observing the estimated MOS.
5) The available bandwidth to the user is immediately increased to recover the network quality when the deterioration of video rate is detected (from 1024 Kbps to 5120 Kbps).
Ratio of video rate deterioration is determined by ratio of the number of times the video rate decreases to the total number of times the experiment is repeated. Meanwhile, average CPU load stands for means of CPU load of the Controller in each experiment’s iteration. Particularly, with each value of t mon , the above procedure was repeated 10 times in total. Given that within 10 times, there is n times the video rate decrease ( n ≤ 10 ), even though control action has already been generated. Then, the ratio of video rate deterioration which is the ratio of n to 10 times of total was calculated for each value of t mon . Alternatively, the average CPU load of the Controller for each interval was also recorded.
The
buffering state than in the steady state. In this figure, during the streaming session, average CPU load showed a clear trend in which it linearly decreased across monitoring interval values from 14.46% to 8.18%.
Particularly, during the buffering state, an increase trend clearly could be seen in ratio of video rate deterioration when the monitoring interval was higher than 2 s. A slight fluctuation was found in range of between 1.5 s and 2 s. However, such fluctuation did not always occur when the whole procedure was repeated several times. Interestingly, the ratio reached to peak of 100% of video rate deterioration when monitoring interval larger than 3.2 s.
When monitoring interval was varied from 1 s to 2 s during steady state, the ratio of video rate deterioration was stable at lowest value of 0.1 of accuracy. However, when the monitoring interval was larger than 2 s, the ratio of video rate deterioration quickly rocketed to 0.6 of accuracy before witnessing a large fluctuation in range of between 2.5 s and 3.5 s. This fluctuation was also explained as the result of limitation of our QoE management algorithm performance. The algorithm frequently called PSQA model (written in MATLAB) by which it could generate some “spike” in Controller’s processing time. Actually, this abnormal fluctuation could not be seen when the experiment procedure was repeated several times.
The reasonable decrease trend of average CPU load was found from the graph. The smaller monitoring interval, the higher average load over time. In this experiment, the performance of computer was not too high and the number of users was small, then average CPU load was not a big problem. However, it will become more serious when the numbers of the users are extremely large. Interestingly, the line of average CPU load crossed by the line of ratio of video rate deterioration for the steady state at the point of interval of 2 s at which the average CPU load was equal 11.45% and the ratio of video rate deterioration was about 0.1.
For the detection time and recovery time criteria, MOS monitoring with defined optimal interval was compared with video rate-based method. The experimental procedure for two scenarios of the evaluation was as follows:
1) A client starts watching a streaming video content.
2) The available bandwidth is reduced on purpose to make the network quality deteriorated.
3) The packet loss, delay and jitter in the network are observed.
4) The deterioration is detected by observing the video rate and the estimated MOS.
5) The available bandwidth to the user is increased to recover the network quality when the deterioration of the video rate (for the first scenario) and estimated MOS (for the second scenario) are detected.
Initially, the capacity of the link from router to server was set to 5120 Kbps. Because there was only one client in the network, thus, the link capacity was equivalent to the available bandwidth of the client. The experiment time was 120 seconds for each scenario. At t = 20 s, t = 60 s and t = 90 s, the available bandwidth of the client was set to low level of 1024 Kbps. During streaming sessions, video rate was continuously captured, whereas, the estimated MOS was monitored in every t mon = 2 s .
at t = 60 s, the video rate also lately reacted to. It decreased to 2056 Kbps at 71.99 s, and even kept staying at that level, although the router had increased the available bandwidth to 5120 Kbps. To make matters worse, when the available bandwidth was reduced to 1024 Kbps t = 90 s, the video rate started decreasing more.
In
It could be seen that that the video rate always takes a large delay to adapt the available bandwidth compared to the estimated MOS. This is because the video rate does not change after detecting the network quality change. In fact, the player reacts, not to the latest fragment download throughput, but to a smoothed estimate of those measurements that can be unrelated to the current available bandwidth conditions. Particularly, in
In this work, an MOS optimal interval was proposed. To sum up, a condition to maximize video rate was established by taking into account playback buffer influence. In order to meet the condition, the monitoring interval was thus required to be equal to the size of video chunk (in second). By applying this interval, QoE management system could effectively maximize video rate during a streaming session. The effectiveness was represented by early detecting video rate deterioration, short recovery time, low CPU load and low ratio of video rate deterioration. The unstable performance of control algorithm was our limitation in this work. It will be improved in the next research.
In the future, an accurate QoE control will be considered which plays an important role in establishing an effective joint QoE management system.
Xuan-Tan, P. and Kamioka, E. (2017) Early Detection of QoE Deterioration in HTTP Adaptive Streaming. Journal of Computer and Communications, 5, 14-27. https://doi.org/10.4236/jcc.2017.514002