ABSTRACT

Data-partitioning of IPTV video streams is a way of providing graceful quality degradation in a form that will work in good and difficult wireless channel conditions, as experienced by mobile devices. This paper’s proposal is to combine redundant slice protection along with an adaptive channel coding scheme that is also proposed in the paper. Adaptive channel coding is achieved by retransmission when necessary of additional redundant data to reconstruct corrupted packets. In the proposal, outright packet loss is provided for by a form of redundant slice protection. The paper finds that it is preferable: not to simply protect only the highest priority packets; that a moderate quantization level should be employed; and that video quality is differentiated by content type. It is important also to configure the partitioning correctly to remove inter-partition dependencies when possible.

1. Introduction

This paper proposes an error-resilience scheme for video streaming over a wireless access network with application to Internet Protocol TV (IPTV). The scheme combines data-partitioning as a form of error resilience with the addition of forward error correction (FEC) using rateless channel coding at the application layer together with retransmission of extra redundant data when required. Thus, the solution offered by this paper’s proposal is source-coded error resilience measures [1] which are employed in combination with adaptive rateless channel coding [2].

However, it is the adaptive channel coding system not the use of rateless coding in itself that is the contribution. In the proposal, adaptive channel coding is achieved dynamically by only sending enough redundant data to match the measured channel conditions. In addition, retransmission of extra redundant data occurs when necessary, as described in Section 3. As these measures are still insufficient if outright packet losses occur, the proposal further includes redundant slices which are transmitted alongside the original video stream. Outright packet losses can either arise because a packet is lost outright before reaching the receiver, as might occur as a result of entering a deep fade, or because a packet is dropped as a result of overflow at the base station’s send buffer. This then is the reason why the paper further proposes combining the aforesaid measures with redundant slice protection.

By way of comparison in [3], for multicast video stream distribution over the wired Internet, FEC was combined with stream replication. In this scalable video scheme [3], stream replication took place at each of the quality layers. The main motivation was that, because of packet loss, if insufficient FEC-bearing packets arrived at the receiver then sending these packets will have been in vain. Because of the multi-layered form of the video stream, the paper [3] concentrated on ways to optimize the mixture of FEC provision and stream replication. However, that paper differs from our one not just in the setting (it is not a wireless system) but in the goal of avoiding feedback.

In [3], as multicast distribution occurs, feedback is not allowed because of the risk of a feedback implosion. In the current paper, rateless coding is used specifically to allow a single request for additional redundant data. This is possible because the proposed protection system is intended for unicast distribution of Internet Protocol TV (IPTV) and because an IEEE 802.16e link, over which the system is demonstrated, represents a low-delay feedback channel by virtue of the time division duplex frames used.

In fact, application-layer FEC, in addition to physical layer protection, has been found necessary [4] for a number of error-prone network environments, because of the stringent anticipated requirements for IPTV [5]. The Digital Video Broadcast (DVB) project has specified [6] optional application-layer rateless coding, though not adaptive as herein. Similarly, 3GPP have also specified a scheme [7], though again not an adaptive coding scheme.

Our proposal may appear to contradict Shannon’s work on the separation of source from channel coding but as pointed out in [8], there are many practical situations, especially in the challenging area of video streaming across wireless links, where a combination of the two is necessary. Similarly, under some restricted conditions, according to Shannon [9], feedback does not necessarily improve the performance but nevertheless feedback can be justified when there is: a high risk of transmission errors; a need to achieve relatively high bit-rates for reasonable video quality; and/or error sensitive predictive coding is used.

IPTV is anticipated to be a key application of broadband wireless access networks such as IEEE 802.16e (mobile WiMAX) [10]. IPTV services include: live TV programs with or without interactivity; video-on-demand unrelated to the streaming of TV programs; as well as streaming of time-shifted TV programs [11]; the latter two of which certainly require unicast streaming.

Capacity studies for WiMAX [12] suggest up to 16 mobile TV users per cell in a “lossy” channel depending on factors such as the form of scheduling and whether Multiple Input Multiple Output (MIMO) is activated. The emerging IEEE 802.16m variant [13] is likely to further increase the capacity available for IPTV services, along with a corresponding improvement in device sophistication.

However, error bursts can still disrupt a fragile compressed bitstream, because of the source-coding data dependencies, which arise both from motion-compensated prediction and entropy coding within the codec. Consequently, sports scenes with high temporal complexity or those news scenes in which there is a high-spatial coding complexity are at risk, because of larger packet sizes and because of the difficulty of reconstructing pictures when prior or neighboring data are missing.

The paper now outlines the data-partitioning background in more detail in Section 2. Section 3 details the proposed adaptive rateless coding scheme for a datapartitioned compressed video stream. Section 4 describes the simulation model employed in the experimental results of Section 5. In Section 5, the proposed scheme is tested with the addition of redundant slice packets, using different levels of redundancy to gauge the impact. Also included in Section 5 are the results of applying a technique to separate out residual dependencies between the data-partitions. Finally, Section 6 makes some concluding remarks.

2. Error Resilience Methods

The H.264/AVC (Advanced Video Coding) codec standard [14] includes three main forms of error resilience: Flexible Macroblock Ordering (FMO) [15]; data-partitioning [16]; and redundant picture slices [17]. FMO allows the macroblocks (MBs) to be rearranged so that one picture slice (a unit of entropy coding synchronization) can aid the error concealment of another. However, in transmitting the FMO mapping there is some overhead, depending on the FMO type. Data-partitioning involves at the slice level the rearrangement of the compressed bitstream according to the reconstruction priority of the data components. There is less overhead than FMO and, hence, data-partitioning can operate during favorable channel conditions as well as unfavorable channel conditions.

On the other hand, redundant pictures slices should be turned off during favorable channel conditions as their transmission involves a significant overhead. To aid in the estimation of channel conditions, the IEEE 802.16e standard [10] specifies that a station should provide channel measurements, which can either be Received Signal Strength Indicators or may be Carrier-to-Noiseand-Interference Ratio measurements made over modulated carrier preambles. Therefore, to aid in this process the proposed method assumes one of these methods is implemented. However, further investigation of channel condition estimation is beyond the focus of this paper, as the subject has many technicalities.

2.1. Data-Partitioning

In H.264/AVC data partitioning, motion vectors (MVs) are packed into a partition-A packet, allowing motioncopy error concealment to partially reconstruct a picture despite missing partition-C packets containing texture data (quantized transform coefficient residuals). Partition-B slices contain intra-coded (spatially encoded) MBs which are substituted for inter-coded MBs according to encoder implementation (only the decoder input format is standardized in H.264/AVC).

To reduce processing complexity at the device, it is possible to specify the distribution of intra-refresh MBs across the video picture sequence. This arrangement avoids the rapid increases in data rate that result from intra-refresh through periodic I-pictures, which can be a problem for fixed capacity wireless links. The increase in data rates also results in increased buffering delay when I-pictures are sent. Gradual decoder refresh is possible, allowing channel swapping at suitable intervals, though this has not been implemented in this paper as it requires signaling to the server when channel swapping is feasible (under intra-refresh). The result of intra-refresh MB provision leads to a moderate increase in partition-B size.

The H.264/AVC codec conceptually separates the Video Coding Layer (VCL) from the Network Abstraction Layer (NAL). The VCL specifies the core compression features, while the NAL supports delivery over various types of network. Table 1 is a summarized list of different NAL unit types. NAL units 1 to 5 contain different VCL data that will be described later. NAL units 6 to 12 are non-VCL units containing additional information such as parameter sets and supplemental information. NAL units are then encapsulated in Real-Time Protocol (RTP) packets, as is normal for Internet video communication.

In the H.264/AVC codec, each video picture can be divided into one or more independently decodable slices; each of which contains a flexible number of MBs. The slices of an Instantaneous Decoding Refresh- (IDR-) or I-picture (i.e. a picture with all intra slices) are located in type 5 NAL units, while those belonging to a non-IDR picture (IP- or B-pictures [18]) are placed in NAL units of type 1, and in types 2 to 4 when data-partitioning mode is active, as now explained in more detail.

In type 1 NALs, MB addresses, MVs and the Discrete Cosine Transform (DCT) coefficient residuals of the encoding blocks, are packed into a packet, in the order they are generated by the encoder. In type 5, all parts of the compressed bitstream are equally important, while in type 1, the MB addresses and MVs are much more important than the DCT coefficients. Notice also that H.264/AVC to prevent reconstruction problems associated with floating-point representation makes use of an integer transform version of the DCT.

In H.264/AVC when data partitioning is enabled, every slice can be divided into three separate partitions and each partition is located in either of type-2 to type-4 NAL units, again as listed in Table 1. A NAL unit of type 2, also known as partition-A, comprises the most important information of the compressed video bitstream of Pand B-pictures, including the MB addresses, MVs,

and essential headers. If any MBs in these pictures are intra-coded, their DCT coefficients are packed into the type-3 NAL unit, also known as partition-B. Intra-coded MBS will occur either because they have been generated at intra-refresh MBs or because the encoder has sought to improve the resulting video quality by including some naturally encoded intra-coded MBs. Type 4 NAL, also known as partition-C, carries the (integer) DCT coefficients of the motion-compensated inter-picture coded MBs.

In order to decode partitions-B and -C, the decoder must know the location from which each MB was predicted, which implies that partitions B and C cannot be reconstructed if partition-A is lost. Though partition-A is independent of partitions B and C, Constrained Intra Prediction (CIP) should be set in the codec configuration [19] to make partition-B independent of partition-C. By setting this option, partition-B MBs are no longer predicted from neighboring inter-coded MBs. This is because the prediction residuals from neighboring inter-coded MBs reside in partition-C and cannot be accessed by the decoder if a partition-C packet is lost. There is a by-product of CIP in increasing packet sizes due to a reduction in compression efficiency but the increase in size may be justified in error-prone environments.

There is another dependency [19] arising from Context-Adaptive VLC (CAVLC) entropy coding, because the number of non-zero coefficients in one MB are predicted from the number of such coefficients in a neighboring MB. By design, setting CIP also results in setting the number of non-zero coefficients in data-partitioned inter-coded MBs to zero when CAVLC is in operation to code intra-coded MBs. Thus, partition-B can be made independent of partition-C. It is not possible to employ the alternative Context Adaptive Binary Arithmetic Coding (CABAC), as this option is not supported in the Extended profile of H.264/AVC, though this is the only profile in which data-partitioning is supported. As CAVLC still predicts from intra-coded MBs when coding partition-C’s inter-coded MBs, partition-C cannot normally be made independent of partition-B. However, the use of CAVLC is not a penalty of the scheme, as CABAC results in approximately 15% more computation than CAVLC, which is a drain on the battery of a mobile device and a challenge to the less powerful processors generally employed on these devices. The coding gain from using CABAC rather than CAVLC is around 5% - 10%.

3. Proposed FEC Protection Scheme

The basis of the proposed FEC protection of data-partitioned video scheme is rateless coding, which is employed in an adaptive manner by retransmission of additional redundant data as and when it is required. Raptor coding [20] is a systematic variety of rateless code that does not share the high error floors [21] of prior rateless codes. It also has O(n) decode computational complexity. Rateless codes are a probabilistic channel code in the sense that reconstruction is not guaranteed.

In order to model Raptor coding, we employed the following statistical model [22]:

(1)

where is the failure probability of the code with k source symbols if m symbols have been received. Notice that the authors of [22] remark and show that for k > 200 the model almost perfectly models the performance of the code. In the experiments reported in this paper, the symbol size was set to bytes within a packet. Clearly, if instead 200 packets are accumulated before the rateless decoder can be applied (or at least equation (1) is relevant) there is a penalty in start-up delay for the video stream and a cost in providing sufficient buffering at the mobile stations.

To establish the behavior of rateless coding under WiMAX the ns-2 simulator was augmented with a module from the Chang Gung University, Taiwan [23] that has proved an effective way of modeling IEEE 802.16e’s behavior. Ten runs per data point were averaged (arithmetic mean) and the simulator was first allowed to reach steady state before commencing testing.

We introduced a two-state Gilbert-Elliott channel model [24] in the physical layer of the simulation to simulate the channel model for WiMAX. To model the effect of slow fading at the packet-level, the P_GG (probability of being in a good state) was set to 0.95, P_BB (probability of being in a bad state) = 0.96, P_G (probability of packet loss in a good state) = 0.02 and P_B (probability of packet loss in a bad state) = 0.01 for the Gilbert-Elliott parameters.

Additionally, it is still possible for a packet not to be dropped in the channel but, nonetheless, to be corrupted through the effect of fast fading (or other sources of noise and interference). This byte-level corruption was modeled by a second Gilbert-Elliott model, with the same parameters (applied at the byte level) as that of the packet-level model except that P_B (probability of byte loss) was increased to 0.165. The main intention of the use of this twofold Gilbert-Elliott model was to show the response of the protection scheme to both types of fading. The Gilbert-Elliott scheme, though it is simple, has been widely adopted [25], as it is thought to realistically model the error bursts that do occur and, more significantly, can be particularly damaging to compressed video streams, because of the predictive nature of source coding. Therefore, the impact of error bursts should be assessed [26] in video streaming applications.

In the proposed adaptive scheme, the probability of channel byte loss through slow fading (PL) in the Gilbert-Elliott model serves to predict the amount of redundant data to be added to the payload. Recall that slow fading represents a change of wireless environment, whereas fast fading is transitory and would result in thrashing if the adaptive scheme responded to short-term variations in wireless conditions.

If P_GB and P_BG are the probabilities of going from good to bad state and from going from bad to good state respectively, then

(2)

(3)

are the steady state probabilities of being in the good and bad states. Consequently, the mean probability of channel byte loss is given by

(4)

that is the mean of a Uniform distribution in this case.

The instantaneous PL (taken from a Uniform distribution with mean PL_mean) is used to calculate the amount of redundant data adaptively added to the payload. In an implementation, PL is found through measurement of channel conditions (refer to Section 1 for an explanation of how is this is accomplished in WiMAX). If the original packet length is L, then the redundant data is given simply by

(5)

which adds successively smaller additions of redundant data, based on taking the previous amount of redundant data multiplied by PL.

Assuming perfect channel knowledge of the channel conditions when the original packet was transmitted establishes an upper bound beyond which the performance of the adaptive scheme cannot improve. However, we have included measurement noise to test the robustness of the scheme. Measurement noise was modelled as a truncated, zero-mean Gaussian (normal) distribution and added up to a given percentage (5% in the evaluation) to the packet loss probability estimate.

If it turns out that the packet cannot be reconstructed, despite the provision of redundant data then extra redundant data are added to the next packet. In Figure 1, packet X is corrupted to such an extent that it cannot be reconstructed. Therefore, in packet X + 1 some extra redundant data is included up to the level that its failure is no longer certain.

Rateless code decoding in traditional form operates by a belief-propagation algorithm [2] which is reliant upon the identification of clean symbols. This latter function is performed by PHYsical-layer FEC which passes up correctly received blocks of data (through a Cyclic Redundancy Check (CRC)) but suppresses erroneous data. Thus, in IEEE 802.16e [27], a binary, non-recursive convolutional encoder with a constraint length of 7 and a native coding rate of 1/2 operates at the physical layer. Upon receipt of the correctly received data, decoding of the information symbols is attempted, which will fail with a probability given by (1) for k > 200.

It is implied from (1) that if less than k symbols (bytes) in the payload are successfully received then a further k – m + e extra redundant bytes can be sent to reduce the risk of failure. In the evaluation tests, e was set to four, resulting in a risk of failure of 8.7% (from (1)) in reconstructing the original packet if the extra redundant data successfully arrives. This reduced risk arises because of the exponential decay of the risk that is evident from equation (1) and which gives rise to Raptor code’s low error probability floor.

4. Simulation Model

To evaluate the proposal, transmission over WiMAX was carefully modeled. The PHYsical layer settings selected for WiMAX simulation are given in Table 2. The antenna heights are typical ones taken from the standard [10]. The antenna is modeled for comparison purposes as a half-wavelength dipole, whereas a sectored set of antenna on a mast might be used in practice to achieve directivity and, hence, better performance. The IEEE 802.16 Time Division Duplex (TDD) frame length was set to 5 ms, as only this value is supported in the WiMAX forum simplification of the standard [12]. The data rate results from the use of one of the mandatory coding modes [10] for a TDD downlink/uplink sub-frame ratio of 3:1. The WiMAX base station (BS) was assigned more bandwidth capacity than the uplink to allow the BS to respond to multiple mobile sunscriber stations (SSs). Thus, the parameter settings in Table 2 such as the modulation type and physical-layer coding rate are required to achieve a datarate of 10.67 Mbps over the downlink. Notice that there is 1/2 channel coding rate at the PHY-layer of IEEE 802.16e, in addition to the application layer channel coding that we add. However, as discussed in Section 1, application layer coding is frequently used in wireless systems because of the high error rates that can occur.

Apart from the mandatory WiMAX PHY channel coding scheme, data is also randomized [27] prior to channel coding as a form of layer 1 encryption. Optional type 1 or type 2 hybrid ARQ is supported by WiMAX, though this was not simulated and, in practice it may be avoided, as it adds complexity to an implementation. Prior to transmission each physical-layer FEC block is interleaved. Interleaving of bits can take place across OFDM sub-carriers and within the modulation constellation. Thus, the purpose of interleaving is to improve the performance of the modulation itself.

In order to introduce sources of traffic congestion into the simulations, an always available FTP source was introduced with TCP transport to a second subscriber station.  Likewise a CBR source with packet size of 1000 B and inter-packet gap of 0.03 s was also downloaded to a third SS. While the CBR and FTP traffic occupy the non-rtPS (non-real-time polling service) quality-of-service queue, rather than the rtPS queue, they still contribute to packet drops in the rtPS queue for the video, if the packet rtPS buffer is already full or nearly full, while the nrtPS queue is being serviced. Buffer sizes were set to fifty packets as is normal to avoid delay and reduce energy consumption at the SSs.

Two video clips with different source coding characteristics were employed in the tests to judge content dependency. The first test sequence was Paris, which is a studio scene with two upper body images of presenters and moderate motion. The background is of moderate to high spatial complexity. The other test sequence was Football, which has rapid movements and consequently has high temporal coding complexity. Both sequences were VBR encoded at Common Intermediate Format (CIF) (352 × 288 pixel/picture), with a Group of Pictures (GOP) structure of IPPP… at 30 Hz, i.e. one initial I-picture followed by all predictive P-pictures. This structure removes the coding complexity of bi-predictive B-pictures at a cost in increased bit rate. Similarly, in H.264/AVC’s Baseline profile, B-pictures are not supported to reduce complexity at the decoder of a mobile device.

As a GOP structure of IPPP... was employed, it is necessary to protect against error propagation in the event of inter-coded P-picture slices being lost. To ensure higher quality video, 5% intra-coded MBs (randomly placed) were included in each frame (apart for the first I-picture) to act as anchor points in the event of slice loss. The JM 14.2 version of the H.264/AVC codec software was utilized according to reported packet loss from the simulator and to assess the objective video quality (PSNR) relative to the input YUV raw video. Lost partition-C slice packets were compensated for by motion copy error concealment using the MVs in partition-A at the decoder.

5. Experimental Results

The Paris sequence was initially simulated for transmission across the IEEE 802.16e channel. Before commencing the main tests, we assessed the relative sizes of the data-partitioned NALs when using a single slice per picture. Table 3 presents the results with and without CIP (refer to Section 2) showing the increase in partition-B and -C sizes that results from the loss in compression efficiency. Notice that NALs that might be above the maximum packet size of 1024 B (Table 2) were constrained to the maximum by the encoder when forming an RTP packet prior to encapsulation by IP/UDP headers. This means that these NALs are not segmented before reaching the link layer, avoiding the possible separation of header information from NAL data.

At lower QPs, i.e. higher quality video, the relative size of partition-C NALs means that the more important partition-A and –B packets are less likely to suffer channel error. However, the larger packet sizes mean that congestion may have more of an impact, because longer packets take longer to transmit and free the channel. At higher QP, the advantage of differential packet sizes is lost but the generally smaller packet sizes compensate to some extent. There is also a small growth in partition-B and –C packet size when CIP is turned on.

The following types of erroneous packets were considered: packet drops at the BS sender buffer and packet drops through channel noise and interference, especially slow fading; together with corrupted packets that were received but affected by Gilbert-Elliott channel noise to the extent that they could not be immediately reconstructed without an ARQ triggered retransmission of piggybacked redundant data. Notice that if the retransmission of additional redundant data still fails to allow the original packet to be reconstructed then the packet is simply dropped. The Raptor code equation (1) was applied to decide if a packet could be recovered, given the number of bytes that were declared to be in error.

Figure 2 shows the effect of the various schemes on packet drops when streaming Paris. Notice that this figure and subsequent figures are slanted towards the