Other Free Encyclopedias » Online Encyclopedia » Encyclopedia - Featured Articles » Contributed Topics from U-Z

Wireless Video - Introduction, Rate Constraints, Energy Constraints, Encoding Complexity and Encoder Power Consumption

communication distortion mobile compression

Zhihai He
University of Missouri-Columbia, Columbia, MO 65201 USA

Chang Wen Chen
Florida Institute of Technology, Melbourne, FL 32901 USA

Definition: Wireless video refers to transporting video signals over mobile wireless links.


The rapid growth of mobile wireless access devices, together with the success of wireless networking technologies, has brought a new era of video communications: transporting video signals over mobile wireless links. Transport of video content over mobile wireless channels is very challenging because the mobile wireless channels are usually severely impaired due to multi-path fading, shadowing, inter-symbol interferences, and noise disturbances. Traditionally, vision has been the dominant medium through which people receive information. Visual information, coupled with intelligent vision processing, provides a rich set of important information for situational awareness and event understanding. Incorporating the video capture, processing, and transmission capabilities into networked mobile devices will enable us to gather real-time visual information about the target events at large scales for situational awareness and decision making. This will create a potential impact through out the society via many important applications, including battle-space communication, video surveillance, security monitoring, environmental tracking, and smart spaces.

During the past decade, the video communication system has evolved from the conventional desktop computing and wired communication to mobile computing and wireless communication, as illustrated in Figure 1. In this scenario, the live video is captured by a camera on a portable device. The video data is compressed on-board and transmitted to remote users through wireless channels. As the communication paradigm evolves from the conventional point-to-point, wired and centralized communication to the current wireless, distributed, ad hoc, and massive communication, the system becomes more and more complex. More specifically, such massive wireless communication networks often involve a large number of heterogeneous devices, each with different on-board computation speed, energy supply, and wireless communication capability, communicating over the dynamic and often error prune wireless networks. How to characterize and manage the communication behavior of each communication devices within the network, and how to coordinate their behaviors such that each operates in a contributive fashion to maximize the overall performance of the system as a whole remain a central challenging research problem.

Over the past few decades, extensive research has been conducted on various elements of the wireless video communication networks, such as video compression, mobile ad hoc protocol design, energy-aware routing, power management and topology control. However, little research work has been done to bridge them into an integrated resource management and performance optimization framework. Developing efficient algorithms for real-time video compression and streaming over wireless networks to maximize the overall system performance under resource constraints has become one of the central research tasks in both signal processing and wireless communication research communities.

The ultimate goal in communication system design is to control and optimize the system performance under resource constraints. In mobile wireless video communication, video encoding and network communication operate under a set of resource constraints, including bandwidth, energy, and computational complexity constraints. To analyze the behavior of the mobile video communication system, manage its resources, and optimize the system performance, we need to study the intrinsic relationships between the resource constraints and the end to end video distortion. This study is called resource distortion analysis. This resource-distortion analysis extends the traditional R-D analysis by considering new resource constraints. In this article, we shall analyze the major resource constraints in real time video compression and streaming over wireless networks, and study the impact of these resource constraints on the overall system performance.

Rate Constraints

In traditional video communication applications, such as digital TV broadcast, and video-on-demand, video signals can be compressed offline, stored on a video server, and transmitted through the wired network to viewers upon request. In this case, the major constraint for video compression and communication is in the form of transmission bandwidth or storage space, which determines the output bit rate of video encoder. Therefore, the ultimate goal in this type of communication system design is to optimize the video quality under the rate constraint. To this end, rate distortion (R-D) theories have been developed to model the relationship between the coding bit rate and signal distortion . The R-D theory describes the performance limit of lossy data compression, and answers the following fundamental question: What is the minimum number of bits needed in compressing the source data at a given distortion level (or reconstruction quality).

During the last 50 years, R-D theories have been actively studied in the information theory literature, mainly focusing on performance bounds, including asymptotic analysis and high rate approximation. It should be noted that theoretical analysis and analytical R-D performance bounds are likely to be found only for simple sources and simple encoding schemes. For complicated sources, such as 2-D images and 3-D videos, and sophisticated compression systems, such as JPEG and JPEG2000 image coding, MPEG-2, H.263, MPEG-4, and H.264 video encoding, this type of theoretical performance analysis is often inapplicable. This is because: (1) Unlike 1-D text and acoustic data, whose compression characteristics can be easily captured by simple statistical models, such as Gaussian and Laplacian models, images and videos often exhibit very complicated source characteristics and correlation structure. The underlying scene structure of the 3-D environment, the time-varying motion patterns of scene objects, as well as the arbitrary camera movement, collectively define a very complicated source correlation structure in the video data. This type of correlation structure is often very difficult to be described by mathematical models. (2) Note that the major effort in image and video compression is to explore the spatiotemporal source correlation with various motion prediction and spatial transform techniques.

To explore the complicated source correlation structure of the video sequence, very sophisticated prediction and data representation techniques, such as multi-frame motion compensation, flexible macroblock (MB) size, intra prediction and mode decision, have been developed. These techniques, often seen to be ad hoc and difficult to be mathematically miodeled, however have significant impact on the overall video compression performance. The difficulty in mathematical modeling of both the source characteristics and the compression system creates a significant gap between the information-theoretic R-D analysis and practices in rate control and quality optimization for video compression. To fill in this gap, over the past two decades, as more and more advanced image and video compression algorithms are being developed and finalized in international standards, a set of R-D analysis and modeling techniques algorithms for practical video compression have been developed.

The analysis and estimation of R-D functions have important applications in visual coding and communication. First, with the estimated R-D functions we can adjust the quantization setting of the encoder and control the output bit rate or picture quality according to channel conditions, storage capacity, or user’s requirements. Second, based on the estimated R-D functions, optimum bit allocation, as well as other R-D optimization procedures, can be performed to improve the efficiency of the coding algorithm and, consequently, to improve the image quality or video presentation quality.

Energy Constraints

In wireless video communication, video capture, compression and network streaming operate on the mobile devices with limited energy. A primary factor in determining the utility or operational lifetime of the mobile communication device is how efficiently it manages its energy consumption. The problem becomes even more critical with the power-demanding video encoding functionality integrated into the mobile computing platform. Video encoding and data transmission are the two dominant power-consuming operations in wireless video communication, especially over wireless LAN, where the typical transmission distance ranges from 50m to 100m. Experimental studies show that for relative small picture sizes, such as QCIF (176×144) videos, video encoding consumes about 2/3 of the total power for video communication over Wireless LAN. For pictures of higher resolutions, it is expected that the fraction of power consumption by video encoding will become even higher. From the power consumption perspective, the effect of video encoding is two-fold. First, efficient video compression significantly reduces the amount of the video data to be transmitted, which in turn saves a significant amount of energy in data transmission. Second, more efficient video compression often requires higher computational complexity and larger power consumption in computing. These two conflicting effects imply that in practical system design there is always a tradeoff among the bandwidth R, power consumption P, and video quality D. Here, the video quality is often measured by the mean square error (MSE) between the encoded picture and original one, also known as the source coding distortion. To find thedist best trade-off solution, we need to develop an analytic framework to model the power-rate-distortion (P-R-D) behavior of the video encoding system. To achieve flexible management of power consumption, we also need to develop a video encoding architecture which is fully scalable in ower consumption.

Many algorithms have been reported in the literature to reduce the encoding computational complexity. Hardware implementation technologies have also been developed to improve the video coding speed. However, little research has been done to analyze the relationship between the encoder power consumption and its R-D performance.

Rate-distortion (R-D) analysis has been one of the major research focus in information theory and communication for the past few decades, from the early Shannon’s source coding theorem for asymptotic R-D analysis of generic information data, to recent R-D modeling of modern video encoding systems. For video encoding on the mobile devices and streaming over the wireless network, it is needed to consider another dimension, the power consumption, to establish a theoretical basis for R-D analysis under energy constraints. In energy-aware video encoding, the coding distortion is not only a function of the encoding bit rate as in the traditional R-D analysis, but also a function of the power consumption P. In other words:

which describes the P-R-D behavior of the video encoding system. The P-R-D analysis answers the following question: for given bandwidth R and encoder power consumption level P, what is the minimum coding distortion one can achieve? Generally speaking, this is a theoretically difficult problem, because power consumption and R-D performance are concepts in two totally unrelated fields. However, for a specific video encoding system, for example MPEG-4 video coding, one can design an energy-scale video compression scheme, model its P-R-D behavior, and use this model to optimize the R-D performance under the energy constraint (See the short article.)

Encoding Complexity and Encoder Power Consumption

In embedded video compression system design, the encoder power consumption is directly related to computational complexity of the encoder. In other words, the encoder power consumption Ps is a function of the encoder complexity C s , denoted by , and this function is given by the power consumption model of the microprocessor. To translate the complexity scalability into energy scalability, we need to consider the energy-scaling technologies in hardware design. To dynamically control the energy consumption of the microprocessor on the portable device, a CMOS circuits design technology, named dynamic voltage scaling (DVS), has been recently developed. In CMOS circuits, the power consumption P is given by

where V , f CLK , and C EFF are the supply voltage, clock frequency, and effective switched capacitance of the circuits. Since the energy is power times time, and the time to finish an operation is inversely proportional to the clock frequency. Therefore, the energy per operation E op is proportional to V . This implies that lowering the supply voltage will reduce the energy consumption of the system in a quadratic fashion. However, lowering the supply voltage also decreases the maximum achievable clock speed. More specifically, it has been observed that fCLK is approximately linearly proportional to. Therefore, we have

It can be seen that the CPU can reduce its energy consumption substantially by running more slowly. For example, according to (3), it can run at half speed and thereby use only 1/4 of the energy for the same number of operations. This is the key idea behind the DVS technology. Variable chip makers, such as Intel, have recently announced and sold processors with this energy-scaling feature. In conventional system design with fixed supply voltage and clock frequency, clock cycles, and hence energy, are wasted when the CPU workload is light and the processor becomes idle. Reducing the supply voltage in conjunction with the clock frequency eliminates the idle cycles and saves the energy significantly. In this work, we just use this DVS technology and the related power consumption model to translate the computational complexity into the energy consumption of the hardware.

Video Compression and Transmission under Energy Constraints

As mentioned before, the energy supply of a mobile communication device is mainly used by video compression and wireless transmission. For the power consumption in video compression and streaming, we have the following two observations. Case A: If we decrease the encoder power consumption P s , the coding distortion D s increases, which is due to lack of enough video processing. That is, Case B: Since the total power consumption P 0 is fixed, and P 0 = P s + P t , where P t is the transmission power. if we increase P s , and then P t decreases. This implies that less bits can be transmitted because the transmission energy is proportional the number of bits to transmit. Therefore, . It can be seen that when the encoding power P s goes too low or too high, the encoding distortion D s will be large. This implies that there exists an optimal power P s that minimizes the video distortion D s . In the following, based on a simplified power consumption model for wireless transmission, we study the performance of mobile video device. More specifically, we assume the transmission power is properly chosen such that the bit error rate at the receiver side is very low and the transmission errors can be neglected. In this case, the transmission power should be given by

where R s is the number of bits to be transmitted, d is the distance between the sensor node and the receiver(e.g., an AFN), and n is the path loss exponent. Therefore, and, Since the transmission errors are negligible, we have D t = 0 and D = D s . According to the P-R-D model,

It can be seen that D is a function of P s , denoted by D(P s ) . Using the P-R-D model, we compute the function D(P s ) in (7), and plot it in Figure 2. Here, the power supply of the wireless video sensor is P 0 = 0.3 watts. This is a typical plot of D(P s ) . It can be seen that D(P s ) has a minimum point, which is the minimum encoding distortion (or maximum video quality) that a mobile device can achieve, no matter how it allocates its power resource between video encoding and wireless transmission, given fixed total power supply. We call this minimum distortion as achievable minimum distortion (AMD). In Figure 3, we plot the AMD as a function of the power supply P 0 . For a given power supply P 0 , the AMD indicates the lower bound on the video coding distortion, or the upper bound on the video quality of a mobile video device.

It should be noted that the AMD bound in (7) is derived based on a simplified model of the mobile video device, and this bound is not tight. More specifically, first, it has not considered the transmission errors. The actual video distortion should consist of both the encoding distortion caused by Quantization loss and the transmission distortion caused by transmission errors. Second, the analysis assumes that the bandwidth of wireless channel is always sufficiently large. Obviously, this is not true for video transmission over the wireless channel which has a limited and time-varying bandwidth. In actual performance analysis for video compression and streaming over mobile devices, it is needed to incorporate the bandwidth constraint and the transmission distortion model into the AMD analysis, and study the achievable minimum distortion in video sensing over the error-prone wireless networks. Another important problem is the mobility of video communication device. The mobility of the device poses new issues in the AMD performance analysis, because it has to deal with various characteristics of the mobile wireless channel, including time varying path loss, shadowing, and small-scale fading.

Wireless Multi-hop Delivery, Video Adaptation, and Security Issues

Wireless video applications often involve transport over a collection of multi-hop wireless nodes to reach the destination. A multi-hop network is dynamically self-organized and self-configured, with the nodes in the network automatically establishing and maintaining mesh connectivity among themselves. This feature brings many advantages to multi-hop networks such as low up front cost, easy network maintenance, robustness, and reliable service coverage. However, limited network resource, severe interference/ contention among neighbor traffic, dynamic changing route, lack of QoS support, direct coupling between the physical layer and the upper layers, etc. pose many challenges for supporting communication over wireless multi-hop networks.

Another significant challenge in wireless video is to effectively deal with the heterogeneity of the wireless links and mobile devices. The needs for video adaptation have becoming more important as the advance of the wireless video has become widespread. With regard to wireless video streaming, there are generally two issues in video adaptation: rate adaptation and robustness adaptation. The objective of rate adaptation is to intelligently remove some information from the video signal itself so that end-to-end resource requirement can be reduced. A popular approach to video rate adaptation is the design of the video transcoding algorithms to bridge between two different networks. The objective of robustness adaptation is to increase the capability of the compressed video for transmission over error prone wireless links. Both error resilient source coding and error control channel coding have been used to increase the robustness.

Because wireless video rely on the wireless networks in which users communicate with each other through the open air, unauthorized users may intercept content transmissions or attackers may inject malicious content or penetrate the network and impersonate legitimate users. This intrinsic nature of wireless networks has several specific security implications. Sensitive and valuable video content must be encrypted to safeguard confidentiality and integrity of the content and prevent unauthorized consumption. A wireless device usually has limited memory and computing power. Battery capacity is also at a premium. Growth in battery capacity has already lagged far behind the increase of energy requirement in a wireless device. The security concerns mentioned above make it even worse since security processing has to take away some of the premium computing resources and battery life. To address the peculiar security problems for wireless video with significant energy constraint, we will need to design lightweight cryptographic algorithms and encryption schemes, to include security processing instructions into the embedded processors, and to integrate scalable coding and encryption with error resilience.

Wireless Video Adaptation [next] [back] Winters, Shelley

User Comments

Your email address will be altered so spam harvesting bots can't read it easily.
Hide my email completely instead?

Cancel or