Other Free Encyclopedias » Online Encyclopedia » Encyclopedia - Featured Articles » Contributed Topics from K-O

Multimedia System-on-a-Chip - Introduction, A SoC for handheld Multi-media, Intel® XScale® Micro-architecture

power data frequency video

Nigel C. Paver and Moinul H. Khan
Intel Corporation, Austin, Texas, USA

Definition: Multimedia system-on-a-chip includes all components necessary to run contemporary multimedia applications; these components are a powerful processor, a SIMD coprocessor, SRAM memory, and a variety of interfaces.


Mobile multimedia is growing at a startling rate. This is fueling the trend toward rich multimedia and communications capabilities on mobile devices. End users in the handheld wireless market segment are demanding multimedia and communication experiences similar to those they enjoy on their desktop-but in a mobile setting. Video playback, multi-player gaming, and video conferencing are a few of the key applications driving the path to higher performance multimedia. The availability of more incoming multimedia data via wireless networks, camera sensors, and audio capture is feeding these ever hungry multimedia applications.

One of the biggest challenges for multimedia on mobile devices is to provide high performance with low power consumption. Playing a richly detailed 3D game on a phone or Personal Digital Assistant (PDA) can be highly enjoyable until the phone or PDA runs out of power. Another critical aspect of enabling multimedia on mobile devices is the availability of compelling applications and content. Providing a productive software design flow is the key to making this happen.

This article presents an architectural overview of the PXA27x processor family as a real world example of a multimedia system on a chip and the key components of the related embedded system.

A SoC for handheld Multi-media

In wireless and handheld platforms area, performance, power, and cost are key metrics for product success. The every shrinking form factor of cell phone and PDA devices is driving increasing levels of on-chip integration in state of the art application processors. The Intel PXA27x processor family is an example of a highly integrated System-on-a-Chip (SoC) targeting wireless and handheld platforms. Figure 1 presents an overall block diagram of the Intel PXA270 processor, and shows the Intel XScale microarchitecture and Intel Wireless MMX SIMD coprocessor as key features. It also includes 256 KB of SRAM memories which is useful in video and graphics applications as a frame buffer. The PXA27x processor also provides multimedia components such as an LCD controller, camera inter face, and a extensive set of peripheral interfaces such as UART, USB, AC97, SSP and I2S.

The level of integration provided in such a system on a chip is intended to provide the mainelements required to support the application subsystem without unnecessary external components thus reducing the system component count.

The primary source of multimedia content on handheld devices is from onboard interfaces such as camera interfaces, audio devices, and content stored on removable media such as flash memory cards. The resolution of camera sensors in handheld platforms is evolvingrapidly from VGA still image capture to megapixel resolutions more typically seen in digital cameras. The resolution and frame rate for video captured from the same sensors is following a similar growth path; today video can be encoded at QCIF (176×144) with QGVA and VGA resolutions on the roadmap for many product lines. The more pixels in an image or image sequence, the more computation power is required to process them.

The capacity of removable storage media is increasing today, 1-gigabyte cards are available today and 4 gigabyte and beyond are being introduced. With this amount of storage available it becomes possible to store many hours of VGA resolution video content. This is driving the need for an increase in both the computing power to process the VGA content and the power efficiency so that many hours of video can be viewed from a single battery charge. The increased capacity of the storage cards is also being utilized for audio content where many hundred of MP3 tracks may be stored. In this case power efficiency becomes critical to ensure playback time meets user expectations.

For a multi-media SoC it is important that interfaces are provided for the input and output of video data and also that enough critical memory card formats are supported.

Intel® XScale® Micro-architecture

The Intel XScale microarchitecture is an implementation of the ARM 1 V5TE architecture. The XScale core supports both dynamic frequency and voltage scaling with a maximum frequency today of 624 MHz. The design is a scalar, in-order single issue architecture with concurrent execution in 3 pipes that support out-of-order return. To support the frequency targets a 7-stage integer pipeline is employed with dynamic branch prediction supplied to mitigate the cost of a deeper pipeline. The memory subsystem contains 32 KByte instruction and data caches with corresponding 32 entries I translation look-aside buffer (TLB) and 32 entries D TLB. The memory subsystem also contains an eight entry write buffer that supports write coalescing and a four entry fill buffer to support multiple outstanding load operations.

Intel® Wireless MMX™ Architecture

Significant research effort and desktop processor development has been under-taken related to SIMD processing for media applications. Wireless MMX technology integrates equivalent functionality to all of Intel MMX technology and the integer functions from SSE 18 to the Intel® XScale© microarchitecture 27. Like MMX technology and SSE, Wireless MMX technology utilizes 64-bit wide SIMD instructions, which allows it to concurrently process up to eight data elements in a single cycle. This style of programming is well known to software developers.

Wireless MMX technology defines three packed data types (8-bit byte, 16-bit half word and 32-bit word) and the 64-bit double word. The elements in these packed data types may be represented as signed or unsigned fixed point integers. Using special SIMD instructions it is possible to operate on data elements in the packed format, where each data element is treated as an independent item.

It should be noted that, for multi-media acceleration other system level solution can be adopted, such as, dedicated HW acceleration solution. Trade-offs between these solutions are in terms of programmablity, cost and efficiency.

Multimedia Interfaces

The growing demand for multimedia processing on the converged platforms is driven by two primary factors. The first is the growing capability and resolution of the display devices. The second factor is the increasing supply of multimedia data arriving over the network and through onboard sensors such as cameras.

Cellular phone handsets in the past had very restricted display capabilities. This was limited to a few lines of monochrome text on a small LCD panel. The recent evolution in both display technology and available computing power is producing more advanced products with higher resolution displays. Figure 2 shows that the trend towards increasing resolution follows two tracks, depending on the physical size, or form factor of the product. The PDA form factor has historically been physically larger than a phone so has supported bigger display resolutions. Today quarter VGA (QVGA) displays (320×240 pixels) are common with VGA displays (640×480 pixels) emerging.

In the smaller physical form factor of the phone handset the common display resolution is around 176×144 size, with a trend towards QVGA (and ultimately VGA) as the data processing capabilities of the phone increase.

As the numbers of pixels in the display increases so does the processing power needed to calculate each pixel value. For example a VGA display typically takes four times the computation that a quarter VGA does to generate the content.

The ability to send, receive, and capture digital images and video has been one of the more important developments in the cell phone and PDA market segment in recent years. Management of data streams to image display and from capture resources becomes a necessary and critical aspect of the system design. Any inefficiency when dealing with video data streams has a direct impact on the user experience. The effect is often manifested with reduction in battery life and decreases in video frame rates and resolution. In order to address these issues, key multimedia features have been introduced with the PXA27x processor family. The features have been integrated with the multimedia interfaces used for image display and image capture.

There are a number of critical flows which target the LCD panel in handheld devices. These include the display of video data following decode for playback mode, the simultaneous display of two streams of video data used in a video conferencing mode, and also the display of a digital viewfinder stream when performing camera or camcorder functions. Figure 3 illustrates some possible multimedia streams.

The PXA27x processor family introduces both flexibility and coherency in the image display and capture peripherals. The quick capture interface provides the flexibility to connect to a wide variety of CMOS sensors using several possible interface options.

In addition, image data formatting using Wireless MMX technology facilitates efficient processing and display using the LCD controller. The LCD controller also provides the ability to connect to a wide variety of LCD panels using a number of programmable options. The image data may be received in several different formats including various flavours of RGB and YCbCr.

1 Other names and brands may be claimed as the property of others

Power Management

To support extended battery life Multimedia SoCs such as the the Intel® PXA27x processor family provide advanced power management. In the case of the PXA27x processor this power management is known as Wireless Intel® SpeedStep Technology. At the hardware level, the technology provides several power domains and modes. A dedicated PMU provides for interaction with the software control components of the system. The software components enable the hardware features to be integrated into embedded operating systems through performance and policy management strategies.

The PXA27x processor family supports six power modes. Three new power modes, Deep Idle, Standby, and Deep Sleep enhance the power management capabilities from previous designs.

The power modes are primarily differentiated by available functionality, total power consumption, and the amount of time to enter and exit a mode. As a function of workload and resource utilization, the device can transition between the power modes to minimze power consumption. Each of the six modes provide different levels of power consumption and resource availablity in addition to variation, the detail of each mode is provide in Table 1.

It is not possible to move arbitratily form one state to the next but a number of well defined transtions between power states are supported that correspond to appropriate platform power states. The state transition diagram between the different modes is shown in Figure 4.

Effective usage of the Power management hardware can be achieved through a software level management solution. The PXA27x family offers a generic framework to implement the management policy. Information is extracted about the system level activity from the OS, application, user preference and various other sources to decide optimal voltage and frequency settings. The performance management unit (PMU) can be used to monitor different system level activities (i.e. CPI-cycles per instruction, cache- efficiency etc.), providing additional information for dynamic voltage and frequency management. The software power management framework is shown in Figure 5.

There are two key components to the power manager software, the policy manager and the performance profiler. The policy manager receives task-level activity information {how many threads are running, or how often the processor is idle) from the OS. In addition, it can receive information from OS’s own power manager. The policy manager decides the power mode in the system, such as, when should the system be in sleep or deep- sleep. When the system is in the run mode the policy manager performs dynamic voltage and frequency management based on the inputs from performance profiler. The performance profiler periodically samples performance monitoring event counters and determines frequency for the different components in the system (i.e. core-frequency, memory frequency, system communication fabric frequency). Based on the periodic profile information, if the current mix of application is computation bound, the core-frequency may be raised and similarly, if the application is memory bound, the memory controller frequency may be raised. On the other hand, as the thread-level activity reduces core and memory frequency can also reduced. Based on the chosen operating frequency the voltage is adjusted as well. The performance profiler also communicates with the software-drivers for different peripherals so that peripherals can be turned off/on based on the usage. The policy manager and performance manager co-ordinate these transition acitivities. Depending on the target platform the policy can be optimized in terms of transition thresholds, frequency changing steps and method of profiling. There are many possible algorithms to sue for power management and the best algorithm to use varies from one platform to the next.

System in a Package

As well as providing high levels of integration in the system on a chip there is also an opportunity to increase levels of integration by incorporating multiple silicon die in a single package. In small form factor platforms such as cell phone the multiple die at stacked on top of each other to provide what is known as stacked multi-chip package or MCP for short. The PXA27x processor family supports a number of configurations of stacked MCP with the other die in the package being memory. A key advantage of the stacked packages in phone and PDA applications is the saving in PCB real estate required to implement the platform. This in turn allows smaller form factor products to be created. Figure 6 shows an example of how such multi-chip stacked package may be constructed.

Multimedia Web Information Systems - World Wide Web History, Multimedia on the World Wide Web [next] [back] Multimedia Synchronization Infrastructure and Engineering - Operating and Real-Time Systems, Middleware and Networking, Database and Data Management

User Comments

Your email address will be altered so spam harvesting bots can't read it easily.
Hide my email completely instead?

Cancel or