Other Free Encyclopedias » Online Encyclopedia » Encyclopedia - Featured Articles » Contributed Topics from K-O

Multimedia Servers - Introduction, Multimedia Server Architecture, Buffer Management, Quality of Service and Admission Control, Client Subsystem

data video time network

Abhaya Asthana
Bell Labs, Lucent Technologies, Westford, MA

Seon Ho Kim
Computer Science Department, University of Denver, Denver, CO

Definition: Multimedia servers store and manage multimedia objects and deliver data streams in real-time, in response to requests from users .


The creation, storage and delivery of ubiquitous, high performance multimedia services present a formidable challenge to the underlying telecommunications and computing infrastructure. Nevertheless, recent advances in computing and communication technologies have made it feasible and economically viable to provide on-line access to a variety of information sources such as books, periodicals, images, video clips, and scientific data. With an efficient standardized data compression technology such as MPEG (Moving Picture Experts Group), huge video files become manageable in computer systems and transferable over networks. Computers with faster processors and storage subsystems make it feasible to support multiple simultaneous displays for large-scale multimedia services. Widely implemented residential broadband network with real-time aware streaming protocols such as RTSP (Real Time Streaming Protocol) are another driving force of the multimedia applications. Multimedia communication also suggests a rich sensory interface between humans and computers that gives the user control over the pace and sequence of the information. While the integration of multiple media, e.g., voice, video, image, and data, provides an effective means of communication and delivery, to effectively support the emerging IP Multimedia Services, great care is required to control the multimedia resources, the transport channels, and the transformation engines.

The architecture of such systems consists of multimedia servers that are connected to client sites via high-speed networks. A generalized environment supporting multimedia applications to homes or to business environments is shown in Figure 1 in which the user at home has access to a service through an access network, while a business user is connected through a local area network.

A multimedia server delivers services over a series of heterogeneous networks to the clients, and the need to provide flawless service to user places significant requirements on a large number of components, which have to work well together. These include:

  • Network subsystems with adequate bandwidth to transport, deliver and adapt the data streams isochronously (i.e., at a specified rate, without observable delay) to the clients.
  • Client subsystems that receive and/or pre-fetch the data streams and manage the presentation of data.
  • Application programs that deal with relationships among data frames and media segments, and manage user navigation and retrieval of this data.
  • High performance storage subsystem with data organization, buffering, indexing and retrieval mechanisms to support multimedia applications.
  • User satisfaction measured by real-time delivery and adaptability to the environment; Quality-of-service (QoS) management.
  • Security, especially management of content rights.

Multimedia applications utilize multiple audio and video streams and time-, user-, and data-dependent behaviors. Multiple applications: operate on the same data, manage information, analyze images, search for appropriate items, and mesh live, pre-recorded, and interactive streams. The multimedia server software framework makes it easy for application developers to create software that satisfies these needs.

Multimedia Server Architecture

Multimedia servers store and manage multimedia objects and deliver data streams in real-time, in response to requests from users. Additionally, they process the stored information before delivery to users. The content may range from long, sequentially accessed videos to composite documents consisting of a mixture of small multiple-media segments (e.g., video, audio, image, text). To manipulate these objects, the servers provide computation capabilities encompassing storage, manipulation, construction, distribution and synchronization. Multimedia servers may range from low-cost, PC-based simple servers that deliver a few streams, to scalable large (either centralized or distributed) servers that provide thousands of streams. Regardless of the server capacity, with given resources, the objectives of multimedia servers are to maximize the number of clients that can be simultaneously supported (high throughput), to minimize the latency between issuing request and initiation of display (low startup latency), and to support a smooth display without any artifacts, disruptions or jitters (continuous display).

Multimedia servers are similar to their predecessors: network file servers. However, they are faced with several added requirements. Traditional file servers are not designed to support the storage and retrieval of “continuous media” data such as video and audio. In a multimedia application, such data must be stored and retrieved at certain guaranteed rates. Clients of such services are permitted to retrieve multimedia objects from the server for real-time playback at their respective sites. Furthermore, the retrieval may be interactive, i.e. clients may stop, pause, resume, and even record and edit the media information. Sometimes many or all of the clients may play the same material. In such cases, multimedia objects must be played continuously at a constant rate once play begins. Therefore, multimedia servers must be scalable in order to serve very large numbers of clients. These challenges are met in part through the use of high-performance computing resources and efficient data storage system. Figure 2 shows a simplified diagram of a typical multimedia server. Servers can be centralized or distributed.

Large-scale servers that serve thousands of streams can be quite complex. The multimedia objects that are large and are played back sequentially require relatively large storage space and playback bandwidth. A common implementation of server software can consist of the following layers:

  • Data placement and retrieval scheduling: for load balancing, fault-tolerant design, efficient retrieval of data from disks to amortize seek time, and scheduling of system resources to avoid jitters.
  • Buffer management: for absorbing variance in data retrieval time and network delays, and for implementing memory-based techniques such as batching.
  • Quality of services and admission control: for maintaining the performance criteria of the server such as the maximum number of concurrent displays with a guaranteed smooth display.

Delivery of time-sensitive information, such as voice and video, and handling of large volumes of data require special considerations to produce successful applications. Digitization of video yields a sequence of frames and that of audio yields a sequence of samples. Since media quanta, such as video frames or audio samples, convey meaning only when presented continuously in time, a multimedia server must ensure that the recording and playback of each media stream proceeds at its real-time rate. Specifically, during recording, a multimedia server must continuously store the data produced by an input device (e.g., microphone, camera, etc.) so as to prevent buffer overruns at the device as shown in Figure 3. During playback, on the other hand, the server must retrieve data from the disk at a rate that ensures that an output device (e.g., speaker, video display) consuming the data does not starve.

Information retrieved from the disk is buffered prior to playback. Thus, the problem of efficiently servicing a single stream becomes one of preventing starvation while at the same time minimizing buffer space requirement and initiation latency. Since the data transfer rates of disks are significantly higher than the real-time data rate of a single stream, employing modest amount of buffering enables conventional file and operating systems to support continuous storage and retrieval of isolated media streams.

Network subsystems create delivery paths from the multimedia servers to the individual clients, and transport the media objects. To avoid long latency and the need for large client buffers, some media objects (e.g., large video and audio files) are streamed isochronously to the clients from the servers.

Raw bandwidth is not enough for effective delivery of multimedia services, especially when this bandwidth is shared among several systems or applications; it is necessary to provide mechanisms that achieve delivery of voice and video information isochronously, so that the corresponding network connections allow users to enjoy flawless delivery of media information. Special processor and disk scheduling is needed to ensure that disk blocks are read and delivered to the network on time. In addition, they must know the total capacity required to deliver a video stream to ensure that when a new play back is started it can be completed without jitter. Network technology and the communication protocol stack(s) used for multimedia information transmission clearly play an important role in achieving the above goals.

Buffer Management

Multiplexed data retrieval from storage subsystem inherently introduces variance in retrieval time of a unit of data, which may result in a jitter. To absorb the variance and provide a smooth display, double buffering has been widely used. One data unit from a buffer is being transmitted to a client while the next unit is being retrieved into another buffer. As long as the next data unit can be retrieved into a buffer before the starvation of data in the other buffer allocated to the same stream, there would be no jitter. Buffering can be extended to have more than two buffers per stream as a trade-off between memory space and a greater tolerable margin of statistical variance of data retrieval time.

In a large-scale service such as a video-on-demand server, multiple requests for the same popular video can arrive within a short time period requiring multiple playback streams from the storage subsystem. The requests arrived within a short duration of time (window) can be batched for service by a single playback stream, resulting in a higher throughput. A longer batching window allows more users to be serviced concurrently but introduces longer startup latency. Adaptive piggy-backing approaches provide a solution for a longer latency by gradually merging multiple streams into one.

Some memory buffer space can also be used as a cache for the underlying storage subsystem. In applications where objects have skewed access frequencies, the server may cache the most frequently accessed objects in memory for an expedited service. The cache performance is determined by the size of cache, the granularity of objects to cache, and the replacement algorithms.

Quality of Service and Admission Control

Given the real-time performance requirements of each client, a multimedia server must employ admission control algorithms to determine whether a new client can be admitted without violating the performance (QoS) requirements of the clients already being serviced. This implies, for example, that a video server has to know in advance the number of streams it can serve without jitter. The admission control relies on finding a delivery path from the storage devices to the selected network interface and reserving appropriate capacities on all components on this path for normal playback. The admission control may also rely on resource-optimization techniques, e.g., batching and caching.

The performance requirement of a multimedia server can be accomplished in two ways: deterministic and statistical guarantee. The performance objectives of an application that doesn’t tolerate any jitter are determined using a deterministic quantification with worst- case scenario. When some applications might tolerate some missed deadlines, one can determine the objectives within a tolerable statistical variation. For example, a few lost video frames, or the occasional pop in the audio may be tolerable in some cases – especially if such tolerance is rewarded with a reduced cost of service. Hence, a multimedia server may be able to accommodate additional clients by employing an admission control algorithm that exploits the statistical variation in the access times of media blocks from disk (or statistical variations in compression ratios, where applicable).

Client Subsystem

The design of the client subsystem is dominated by cost, functionality, and comfort considerations. The client device could be an upgraded television set (with internal or external “set-top” capability), or a full-fledged personal computer. The restrictions on local capabilities certainly affect which functions are performed well locally and which depend on server or network support. Limiting buffer sizes and local computing affect the load on the network. Classic VCR control operations can be implemented in many different ways. In addition, audio and video inputs (for video conferencing) result in significant implications for upstream bandwidth and control.

Searching image, audio, and video data is far more difficult than doing the same for text. Approaches to finding desired information include creation and analysis of text annotations, examining reduced versions of the video (thumbnails or key frames), and full- scale analysis of high-resolution data. Practical considerations force us to maximize the value received from a small amount of computation. The requirements become particularly onerous in interactive and real-time situations such as push technology, browsing, and interactive navigation in large sets of multimedia data.

Presenting the information is also difficult, especially when system and network capabilities are limited. Managing controlled collaborations and teleconferences with many audio, video, and data streams is even more challenging. In the future, people will want even more complicated composite moving media, with complex constraints on location and timing of the various parts.

The computer industry is providing demand and an expanding market for the key hardware technologies that underlie multimedia. Now that the multimedia server technology has been developed the marketplace will determine future direction. As a result, the application of multimedia, which appears expensive now, will become less expensive and more attractive. Finally, a fundamental understanding about composition and flow of multimedia services will help to define appropriate strategies for the development of next generation mixed mode communication, presence, location, and telephony services.

Multimedia Storage Organizations - Introduction, Data Placement,   [next] [back] Multimedia Sensor Networks - Multimedia Sensor Network Systems

User Comments

Your email address will be altered so spam harvesting bots can't read it easily.
Hide my email completely instead?

Cancel or

Vote down Vote up

about 7 years ago

no images are given

Vote down Vote up

about 7 years ago

You really helped me, thanks.

Vote down Vote up

over 4 years ago


Vote down Vote up

almost 8 years ago

whenever a new research comes about multimedia server just make mail me