OS/2 Multimedia Chapter – Video Extensions

by Maria Ingold

X.9 VIDEO EXTENSIONS

Before OS/2 Warp, the only ways to display video under OS/2 was by using the 16-bit full-screen VIO APIs from OS/2 1.X or by using the Presentation Manager (PM) GPI interfaces. While the VIO APIs do permit fast screen updates, they do not take advantage of the synchronization, device independence, and file format independence provided by OS/2 multimedia. The GPI provides device-independent graphics, but its performance is unacceptable for applications that require fast screen updates, such as software motion video or games.

X.9.1 Accessing Video Memory

In direct video output, as opposed to the indirect video output used by GPI’s GpiBitBlt, the OS/2 multimedia video stream handler works in cooperation with the display driver. The display driver provides an interface that allows video memory (VRAM) to be accessed directly. This is accomplished by querying the address to video memory, and determining how the memory should be accessed. Video memory can be accessed as either a single view, or aperture, or it may be split into several banks, and must be accessed one bank at a time. Remapping the current video bank is referred to as bank-switching.

While direct video display provides fast screen updates, the disadvantage is that each application or CODEC that displays directly to video memory has to handle its own bank-switching, clipping, color conversion, video scaling, and exploitation of any special hardware capabilities. With clipping, the application or CODEC has to know when other objects, such as windows, are partially or fully covering the video playback window. Only in the uncovered areas of the video playback window should video be seen. Color conversion is necessary since the video format used by the application or CODEC may not be the same as the encoding format required by the display device. A display may require 24-bit RGB values, but the image may be stored as 8-bit (256 color), or in a YUV format that stores luminance (brightness values) and chrominance (color) rather than RGB values. The image has to be converted between the different color formats to be displayed. Any scaling such as pel-doubling, pel-halving, or continuous scaling of the video has to be done at runtime by each application or CODEC that is doing the displaying. Some video cards provide scaling and overlay capabilities. Since these features can be done more quickly in hardware than in software, hardware accelerators are used when they are available.

Some applications may want direct access to video memory, since they may be able to make certain assumptions that optimize performance. Other applications or CODECs may want the ability to display quickly to the screen, but also want the convenience of having bank-switching, clipping, color conversion, scaling, and exploitation of hardware handled for them.

X.9.2 Direct Interface Video Extensions

With OS/2 Warp, a new multimedia API set called Direct Interface Video Extensions (DIVE) was introduced. The DIVE APIs evolved from the bank-switching, color conversion, and clipping technology developed for the software motion video stream handlers and Ultimotion and Indeo CODECs. With DIVE, applications can either write directly to video memory by gaining access to a pointer to the frame buffer, or use the DIVE blitter to perform fast screen updates, image scaling, clipping, color space conversion, bank-switch display support. While DIVE provides device-independence to the application developer, DIVE enables the use of special hardware features when they are available.

The DiveQueryCaps function provides information about the characteristics and capabilities of the display hardware. This includes determining if the display subsystem is bank-switched, and if direct access to memory is possible. Whether an application is going directly to video memory or using the DIVE blitter, the application then calls DiveOpen so that it can use the rest of the DIVE functions. DiveOpen also provides access to the frame buffer for applications that want to go directly to video memory. DiveCalcFrameBufferAddress gets the location in the frame buffer/aperture bank that corresponds to the area on the screen where the video, game, or other imagery is to be displayed. The information to be displayed is then written to that location in the frame buffer. Writes to the frame buffer must be bracketed by calls to DiveAcquireFrameBuffer and DiveDeacquireFrameBuffer. This ensures that only one instance or process can access the frame buffer at a time by locking the frame buffer for use by the current instance. Any other calls to access the frame buffer are blocked until the buffer has been deacquired. Again, when writing to the frame buffer, the application itself must handle any clipping, color conversion, sizing, and bank-switching. The DiveSwitchBank call selects the VRAM bank to be used on bank-switched displays.

When the DIVE blitter is used instead of direct VRAM access, the application or CODEC does not need to be concerned with how the displaying, sizing, clipping, and color conversion functions are performed as DIVE abstracts the actual implementation. Internally, the DIVE blitter may use its software emulation for these functions, or it may use video acceleration hardware if the hardware is present and it supports the function that is being performed.

Acceleration for the video graphics plane is provided to DIVE through EnDIVE (Enhanced DIVE). EnDIVE is a low-level interface that communicates via DevEscape calls to the display driver that is providing the hardware acceleration. As shown in Figure X.10, the application or CODEC is unaware of whether or not hardware assistance is present.

DIVE

Figure X.10 DIVE

To ensure that any available acceleration features are used, an application or CODEC that is using the DIVE blitter should allocate all source blitting buffers from DIVE using DiveAllocImageBuffer. The application also specifies the color format of the source image when making this call. See Tables X.14 and X.15 for a list of input and output color formats that are supported by DIVE. Before color data can be written to the buffer, the buffer must be accessed using DiveBeginImageBufferAccess. When all the data has been written, DiveEndBufferAccess is called to deaccess the buffer. DiveFreeImageBuffer is called to free the buffer when it is no longer needed.

Color Input Format Encoding Value
CLUT 8 (256 color lookup table) LUT8
8-bit greyscale GREY
RGB 16 (5-6-5, 5-5-5, 6-6-4) R565, R555, R664
RGB 24 (R-G-B, B-G-R) RGB3, BGR3
RGB 32 (R-G-B, B-G-R) RGB4, BGR4
YUV9 (Indeo three-plane color subsampled) YUV9
YUV 411 Y411
YUV 422 Y422
YUV CCIR601 (Three-plane 2×2 color subsampled – MJPEG, MPEG) Y2X2
YUV CCIR601 (Three-plane 4×4 color subsampled) Y4X4

Table X.14 Color Input Formats

Color Output Format Encoding Value
CLUT 8 (256 color lookup table) LUT8
RGB 16 (5-6-5, 5-5-5) R565, R555, R664
RGB 24 (R-G-B, B-G-R) RGB3, BGR3
RGB 32 (R-G-B, B-G-R) RGB4, BGR4

Table X.15 Color Output Formats

The application uses the DiveSetSourcePalette call if it needs to set the palette associated with the source color data. This is necessary, for example, if the application is using DIVE to blit 256-color palletized images to the screen. If an application is using DIVE to transfer 24-bit direct color images to an 8-bit display and the physical palette changes, the application uses the DiveSetDestinationPalettefunction to notify DIVE of this change.

Whenever the visible region, source color format, or image source or destination size changes (due to stretching the image), the application needs to notify DIVE by calling DiveSetupBlitter. Only one blitter setup is active per instance. This means that multiple source image buffers may be allocated for that instance, but they must either all have the same size and color space, or DiveSetupBlitter must be called between their usages.

The image is actually transferred from its source to its destination by using the DiveBlitImage call. The destination can be a non-screen destination buffer used for color format conversion, or it can be the screen. The screen can be either the graphics plane, or it can be an alternate plane such as an overlay plane. For a complete list and description of all of the DIVE functions see Table X.16.

Function Description
DiveQueryCaps Queries display capabilities
DiveOpen Opens a display engine instance
DiveClose Closes instance
DiveCalcFrameBufferAddress Calculates linear frame buffer address
DiveAcquireFrameBuffer Allows the frame buffer to be accessed serially
DiveDeacquireFrameBuffer Releases exclusive access to frame buffer
DiveSwitchBank Selects VRAM bank for bank-switched dispalys
DiveAllocImageBuffer Allocates a buffer to contain an image
DiveFreeImageBuffer Deallocates an image buffer
DiveBeginImageBufferAccess Begins access to the image buffer
DiveEndImageBufferAccess Ends access to the image buffer
DiveSetSourcePalette Sets palette associated with source data
DiveSetDestinationPalette Sets palette associated with destination data
DiveSetupBlitter Set up blitter operations
DiveBlitImage Transfers an image from the source to the destination

Table X.16 DIVE Functions

To ensure compatibility with the other PM windows on the screen, DIVE requests to be notified of visible region changes by calling WinSetVisibleRegionNotify at window initialization time. Once this has been set, the window procedure receives one of two window messages: WM_VRNENABLED or WM_VRNDISABLED. When the visible region changes because of a move or a size, or another window or icon is overlaying the DIVE window, the window procedure in which the video is being displayed receives a WM_VRNDISABLED message. This means that DIVE should not blit anything at this time. When PM is done, a WM_VRNENABLED message is posted. When this message is received, the DIVE application would query the visible regions, or areas that are left uncovered, by getting the presentation space, getting the rectangles that compose the visible regions, calling DiveSetupBlitter with this information, and then calling DiveBlitImage to display the image.