Performance Tuning on DirectShow Camera Application

Windows CE 6.0 supports DirectShow® components to develop a rich camera application. A minimum requirement of the camera application is to show preview, capturing the still images which is stored as a compressed file (JPG) format, capturing/multiplexing the video with or without audio i.e. stored as a compressed file (WMV) format.

Windows CE 6.0 provides sample applications for showing preview, capturing still and video using DirectShow® components. However these sample applications doesn’t cover all the aspects to develop a commercial application. A commercial camera application at least satisfies the following performance criteria.

1. Video Captured with sufficient frame rate.
2. User acceptable video processing time. Video processing involves converting and compressing the video frames to WMV file.
3. Quality and size of the captured video.
4. There shouldn’t be breaking in audio.
5. Still Image compression quality.

The performance criteria mentioned here is related to audio, video and still encoders. I have taken the WMV encoders for video, audio source filter and JPEG encoder for still image as an example for performance tuning.
DMO & WMV Encoder

Windows CE 6.0 supports WMV encoder as a DirectX Media Object (DMO) and it can be accessible through DMO wrapper filter. The DMO Wrapper filter enables DirectShow application to use the DMO within a filter graph. The filter wraps the DMO and handles all the details such as passing data to and from the DMO. Let us take WMV encoder. Performance tuning can be done by understanding and playing with properties of WMV encoder.

Accessing WMV encoder properties:
All WMV encoder properties can be retrieved by querying the IPropertyBag interface. Following steps are involved to retrieve and set each filter properties.

• Create the DMO Wrapper filter by the following way:

CComPtr pVideoDMOWrapperFilter;
CoCreateInstance( CLSID_DMOWrapperFilter, NULL, CLSCTX_INPROC_SERVER, IID_IBaseFilter, (void**) &pVideoDMOWrapperFilter );

• Retrieve the DMO wrapper filter as follows:

CComPtr pVideoEncoderDMO;
pVideoDMOWrapperFilter.QueryInterface( &pVideoEncoderDMO );

• Initialize the WMV video encoder filter using its class ID on the DMO wrapper filter.

pVideoEncoderDMO->Init(CLSID_CWMV9EncMediaObject, DMOCATEGORY_VIDEO_ENCODER );

• Add the filter to filter graph by using the following interface.

CComPtr pGraph;
pGraph-> AddFilter(pVideoDMOWrapperFilter, TEXT(“Video Encoder”));

• Retrieve the IPropertyBag interface by the bellow given way.

CComPtr pPropertyBag = NULL;
pVideoEncoderDMO.QueryInterface( &pPropertyBag);

• Each property value can be read by using IPropertyBag::Read ().

• Each property value can be modified by using IPropertyBag::Write ().

Tuning WMV encoder
There is a list of properties available for WMV filters, following properties are mainly considered to tune the performance.

Encoder algorithm complexity
Encoder algorithm complexity can be controlled through “_COMPLEXITYEX” property. This property covers the first two points in the performance criteria list. This property value is started from 0 to Maximum value supported by the codec.

Lowest value provides the real time processing of captured frames using simple encoding algorithm and it will provide user acceptable video processing time after finished the video capturing. But there is a possibility of missing frames and it is depends on the processing capability of the hardware.

Higher values provide quality output and no frames will be missed for encoding. In this case, it will use complex encoding algorithms but the processing time is more. After stop the video capturing, user has to wait for a long period to receive the video file.

Variable Bitrate Quality
Quality can be controlled through “_VBRQUALITY” property. This property covers the third point in the performance criteria list. This property indicating the quality level for quality based video encoding. The property value is started from 0 to 100. In this case, Quality is directly proportional to size of the video file. Higher property values increase the quality as well as the size of the video file. Users can fix the value based on their requirement on quality as well as requirement on size of the video file.

Tuning Audio
Audio quality can be tuned up by changing the audio input bit rate. Audio capturing can be multiplexed with video capturing and stored in the WMV file. Again increasing the bitrates will increase the audio quality and also the file size and lowering the bit rate reduce the file size and it may cause breaking on sounds. Changing bitrates are left to the user, based on their requirement.

Following steps involved in changing the bit rate of the audio pin:

• Load the audio capture filter.

CComPtr pAudioCapture;
CcomPtr< IPersistPropertyBag > pPropertyBag;
hr = CoCreateInstance( CLSID_AudioCapture, NULL, CLSCTX_INPROC_SERVER, IID_IBaseFilter, (void**) &pAudioCapture );
pAudioCaptureFilter.QueryInterface( &pPropertyBag );
pPropertyBag->Load( NULL, NULL );
pGraph->AddFilter(pAudioCapture, L”Audio Capture Filter” ); // Assume Filter graph manager is already created

• Retrieves the IAMStreamConfig interface for Audio capture filter. Assumed that Capture Graph filter is already created.

CcomPtr< IAMStreamConfig> pAudioStreamConfig;
hr = m_pCaptureGraphBuilder->FindInterface(&PIN_CATEGORY_CAPTURE, &MEDIATYPE_Audio, m_pAudioCapture, IID_IAMStreamConfig, (void **) &pAudioStreamConfig);

• Retrieves the current format of waveform audio data and set the new bit rate. New bitrates value can be assigned using WAVEFORMATEX structure. AM_MEDIA_TYPE contains the audio configuration data provided, when the IAMStreamConfig is retrieved from audio capture filter.

AM_MEDIA_TYPE *pmt; WAVEFORMATEX *pWft; pAudioStreamConfigDMO->GetFormat(&pmt);
pWft = (WAVEFORMATEX*)pmt->pbFormat; //Current waveform audio configurations
pWft->wBitsPerSample=8;
pWft->nBlockAlign=1;
pWft->nAvgBytesPerSec=8000;
pWft->nSamplesPerSec=8000;
pWft->nChannels=1; //Mono channel
pWft->wFormatTag=1; // Normally WAVE_FORMAT_PCM
pAudioStreamConfigDMO->SetFormat(pmt); // Assign the new waveform audio configurations

Average bytes per second (nAvgBytesPerSec) is calculated as follows:
nAvgBytesPerSec = nSamplesPerSec* wBitsPerSample/8 (since wBitsPerSample is maintaining the bit value)
Playing with these values make you to change the bit rate of the audio input.

Tuning the Still image quality
Windows CE supports image sink filter for capturing still image. Since image sink filter is using Imaging API, it will support JPG, BMP, TIFF and GIF etc…. Image sink filter uses the following registry to obtain parameter values that are passed on to the image encoding factory.

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\DirectShow\ImageSink\Quality\ <Quality Level> \ <File Extension> \ <Parameter Name>

GUID – BINARY (The GUID for the encoding parameter)
Value – DWORD (The number of parameter values that are contained in Value)
Type – DWORD (The data type for the parameter)
NumberOfValues – DWORD (The value of the parameter)

There are set of encoding parameters such as Quality, compression and scan method etc… All these encoding parameters can be added through registry.Following registry setting is an example for tuning the Quality for JPG encoding.

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\DirectShow\ImageSink\Quality\ <Quality Level>>\jpg\Quality
“GUID”=”1D5BE4B5-FA4A-452D-9CDD-5DB35105E7EB”
“Value”=dword:00000019
“Type”=dword:00000004
“NumberOfValues”=dword:00000001
is started from 0 and it represents the lowest quality level and each represents the unique quality level. The sub keys GUID, value, Type and NumberOfValues are passed as a parameter to image encoding factory using the structure StructEncoderParameter. Required Encoding parameters can be set through registry for the same by changing the GUID, Value, Type and NumberOfValues.

GUID represents that the encoding parameter is quality. The range of the value is from 0-100. Lower value represents the lower quality and increasing this value will increase the quality. User can set the required quality level using IImageSinkFilter::SetQuality(). Quality parameters for the current image encoding can be taken from the above given registry based on the . The Value ‘n’ passed as an argument to the SetQuality() is used to select the corresponding registry by comparing with of the above given registry.
Steps to use the IImageSinkFilter::SetQuality()

• Load the Image Sink Filter

CComPtr pStillSink;
CComPtr pImageSinkFilter;
hr = CoCreateInstance( CLSID_IMGSinkFilter, NULL, CLSCTX_INPROC_SERVER, IID_IBaseFilter, (void**) &pStillSink );
hr = m_pGraph->AddFilter( pStillSink, TEXT(“Image Sink Filter”)); //Add the filter to filter graph.
pStillSink.QueryInterface( &pImageSinkFilter );

• Set the required quality parameter.

hr = pImageSinkFilter->SetQuality(n);
Where ‘n’ represents the <Quality Level>