Frequently asked Questions (FAQ)

1. What is the RGB-IR format and how it is different from Bayer RGB format?

The RGB-IR format is a novel image format in the CMOS image sensor technology. The traditional CMOS image sensor usually employ a color filter array (CFA) called as Bayer format. This Bayer format CFA allows each pixel of the CMOS sensor to be excited by a specified wavelength of light. More information about the Bayer format is provided below.

In the RGB-IR format, the CFA has been modified to pass infra-red light only (no visible light) to one of every four-pixel group. In a de-facto Bayer CFA, there will be 50% of Green pixels, 25% of Red pixels and 25% of Blue pixels. In the RGB-IR format, there will be 25% of Green pixels, 25% of Red pixels, 25% of Blue pixels and 25% of IR (only IR) pixels. The pixel format is given below:

The following table summarizes the difference between the Bayer CFA and RGB-IR CFA formats for a quick reference.

CFA Format	Green Pixels	Red Pixels	Blue Pixels	Infrared Pixels
Bayer	50%	25%	25%
RGB-IR	25%	25%	25%	25%

Note: In Bayer format even though there are no pixels for Infrared, all the Red, Green and Blue pixels will be sensitive to respective color wavelengths and also for Infrared. Hence in a Bayer format sensors, an Infra-red cut-off filter is placed to avoid the direct influence of infrared over other colors.
In the RGB-IR format, all the pixels are sensitive to Infrared light, except that the IR pixels are sensitive only to Infrared light.

The following two images differentiate the Bayer CFA and the RGB-IR formats.

As it can be seen from the above, the ‘G'  pixel neighboring the ‘R' pixel has been converted to Infrared pixel.

2. What is the RAW Bayer format and how it is different from standard RGB format?

The RAW Bayer format is not the same as the standard RGB format. The RAW format refers to the raw output from the CMOS image sensor which is based on the Bayer Color Filter Array (CFA). Each pixel of the CMOS image sensor picks only one color (either Red or Green or Blue) of the incident light. The Bayer pattern is one type of color filter array, where 50% of the pixels respond to Green color, 25% of pixels respond to Red and remaining 25% to Blue color. The sensor has a micro-lens array that filters the respective color (Red or Green or Blue) for the respective pixels. Each pixel of the CMOS sensor array responds to one of these primary colors and the other two colors for the same pixel has to be interpolated.

The above picture shows the Bayer filter geometry. For more information please visit http://en.wikipedia.org/wiki/Bayer_filter

3. How the RAW Bayer format is different from the standard RGB format?

The RAW Bayer format contains one color information per pixel. That color information may be Red or Green or Blue, depending on the position of that pixel.

In the standard RGB format, each pixel contains all the three primary color information.

4. How to get the standard RGB image from a Bayer RGB image?

As explained in questions above, the Bayer RGB format contains only one color information per pixel. The other two color information has to be interpolated from the near pixels. There are various interpolation algorithms available and this process of obtaining RGB image from a Bayer pattern image is called as “demosaicing”. There are lots of demosaicing algorithms each claiming its own advantages on image quality, noise, performance, chromatic noise reduction etc.

5. Learn how an RGB-IR camera works, what it can do for you, and why it's better than regular cameras in several embedded vision applications. Know more

6. Is the Demosaicing process alone sufficient to get the good quality RGB image from the Bayer pattern sensor?

No, the demosaicing is just one of the processes, but the most important process affecting the image quality. All the processing required creating a RGB image out of Bayer pattern image is collectively known as Image Signal Processing (ISP). The ISP pipeline has multiple processes and demosaicing is one of critical process. Other processes include Color Correction, Dead/Bad Pixel correction, Black Level compensation, Noise reduction, Gamma correction etc. And the ISP can also have Auto Exposure, Auto White Balance, Auto focus etc.

7. How do we get the YUV image from Bayer pattern sensor?

The standard method to get the YUV image from Bayer pattern sensor is to first reconstruct the RGB image and then convert the RGB image to YUV image.

8. How do we get the standard RGB image from the RGB-IR format image from e-CAM40_CUMI4682_MOD?

As explained, all the pixels are sensitive to Infrared and the IR pixels are sensitive to IR pixels only. The illustration of spectral characteristics of the OV4682 image sensor is given below. This is not an accurate graph, but can be used for illustrative purposes only. For more accurate spectral characteristics waveform, please contact camerasolutions@e-consystems.com

OmniVision recommends the following operation to get the RGB information from the RGB-IR image. Subtract the IR pixel value from neighboring R,G and B pixel values and then do the demosaicing operation for the R,G and B pixels to get the remaining colors for these pixels. The IR pixel will then be extrapolated from the neighboring R,G and B pixels. This is one suggestion and customers can play with this format and optimize this.

There are lot of algorithms on demosaicing and extrapolation and customers can use the algorithm that fits the most for their application.

9. What is the spectral response chart for the OV4682 sensor?

The spectral response curve of the OV4682 sensor is available from OmniVision under NDA. However, e-con has arrived at the OV4682 sensor spectral characteristics as illustrated below.

This is only for the illustration purposes only and is not accurate. However this spectral response curve illustrates the spectral response for R,G, B and IR pixels.

e-CAM40_CUMI4682_MOD Camera Module

10. Does the e-CAM40_CUMI4682_MOD support parallel interface?

No, the e-CAM40_CUMI4682_MOD supports only MIPI CSI-2 interface. In fact, the OV4682 sensor also does not have parallel data interface.

11. Does e-CAM40_CUMI4682_MOD support Autofocus?

The e-CAM40_CUMI4682_MOD camera module does not have autofocus mechanism. The current version supports only the S-mount (M12) lens holder only. Please contact camerasolutions@e-consystems.com for support on autofocus for the same module.

12. What kind of lenses can be used with this camera?

The camera comes with the M12 lens holder (sometimes called S-mount). Customers can choose any M12 lens based on their application and use with our camera.

13. What kind of optical filters do you provide with your default lens?

The default lens does not have any optical filter. The lens allows both the visible and IR light.

14. Does your default lens contain the IR cut-off filter?

No, the default lens that we supply does not have IR cut-off filter. It passes all the visible and IR light.

15. If I want to get both visible and IR images, which filter you recommend?

To get both the visible and IR, we recommend using a Dual band-pass filer that allows the visible light and the 850nm IR light. The typical spectrum of such filter will be as below.

The above filter is recommended for use with visible+IR applications.

16. Can I get 4MP IR image from e-CAM40_CUMI4682_MOD?

Yes, we can get full 4MP (2688x1520) IR images from the e-CAM40_CUMI4682 camera. All the 4MP pixels of the sensor are sensitive to Infrared and we can get 4MP IR images.

17. I want to use this e-CAM40_CUMI4682_MOD only with IR illumination. Is this ok?

Yes, the camera can be used only with IR illumination to get the IR image. Or alternatively, you can also use IR-only-pass filter to stop the visible light to get the IR image.

Evaluation Kit

18. How can I evaluate the e-CAM40_CUMI4682_MOD?

Customers can evaluate the e-CAM40_CUMI4682_MOD with the USB 3.0 interface on PC using our See3CAM_CU40 camera or on the NVIDIA Jetson board using our e-CAM40_CUTK1 kit.

19. Can I evaluate this on Denebola CX3 RDK?

No, at this moment we don't have any plans to support this camera module on Denebola CX3 RDK.

20. What software will I get to evaluate the See3CAM_CU40 on PC?

e-con provides the e-CAMView application with a basic Image Signal Processing filter to capture the RAW video from the See3CAM_CU40 camera and display both the RGB color video and IR video side-by-side. This application is only for the evaluation purposes and customers can develop their own applications using DirectShow or Windows Media Foundation framework provided by Microsoft for Windows platforms. For linux the camera is exposed as V4L2 device. e-con Systems® provides QtCAM application compatible with V4L2 to capture images from the See3CAM_CU40.

21. What software will I get to evaluate the e-CAM40_CUTK1 on Jetson board?

e-con provides the test application that captures the RAW RGB-IR frames from the camera. This is based on V4L2 API and customers can develop their own applications and implement the image processing software on their own.

22. What is the ISP library?

The ISP library performs all the image signal processing for the RGB and RGB-IR format inputs. At this time, e-con's ISP library has the support for the Bayer demosaicing and color correction and other modules are being developed. This library can do the Bayer demosaicing and also the RGB-IR demosaicing. This is a proprietary ISP library developed by e-con and the source code for this is not available along with the e-CAM40_CUTK1 camera board.

e-CAMView and DirectShow based Application Development

23. What is e-CAMView Application?

The e-CAMView application is a simple video viewer application built on DirectShow framework provided by Microsoft. This video viewer application is generic and can work with almost any UVC compliant video source such as integrated webcam etc. The e-CAMView application also supports all our USB UVC camera products.

24. What is DirectShow?

The Microsoft DirectShow application programming interface (API) is a media-streaming architecture for Microsoft Windows. Using DirectShow, your applications can perform high-quality video and audio playback or capture.

The DirectShow headers, libraries, SDK tools, and samples are available in the Windows SDK.

25. What is QtCAM Application?

QtCAM is e-con's open source Linux webcam software with advanced features. This Qt Linux camera software application provides easier user interface for capturing and viewing video from devices supported by Linux UVC driver. It would also work with any V4L2 compatible device.

For more information visit QtCAM Web page.

26. Will I get QtCAM Source Code?

Yes. You can download the QtCAM Source Code in the following link https://github.com/econsysQtCAM/QtCAM

27. What you mean by DirectShow filter? You refer Software ISP filter – what is it?

The building block of DirectShow is a software component called a filter. A filter is a software component that performs some operation on a multimedia stream. For example, DirectShow filters can

Filters receive input and produce output. For example, if a filter decodes MPEG-1 video, the input is the MPEG-encoded stream and the output is a series of uncompressed video frames.

When we refer Software ISP filter, this filter receives the RAW Bayer format video and converts the same to YUV422 format video.

In DirectShow, an application performs any task by connecting chains of filters together, so that the output from one filter becomes the input for another. And hence, the DirectShow application is just a place holder where the filters are connected together to perform a particular job(s) and then offers handle to control the operation of these filters.

28. I am not seeing the best image quality with e-CAMView and the evaluation kit for e-CAM40_CUMI4682_MOD.

As mentioned earlier, the e-CAMView employs a basic implementation of Sofware ISP for converting the RGB-IR output of e-CAM40_CUMI4682_MOD to YUV or RGB. This also employs a basic software extrapolation algorithm for the missing Green pixels. This Software ISP filter is a very primitive implementation and contains just the bare minimal modules of the ISP pipeline and this is not optimized for the high quality Image Signal Processing.

29. Will I get the source code of the Software ISP Filter?

No, we don't provide the source code for the Software ISP filter.

30. Can I get the source code of the e-CAMView application (without the Software ISP filter)

Yes, you can license the source code of the e-CAMView application by executing a Source Code licensing Agreement with e-con Systems® and paying an one-time royalty fee. Please contact camerasolutions@e-consystems.com to get the copy of SLA and the quotation for one-time license fee. Please note that e-con does not provide the source code of the Software ISP filter.

31. I don't want to pay and license the e-CAMView. I want to build my own viewer application. What should I do?

The first step is to download the Windows SDK which also contains the DirectShow SDK. Once you download and install the Windows SDK, then you can go the samples located at [SDK Root]\Samples\Multimedia\DirectShow folder. There are a number of sample applications that are provided by Microsoft as part of the SDK, which are used as a starting point for custom applications.

There are two capture applications that may be of interest for you.

The first sample application is called PlayCap, which is a very basic capture application that displays the video from the first capture device it locates in your system. This sample is available at the following location of your SDK installation:

[SDK Root]\Samples\Multimedia\DirectShow\Capture\PlayCap

The secnd capture sample shall be AmCap. This sample application demonstrates the following tasks related to audio/video capture:

This sample is available at the following location of your SDK installation:

[SDK Root]\Samples\Multimedia\DirectShow\Capture\AmCap

Both the samples are available with source code and you can use these samples to build your own application. For more information and help, please refer the SDK documentation or http://msdn.microsoft.com/en-us/library/windows/desktop/dd375454%28v=vs.85%29.aspx and http://msdn.microsoft.com/en-us/library/windows/desktop/dd375849%28v=vs.85%29.aspx

32. I don't want to use your evaluation kit for e-CAM40_CUMI4682_MOD. I want to get the RAW RGB-IR images directly from the sensor.

You can still use See3CAM_CU40 along with the associated firmware binary. But instead of using e-CAMView, you can write your own DirectShow application to grab the RAW RGB-IR format video directly from the sensor. You can implement a grabber filter (a type of DirectShow filter) and grab the RAW RGB-IR images and you can do any processing.

e-CAM40_CUMI4682_MOD Camera module customization

33. I want to customize the e-CAM40_CUMI4682_MOD camera for my application. I want to use e-CAM40_CUMI4682_MOD in my project. But I want to customize the camera module pinout, dimensions and also PCB shape. Can you supply customized versions of e-CAM40_CUMI4682_MOD?

Yes, we can customize the PCB, sensor orientation, lens mount or even the PCB shape. These are minimal level changes. All these customizations attract NRE charges. Please contact camerasolutions@e-consystems.com along with your customization request to get the quote for the same.