Most of the modern color cameras come with a CFA (Color Filter Array) with Bayer BGGR pattern. But the sensitivity of the RGB channel is such that it can capture both the visible and IR wavelength. This results in the reduced quality of the final RGB image output owing to color corruption. Also, this makes it difficult to measure the intensity of IR light in the captured image.
The traditional approach to overcome this was to use an IR cut filter for daytime imaging that would prevent any IR light from falling on the sensor. During the night, the IR cut filter is removed mechanically to allow IR light to come in to assist with low light imaging. However, this mechanical component leads to wear and tear over time and adversely affects the lifetime of a camera system.
An RGB-IR camera helps to overcome these challenges faced in an embedded camera system. An RGB-IR camera uses a new type of CFA with dedicated pixels for both visible and IR light. This way, images in both the visible and IR spectrum can be captured without having to use a mechanical switch, at the same time preventing any form of color corruption. In this article, we look at the working principle of an RGB-IR camera, its benefits, and some key embedded vision applications where it is recommended over normal cameras.
How does an RGB-IR camera work?
As mentioned before, the standard Bayer CFA comes with a BGGR pattern. The below image represents the pixels in the Bayer format.
But an RGB-IR camera comes with an additional set of pixels that are dedicated to allow only light in the IR spectrum to pass through them. Presence of these pixels facilitates multi-band imaging. Many sensor manufacturers have developed this new CFA that comes with a combination or R,G,B, and IR pixels as shown in the image below:
Benefits of using an RGB-IR camera
Given below are some of the benefits of an RGB-IR camera:
- Easily adapts to changing day and night conditions. This is very useful in applications that require round the clock image and video capture.
- Avoids the mechanical filter to switch between visible and IR imaging – increasing application lifetime and robustness
- Provides a dedicated IR channel to separate the visible and IR image data cleanly. This improves the quality of the RGB output by helping to measure the amount of IR light in the RGB component of the image accurately and do proper color correction.
How to leverage visible and IR imaging CFA
Having an RGB-IR filter alone is not enough to effectively use the imaging technique. It is important to choose the right camera components such as the sensor, lens, and ISP (Image Signal Processor) that support RGB-IR imaging.
Let us look at each of them in detail.
Sensor
You have to choose a sensor with IR sensitivity pixels on the CFA. Both onsemi and OmniVision offer various sensors that come equipped with RGB-IR imaging capabilities.
Have a look at the below image to understand the pixel pattern of a sensor that supports RGB-IR imaging
Optics
Generally, most lenses for color cameras come with an IR cut filter to block wavelength above 650nm. To facilitate RGB-IR imaging, it is pertinent to use a dual-band pass filter – allowing 400-650nm as well as 800-950nm for the IR wavelength.
Given below is the transmission diagram of a dual band pass filter:
Image Signal Processor (ISP)
On the ISP end, you have to implement an algorithm to separate the image data that contains both RGB and IR into two different frames. You have to interpolate the processed RGB output and IR frame simultaneously. It should also have an algorithm to subtract the IR contamination on the RGB channels to output the right colors. In addition, the ISP should have the capability to output only the processed RGB or IR frames as per the host system requirements.
Popular embedded vision use cases of RGB-IR cameras
Biometric (facial and iris) recognition
Advanced 24X7 security
With RGB-IR cameras, security and smart surveillance applications can overcome the challenge of color inaccuracies – which hinder the detection of objects or people. No matter the lighting condition – day or night – these cameras use an RGB-IR sensor and a dual band pass filter to capture high-quality images that help extract accurate information for analysis.
Automated Number Plate Recognition (ANPR)
ANPR applications need to detect letters, symbols, and colors of registration plates in varying lighting conditions. Using an RGB-IR camera, ANPR systems can reliably capture visible and IR images with an increased lifetime and accuracy.
To learn more about how to choose the right sensor for your ANPR system, have a look at the article How to choose the right image sensor for Automatic Number Plate Recognition (ANPR).
Also, if you are planning to develop an ALPR system with Raspberry Pi 4, the article Automatic number plate recognition with e-con Systems’ global shutter camera and Raspberry Pi 4 will be a good read.
e-con Systems and RGB-IR cameras: What’s happening now?
e-con Systems is currently developing a camera module based on an RGB-IR sensor to address the problems faced by customers in application that require both visible and infrared (IR) imaging. The camera comes with the right combination of sensor, optics, and ISP for seamless RGB-IR imaging.
At e-con Systems, we also know that building lighting-agnostic applications for biometric recognition or ALPR can be a complex process. Our deep experience in customization ensures that you can reduce design lifecycles that typically take a long time.
Keep an eye on our website and social media profiles to get the latest updates on what’s happening on the RGB-IR camera development front at e-con Systems.
We hope this blog helped you understand the intricacies of why RGB-IR cameras are a blessing when it comes to certain embedded vision applications.
RGB-IR cameras from e-con Systems.
- See3CAM_CU83 – 4K AR0830 RGB-IR USB 3.2 Gen 1 Camera
- e‑CAM85_CUMI0830C_NIR_MOD – 8MP AR0830 RGB-IR Camera Module
- e-CAM227_CUMI2312C_MOD – 2MP OV2312 Global Shutter RGB-IR Camera Module
- e-CAM27_CUiMX8 – 2MP OV2312 Global Shutter RGB-IR camera for i.MX8 Processors
- e-CAM27_CUOAGX – 2MP OV2312 Global Shutter RGB-IR camera for Jetson AGX Orin™ / AGX Xavier™
- See3CAM_27CUG – 2MP OV2312 Global Shutter RGB-IR USB 3.2 Gen 1 Camera
Prabu is the Chief Technology Officer and Head of Camera Products at e-con Systems, and comes with a rich experience of more than 15 years in the embedded vision space. He brings to the table a deep knowledge in USB cameras, embedded vision cameras, vision algorithms and FPGAs. He has built 50+ camera solutions spanning various domains such as medical, industrial, agriculture, retail, biometrics, and more. He also comes with expertise in device driver development and BSP development. Currently, Prabu’s focus is to build smart camera solutions that power new age AI based applications.