RGB-IR sensors are one of the major evolutions in the embedded vision space. While traditional RGB color cameras are equipped with a CFA (Color Filter Array) with Bayer BGGR pattern, RGB-IR cameras come with dedicated IR pixels. With the RGB channel’s sensitivity, color cameras capture both the visible and IR wavelength – causing color corruption and triggering a dip in the image quality of the RGB output. Furthermore, measuring the IR light intensity in the captured image is significantly hard. This is where RGB-IR cameras brought about a revolution. They solve both the challenges making them more suitable for certain modern-day embedded vision applications than traditional RGB cameras. In this article, we look at some of the most popular embedded vision applications where RGB-IR cameras can be a game changer.
The rise of RGB-IR cameras
While most of the traditional camera systems use IR cut filters (for instance, most of the night vision USB cameras come with IR filters) to prevent IR light from falling on the sensor during the daytime, these had to be removed to adjust to capture low-light images during nighttime. But it led to serious ‘wear and tear’ of the camera systems’ mechanical components. This was because of the fact that this operation had to be done using a mechanical switch. Today, thanks to RGB-IR cameras. As shown in the image below, they come with separate IR pixels that allow only IR light to pass through them. This avoids the need for using mechanical filters.
Figure 1 – RGB-IR filter pattern
Looking for more deep-dive information on how RGB-IR cameras work? Read: What is an RGB-IR camera? How does an RGB-IR camera work? Now, let’s look at the most popular applications that rely on RGB-IR cameras and how they leverage this technology to execute their respective business use cases.
Top embedded vision applications of RGB-IR cameras
Smart traffic management
As you may be aware, progress can be paradoxical. So, with cities becoming more livable – it caused overcrowding, which in turn has led to traffic congestion and even a measurable spike in fatal accidents. Hence, smart traffic management systems powered by camera modules have gone from being nice-to-haves to must-haves as part of the urban planning process. One of the crucial features of such intelligent systems is cameras enabled with night-vision capabilities so that they can monitor vehicles during nighttime. You may assume that the latest low-light cameras can serve this purpose, but the reality is that many traffic management systems rely on RGB-IR cameras due to their IR lighting/imaging needs. As some of their functions include Automatic Number Plate Recognition (ANPR) and facial recognition, the camera must easily adapt to changing day-night conditions. Any loss in quality or color corruption would end up affecting the accuracy of recognition and the validity of the analysis. Furthermore, using a camera solution with RGB and IR capabilities can offer better longevity to the camera system.
Smart surveillance
Smart surveillance systems are camera-based devices placed in parking lots, shopping malls, supermarkets, airports, etc. In fact, smart traffic devices are a type of smart surveillance systems. These systems can be used for purposes such as people & vehicles counting, facial recognition, ANPR (Automated Number Plate Recognition), theft detection, etc. These systems have to operate in both day and night conditions, and hence become a perfect use case for RGB-IR cameras. They ensure superior IR sensitivity while at the same time preventing color corruption that typically occurs in RGB cameras.
Smart farming
Artificial intelligence and data analytics have enabled smart farming systems to reinvent the agricultural industries. There are many cost optimization opportunities for farmers and agritech companies to miss out on, not to mention the promise of faster and more streamlined day-to-day operations. While features like multi-camera synchronization and external trigger support can be important, NIR sensitivity is a feature that can’t be compromised upon in smart farming devices that need to carry out crop analysis using NDVI (Normalized Difference Vegetation Index) measurement. It can only be performed if the smart farming system is able to capture images in the Near-Infrared (NIR) region. Also, similar to smart traffic systems, they come with edge AI processors to run ML and DL algorithms for data analysis, which demand imaging clarity – day or night. RGB-IR cameras are perfectly suited for this as they come with high quantum efficiency in the NIR spectrum while ensuring 24X7 superior imaging quality – irrespective of changing light conditions.
Gesture recognition
RGB-IR cameras offer gesture recognition systems the ability to get RGB and IR data through dedicated pixels. While the IR data makes recognition of movement or gestures easier, the RGB data can be used to identify which part of the body is moving. This can be an added advantage in gesture recognition systems like gaming consoles, luxury vehicles and cars, medical systems, etc.
Facial anti-spoofing systems
Many anti-spoofing systems that are used to detect fraud need to capture both RGB and IR information. With the help of structured light (which essentially operates in the IR spectrum), these systems can detect whether an image or video is used to simulate a real person’s presence. This information combined with the RGB data obtained from the scene can be used to identify spoofing and false facial verification. RGB-IR cameras make for a perfect solution in this scenario, where the data from the RGB pixels can be used for object identification while the IR data can be used for analyzing variance of depth in the target scene (one of the techniques using which spoofing is detected is by checking for depth variations. An image or video shown on say a mobile screen will be flat whereas a real person’s face will be uneven).
Driver monitoring
Recent advancements in facial recognition have accelerated the time-to-market of monitoring systems for in-vehicle surveillance. They make sure that drivers remain alert while on the road – thereby avoiding collisions and creating a better driving experience. Driver monitoring systems have to be able to operate under dim light as vehicles are not well-lit inside. With RGB-IR cameras, they can capture the images of the driver in both visible and IR light. It helps further the device lifetime since there’s no need to use a mechanical filter to switch between visible and IR imaging. The RGB-IR camera also captures image/video 24X7 – collecting driver-centric data points in real-time for continuous improvement.
Biometric and access control systems
Biometric and access control systems provide people analytics – one of the most valuable commodities across several industries – from smart cities and border control to enterprise surveillance and modern retail. Obviously, the inability to effectively capture IR imaging data is a clear deal-breaker for such systems whose primary purpose is to enable accurate identification and verification of people. RGB-IR cameras come with high NIR spectrum response and a dedicated IR channel to distinguish the visible and IR image data with precision. This makes them suitable for such applications. However, it is to be noted that if the biometric system uses IRIS recognition for people identification and verification, a dedicated IR camera is more recommended since it offers better sensitivity. On the other hand, if the system uses facial recognition as the method of identification, an RGB-IR camera works better.
What is e-con Systems doing in the RGB-IR camera space?
As of today, e-con Systems is working on a state-of-the-art RGB-IR camera module for applications that require simultaneous visible and IR imaging. We are confident that our combination of sensor, optics, and ISP will majorly help overcome some of the biggest industry challenges. Also, with our track record of customization, there would be no need to sit through painfully long design lifecycles or complicate deployment dates. Don’t forget to stay tuned to our website and social media profiles for the latest updates! Please browse our Camera Selector page to get a full view of our portfolio. We certainly hope you were able to draw some insights to understand why RGB-IR cameras are such a big deal across industries. If you need any help in selecting and integrating the ideal camera solution into your embedded vision devices, please write to us at camerasolutions@e-consystems.com.
RGB-IR cameras from e-con Systems.
- See3CAM_CU83 – 4K AR0830 RGB-IR USB 3.2 Gen 1 Camera
- e‑CAM85_CUMI0830C_NIR_MOD – 8MP AR0830 RGB-IR Camera Module
- e-CAM227_CUMI2312C_MOD – 2MP OV2312 Global Shutter RGB-IR Camera Module
- e-CAM27_CUiMX8 – 2MP OV2312 Global Shutter RGB-IR camera for i.MX8 Processors
- e-CAM27_CUOAGX – 2MP OV2312 Global Shutter RGB-IR camera for Jetson AGX Orin™ / AGX Xavier™
- See3CAM_27CUG – 2MP OV2312 Global Shutter RGB-IR USB 3.2 Gen 1 Camera
Gomathi Sankar is a camera expert with 15+ years of experience in embedded product design, camera solutioning, and product development. In e-con Systems, he has built numerous camera solutions for robots, industrial handhelds, quality inspection systems, smart city applications, industrial safety systems, and more. He has played an integral part in helping hundreds of customers build their dream products by integrating the right vision technology into them.