Everything you need to know about split-pixel HDR technology

Split-pixel HDR (High Dynamic Range) technology holds significant importance in the realm of embedded vision as it enables camera systems to capture and reproduce a wider range of brightness levels. It is also known as sub-pixel HDR technology as it involves splitting each pixel on the sensor into two sub-pixels to capture the brighter and darker areas of the scene. Since the camera system simultaneously captures images – not sequentially, this results in reduced motion blur and delivers more vibrant and true-to-life images. Basically, it provides a more accurate representation of colors – thereby delivering images that closely resemble the natural color tones of the scene.

Some of the popular use cases of split-pixel HDR include automotive imaging for ADAS and autonomous vehicles, medical imaging for accurate diagnostics, and more.

In this blog, let’s look at the different HDR modes, find out how split-pixel HDR works, and get insights into why this technology is gaining prominence.

What are the different HDR modes?

1. Multi-exposure or traditional HDR

By using both short and long exposures, this mode makes it easy to effectively capture a wide range of brightness levels, from the brightest highlights to the darkest shadows, in a single image. This technique helps preserve details in high-contrast scenes and low-light conditions, resulting in images with improved dynamic range and enhanced visual quality.

The short-exposure image is captured to preserve details in the highlight areas of the scene, while the long-exposure image is captured to retain details in the shadow areas. These multiple exposures are then processed and merged using specialized algorithms to create a final HDR image that combines the properly exposed parts of each exposure.

2. Split-pixel HDR

As previously mentioned, split-pixel HDR technology works by splitting each pixel on the sensor into two sub-pixels. One has a higher sensitivity to light (typically referred to as the “highlight” pixel), and the other has a lower sensitivity to light (typically referred to as the “shadow” pixel).

The highlight pixel captures details in the brighter areas of the scene, while the shadow pixel captures details in the darker areas.

How does the split-pixel HDR technique work?

A single pixel consists of one each of large and small on-chip microlens (OCL). The OCL of SP2 is located in the gap section of the OCL of SP1. One pixel is equipped with a “high sensitivity pixel that brightly projects a dark subject” and a “pixel that does not easily become saturated, even with bright subjects.”

The combination of a signal from those two pixels has made it possible to obtain a signal with a wide dynamic range so that even if the scene has high contrast, such as backlit conditions or tunnel entrances and exits in the daytime, there is no loss of blackout in the dark parts and no blowout in the bright parts.

A large sub-pixel is used for darker parts of an image, and a small sub-pixel for the mid-to-high light parts of image captures to extend the dynamic range. The big pixel collects more light and therefore saturates in brighter conditions.

Conversely, the small pixel can be exposed for longer but will not saturate easily because much less light is collected. The overall effect of having the large pixel working in low-light conditions and the small pixel in brighter conditions is to extend the dynamic range.

Top view of the pixel architecture

For the HDR merging, there are totally four 12-bit readouts from each pixel:

• SP1_HCG — Sub pixel 1 High Conversion Gain
• SP1_LCG — Sub Pixel 1 Low Conversion Gain
• SP2_HCG — Sub Pixel 2 High Conversion Gain
• SP2_LCG — Sub Pixel 2 Low Conversion Gain

For each frame exposure, four lines – SP1_HCG, SP1_LCG, SP2_HCG and SP2_LCG – are combined and generated as a 24-bit output value. It is then processed by the ISP, which is part of the camera module and outputs the 8-bit YUV data used by most applications.

This split or sub-pixel architecture helps capture a High Dynamic Range of 120dB – using the above technique.

 

Advantages of leveraging the split-pixel HDR technique

 Split-pixel HDR technology offers several advantages over traditional methods when it comes to capturing dynamic scenes with moving objects. While traditional HDR techniques using different exposures can result in motion artifacts or blur, split-pixel HDR technology overcomes this limitation. By capturing images at the same timing, it eliminates motion blur, ensuring that moving objects appear sharp and well-defined.

However, it’s important to note that split-pixel HDR technology may still have some rolling artifacts due to the nature of the capture process. Despite this, this technology remains advantageous compared to traditional HDR techniques as it effectively addresses the issue of motion blur, resulting in sharper and more detailed images of moving objects.

HDR cameras offered by e-con Systems

e-con Systems, having been designing, developing and manufacturing custom and off-the-shelf camera solutions since 2003, has developed a vast portfolio of HDR cameras.  We have a proven track record of successfully delivering camera solutions, as well as providing customization support, such as ISP fine-tuning, lens gluing/alignment, etc.

e-con Systems has released the STURDeCAM31 – a 3MP GMSL2 HDR IP69K camera, based on an ISX031 sensor and equipped with sub-pixel technology.

See3CAM_CU31 – 3MP Sony® ISX031 HDR Low light USB Camera

Some of the other HDR cameras in our portfolio are:

HDR USB Cameras

• See3CAM_CU81 – AR0821 4K HDR USB Camera with 1/2″ sensor
• e-CAM83_USB – 4K High-resolution HDR USB camera based on Sony IMX317
• See3CAM_CU20 – Full HD HDR USB3.1 Gen1 Camera
• See3CAM_CU22 – Full HD HDR USB Camera with LFM
• Hyperyon® – 2MP SONY STARVIS IMX290 Ultra Low-light USB camera

HDR Camera Modules

• Sony ISX031 Camera Module
• e-CAM82_CUMI0821_MOD – 8MP (4K) HDR MIPI Camera Module
• e-CAM27_CUMI290_MOD – 2MP IMX290 Camera Module
• e-CAM20_CU0230_MOD – 2MP Industrial grade HDR Camera Module

NVIDIA HDR Cameras

• STURDeCAM31_CUOAGX – 3MP 120dB HDR NVIDIA® Jetson Orin™ Camera for Autonomous Mobility
• e-CAM31_CUOAGX – SONY ISX031 120dB HDR camera for NVIDIA® Jetson AGX Orin™
• e-CAM81_CUNX – 8MP HDR camera for NVIDIA Jetson Xavier NX/TX2 NX/Nano
• NileCAM21_CUXVR – HDR GMSL2 multi-camera solution for NVIDIA Jetson AGX Xavier
• e-CAM20_CUXVR – Sony STARVIS IMX290 synchronized HDR multi-camera system
• SmarteCAM – IP66 rated AI HDR smart camera

GMSL2 HDR Cameras

• STURDeCAM31 – 3MP 120dB HDR Camera for Autonomous Mobility
• NileCAM21 – Full HDR GMSL2 HDR camera module with LFM
• STURDeCAM20 – IP67 rated Full HD GMSL2 HDR camera module

Smart HDR Cameras

• SmarteCAM – IP66 rated AI HDR smart camera

Visit our HDR cameras hub page for the full list or our Camera Selector page to view our entire camera portfolio.

If you are looking for help integrating HDR cameras into your embedded vision products, please write to us at camerasolutions@e-consystems.com.