Enabling auto exposure control for monochrome sensors using NVIDIA ISP

Many embedded vision applications today require monochrome cameras to be placed in varying lighting conditions. Some of these applications include smart traffic management, auto farming, outdoor AMRs (Autonomous Mobile Robots), etc. Unfortunately, many monochrome sensors do not come with an inbuilt auto exposure engine, and depend on an external ISP to enable AEC (Auto Exposure Control). However, adding additional hardware or ASCII ISP is a costly affair. The alternate method is to go with an inbuilt ISP that works seamlessly with powerful host processors like the NVIDIA Jetson series which are capable of running heavy workloads and algorithms on them.

At the same time, it is to be noted that using an external ISP does come with certain advantages. Since it is beyond the scope of this article, read about it in the article Camera ISP and significance of using an external ISP in imaging solutions.

Now coming back to using an inbuilt ISP. For product engineers using NVIDIA as the host platform, e-con Systems offers AEC-enabled camera solutions. These include camera systems – that leverage the NVIDIA ISP – such as e-CAM222_CUMI2311_MOD (2MP OV2311 monochrome global shutter camera module) and cameras with monochrome sensors including AR0521. If you wish to look at the entire portfolio of e-con’s cameras for NVIDIA Jetson, please visit the NVIDIA Jetson cameras page.

In this article, we look at the pipeline used to enable AEC using the NVIDIA ISP for such camera solutions.

Auto exposure vs manual exposure

Before we even look at the AEC enablement methodology using NVIDIA ISP, let us first try to understand the terms exposure, auto exposure, and manual exposure.

Camera exposure is nothing but the overall brightness or darkness of an image. It is the amount of light that reaches the film or camera sensor when an image is captured. The more you expose the film or camera sensor to light, higher would be the brightness of the image. On the other hand, a low exposure time implies lesser light falling on the sensor, which in turn results in a darker image.

There are two methods to set the exposure of a camera – manual exposure and auto exposure. In the manual exposure mode, exposure time is fixed by means of manually adjusting the shutter speed of the camera. Auto exposure relies on the ISP block to set and manage shutter speed.

Setting auto exposure using the NVIDIA ISP

Setting auto exposure in a camera requires you to understand three things or activities:

1. NVIDIA camera architecture
2. Device tree settings for exposure control
3. Exposure handling in NVIDIA platform driver

Let us now look at each of them in detail.

NVIDIA camera architecture

To understand how auto exposure is enabled in NVIDIA ISP, it is important to know how the architecture of an NVIDIA based camera system looks like. Have a look at the below diagram which represents the ISP pipeline of an NVIDIA based camera.

A camera sensor can be accessed using two pipelines – camera core library interface and direct V4L2 interface.

Device tree settings for exposure control

In this section, let us look at the device tree settings required for enabling AEC.

1. Setting use_sensor_mode_id property value to “true” makes the NVIDIA platform drivers to use the camera exposure, gain, and frame rate related settings provided in the device tree. The below image depicts this configuration:
   
   

2. Following entries determine the minimum exposure time, maximum exposure time, step exposure time, and exposure factor for the camera sensor in that particular resolution.
   
   

3. The exposure_factor property determines the unit in which minimum, maximum and step exposure time are mentioned in the device tree. For example, in the code snippet above, the parameter ‘exposure factor =1000000’ defines that all the exposure time values are in the unit microseconds (us).

Exposure handling in NVIDIA platform driver

Now that we understand how to configure device tree settings, let us learn how to handle exposure at the NVIDIA platform end. Following are the steps required for the same:

1. Based on the scene, the NVIDIA ISP algorithm calculates the exposure and gain needed for the camera and communicates the values to the sensor driver through cam_ctrl_ops set_exposure ioctl as given below:
   
   

2. The cam_set_exposure function implemented in the sensor driver then sets the exposure for the camera sensor.

So that wraps up the entire process to enable auto exposure control in monochrome sensors using NVIDIA ISP. If you have any further queries on the topic, please feel free to leave a comment, or drop a note to camerasolutions@e-consystems.com.

Monochrome cameras from e-con Systems

Following is the list of monochrome cameras from e-con Systems:

• See3CAM_20CUG: 2MP OV2311 monochrome USB global shutter camera
• e-CAM222_CUMI2311_MOD: 2MP OV2311 monochrome global shutter camera module
• See3CAM_10CUG: 1.3MP monochrome USB global shutter camera
• See3CAM_CU135M: 4K monochrome USB camera
• See3CAM_CU55M: 5MP monochrome USB NIR camera

To enable AEC in e-con Systems’ monochrome cameras on NVIDIA platforms, an additional driver change is required. For any support on this, please write to camerasolutions@e-consystems.com.