Achieving a frame rate of 800 fps with e-con’s 13MP camera module

As embedded vision applications get more complex, the requirements on resolution and frame rate are also getting tougher. Traditionally, it was machine vision applications that required a very high frame rate. Though a frame rate of 30 to 60 fps is adequate in majority of the embedded vision applications, there are some specialized use cases that demand a frame rate in excess of 500 fps.

However, as you might be knowing, achieving a high frame rate is limited by factors such as the sensor resolution, bandwidth of the camera interface, platform processing capability, etc (we will look at these factors in detail in an upcoming section). Hence, achieving a high frame rate requires a different approach.

 

Why and where high frame rates are required?

Before we look at how to achieve a frame rate of more than 800 fps, let us understand why some embedded vision applications require it in the first place.

Simply put, a high frame rate camera module is required to capture objects which are moving or rotating at a very high speed. The phenomenon that leads to blurring in the image output due to the fast motion of the target object is called motion blur. A high frame rate camera with a low exposure time helps to eliminate motion blur. Examples of such use cases include a fast-moving car, a rotating fan, or a production line with items in fast motion.

However, it is to be noted that if the image output has rolling shutter artifacts, a high frame rate camera might not suffice. In such cases, a global shutter camera is recommended. To learn more about the two concepts, please visit the article Differences between rolling shutter artifacts and motion blur.

 

Factors that determine the frame rate of a camera

As mentioned before, increasing the frame rate beyond a particular value is limited by various hardware constraints. Following is the list of key parameters the frame rate of a camera depends on:

 

1. Sensor resolution: The higher the resolution of a camera, the larger will be the size of the data each frame has to carry. Since for a given bandwidth, a camera interface can only transmit a limited amount of data, a high resolution would imply a lower frame rate.
2. Interface bandwidth: Different camera interfaces have different bandwidths. For instance, USB 3.0 has a maximum bandwidth of 400 MB/s whereas MIPI offers a bandwidth of up to 320 MB/s (per lane).
3. Platform processing capability: There are various processors available in the market which are targeted for low-end as well as high-end applications. For example, NVIDIA’s processor series spans Jetson Nano to the more advanced Jetson AGX Orin with a wide range of computing performance ranging from 0.472 TFLOPS to 200 TOPS. They can be used for low-end applications to extremely complex edge AI and deep learning-based applications. The higher the performance, the higher will be the frame rate you can achieve, provided the other factors remain constant.
4. Exposure time: The maximum frame rate that can be achieved cannot exceed 1 divided by the exposure time in seconds. This would mean that if your exposure time is 50 milliseconds, the maximum frame rate you could achieve would be 1/0.05 = 20 fps. So the exposure time has to be set in a way that the desired frame rate can be achieved.
5. Pixel format: The format in which the output image is delivered (Bayer, RGB, mono) will also impact the frame rate. An RGB output consumes more bandwidth and processing compared to a raw Bayer output, and hence tends to deliver lower frame rates.
6. ROI width and height: The larger the ROI (Region Of Interest) width and height, the higher will be the number of pixels included in a frame, and hence larger will be the frame file size. So, a high ROI width or height can limit the maximum frame rate you could achieve.
7. Sensor bit depth: With increase in bit depth, the maximum frame rate that can be achieved will reduce.

 

Achieving a frame rate of 800 fps using a 13MP camera module

Now that we have looked at the factors that determine the frame rate of a camera in the previous section, it would be much easier to understand how e-con Systems’ expert engineering team was able to achieve a frame rate of 816 fps using its 13MP camera module.

This particular customization was done for the first time by e-con for one of the largest industrial automation companies in the world. So here, we will be understanding the process with respect to the values we considered for their camera-enabled quality inspection solution.

 

The application, camera module, and development kit

The end application of the client was to inspect the alignment of various parts on the mechanical shaft used in a factory. The shaft rotates very fast and hence required the camera to capture images at a very high frame rate. As mentioned before, we used the 13MP autofocus camera module with Variscite’s IMX8M-Mini dart development kit to develop this high frame rate solution.

 

Steps involved in achieving 816 fps

For this particular customer, e-con’s customization efforts were able to attain a maximum frame rate of 816 fps. Let us now look at the detailed steps involved in doing this:

 

1. Reducing the ROI (Region Of Interest): One of the ways to achieve a high frame rate is to reduce the resolution. Since the ROI required for this application was low, we were able to consider a resolution as low as 80×50.
2. Determining pixel format, sensor bit depth, and binning mode: To reduce the frame size, the image was delivered in the raw Bayer format at 8 bits. Also, 4×4 binning was enabled to enhance the sensitivity of the sensor.
3. Setting focus, brightness, and field of view: Once the above parameters are finalized, the camera has to be configured for brightness, focus, and field of view to achieve the desired quality.

In addition to the 80×50 resolution mode with 4×4 binning, we also tried the following modes for achieving higher resolutions:

 

• 320×64 resolution with 8-bit and no binning at 600 fps
• 648×480 with no binning at120 fps
• 648×480 with 2×2 binning at120 fps

For all these modes, GStreamer pipelines were provided. Also, a fast switch with a switching time of less than 0.7 seconds was provided to switch between these modes and the normal mode.

 

e-con Systems cameras with custom ROI support

Depending on the use case requirements, extensive customization can be done to meet the resolution, frame rate, and image quality requirements. Following is the complete list of camera modules from e-con Systems that can be customized for resolution, ROI, and frame rate:

 

• e-CAM131_CUMI1335_MOD – 13MP 4K Camera Module
• e-CAM130_MI1335_MOD – 13MP AutoFocus Camera Module
• See3CAM_50CUGM – 5MP IMX264 Monochrome Global Shutter USB Camera
• e-CAM56_CUOAGX – 5MP Sony® Pregius S IMX568 Global shutter Camera for NVIDIA® Jetson AGX Orin

Since building and integrating a custom ROI camera solution can be often complex, it is recommended for you to take the help of an imaging expert who has experience in building customized camera solutions to suit your application’s requirements.

If you are looking for help in integrating a camera into your product, please write to us at camerasolutions@e-consystems.com. You could also visit the OEM camera customization page to learn more about e-con Systems’ customization services.

To get a complete view of e-con’s camera portfolio, please visit the Camera Selector where you can browse through our cameras by features, interfaces, applications, etc.