Inarguably, camera performance has become increasingly important across multiple industrial applications to solve real-world business challenges and meet customer expectations. Hence, camera manufacturers have been working doubly hard to enhance image quality while adding new embedded functionalities.
One of the important yet complicated parts of the process is capturing light from the camera sensor before transforming it into high-quality images. It involves a significant amount of processing expertise. Fortunately, embedded vision cameras come equipped with a dedicated ISP to handle the process effectively and effortlessly.
As the name implies, the Image Signal Processor (ISP) is used for processing images in embedded vision camera systems. The ISP also performs other operations on the captured image such as demosaicing, denoising, and auto functions that help deliver an enhanced image. Through this blog, we hope to bring to light some of the basics regarding ISP tuning, as well as the significance of using an external ISP in your imaging solution.
What is ISP Tuning?
ISP tuning is the process of adjusting an image signal processor to optimize performance and convert RAW images from image sensors into high-quality images.
Let’s look at the image processing pipeline used by ISPs.
Demosaicing
In an image sensor, each pixel records only one of three colors (Red, Green, Blue) filtered by the Bayer filter mosaic. ISP interpolates a set of complete red, green, and blue values for each point. Demosaicing is, basically, a part of the image processing pipeline used by ISPs to reconstruct a full-color image from the Bayer pattern images.
Denoising
With the presence of noise, possible subsequent image processing tasks, such as video processing, image analysis, and tracking, are adversely affected. Image signal processors apply denoising techniques to the data of images to mitigate noise.
Auto exposure
ISP ensures auto-exposure capabilities, which is the automatic fine-tuning of image brightness – according to the amount of light that reaches the camera sensor.
Auto white balance:
ISP applies color correction matrices in the captured camera image to adjust the colors and ensure proper color fidelity in the captured image.
The Architecture of an Image Signal Processor (ISP)
The architecture of an ISP is based on several key components that help process raw image data into a final output suitable for display.
Core components of the ISP architecture
The Image Signal Processor (ISP) is built around three main components: the A/D converter, the digital signal processor, and the memory unit. The process begins with the image sensor, which captures raw image data and converts it into an analog signal.
This analog signal is then transformed into a digital signal by the A/D converter, preparing it for further processing within the ISP.
Within the digital signal processor, several blocks handle specific tasks to refine the image. The color correction block adjusts the color balance to ensure a natural appearance, while the noise reduction block minimizes unwanted noise, enhancing image clarity. The demosaicing block converts the image from a mosaic pattern into a full-color image.
The white balance block fine-tunes the overall color temperature, ensuring accurate color representation, particularly for white objects. Lastly, the gamma correction block adjusts brightness and contrast to optimize the image for display.
The ISP processes the raw image data in a series of stages. It begins with the A/D conversion, where the analog signal is converted to digital. The digital signal is then processed through the various signal processing blocks, each performing its designated function. Finally, the processed digital signal may be converted back into an analog signal in the D/A conversion stage for display purposes.
External ISP vs. Internal ISP
While processors with internal ISPs are commonly available, they have yet to reach the level of advancement offered by external ISPs. External ISPs typically come with a broader range of features and greater flexibility. For instance, in situations where multiple cameras need to be
synchronized—such as a setup involving as many as six cameras—an external ISP is recommended to achieve better image output.
External ISPs are also important for USB cameras since these cameras need dedicated processing power that an internal ISP might not provide. Moreover, when dealing with processors like those from NVIDIA, developers often opt for external ISPs to conserve GPU bandwidth.
The internal ISP in such processors consumes additional resources, which could otherwise be allocated to algorithm processing. By using an external ISP, developers can reserve the GPU’s power for tasks that directly impact performance rather than image processing.
Ultimately, the decision between using an internal or external ISP depends on the application’s priorities. For applications where cost and power consumption are critical, an internal ISP might be preferable due to its integration and lower resource demands. However, if the goal is to maximize image quality and require a high degree of customization and flexibility, an external ISP would be the better choice.
Importance of Using an External ISP in Imaging Solutions
Not all application processors come with ISPs. Using one which doesn’t have an ISP can lead to decreased application performance. In such scenarios, an external image signal processor can be used to get the best out of the image sensor.
Since ISPs in processors have limited functionalities and tuning is not supported to a great extent, it causes time-to-market delays and disruptions. In the case of Internal ISPs, some may not support key features such as High Dynamic Range (HDR).
Therefore, it is recommended that an external ISP should be leveraged to get the desired quality images.
External ISP Camera Modules from e-con Systems
e-con Systems has 20+ years of experience in designing, developing, and manufacturing OEM cameras. Our proven expertise in ISP tuning and image processing is reflected in TintE™ – our FPGA-based Image Signal Processor (ISP) that delivers exceptional image enhancement to specific processing needs.
We offer a comprehensive, ready-to-use imaging pipeline that integrates smoothly into diverse applications, ensuring superior image quality. This pipeline comes with customizable blocks such as debayering, auto white balance, auto exposure, and gamma correction. TintE™ can also be adapted to various FPGA platforms, driving high-quality imaging across different system configurations without compromising performance.
Features of TintE™
- Support for 4k at 60 fps
- Scalability up to 8k (MIPI, USB, PCI)
- Support for 2, 4, 8 pixels per clock
- Low latency
- Support for I2C and APB interface
- Enhanced Debayer algorithm
- Black level and gamma correction
- Support for RAW 8, 10, 12 bit input data width from sensors
- Support for crop and binning
Other external ISP camera modules from e-con Systems include:
- e-CAM137A_CUMI1335_MOD – AR1335 Camera Module
- e-CAM131_CUMI1335_MOD – 4K Camera Module
- e-CAM130_CUMI1820_MOD – AR1820HS MIPI Camera Module
- e-CAM130_MI1335_MOD – 13MP AutoFocus Camera Module
- e-CAM55_CUMI0521_MOD – 5MP MIPI Camera Module
- e-CAM52A_MI5640_MOD – 5MP MIPI Camera Module
- e-CAM52A_5640_MOD – 5MP Parallel Camera Module
Use our Camera Selector to see our full portfolio
If you are looking for help integrating an external ISP-based camera into your embedded vision application, please write to camerasolutions@e-consystems.com