An ISP is a key component in an embedded camera system since a sensor provides the output only in the RAW format. An ISP (Image Signal Processor) is a dedicated processor that converts this RAW image data into a high-quality workable output through various processes such as noise reduction, lens shading correction, gamma correction, auto exposure, auto white balance, etc.
That said, a logical question that might arise is why image sensors don’t come integrated with an ISP. Why don’t sensor manufacturers such as Sony, ON Semiconductor, or OmniVision consider adding value to their sensor products by integrating a dedicated ISP?
In this blog, we break down the reasons why sensor manufacturers don’t offer a combination of a sensor and ISP as an integrated solution.
Why Image Sensors Are Not Offered With an ISP Anymore
Before we start looking at the reasons, a bit of history might help.
When we say that sensors don’t come with ISPs, was it always the case? Not really.
Sensor manufacturers used to offer ISPs as well in the past. The last known image sensor that continues to have an integrated ISP and is still selling in volumes is OmniVision’s OV5640, which is a 1/4-inch 5MP camera. However, as some of you may be already aware, OmniVision has decided not to launch image sensors with integrated ISP beyond 5MP anymore.
Hence, image sensors do not come integrated with ISPs anymore because of predominantly two reasons. Let us look at each of these in detail.
Growth of microprocessors with built-in ISP
Earlier, processors didn’t have a built-in ISP. But today, most of the modern manufacturers like Qualcomm, NXP, and NVIDIA offer an ISP with their microprocessors. Due to this, sensor manufacturers do not want to add to their product cost by integrating an ISP within. Since an ISP is not a value add anymore, sensor manufacturers do not want to run the risk of losing a design win or a bulk order on a budget or cost grounds.
Product developers want to choose the ISP based on tuning requirements
The second significant reason for manufacturers to come up with raw Bayer filter sensors is that many product developers and design engineers want to pick the ISP on their own based on the tuning they would require. It is also dependent on the features and the interfacing supported by the ISP.
Different ISPs come with different feature sets, and picking and integrating an ISP many times requires a thorough understanding of the sensor as well. One of e-con Systems’ key strengths is having a suite of products that come with different ISPs for different types of applications.
For example, Hyperyon – a 2MP ultra low light USB camera based on the Sony STARVIS IMX290 sensor – uses the Socionext ISP. On the other hand, e-CAM21_CUTX2 – Sony STARVIS IMX290 camera for NVIDIA Jetson TX2 – uses the NVIDIA Jetson ISP. While Hyperyon is targeted at low-light HDR USB 2.0 applications, e-CAM21_CUTX2 is suited only for low-light MIPI CSI-2 based applications.
External ISP vs Internal ISP
Now that we understand that an ISP comes integrated with a processor (which is when it is called an internal ISP), we need to look at why we might need an external ISP in some cases.
It is well understood that USB cameras definitely need an external ISP. So, the question of choice between an internal and external ISP arises only in the case of other cameras.
Even though processors come with an internal ISP, they have not yet become as sophisticated as external ISPs. External ISPs offer more flexibility and an array of features in comparison with internal ISPs. In complex use cases where multiple cameras (say as many as 6) need to be synchronized, an external ISP is recommended for better image output.
Also, some product developers who use NVIDIA processors do not want to use the internal ISP since it consumes additional GPU bandwidth for processing. They would instead prefer to use the processor for algorithm processing alone with the help of GPUs.
Therefore, the choice between an internal and external ISP depends on your application. The more complex your application, the higher the need for an external ISP.
Also, if you want to learn more about ISP tuning, and the significance of using an external ISP, read this blog.
A Cutting-Edge FPGA-based Image Signal Processor by e-con Systems
Since 2003, e-con Systems has been designing, developing, and manufacturing OEM cameras. We also provide TintE™, an FPGA-based ISP that helps deliver high-quality imaging for camera systems. It features a complete imaging pipeline, making it a ready-to-deploy ISP solution. With functionalities like debayering, auto white balance, auto exposure, gamma correction, and more, TintE™ offers a reliable image processing framework. It is built to work seamlessly across a variety of FPGA platforms, ranging from cost-effective options to high-end SoCs, all while ensuring consistent image quality.
With extensive customization options, TintE™ can be customized to meet unique image processing requirements.
Explore our Camera Selector Page to check out e-con Systems’ full portfolio.
If you require help in integrating the right camera with the right sensor and ISP into your embedded vision application, please write to camerasolutions@e-consystems.com.
Prabu is the Chief Technology Officer and Head of Camera Products at e-con Systems, and comes with a rich experience of more than 15 years in the embedded vision space. He brings to the table a deep knowledge in USB cameras, embedded vision cameras, vision algorithms and FPGAs. He has built 50+ camera solutions spanning various domains such as medical, industrial, agriculture, retail, biometrics, and more. He also comes with expertise in device driver development and BSP development. Currently, Prabu’s focus is to build smart camera solutions that power new age AI based applications.