When cameras first found their way into commercial use, they were embedded into CCTV cameras and machine vision systems. However, advancements in processing platforms augmented by innovations in camera technology gave rise to embedded vision – where camera modules are integrated into embedded systems with the processing of image and video data done on the edge using high-performance processors.
Along with this evolution, the advent of artificial intelligence also led to product developers coming up with innovative use cases where a single camera was not enough. And surround view system is such an application where a multi-camera system has to be used. In this article, we will learn in detail about the key factors you need to consider while integrating a multi-camera solution into a Surround View System (SVS).
What is a Surround View System?
As the term suggests, a surround view system is used to create a 360-degree view of the surroundings. This can be used in large vehicles or fleets for the purpose of eliminating blind spots to ensure a safe driving experience and prevent any possible accidents. The vehicle will have cameras covering the two sides, the rear, and sometimes the front. The videos from the cameras are fed into a dashboard display, using which the driver can easily take navigational decisions.
A surround view system can also be used in certain autonomous vehicles such as autonomous tractors and patrol robots. The key difference between an SVS used in an autonomous vehicle and a guided vehicle (like a heavy mining equipment) is that in the former, navigational decisions are taken automatically using edge-based analytics.
Integrating a multi-camera into a surround view system
To learn in general about the key considerations for a multi-camera integration, please have a look at the article ‘What are the crucial factors to consider while integrating multi-camera solutions?’
Here, we will dive deep into these parameters specifically for this use case.
Number of cameras
Wherever it is used, a surround view system can have anywhere between 3 to 10 cameras in order to ensure the complete surrounding of the vehicle or the device is covered. This will also depend on the extent of the area around the vehicle to be covered. In some cases, an SVS will have to cover only the sides and the rear, whereas in some cases the system will also have to offer a 360-degree view.
The number of cameras will also depend on the size of the vehicle or the device. The larger the vehicle, the higher the likelihood of the system needing more cameras. Another factor to consider is the desired quality of the video stream. To get a clearer view with more details and to avoid lens distortions, it is recommended to go with a higher number of cameras – whether you are using the SVS in a fleet management system or an autonomous mobile robot.
The camera interface
In order to transmit video data over long distances, it is recommended to go with a reliable interface like the GMSL. Hence, GMSL is the best choice for large vehicles and autonomous tractors. Using GMSL cameras also enables the transfer of high-resolution data at high frame rates to facilitate real-time surround view.
In robotic systems, if the cameras are placed at 1-2 meters from the host processor, a USB interface can be used. However, for better reliability, GMSL is preferred over USB.
The host platform
The host platform should be capable of handling the throughput delivered by the cameras. Considering the number of camera pipelines, it is recommended to go with high-performance processors like NVIDIA Jetson AGX Xavier or AGX Orin.
Method of synchronization
The key objective of a surround view system in a heavy vehicle is to eliminate blind spots and help the driver navigate the vehicle better. Similarly, in a patrol robot, the videos from all the cameras have to align perfectly with each other to create a 360-degree of its surroundings. Both these require the data from the three cameras to have frame-level synchronization. This in turn demands the system to have hardware-based synchronization, as the software-based method does not guarantee frame-level synchronization.
Latency
One key factor that is more critical in surround view systems unlike a lot of other multi-camera applications is latency. This is true especially in SVS in large trucks and vehicles. Since either the driver or the AI algorithm – in the case of autonomous vehicles – has to take real-time navigational decisions, the glass-to-glass latency has to be minimum. This is also one of the reasons why it is recommended to go with the GMSL interface, in addition to having the need to transfer data to long distances.
That wraps up the key factors you need to consider while integrating multi-camera solutions into your surround view system.
e-con Systems’ multi-camera solutions for surround view systems
e-con’s expertise spans USB, MIPI, and GMSL cameras with a wide range of options for each. Being an elite partner to NVIDIA, we have camera modules for all the Jetson processors including the latest NVIDIA Jetson AGX Orin. Other Jetson platforms supported by our multi-camera solutions include Xavier NX, TX2, TX2 NX, and AGX Xavier. Their resolutions range from Full HD to 18 MP with sensors from Sony, Onsemi, and Omnivision. The portfolio also features some of the most popular sensors available in the market including the Sony STARVIS series.
Let us first look at some of e-con’s flagship multi-camera solutions.
The first solution is STURDeCAM20 – IP67-rated Full HD GMSL2 multi-camera solution that comes with the high dynamic range feature. Based on the AR0230 sensor, STURDeCAM20 can transmit image and video data up to a distance of 15 meters from the host processor, making it suitable for applications such as autonomous mobile robots, fleet management systems, smart traffic systems, etc. This multi-camera is also compatible with our partner carrier boards from Connect Tech, Diamond Systems, Aetina, and Syslogic.
The next notable multi-camera solution from e-con Systems is STURDeCAM25 – IP67-rated Full HD color global shutter camera based on the AR0234 sensor from Onsemi. It is a GMSL2 camera that can transfer data reliably up to a distance of 15 meters with very low latency. With its global shutter efficiency, it can capture images and videos of fast-moving objects without any rolling shutter artifacts. The IP67-rated enclosure makes it suitable for use in harsh industrial and outdoor environments as well. With these features, STURDeCAM25 can be used in autonomous tractors, autonomous mobile robots, fleets & heavy vehicles, smart traffic systems, etc.
The last multi-camera solution we will look at is NileCAM81. It is a 4K GMSL2 multi-camera solution based on the AR0821 sensor from Onsemi. Coming with a large pixel side, this long-distance camera solution also has the HDR feature making it suitable for capturing images in challenging lighting conditions. Though the off-the-shelf solution comes without an IP67 enclosure, e-con will be able to customize the camera to add it depending on the end application requirement.
e-con’s unmatched engineering expertise helps us design multi-camera solutions using any of our other camera modules with ease and speed. In addition to the sensors mentioned above, our camera modules also house sensors such as:
- AR0233, AR0330, AR0521, AR1335, and AR1820 from Onsemi,
- IMX327, IMX290, IMX462, IMX415, IMX485, IMX412, and IMX298 from Sony,
- And OV5640 and OV2311 from Omnivision.
e-con Systems also offers a 3D time of flight camera and stereo cameras that can be used to measure depth for the purpose of autonomous navigation in systems like robots, automated wheelchairs, and autonomous tractors.
We have made camera evaluation easy for our customers by offering readily available multi-camera solutions with 2, 3, 4, 6, and 8 cameras in a single system. We also offer extensive customization services to make sure that the camera can be tailored to your unique application needs. Some of these services include form factor changes, firmware modifications, lens mount customization, and ISP finetuning.
In addition, our platform side expertise especially on NVIDIA has enabled us to integrate pretty much any sensor available in the market today with a wide variety of host platforms to build readily usable multi-camera solutions for product developers.
Check out our Camera Selector to have a look at our entire portfolio. If you need help in integrating a multi-camera into your surround view system, please write to us at camerasolutions@e-consystems.com.
Ranjith is a camera solution architect with over 16 years of experience in embedded product development, electronics design, and product solutioning. In e-con Systems, he has been responsible for building 100+ vision solutions for customers spanning multiple areas within retail including self service kiosks, access control systems, smart checkouts and carts, retail monitoring systems, and much more.