How embedded vision is modernizing patrol robots

With the need for edge-based AI and analytics on the rise, the role of embedded vision in automating various tasks across industries has also become more critical. And the use of camera technology in AMRs (Autonomous Mobile Robots) has been pivotal in reducing human dependency in making intelligent decisions with the aid of automation.

Of the many types of camera-enabled AMRs, patrol robots have gained more popularity recently. They are used for outdoor security and inspection purposes in places such as:

• Warehouses
• Hospitals and other medical facilities
• Large office buildings
• Factories and manufacturing plants
• Mines and construction sites
• Chemical plants and oil & gas facilities
• Hotels, gated communities, and amusement parks
• Large sports arenas as such as stadiums and golf clubs

To perform their tasks effectively, patrol robots need to use embedded cameras that integrate perfectly into their system. In this article, we will look at how cameras and embedded vision are helping patrol robots redefine security and surveillance across multiple industries.

The role of cameras in patrol robots

As mentioned above, patrol robots are deployed for unmanned security and surveillance. And this for sure requires them to have cameras for the purposes of:

• Autonomous navigation
• Video capture for real-time detection of theft and intrusion
• Video capture (most of the time 360-degree) for remote surveillance or analysis
• Facial recognition for identifying intruders and other people

Let us dive deep into each of these.

Autonomous navigation

Patrol robots need to be able to seamlessly move around their environments without running into any object or human. This is made possible by using 3D depth cameras that help them measure depth, thereby detecting any nearby obstacles. A combination of camera-enabled techniques – such as mapping, localization, navigation/path planning, obstacle detection & avoidance – and odometry is used by robots to navigate with minimal or no human supervision. The different types of depth cameras used in robots include time-of-flight cameras, stereo cameras, and structured light cameras.

Customer Testimonial: Minus Zero’s zPod – the revolutionary autonomous vehicle features e-con’s GMSL2 HDR NVIDIA Jetson AGX Orin camera

e-con Systems is delighted to share the heart warming customer testimonial video from Minus Zero, one of our esteemed customers. They have generously appreciated our GMSL2 HDR NVIDIA Jetson AGX Orin camera – NileCAM21_CUOAGX used in their recently launched revolutionary zPod autonomous vehicle. We invite you to watch the video.

Video capture for real-time detection of theft and intrusion

This function is what makes patrol robots really intelligent. Using the video data captured using embedded cameras, these robots can detect and track people within a distance of 100 to 200 meters (depending on the range of the robot). The data is analyzed on the edge using an AI algorithm that can decipher the video and image data to identify the presence of undesired objects and people.

Here, depending on the FoV (Field of View) to be covered, a single camera or a multi-camera system may be used. In the case of a multi-camera system, 3-4 camera modules can be used to cover the entire area of interest.

To learn how a successful multi-camera integration is done, check out the article What are the crucial factors to consider while integrating multi-camera solutions?

One of the key differentiators of e-con Systems when it comes to multi-camera integration is its proprietary 180-degree stitching algorithm that can create a panoramic view using multiple images. This can be a very useful feature for any device that has a surround view system like a patrol robot.

Video capture for remote surveillance or analysis

Though patrol robots are predominantly used for automated surveillance, there might be cases where they will have to be handled by a human supervisor. These can include:

• Manually analyzing and interpreting the data captured for identifying patterns or issues
• Manoeuvring the robot remotely for inspection or manual surveillance
• Capturing videos of a specific site during a particular time frame

In addition to this, the robot can also be used for other purposes such as collecting data on vehicles, pedestrians, and other objects. As in the previous case, the field of view primarily determines the number of cameras to be used.

Facial recognition for identifying intruders and other people

This is an extension of the automated surveillance function. While navigating and surveilling an environment, the robot might have to identify people. This would mean that the camera should have the capability to deliver clear images for the facial recognition algorithm to be able to recognize the person with high levels of accuracy. At times, this might have to be done with a limited light supply, which in turn means that the camera should come with excellent low light performance.

While patrol robots can usually detect people up to a distance of 100 to 200 meters, facial recognition could be limited to around 40 to 50 meters.

How e-con Systems can change the game for patrol robot manufacturers

e-con Systems has been in the embedded vision space for 18+ years now. Leveraging our wide portfolio of products and engineering expertise, we have smoothly integrated cameras into a vast variety of robots including patrol robots. Some of the differentiators e-con brings to the table when it comes to enabling vision for patrol robots are:

• A portfolio including specialized camera solutions for patrol robots including GMSL2 cameras, time of flight cameras, stereo cameras, GigE cameras, global shutter cameras, etc. These cameras come with features such as HDR, high resolution & frame rate, excellent low light sensitivity, superior NIR performance, etc.
• Customized camera solutions to meet the unique camera requirements of a robot instead of merely offering a standard off-the-shelf solution (an example of this is customizing a single camera to form a stereo pair for depth measurement).
• Unmatched experience and expertise when it comes to ensuring reliable long-distance transmission, especially using the GMSL2 interface.
• The ability to achieve excellent image quality by leveraging our ISP finetuning expertise.
• Cameras compatible with all the popular processing platforms such as the NVIDIA Jetson, Qualcomm, Texas Instruments, NXP i.MX series, etc.

If you are interested in learning more about e-con’s camera solutions for autonomous mobile robots, please visit the AMR markets page. You could also check out the Camera Selector to look at our complete portfolio of cameras.  Meanwhile, if you are right away looking for help in integrating camera modules into your robot, please write to us at camerasolutions@e-consystems.com.