Imagine a mobile robot that navigates through the aisles in a warehouse and assists in transporting goods across a huge space. They perceive the path efficiently to move past obstacles swiftly and safely to complete monotonous tasks with high reliability. Now imagine another breed of such sophisticated robots in an agricultural environment such as a greenhouse, examining the plants and plucking the ripe fruits and vegetables autonomously. These robots may also aid in crop maintenance and pesticide spraying on the fields. Such robotic systems that have the capability to navigate, detect obstacles, and identify objects completely autonomously are called AMRs or Autonomous Mobile Robots.
Most of you reading this would have seen these kinds of robots or are probably developing such systems to be tasked in warehouses, agricultural farms, retail stores, roads, etc. However, they need ‘eyes’ to see and perceive their surroundings to carry out their respective tasks autonomously. This function is performed by cameras.
In this article, we look at the most common types of cameras used in Autonomous Mobile Robots, and how they help robots perform even some of the most complex tasks in industrial, medical, and retail environments. So if you are a product engineer or developer working on robotic vision systems, or are just looking to learn more about the role of cameras in robots, this piece of content is for you.
The emergence of Autonomous Mobile Robots and robotic vision
First, let’s look back at some of the factors that can be attributed to the rise of the robotic workforce across several industries.
On the one hand, the increased cost of labor and lack of skilled resources are forcing companies to modernize their operations with robotic vision. On the other hand, the customer demand for faster production and delivery is putting pressure on them to meet delivery expectations. Plus, the pandemic has brought a new wave of health and safety protocols that have made it important to limit the usage of human resources.
Especially in labor-intensive sectors like warehouse management and smart farming, accelerating productivity and performance can significantly impact the business. Traditionally, their daily processes involved manual tasks prone to time-consuming delays and high costs. But today, Autonomous Mobile Robots (AMRs) are helping reinvent their operations – enabling them to maximize their investments and even enter new market segments.
And with advancements in embedded vision and camera technology, robots today have become faster, more reliable, and accurate. This shift has seen its biggest impact in Industry 4.0. To learn more about how embedded vision has helped in improving the outcomes of Industry 4.0 through robotics, have a look at the whitepaper Embedded vision: its role in transforming Industry 4.0 using robotics.
That said, in the next section, let us look at in detail about the role of cameras and vision in AMRs and the different types of camera systems used in them.
The important role of vision in Autonomous Mobile Robots
Undeniably, the quality of the camera system plays a major role in deciding the effectiveness of Autonomous Mobile Robots. Such robots leverage single, multiple, and stereo camera systems. Each has its unique purpose and comes integrated with the functionalities to help the robot to function intelligently in the specified environment. These cameras, coupled with the right sensors and processing platforms, ensure that AMRs are able to perform all their necessary tasks without fail.
Now let us look at the different types of camera systems used in AMRs.
Customer Testimonial: Minus Zero’s zPod – the revolutionary autonomous vehicle features e-con’s GMSL2 HDR NVIDIA Jetson AGX Orin camera
e-con Systems is delighted to share the heart warming customer testimonial video from Minus Zero, one of our esteemed customers. They have generously appreciated our GMSL2 HDR NVIDIA Jetson AGX Orin camera – NileCAM21_CUOAGX used in their recently launched revolutionary zPod autonomous vehicle. We invite you to watch the video.
Learn how e-con Systems helped a leading Autonomous Mobile Robot manufacturer enhance warehouse automation by integrating cameras to enable accurate object detection and error-free barcode reading.
Single camera systems in Autonomous Mobile Robots
A single-camera system relies on data acquisition, and management and inferencing techniques to create highly intelligent systems. They are typically used in robots for the purpose of obstacle detection. For highly accurate detection, a camera with a wide HFOV (Horizontal Field of View) is recommended in order to get maximum coverage of the scene. In addition, these cameras are characterized by high resolution and adaptive imaging capabilities like auto-exposure and HDR (High Dynamic Range) to provide accurate imaging in varied light conditions and environments.
Multi-camera systems in Autonomous Mobile Robots
In the case of robots like patrol robots, multiple camera systems are required for them to be continuously aware of their environments with ease. These robots are deployed for perimeter and neighborhood monitoring to provide uncompromised 24X7 security. A synchronized multi-camera system gives Autonomous Mobile Robots a 360-degree view of their surroundings. Using AI and inferencing techniques, they can raise warnings whenever any unusual activity or threats are detected. The surround-view (360-degree) data captured by these robots are also recorded for legal evidence and transmitted over wireless networks for manual monitoring. Other types of robots that typically use a 360 degree surround view system with multiple cameras include agricultural robots, warehouse robots, and delivery robots. It is also important to note that camera synchronization becomes a critical requirement in such systems. Without this feature, they would be unable to achieve reliable inferencing. Other key features are HDR, auto-focus, and auto-color balance – especially for outdoor AMRs to get adaptive imaging capabilities to deal with unpredictable lighting conditions. In cases where either the robot or the object to be captured is in fast motion, a global shutter camera might be required. To learn how global shutter cameras can enhance the performance of robots, please have a look at the article How global shutter cameras can improve robotic vision systems. However, in many cases a rolling shutter camera with high frame rate should do the job. You could learn how to choose the right one for your application by going through the article What are global shutter and rolling shutter? How to choose the one that fits the application?
Stereo cameras in Autonomous Mobile Robots
A single camera or an ordinary multi camera system can only help in basic obstacle detection or surround view. However, in order to measure depth to infer and analyze the distance to and between objects, cameras need to act as a stereo pair. Stereo camera systems typically involve two cameras being placed horizontally at a particular distance from each other. It helps the cameras to view the same area and assess the depth and distance of the object by using the pixel disparity technique. They are also paired with Light Detection and Ranging (LiDAR) sensor capabilities to improve reliability and accuracy. To learn more about stereo cameras and the various technologies involved in it, please have a look at What is a stereo vision camera? Delivery robots rely on depth mapping technologies to move autonomously through traffic, avoid obstacles, and finally deliver packages to the customers’ doorstep. Other similar AMRs are cleaning robots that operate autonomously in indoor environments. They carry out laborious tasks like sweeping and vacuuming floors in malls and retail outlets. These robots depend on 3D depth cameras to derive their locating and mapping abilities. One of the most popular depth mapping technologies that has gained popularity in the recent years is time of flight. PFB a couple of articles that will help you learn more about it:
- What is a time of flight sensor? What are the key components of a time of flight camera?
- How time of flight compares with other 3D depth mapping technologies.
Choosing various camera components for Autonomous Mobile Robots
To ensure optimal performance of the camera system in an AMR, all the components need to work well together. You need to choose not only the right sensor, but a processor which is capable of supporting the required level of image processing. To learn more about this, we recommend you visit the article A quick guide to selecting the best-fit processor for your robotic vision system. Also, you need to understand the difference between using an in-built ISP and an external ISP in your robotic vision system. Here is an article that will delves deep into the topic and analyzes the pros and cons of each: Camera ISP and significance of using an external ISP in imaging solutions. Your camera selection process also involves selecting the right lens depending on your application requirements. Choosing the right camera interface is also key to achieving the desired performance in your embedded camera system. It is always recommended to take the help of an imaging partner who has extensive experience in camera design, manufacturing, and solutioning to help you with these.
e-con Systems – empowering new-age Autonomous Mobile Robots (AMRs)
e-con Systems, with 19 years of experience and expertise in embedded vision, has been helping clients to build cutting-edge Autonomous Mobile Robots to deliver exceptional customer experiences and create operational efficiency. See how we helped a top Europe-based manufacturer of Autonomous Mobile Robots by deploying a 3D camera for depth measurement and obstacle detection. We provide single vision cameras, depth cameras, barcode scanning cameras, and surround-view cameras to power your AMRs. Here is a comprehensive list of cameras from e-con Systems suitable for Autonomous Mobile Robots:
- e-CAM130_MI1335_MOD: 13MP autofocus camera module
- STEEReoCAM™: 2MP stereo camera for NVIDIA Jetson Nano/AGX Xavier/TX2
- e-CAM217_CUMI0234_MOD: Full HD color global shutter camera module
- STURDeCAM20: IP67 rated Full HD HDR GMSL2 camera module
- STURDeCAM25: IP67 rated Full HD color global shutter GMSL2 camera module
- NileCAM25: Full HD color global shutter GMSL2 camera module
- NileCAM21: Full HD HDR GMSL2 camera module with LED flicker mitigation
In addition to the above camera solutions, e-con Systems is also working on a time of flight camera that will hit the market by the end of FY22. Follow our blog, YouTube channel, and social media posts to get updates on the same. If you need any help in picking and integrating the right cameras into your AMRs, please write to us at camerasolutions@e-consystems.com.
Gomathi Sankar is a camera expert with 15+ years of experience in embedded product design, camera solutioning, and product development. In e-con Systems, he has built numerous camera solutions for robots, industrial handhelds, quality inspection systems, smart city applications, industrial safety systems, and more. He has played an integral part in helping hundreds of customers build their dream products by integrating the right vision technology into them.