Embedded vision in telepresence robots – how cameras are easing communication and reducing human labor

In this increasingly digital world, remote communication and interaction have become a necessity across multiple domains such as healthcare, education, industrial, retail, etc. However, traditional video conferencing systems are tedious to operate and do not offer a smooth experience to the users.

This is where telepresence robots have become a game-changer. They make it possible for humans to be virtually present at any location with the ability to control and maneuver the robot remotely. From schools to hospitals to corporate offices and retail stores, telepresence robots are widely used today for:

• Seamless remote communication.
• Perform certain tasks (such as picking and placing) remotely.

And these would not have been possible without using cameras. While microphones become the ‘ears’ of the robot, camera form the ‘eyes’. In this article, we will learn in detail how camera modules help telepresence robots to ease communication and reduce human labor.

Different types of telepresence robots and how cameras enable them with vision

Every telepresence robot is aimed at simulating the real presence of a human in a location. However, depending on their function and the level of safety required, they can be broadly classified into three:

1. Telepresence robot that is fully controlled manually.
2. Robots that have the ability to move autonomously.
3. Robots that can perform certain tasks autonomously or with human supervision (teleexistence robots).

Here, it is to be noted that the above categorization is also done based on the type of cameras used in the robots. We will discuss this more in the next section where we look at the details of how cameras make telepresence possible.

Telepresence robot that is fully controlled manually

These are telepresence robots that are purely used for telecommunication and virtual presence. The use cases include:

• Attending and delivering virtual meetings, talks, and classes.
• Conducting remote office/real estate walkthroughs.
• Remote inspection and surveillance.

These robots make virtual interactions smoother thereby enhancing the overall communication experience. They also reduce the inconvenience caused by ‘old-school’ video conferencing systems and help save the time required for setting them up for meetings and virtual walkthroughs.

In this type of robot, cameras are used for teleconferencing and previewing purposes. While a video conferencing camera facilitates remote communication, it can also be used to aid manual navigation. There could also be a camera placed facing the bottom of the robot or the floor (usually called a foot camera) that helps the remote user to have a better view of the vicinity to avoid accidents.

Here, features like high resolution & frame rate, a wide field of view, an interface that can transfer high throughputs, etc., are some of the most critical camera features. For instance, e-CAM83_USB – a 4K HDR USB camera from e-con Systems – is specially designed for video conferencing and telepresence applications.

Robots that have the ability to move autonomously

When it comes to telepresence robots, since there is always some form of human supervision, navigation is never done fully autonomously. This capability is used for one of three purposes:

1. For point-to-point navigation guided by the user remotely.
2. For adding an additional layer of safety to automatically detect obstacles.
3. For autonomously moving back to the charging station after use.

In the first case, instead of maneuvering the robot completely manually, the user guides the robot by marking the location or point to which it has to move. The robot will accordingly plan the right path (navigation/path planning) to reach the point without any collisions (obstacle detection and avoidance).

In the second scenario, autonomous navigation helps to prevent accidents by automatically detecting obstacles (in case the remote user fails to spot any nearby object).

When it comes to the third case, the robot leverages autonomous navigation to move and plug back into its charging station. This is most often done towards the end of a day (since a robot can last up to 6-8 hours with a single charge).

In all the above cases, autonomous navigation can be done only with the help of cameras. 3D depth cameras are used for this where they help in detecting obstacles by measuring depth. The three types of cameras used for depth measurement are stereo cameras, time-of-flight cameras, and structured light cameras.

These camera-enabled telepresence robots with autonomous navigation capabilities help reduce the human effort required not only in enabling a seamless telecommunication experience, but also in moving the robot around.

Robots that can perform certain tasks autonomously or with human supervision

These are robots that can perform certain other functions in addition to telepresence. An example could be a robot that can automatically monitor retail shelves and replenish the stock as and when required. Here, the robot can do the task fully autonomously. However, in case the robot faces any trouble, the human supervisor can guide it from a remote location. Such robots are known as teleexistence robots.

In teleexistence robots, in addition to video conferencing and navigation, camera modules will be needed to perform the additional specialized tasks. For instance, if we take the above example of retail shelf replenishment, the robot will usually need:

• A camera to track the shelf for stock.
• A camera to locate the object to be picked and to accurately place it at the right location.

Depending on the complexity of the task to be performed, the type and number of cameras will also change.

Teleexistence robots are a life-saver when it comes to eliminating a task as mundane as filling retail shelves. This helps to improve retail staff motivation and productivity. It also makes it possible for the staff to sit at the comfort of his/her home while still supervising the activities at the store.

How e-con Systems can help your telepresence robots ‘see’

e-con Systems is an OEM camera solution provider that offers a wide variety of USB cameras, MIPI cameras, GMSL2 cameras, and GigE cameras for AMRs (Autonomous Mobile Robots). And we have worked with numerous telepresence robot manufacturers to integrate our camera solutions into their robotic systems. Whether it’s a robot used in teaching, telehealth, or office communication, we have seen it all. Our portfolio of cameras designed for telepresence robots comes with features like depth sensing, high resolution, wide field of view, compatibility with all the popular host platforms (including the NVIDIA Jetson), etc.

To have a look at our cameras suitable for autonomous mobile robots, please visit the robot camera page. Alternatively, you can visit the Camera Selector to browse through our complete portfolio or products by features. Meanwhile, if you are right away looking for help in integrating embedded cameras into your robots, please write to us at camerasolutions@e-consystems.com.