What are RGBD cameras? Why RGBD cameras are preferred in some embedded vision applications? In recent years, 3D vision technologies have evolved significantly improving their accuracy, range, low light performance, etc. And there has been a surge in demand for depth sensing in many embedded vision applications. Some of the most popular applications where a depth camera is used are autonomous mobile robots, automated guided vehicles, remote patient monitoring, and people counting systems. While we all understand what a depth camera is, the concept of an RGB depth camera (or RGBD camera) is a mystery to many. In this blog post, we will attempt to learn what an RGBD camera is and why it is a game-changer in some of the most popular embedded vision applications.
What is an RGB camera?
Before we learn about an RGBD camera, we need to first understand what an RGB camera is. The term RGB refers to the color model in which the red, green, and blue primary colors of light are added to create different colors that we perceive. An RGB camera is used to deliver colored images of people and objects by capturing light in red, green, and blue wavelengths (RGB). This camera utilizes visible light with a wavelength ranging from 400 to 700nm.
What is an RGBD camera?
An RGBD camera is a type of depth camera that provides both depth (D) and color (RGB) data as the output in real-time. Depth information is retrievable through a depth map/image which is created by a 3D depth sensor such as a stereo sensor or time of flight sensor. RGBD cameras are able to do a pixel-to-pixel merging of RGB data and depth information to deliver both in a single frame. For instance, DepthVista – a time of flight based RGBD camera from e-con Systems – has the capability to deliver both the depth and RGB data in a single frame. 
Figure 1 – DepthVista: 3D time of flight RGBD camera from e-con Systems
For the purpose of understanding how this works better, please have a look at the below figure that illustrates a sample application used to stream image and depth data from DepthVista. Here, the camera displays a depth frame (640 x 480), an IR frame (640 x 480) and an RGB frame (640 x 480) at 30 fps along with a 3D viewer. 
Figure 2 – sample application streaming RGB and depth data from e-con’s DepthVista
Why RGBD camera is one of the most preferred cameras in certain embedded vision applications?
Adding RGB data to a 3D depth sensing camera’s point cloud or depth map allows to pinpoint objects more effectively for pattern recognition or detection. This is particularly useful in applications that need to identify the type and nature of objects in the scene along with measuring the depth to them. And the ability to deliver these two types of data in a single frame makes the camera even more suitable for applications such as facial recognition based anti-spoofing systems and people counting devices. To learn more about how RGB depth cameras can enhance the performance and effectiveness of more such embedded vision applications, we recommend you check out the two articles given below:
- How does a Time-of-Flight camera make remote patient monitoring more secure and private?
- How does an Autonomous Mobile Robot use time of flight technology?
RGBD camera with a combination of Time of flight and RGB sensors
A time of flight based RGBD camera is one of the most common types of RGBD cameras that uses a ToF 3D sensor for depth measurement and an RGB sensor for capturing color information. The ToF sensor calculates the distance based on the total flight time of the light (considering the velocity of light) which can be done even in complete darkness.
To learn more about the concept of ToF and how ToF camera systems work, have a look at the article What is a Time-of-Flight sensor? What are the key components of a Time-of-Flight camera? If you are interested in learning how time of flight compares with other depth-sensing technologies, please read the article How Time-of-Flight (ToF) compares with other 3D depth mapping technologies.
The following block diagram shows the architecture of a ToF + RGB camera:
Figure 3 – architecture of ToF based RGBD camera
Time of flight based RGB depth camera from e-con Systems
DepthVista from e-con Systems is an RGB depth camera that uses a CCD sensor for 3D depth measurement and the AR0234 color global shutter sensor from Onsemi for capturing RGB data. This RGBD camera uses a VCSEL of 850nm for high-speed data communication and is capable of streaming VGA (640X480) at 30 fps. The 2D RGB video is integrated with the 3D depth video to create the colored point cloud (a collection of points in three-dimensional space). Then the post processing of 3D depth data can be performed using post processing filters such as planarization, temporal filter, edge detection, and spatial filter. The depth frame with all the post processing filters applied is shown in the following figure: 
Figure 4 – Comparison between a normal depth frame and a depth frame with all the post processing filters applied
To learn more about the features, benefits, and target applications of the camera, please visit DepthVista’s product page. You could also watch the below video to get a quick overview of the product.
3D Time-of-Flight (ToF) camera for accurate 3D depth imaging
We hope this blog gave you a good understanding of what an RGBD camera is and how it works. If you need any help in picking and integrating the right cameras into your camera-enabled device, please write to us at camerasolutions@e-consystems.com. Also do checkout the Camera Selector in case you wish to look at e-con Systems’ complete portfolio of cameras.
References:
Prabu is the Chief Technology Officer and Head of Camera Products at e-con Systems, and comes with a rich experience of more than 15 years in the embedded vision space. He brings to the table a deep knowledge in USB cameras, embedded vision cameras, vision algorithms and FPGAs. He has built 50+ camera solutions spanning various domains such as medical, industrial, agriculture, retail, biometrics, and more. He also comes with expertise in device driver development and BSP development. Currently, Prabu’s focus is to build smart camera solutions that power new age AI based applications.