What is the NVIDIA® Isaac ROS? How to get started with it?

 In one of the previous articles, we learned what a Robotic Operating System (ROS) is. Today, let us attempt to understand what the NVIDIA Isaac ROS is. We will also look at some of its key functions/use cases, and the steps involved in installing it.

NVIDIA Isaac GEMs ROS provides a set of GPU-accelerated packages for your ROS2 application – making it convenient for developers to create high-quality NVIDIA hardware-based solutions. NVIDIA Isaac ROS helps improve the throughput on image processing and DNN-based perception models.

These ROS2 packages are built from ROS2 Foxy, the first Long Stable Release (LTS) to come out of the robotics community. Some of the key advantages of Isaac ROS GEMs include:

• Optimized processing (image, CV & DNN) for NVIDIA GPUs and Jetson.
• Modular packages for developers to pick what they want to integrate – whether a single algorithm or the entire pipeline.
• Easy-to-integrate capabilities that ensure rapid prototyping and deployment.

Perception AI Packages for ROS Developers

It is important to note that the ROS 2 nodes should address common image, computer, and DNN processing functionalities in order to deliver high-performance perception to AI-based ROS robotics applications. The below image shows the architecture of NVIDIA Isaac ROS:

Figure 1 – architecture of NVIDIA Isaac ROS

(Source: https://developer.nvidia.com/isaac-ros-gems)

Important use cases of Isaac ROS GEMs

Visual SLAM based Localization

Isaac ROS GEMs are hugely beneficial for autonomous systems that move within an allocated area. Leveraging Visual odometry, Isaac ROS GEMs help to assess the position of the camera with regard to its starting point. ROS developers can also make sure that the SLAM algorithms can be run – in real-time – with HD resolution (1280×720) on NVIDIA Jetson Xavier AGX.

3D Scene reconstruction

Isaac ROS GEMs empower robotic applications to maneuver and navigate through complicated environments. Using the nvblox (preview), they can ensure that the RGB-D data creates a solid 3D model of the selected environment. It can also help the robot avoid unexpected obstacles on the course.

DNN Inference Processing

ROS2 packages can equip developers to harness, fine-tune, and optimize NVIDIA’s inference models before deploying the packages by TensorRT or Triton.

Camera/Image Processing

Camera/image processing GEMs are built to make use of Jetson’s CV hardware. If the robotic application is integrated with camera solutions that are connected through a CSI interface, it makes sense to capitalize on NVIDIA’s hardware-accelerated Argus package.

System requirements of Isaac ROS GEMs

Jetson

• Jetson AGX Xavier or Xavier NX
• JetPack 4.6.1

x86_64

• Ubuntu 20.04+
• CUDA 11.4 supported discrete GPU
• VPI 1.1.11

You must first install the NVIDIA Container Toolkit to make use of the Docker container development/runtime environment.

Configure nvidia-container-runtime as the default runtime for Docker by editing /etc/docker/daemon.json to include the following:

“runtimes”: {
“nvidia”: {
“path”: “nvidia-container-runtime”,
“runtimeArgs”: []
}
},
“default-runtime”: “nvidia”

and then restart the Docker: sudo systemctl daemon-reload && sudo systemctl restart docker

Note: For best performance on Jetson, ensure that power settings are configured appropriately (Power Management for Jetson).

ROS – Foxy

• To Install ROS 2 – foxy. Ubuntu should be upgraded to 20.04. For reference follow: https://qengineering.eu/install-ubuntu-20.04-on-jetson-nano.html
• After upgrading to 20.04, to install and test ROS 2 – fox, follow the instructions given in: https://docs.ros.org/en/foxy/Installation/Ubuntu-Install-Debians.html

All set? Now, let’s get started with the installation.

How to install and work on Isaac ROS

• Create a ROS2 workspace

mkdir -p ISSAC_ROS/src
Note: The workspace can have any name; this guide assumes you name it ISSAC_ROS.

• Clone the Isaac ROS Common repository to ISSAC_ROS/src/. Check that you have Git LFS installed before cloning to pull down all large files.
  sudo apt-get install git-lfs
cd ISSAC_ROS /src

git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common.git

• Source and Build the workspace:
  source /opt/ros/foxy/setup.bash

colcon build --packages-up-to isaac_ros_common

cd ../

Clone the Isaac ROS Argus camera repository to ISSAC_ROS/src/ and build the source

git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_argus_camera.git

cd isaac_ros_argus_camera/

colcon build

• Source the environment and Launch the node:

source ./install/setup.bash
ros2 run isaac_ros_argus_camera_mono isaac_ros_argus_camera_mono --ros-args -p device:=0 -p sensor:=0 -p output_encoding:="mono8"

or

ros2 run isaac_ros_argus_camera_mono isaac_ros_argus_camera_mono --ros-args -p device:=0 -p sensor:=0 -p output_encoding:="yuv422"

Launch the new terminal and source the environment to launch the rqt

Note; make sure you are in the “isaac_ros_argus_camera/” directory

source ./install/setup.bash

rqt

To view the stream

In the rqt pop window go to Plugins->Visulization->Image View

Select the /image_raw

Reference cameras from e-con Systems

e-con Systems, with 18+ years of experience as an  OEM manufacturer, has been a long-standing partner of NVIDIA. We have worked with our camera partners to provide the modules – compatible with the NVIDIA Isaac ROS Argus Camera packages.

• Sony STARVIS IMX415 based camera: An 8MP ultra-low-light MIPI CSI-2 camera capable of streaming 4K @ 44 fps .
• Sony STARVIS IMX462 based camera: A Full HD ultra-low-light MIPI camera with superior NIR performance for NVIDIA® Jetson Xavier™ NX developer kit.

We hope you now better understand the NVIDIA® Isaac ROS GEMs, its major use cases, and how you can get started with it. If you are interested in integrating cameras to your robotic system using the NVIDIA Isaac ROS, please write to us at camerasolutions@e-consystems.com. Visit the Camera Selector to see our complete portfolio.