Introduction to IsaacSim: GPU-Based Robot Simulation Tool

IsaacSim is a GPU-based robot simulation tool used for verifying algorithms and training neural networks, integrated with ROS.

### Introduction to NVIDIA IsaacSim: Installation Process and Overview
* 🤖 NVIDIA IsaacSim is a GPU-based robot simulation tool developed by NVIDIA, allowing users to **simulate robots in a 3D environment** before working with physical robots.
* 🔧 One of the key advantages of IsaacSim is that it uses GPU for simulation and training, **reducing data transfer time between CPU and GPU**.
* 🎮 The simulation environment in IsaacSim is highly realistic, leveraging the **latest GPU technology** to create **lifelike simulations**.
* 🖥 To install IsaacSim, you need to download the Omniverse Launcher, which is a platform where **various NVIDIA tools** are available.
* ⚙ ️ After installing the Omniverse Launcher, you can **download and install IsaacSim** through the launcher.
* 📈 The ideal GPU specification for running IsaacSim is a **GeForce RTX 3080**, with a minimum requirement of an **RTX 2070**.
* 🌐 IsaacSim is integrated with ROS (Robot Operating System), and it uses **USD files** for working with **3D objects on the screen**.

Hello, from today, I will gradually explain about a robot simulator called IsaacSim. I am also currently studying how to use it for my research and will introduce the studying process to you little by little. So, there may be some mistakes in my explanations, and I would appreciate your understanding. First of all, let me explain the tool we will be studying. When you search on Google or similar platforms, you will find tools related to IsaacSim. As for Hangul Day, I haven't seen much about it, and most likely the materials are in English, so I will focus on that for the explanations. There are excellent tutorials available, and the software is continuously being updated. Therefore, I will briefly introduce what IsaacSim is.

Today, I will explain the process of installation. First of all, our tool called IsaacSim is a robot simulation tool that comes from MBTI. When we talk about this robot simulation tool, as you may already know, let's take a look at Gazebo. Gazebo is a tool that allows us to simulate robots in a 3D simulation environment on a computer by integrating with ROS. Therefore, when we develop and research actual robots, before we proceed to work with the physical robot, we need to verify if our algorithms work well by simulating the robot in a simulation environment. So, Gazebo, PyBullet, CoppeliaSim, etc., there are various types of tools for robot simulation, and among them, IsaacSim is one of the tools to use.

I'm not sure if the first thing that came out on NDB was in 2020. But it's one of the latest developments. One big difference is that while other simulations were done based on the CPU, NVIDIA i30 allows GPU-based simulations. So, when you compare it with the existing simulations, you will notice a significant difference. When you see this, in environments like the 3D gaming environment that we are familiar with, or in virtual environments, it will catch your eye. People often say it looks similar to real life, but if you watch Isaac's video for a moment, I will remove the subtitles. When you see this simulation environment, it shows a very realistic picture. The one above is a simulation, and the one below is real. When you look at it this way, the depiction in the simulation is very high. The top is the simulation, and the bottom is real. This is one of the latest technologies in graphics cards.

It's about rate lessons, utilizing that to create a robot simulation environment that is closer to reality has its advantages. One of the main merits of using the Isaac team, which is a representative team, is that it can use GPUs for optical learning. In the past, when it was based on CPUs, we had to transfer data back and forth between the CPU and GPU to train neural networks for deep learning, which took a lot of time. By simulating within the GPU and training within the GPU, the phenomenon of data bottleneck is reduced. As you can see here, you can create multiple environments, which is called gym. By utilizing this, you can perform simulations and training within the GPU, reducing the time needed for data transfer between the CPU and GPU.

You can run a much larger number of simulations at once compared to simulations. These differences and advantages make it one of the tools that attracts a lot of interest. Also, one of the things that are actually using robot simulations is that once the simulation is done, we need to transfer it directly to some hardware like Raspberry Pi or similar environments, install ROS, and run it. First of all, NVIDIA is a company that is a giant in the hardware industry, so they have products like Nano boards that can use GPUs, making it possible to connect them directly. This can also be seen as an advantage. So, in order to utilize these, I am also studying. Now, let's take a look at the Hello world that appears here for the first time. If you look here, the content here is the tutorial provided by MBTS.

There are a lot of tutorials as I mentioned earlier. Now, in this environment where multiple simulations can be run optically, there are tutorials available. Of course, since it is a robot simulation, it should be integrated with ROS. Among them, there are simulations that are integrated with ROS such as ROS 1 and ROS 2. So, learning about those things like the URDF reporter would be very helpful for you. There may be some limitations due to the language being in English, but we will gradually study and explore those aspects. Originally, in the previous versions, it supported Windows more than now, originally it supported Windows, Mac OS, and Linux. It seems that it was used quite a lot in the past. In fact, for ROS 2, from the 2022.2 version, it seems that Windows version is being supported a lot, so we will provide tutorials for you.

When showing it, I think I can show it well enough on Windows. Well, then, probably, in order to install this, you all need to download something called Omnibus. First, you can download it as an indie visual version, and I think I also used it as an indie visual version, and there wasn't a big problem. I'm not sure exactly how it is compatible with the enterprise version. So, you might test it, or for research purposes, the indie version should be sufficient without any problems. When you download this, you need to fill in things like developer, English name, email address, and then a link to download the service will probably appear. I won't disclose personal information, so I won't put that here. By doing this, when you download it, I will briefly show you the downloaded file.

If you look here, I already have the omnibus lunch version installed, but you will be able to download this. So if you run it, it's currently running. Yes, once you finish installing this, an icon called Omnibus Launcher will appear. I have copied this for your convenience. When you click on this, a window like this will pop up. Let me briefly explain about this window. Probably, initially, the cache may not be installed. I have it installed now, so first, you will probably install the cache, and then various installation processes will follow. Basically, follow the basic steps and then I will explain how to install the main program. First, you probably need to log in like this, so you need to create an account with Naver and log in, and then all the processes.

Once everything is completed, I initially installed this thing called Cache, and maybe I'm not exactly sure, but it seems to make it more convenient to use some apps that are not on the bus. So, I installed this and also installed R even numbers. If you go to the Exchange, you will find these things called Ajax V. In fact, in NBDR, Omnibus is now NVIDIA, originally a company that made GPUs. So, based on GPU technology, you can think of it as a place where various tools that can be used in the metaverse, virtual reality, and so on, on the 3000 platform are gathered together. So, there are tools related to Omnibus now, and besides this, there were also things used in the drive auto vehicle sector, quite a variety of tools. So, now, various tools are available, and even those who create 3D characters use them separately.

Alternatively, there are various tools available, so we will start by using tools related to robots. Since our purpose is robot simulation, we will try using AjaxView among various tools. So, if you click on this, I have the latest version installed now, so when it pops up as Launch, I will choose what was there before. Then, you will see an iPhone-like sound here. So, if you click on Install, it will proceed here, and the installation will take place. As far as I remember, the version I installed, 2022.2, was almost 6-point-something gigabytes, almost 7 gigabytes on Windows. So, you need to download and install that, which will take quite some time. Also, just a quick note from the tutorial, when using Ajax, you need to have an NVIDIA GPU for it to run. So, the basic specifications will be explained briefly later on.

Let me explain. So, when we install this library, you will be able to see it, and another thing called NewClass is a file server used in Isaac Omniverse. Typically, I have also installed servers in Omniverse for services. You will also install these things, and probably related to Isaac Sim, demos or such things will be provided through the local server. Therefore, if you install it along with UPlus, the installation process will be like this. I will show you what I have installed to you, so even if the Next button is not showing, I will check if there is any issue on the page. So, first, if the installation is done like this, by using this launcher, let's try running it once. When you click on Launch, a dialog box will appear, showing the console by default.

There are headless methods and headless methods. Headless means that the GUI is not visible, and you can think of what we have selected now as a version where the GUI is visible. So when you click start, the consultation will be executed like this. This is the part that appears because we have set it to show the console window with checkboxes. Now the loading is completed, so it is loading right now. Probably, it will take some time when you first start it, so now it is loading RTX. Based on the fact that it is loading RTX, you can see that I am zooming in and out using the wheel. If you press the wheel and move, it will move like this. This is the window. For example, let me briefly load it and it may take some time. If you see, it is loading bolts and such. This is the loading part, and I will run the simulation here. Then the arm moves.

We are showing the process of capturing a yacht and entering into a contract as a demo. Actually, if you look at the demo in your tutorials, this is the content to be covered in Advanced Tutorials 23. So, you can actually try out the content, there are explanations about the code, and so on. By following these methods little by little, we will create videos in the tens of millions range to show you. So, in the beginning, you might see some installation processes or something like that in the interactions. It would be nice if I could show you each step one by one, but for now, I haven't done that, so I will explain a bit more while looking at these pictures. This is now something that can be integrated with ROS and such, and basically, in this iXmap, we can use USD files for assignments or when dealing with 3D objects on the screen.

What is a URDF file and what is an SDF file? It means using USD files. So, there are tools to convert the STM file or URDF file to USB before that. Now, I will explain and provide a tutorial. First of all, the ideal state is having a GeForce RTX 3080. Currently, I have a Windows running on a 3090. So, it needs to be at a certain level to run. The best idea is the RTX A6000 series, but you don't have to go that far. The minimum specification is 2070. Therefore, a GeForce RTX series graphics card needs to be installed. I had a gaming laptop with an RTX 3060 before, and even up to 3060, basic tutorials were fine. However, in reality, because GPUs are expensive, you need to have a GPU installed to use it. And there might be some installation steps, let me briefly explain.

I will take a look. The launcher, if you sign up and download it, there are versions for Linux and Windows. So, the Windows version that briefly appeared on the screen earlier can be installed. So, install it, run the launcher, choose and proceed, and it will launch like this, indicating that the installation is complete. And as I showed you earlier, in the selector tool, maybe I also installed it. Depending on the environment, there were several issues, so it might work at once in the beginning, or you might struggle a bit with the installation. It would be good to try installing it once. This is probably based on Docker standards or there will be explanations. So, these things, and this is cloud-based, and Python. Since almost everything is based on Python, after installing Python and setting up the Python environment, we can also debug, you know.

I am explaining the process to you, and I will provide additional explanations when that time comes. This is using Ross OneTwo. If you look at the picture now, there is something called Ross Bridge. There are One Bridge and Two Bridge, depending on the in-app at D7. We can use Ross One and Ross Two. Regarding the setup, if you proceed with the installation like this, install it, and as I showed you in the picture earlier, once you load this Ignition file, you can continue to the next step without any major issues. So, I have briefly introduced basic installation, execution scenes like this to you today. From now on, I will explain how to use it through the actual process or, basically, following the tutorial provided by Isaac. Thank you.

[Isaac Sim - 초급] #1 Isaac Sim 소개 및 설치

동아대학교 기계공학과 
지식-지능형 메카트로닉스 실험실
(Knowledge-Intelligence Mechatronics Laboratory)


KIMe Lab

Introduction to IsaacSim: A GPU-Based Robot Simulator

IsaacSim is a robot simulation tool that allows for GPU-based simulations, providing a more realistic 3D environment compared to CPU-based simulations like Gazebo.

It is a latest development that offers high-fidelity graphics and the advantage of using GPUs for deep learning, reducing data transfer time and bottleneck phenomena.

IsaacSim also allows the creation of multiple environments (gym) for simulations and training within the GPU, ultimately cutting down the time needed for data transfer between the CPU and GPU.

Advantages of Robot Simulations and Integration with ROS

Robot simulations offer the advantage of running a larger number of simulations at once, attracting significant interest.

NVIDIA's products, like Nano boards with GPUs, allow for direct connection, providing a hardware advantage.

The simulations are integrated with ROS, such as ROS 1 and ROS 2, making it beneficial to learn about aspects like the URDF reporter for ROS integration.

Installing Omnibus Lunch Version: A Step-by-Step Guide

The Omnibus Lunch Version can be downloaded and installed, with a subsequent need to create a Naver account and log in.

The installation process includes installing the cache, R even numbers, and AjaxView for robot simulation.

The program offers a variety of tools, including those for the metaverse, virtual reality, and 3D character creation, and requires an NVIDIA GPU to run AjaxView.

The installation file for the latest version is about 6-7 gigabytes, so it may take some time to download and install.

Installation Process for the NewClass Library in Isaac Omniverse

The NewClass library is installed using the launcher and can be run with a visible GUI interface.

During the installation, the console window can be displayed, and the process involves loading RTX and running simulations.

The library's capabilities include capturing a yacht, entering into contracts, and providing advanced tutorials for code and demo exploration.

The NewClass library also offers integration with ROS and the use of USD files for 3D object manipulation in iXmap.

Understanding URDF and SDF Files for Robotics: A GPU Installation Tutorial

URDF and SDF files are used in robotics to describe the structure and properties of robots, with the need for a GeForce RTX series GPU for installation.

The installation process involves downloading the launcher, choosing the Windows version, and potentially facing some initial struggles.

The setup is based on Docker standards, cloud-based, and Python-dependent, and once installed, the files can be used for robot simulation and debugging.