uh, thank you, Madison. uh, I'm Leela. As Madison said, I'm the product manager of Isaac Sim. um, I know that you heard a couple of times Frank mentioning Isaac Sim and also Damian. So, I'm gonna give it like a brief description of what actually Isaac Sim is and act how does uh embraces Omniverse to create such a good simulator platform. So, let's get started with what Isaac Sim is. Isaac Sim is a robotic simulator and a synthetic data generation tool. And, uh, for so in if if I want to go more in deep to add let me slide my mouse here just a sec. So, for robotics application, that means, uh, you can assimilate your robot and or a couple of robots and the way it interacts with the environment. For a non-robotic application, Isaac Sim as a synthetic data generation tool creates a good set of features to create a large-scale photorealistic and diverse set of data which you can feed into your uh perception training tool pipeline.

so you could I'm going to get more into the details of this. But first, the question is: why Omniverse is the platform to build such a simulator on top of? I'm sure that Frank said a lot of detail, so I'm not going to get into that. I'm sure I cannot do a good job as Frank did, but Omniverse does provide photorealistic visuals and physically accurate virtual environment, which is essential and key for building any digital twin or virtual environment for any practice. And also, Omniverse is the metaverse for 3D workflow. What I'm going to explain here, because again Frank explained in details about Nucleus and the connectors and how these works and what I'm going to say and why these matters for building a robotic application and an AI application in general. For building a robotic application, we would want to import the building that the robot will live and perform tasks in. Okay. And so, one thing that you want to import is the building model. Also, you want to import your robot model into it.

into the building and across that you want to bring all the objects and the assets that you have into the building. So there's this means that you want to bring many objects from different workflows. If you're building a factory or like a manufacturer, a warehouse, you would want to import some engineering machines into it, and those would have their own tool sets and their own workflow and their own format, and you would need also this to create your virtual environment. So you can see that there are different objects from different workflows. Also, you one would want to bring the 3D scan of the environment to because there is no already like a CAD design for it. You just have the scanner that you want to build it. So you could see that for building such a complex virtual environment that includes the robot, the assets, the objects, the building, you would need a metaverse for 3D workflow. And that's why Omniverse is key for building such a simulator, toolset. So again like this is its rating that

these objects or these assets could come from any workflow, and many times these workflows have these three industries. Will have their own workflow, their own standards, their own format of 3D modeling. We all know that this AI is going to be everywhere soon, and everything will become smarter. So that includes the robots. Robots need to be, need to get smarter, and they have to be autonomous. So virtually anything that moves will be autonomous. For that, we need the robots to have a strong perception, understand the environment. And that doesn't come easy, and you need a lot of training and a lot of data to get to that level. And I think it's obvious that we cannot get that in a real practice, practicing in a real world. We needed, we need to train it in a scalable, photorealistic, that debates the laws of physics. Well, actually we need a two-to-reality simulator to do all this training, to make those autonomous robots, on those smart robots, to perform complex taxes, the accomplice, uh, tasks. So that's why we, we need for

building such a robots, we need uh to build these complex environments and complex digital twins to perform such tests. So, what kind of application you can run in isaac sim? You can do navigation and manipulation both. Navigation is, if you're not very familiar with it, when you have a robot that moves from point a to b. And that is not only for a robot in a warehouse, that could be your last mile delivery robot or it could be your smart uh vacuum cleaner at home. And also, we could simulate the manipulation. Manipulation is where your robot looks like an arm and usually performs a pick and place. But again, this is not limited to factory or industries application. Your shopping body in future that is going to be a robot picking items from the aisle from the shelves is going to be like a manipulator. So, we need manipulation and navigation in many uh robotics application other than robotics application. As I said, isaac sim is a synthetic data generation tool and for that, it provides a complex set of features that makes it

easy to create synthetic data to feed it into your training pipeline. Once one for sure knows that the real-world data not only is expensive, it's hard to get many times, is like it's hard to get the corner cases. So, we would rely heavily on synthetic data in future. Isaacson provides a ground rule from RGB, depth, bounding boxes, 2D, 3D, 3D bounding boxes. And also, we have domain randomization tools that you can randomize many components in your scene, from texture to the color, to the position or to the placement, time of the day, the occlusion, many things that with that you can create a diverse set of images for your training. And with all these being realistic and photorealistic makes it more important for your training. So, I'm going to give a summary because this is going to supposed to be a very short description. Isaacson, as I said, is an app built on top of Omniverse. So, we do rely on RTX rendering for the visuals.

need the photo. We need the realistic physics to uh simulate robots and isaac's in. We have made it easy to import the models. This could be a robot model, the environment, the building, or all the 3d assets and the objects in the scene, and all of this is essential for building the digital twin. Once you have all the whole environment set up, you can use the synthetic data generation tool to create a large data set for yourself, and actually you can connect it to the transfer learning toolkit, which is against built-in Nvidia, to iterate faster on the model that you already have. Let's say you had a model, you trained it using a set of real data, but now you want to make it more precise, like improve it a bit. You would feed to it more synthetic data to enhance that. And for controlling the robot, you could use ROS or Python, so you can train it, control it, or program your robot to do all these cool stuff with the robot.

Overview of NVIDIA Isaac Sim

NVIDIA Omniverse is a scalable development platform for building and operating custom 3D pipelines and industrial metaverse applications. Based on Universal Scene Description (USD), Omniverse fundamentally transforms complex 3D workflows, allowing individuals and teams to connect disparate 3D tools and datasets, and simulate large-scale, physically accurate virtual worlds for industrial and scientific use cases.

On this channel, we are connecting creators, designers, artists, and engineers, their favorite content creation tools and assets, in real time. Join us for exclusive teasers, demos, and feature sneak-peeks. Tag us with #NVIDIAOmniverse, #OpenUSD, and @NVIDIAOmniverse to show us your work!

Join our Omniverse livestream every Wednesday, and review the full livestream schedule and add specific streams to your calendar here: https://www.addevent.com/calendar/ae483892


Overview of NVIDIA Isaac Sim

Overview of NVIDIA Isaac Sim

Scripts