Simulation environment
General Description
Isaac Sim extension that allows photorealistic 3D representations of a collaborative manufacture environment, including robots, industrial equipment, and human agents. It will help industrial partners to simulate processes in a more realistic way, considering human operators in the simulation. With the SE (simulation Environment), users will be able to recreate production plants and lines, taking into consideration interactions between the existing agents in the environment and integrating human avatars.
| Resource | Link |
|---|---|
| Source code | Link to source code |
| Demo Video | Link to video |
Contact
The following table includes contact information of the main developers in charge of the component:
✔️ Edit the names and contact info of the main developers in charge and update the logo image and link to your website.
| Name | Organisation | |
|---|---|---|
| Angela Lago Alvarez | angela.lagoalvarez@tuni.fi |  |
| Aitor Toichoa Eyam | aitor.toichoaeyam@tuni.fi | |
License
Integrated and Open Source Components
The SE is an extension based on Isaac Sim (NVIDIA Omniverse framework), consequently to work with it is necessary to fullfil some requirements.
Overview
Isaac Sim is a simulation platform developed by NVIDIA, built on the NVIDIA Omniverse framework. It enables developers to design, simulate, and test AI-driven robots in highly realistic virtual environments.
Key features include: - High-Fidelity Simulation: Utilizes NVIDIA's PhysX engine for accurate physics simulation, supporting various sensors like cameras and LiDAR. - Synthetic Data Generation: Facilitates the creation of synthetic data for training AI models. - Extensibility: Allows integration with existing pipelines and customization using Universal Scene Description (USD). - Robotics Workflows: Supports workflows for different types of robots, including humanoids, manipulators, and autonomous mobile robots. - Deployment: Provides tools for deploying trained agents to real robots and integrating with systems like ROS and ROS2.
To work with it, it is necesary to fullfil some requirements. Software Requirements: - NVIDIA Omniverse Platform: NVIDIA Omniverse Platform: Download and install the NVIDIA Omniverse. There will be available all NVIDIA Omniverse specific apps.
Hardware Requirements: - NVIDIA GPU: Omniverse applications are GPU-intensive and require an NVIDIA RTX GPU with dedicated ray-tracing cores, such as GeForce RTX 5080. Consult the up-to-date GPU requierements in the official documentation - System requierements: Consult the system requierements in the official documentation
Recommendations: - Most of the specifications before mentioned refers to Windows computers. In case you are planning to use Linux or any specific deployment, please refer to the NVIDIA Omniverse Technical Requirements to get the suggested technical requirements.
How to install
In order to use the component, it needs to be added as a Third Party Extension in Isaac Sim. Follow the instructions on how to add or update an extension in Omniverse.
How to use
Isaac Sim NVIDIA Omniverse is the foundation for the Simulation Environment (SE). Please refer to the documentation provided above to check the technical requirements of this software and set it up. To get access to the SE, users need to install the given extension as follow.
Open NVIDIA Isaac Sim, from the Omniverse launcher. This first mock-up represents the software open with an empty project.
From the top menu, go to “Window” and click in “Extensions” from the drop-down menu.
A window call Extensions will appear near the Viewport. It includes all Extensions available in the system. The simulation Environment Extension will be there too.
Keep in mind that two Extensions are available inder this component. The first one, called Assets Simulation allows the user to simulate the human-robot collaboration workflow of Silverline use case (pilot of AI-PRISM project). The second one, called Simulation Environment, permits the user to create their own set ups by adding human workers, robots and cobot to an scenario and, evaluate the assets physical interactions.
Assets Simulation Extension
Once the extension is clicked, some information regarding it will appear in the screen. Install the extension.
Once it is installed, the user can activate it by clicking the toggle button of the Extension.
Click on the extension to open it.
The side panel of the extension will appear. Click on Load extension.
Once the scenario is loaded, press the Run Scenario drop down.
Press on Run scenario, to simulate the human-robot collaboration workflow of Silverline use case
Click Stop to freeze the simulation.
Press Reset button to reset the scenario.
Simulation Environment Extension
Once the extension is clicked, some information regarding it will appear in the screen. Install the extension.
Once it is installed, the user can activate it by clicking the toggle button of the Extension.
Click on the extension to open it.
The side panel of the extension will appear. Click on the Assets configuration drop down.
This drop down will allow the user to configurate all the assets (human, robots and cobots) that are loaded to the scenario. First, select the number of assets to be loaded.
Click Load Assets. Initially, the assets will be loaded in the shape of color coded cylinders (pink for human workers, yellow for cobots and blue for robots).
Once the cylinders appear in the scenario, position them in their final location.
Click Fix Positions.
Once the cylinders are positioned, the user can start to configurate them. In the Asset selection drop down, choose the first cylinder to configure. Users can double-check the cylinders in the Stage panel (right side of the screen)
Click the second drop-down called Asset assignment and choose the asset to substitute the cylinder selected before.
Once selected, click Upload Asset.
The asset will appear in the stage inder a folder called Assets.
Repeat the 3 previous steps until all the assets are in the scenario. Then click Configuration Done.
Click on the Assets interaction drop down.
Group the staged assets in workstations by creating workstations names and selecting the asset belonging to it.
Click Assign Asset to include the selected asset under the workstation. Complete these 2 previous steps for all the assets in the scenario.
Click Show Interaction to see the working envelopes of the assets and the p-HRI KPIs.
