Unreal Engine
Building virtual worlds using Unreal Engine3D Reconstruction Visualization
3D reconstruction trained using Gaussian Splatting can be showcased through Unreal Engine. Leveraging the extensive resources available in Unreal Engine enables rich interaction with the scenes. For instance, the training results can be freely observed using virtual reality devices (I used Pico 4).
In addition, I can use Unreal Engine to visualize Gaussian Splatting reconstructions within a global context. By integrating the reconstructed scenes into their corresponding real-world locations on a map, users can explore and interact with these scenes seamlessly. Additionally, these trained scenes can also serve as backgrounds for all simulations mentioned later in this document.
Autonomous Driving Simulation
I can perform autonomous driving simulations based on Airsim, including simulations for unmanned vehicles and drones. During the simulation, it is possible to obtain real-time RGB images from the payload, along with corresponding semantic segmentation and depth estimation ground truth. In future work, I can modify the above scenarios slightly to enable simulations for embodied intelligence applications.
Real-Time Scene Manipulation
I can use Unreal Engine’s visual scripting tool (Blueprint) to make simple adjustments to the scene, and I can use C++ for more complex control. On this basis, I can provide an interface for any programming language via TCP communication (I have already experimented with controlling the scene using Python for network training). In the future, I can use Unreal Engine as a simulation environment to complete the training process for embodied intelligence or to customize datasets for dynamic 3D reconstruction.
Related Works
Here are some of the articles I’ve published that relate to this skill!
