For the 3D Scanning and Motion project at the Technical University of Munich, we propose a real-time marker-less tracking and 3D reconstruction implementation and show how it can be used to create a mixed reality game using the Unity3D game engine. The concept was inspired by the spatial mapping technology behind Microsoft HoloLens and our 3D reconstruction pipeline is based on the Kinect Fusion pipeline. Tracking and reconstruction is performed in real-time using GPGPU acceleration (Cuda 10, DirectX 11 compute shaders). Finally, several Unity modules (e.g. physics, particle systems, etc) can be used to create interesting mixed reality projects.
Wojciech Zielonka, Patrick Radner and Mustafa Isik worked later on a really cool project for the 3D Scanning and Spatial Learning Practical Course at the Visual Computing lab. Take a look at RGB Face Tracking and Reconstruction using CUDA!