RGB-D Reconstruction for Mixed Reality - Juan Raul Padron Griffe

Epale! I am a Marie Sklodowska-Curie fellow of the EU Project PRIME and a PhD candidate at the Graphics and Imaging Lab (Universidad de Zaragoza). My PhD thesis under the supervision of Prof. Adolfo Muñoz and Adrian Jarabo focuses on developing theory and methods for accurate and efficient rendering of complex volumetric appearances. Previously, I obtained my Bachelor degree in Computer Science (2015) at the Central University of Venezuela. Later, I received my Master degree in Informatics (2020) at the Technical University of Munich. During my Master studies I focused mostly on the Computer Graphics and Vision subjects, where I was fortunate enough to be advised by Prof. Matthias Niessner and Dr. Justus Thies at the Visual Computing lab to conduct my research on 3D Scanning and Neural Rendering. I am currently looking for a postdoctoral position.

I am a computer scientist enthusiastic about the intersection of realistic image synthesis, graphics-based vision and machine learning for the digital acquisition, representation and understanding of the visual world. I am currently interested in pushing the state of the art on physically-based rendering of complex multi-scale materials like biological tissues. In my research, I rely on powerful tools like Monte Carlo simulation and gradient-based optimization. In the long term, I believe the combination of powerful forward models (simulation algorithms) and inverse models (gradient-based models) could be impactful in other interesting domains too like computational biology.

For the 3D Scanning and Motion project at the Technical University of Munich, we propose a real-time marker-less tracking and 3D reconstruction implementation and show how it can be used to create a mixed reality game using the Unity3D game engine. The concept was inspired by the spatial mapping technology behind Microsoft HoloLens and our 3D reconstruction pipeline is based on the Kinect Fusion pipeline. Tracking and reconstruction is performed in real-time using GPGPU acceleration (Cuda 10, DirectX 11 compute shaders). Finally, several Unity modules (e.g. physics, particle systems, etc) can be used to create interesting mixed reality projects.

Results:

Original Scene Reconstruction Unity Animation

Team Members: Juan Raul Padron Griffe, Wojciech Zielonka, Patrick Radner, Baris Yazici
Instructors: Angela Dai, Justus Thies

Github repository Poster

Wojciech Zielonka, Patrick Radner and Mustafa Isik worked later on a really cool project for the 3D Scanning and Spatial Learning Practical Course at the Visual Computing lab. Take a look at RGB Face Tracking and Reconstruction using CUDA!