RGB-D Reconstruction for Mixed Reality - Juan Raul Padron Griffe

About me
I am a Marie Sklodowska-Curie Fellow of the EU Project PRIME and a PhD candidate at the Graphics and Imaging Lab. My PhD thesis under the supervision of Prof. Adolfo Muñoz and Prof. Adrian Jarabo focuses on physically-based rendering and appearance modeling of multi-scale materials, such as biological tissues (skin, scales and feathers) and intricate human-made objects (cosmetics). Previously, I earned my Bachelor of Sciene degree in Computer Science at the Universidad Central de Venezuela, where I specialized in computer graphics and imaging processing. My undergraduate thesis explored the generation and visualization of procedural terrains. Later, I received my Master of Science degree in Informatics at the Technical University of Munich, concentrating on computer graphics, computer vision and machine learning. During my Master studies, I conducted research on 3D Scanning and Neural Rendering for object and face relighting advised by Dr. Justus Thies. Beyond my academic experience, I have two years of software development experience in backend technologies (.NET, Service Stack, Java, Spring).

Looking for Opportunities
I recently submitted my Ph.D. dissertation under the title “Modeling and Rendering of Multiscale Materials”. I am currently seeking both postdoctoral and industry opportunities where I can apply my expertise in computer graphics, computer vision and artificial intelligence for the digital acquisition, representation and understanding of the visual world. My combined expertise in computer graphics, computer vision, machine learning, and software engineering allows me to tackle complex technical challenges from both a research and implementation perspective, If you're interested in collaboration or have an opportunity that aligns with my expertise, please feel free to reach out!

For the 3D Scanning and Motion project at the Technical University of Munich, we propose a real-time marker-less tracking and 3D reconstruction implementation and show how it can be used to create a mixed reality game using the Unity3D game engine. The concept was inspired by the spatial mapping technology behind Microsoft HoloLens and our 3D reconstruction pipeline is based on the Kinect Fusion pipeline. Tracking and reconstruction is performed in real-time using GPGPU acceleration (Cuda 10, DirectX 11 compute shaders). Finally, several Unity modules (e.g. physics, particle systems, etc) can be used to create interesting mixed reality projects.

Results:

Original Scene Reconstruction Unity Animation

Team Members: Juan Raul Padron Griffe, Wojciech Zielonka, Patrick Radner, Baris Yazici
Instructors: Angela Dai, Justus Thies

Github repository Poster

Wojciech Zielonka, Patrick Radner and Mustafa Isik worked later on a really cool project for the 3D Scanning and Spatial Learning Practical Course at the Visual Computing lab. Take a look at RGB Face Tracking and Reconstruction using CUDA!