Juan Raúl Padrón Griffe

Epale! I am a Marie Sklodowska-Curie fellow of the EU Project PRIME and a PhD candidate at the Graphics and Imaging Lab (Universidad de Zaragoza). My PhD thesis under the supervision of Prof. Adolfo Muñoz and Adrian Jarabo focuses on developing theory and methods for accurate and efficient rendering of complex volumetric appearances. Previously, I obtained my Bachelor degree in Computer Science (2015) at the Central University of Venezuela. Later, I received my Master degree in Informatics (2020) at the Technical University of Munich. During my Master studies I focused mostly on the Computer Graphics and Vision subjects, where I was fortunate enough to be advised by Prof. Matthias Niessner and Dr. Justus Thies at the Visual Computing lab to conduct my research on 3D Scanning and Neural Rendering.

I am a computer scientist enthusiastic about the intersection of realistic image synthesis, graphics-based vision and machine learning for the digital acquisition, representation and understanding of the visual world. I am currently interested in pushing the state of the art on physically-based rendering of complex multi-scale materials like biological tissues. In my research, I rely on powerful tools like Monte Carlo simulation and gradient-based optimization. In the long term, I believe the combination of powerful forward models (simulation algorithms) and inverse models (gradient-based models) could be impactful in other interesting domains too like computational biology.

Projects

Game Capture

2018, Sep 28    

The main goal of the Hiwi project at the chair of Remote Sensing Technology at the Technical University of Munich is collecting potentially useful information from video games in order to train computer vision models for autonomous driving applications. The project is inspired in the seminal publication Playing for Data, where the authors show that acquired data from video games supplemented with real-world images significantly increases the accuracy of deep learning models for the semantic segmentation task. In addition, the acquisition pipeline reduces the amount of hand-labeled real-world data. We develop a prototype in C++ to collect in realtime the DirectX Frame Buffers (RGB, Stencils, Albedo, Irradiance, Specular, Normal) from the video game Grand Theft Auto V. Finally, we could also extract further internal game states (e.g. time of day, location, vehicle speed, etc) using the Script Hook V mod.

In the second phase, we tried to capture data from other games based on the Free Supervision from Video Games paper. The GameHook library wraps DirectX 11 to intercept and modify the rendering code of a particular game, including the injection of code into the vertex or pixel shaders. Unfortunately, the library failed to hook games like Project Cars 2 or Sebastian Loeb Rally.

Results:

Disparity Map Normal Map Instance Segmentation

Albedo Map Specular Map Irradiance Map

Advisors: Sandra Aigner, Lukas Liebel
Supervisor: Marco Körner

If you want to know more about the project, please contact Lukas Liebel and Sandra Aigner. They are really nice advisors and they are working in interesting projects. I would also suggest you to read this amazing post GTA V - Graphics Study and play with the amazing RenderDoc tool.

Twitter
# Video Games # Data Capture