Juan Raúl Padrón Griffe

Epale! I am a Marie Sklodowska-Curie fellow of the EU Project PRIME and a PhD candidate at the Graphics and Imaging Lab (Universidad de Zaragoza). My PhD thesis under the supervision of Prof. Adolfo Muñoz and Adrian Jarabo focuses on developing theory and methods for accurate and efficient rendering of complex volumetric appearances. Previously, I obtained my Bachelor degree in Computer Science (2015) at the Central University of Venezuela. Later, I received my Master degree in Informatics (2020) at the Technical University of Munich. During my Master studies I focused mostly on the Computer Graphics and Vision subjects, where I was fortunate enough to be advised by Prof. Matthias Niessner and Dr. Justus Thies at the Visual Computing lab to conduct my research on 3D Scanning and Neural Rendering.

I am a computer scientist enthusiastic about the intersection of realistic image synthesis, graphics-based vision and machine learning for the digital acquisition, representation and understanding of the visual world. I am currently interested in pushing the state of the art on physically-based rendering of complex multi-scale materials like biological tissues. In my research, I rely on powerful tools like Monte Carlo simulation and gradient-based optimization. In the long term, I believe the combination of powerful forward models (simulation algorithms) and inverse models (gradient-based models) could be impactful in other interesting domains too like computational biology.

Projects

Face Relighting In The Wild

2020, Jul 31    

Relighting plays an essential role in realistically transferring objects from a captured environment into another one. In particular, current applications like telepresence need to relit faces consistently with the illumination conditions of the target environment to offer an authentic immersive experience. Traditional physically-based methods for portrait relighting rely on an intrinsic image decomposition step, which requires to solve a challenging inverse rendering problem in order to obtain the underlying face geometry, reflectance material and lighting. Inaccurate estimation of these components usually leads to strong artifacts (e.g. artificial highlights or ghost effects) in the subsequent relighting step. In recent years, several deep learning architectures have been proposed to address this limitation. However, none of them are free from these artifacts.

In my Master thesis, we propose a general framework for automatic relighting enhancement using the StyleGAN generator as a photorealistic portrait prior. Specifically, we apply the ratio image-based face relighting to an artificial portrait dataset generated using the StyleGAN model. Next, we refine this dataset by projecting back the relit samples into the StyleGAN space. Then, we train an autoencoder network to relit portrait images from a source portrait image and a target spherical harmonic lighting. We evaluate the proposed method on our synthetic dataset, the Laval face and lighting dataset and the Multi-PIE dataset both qualitatively and quantitatively. Our experiments prove that this method can enhance the state of the art single portrait relighting algorithm for synthetic datasets. Unlike this algorithm, we achieve these results relying on a synthetic dataset five times smaller employing a traditional training scheme.

Results:

Obama Lights Obama Relighting

Advisor: Justus Thies Supervisor: Matthias Niessner

Github repository Document

Twitter
# Neural Rendering # Relighting