Structurally and semantically coherent deep image inpainting

Abstract

This thesis proposes an augmented method for image completion, particularly for images of human faces by leveraging on deep learning based inpainting techniques. Face completion generally tend to be a daunting task because of the relatively low uniformity of a face attributed to structures like eyes, nose, etc. Here, understanding the top level context is paramount for proper semantic completion. Our method improves upon existing inpainting techniques that reduces context difference by locating the closest encoding of the damaged image in the latent space of a pretrained deep generator. However, these existing methods fail to consider key facial structures (eyes, nose, jawline, etc) and their respective locations. We mitigate this by introducing a face landmark detector and a corresponding landmark loss. We add this landmark loss to the construction loss between the damaged and generated image and the adversarial loss of the generative model. After several experimentation, we concluded that the added landmark loss attributes to better understanding of top level context and hence more visually appealing inpainted images.