Browsing by Author "Cho, Yeon Joo"

Filter results by typing the first few letters
Now showing 1 - 1 of 1
  • Thumbnail Image
    ItemOpen Access
    A 3D-aware conditional diffusion model for gaze redirection
    (2024) Cho, Yeon Joo
    Gaze redirection refers to the task of modifying the direction of eye gaze and its corresponding facial counterparts to a targeted direction, while preserving the original identity of the subject. An effective gaze redirection approach must (i) be aware of the 3D nature of the task, (ii) accurately redirect the gaze into any specified direction, and (iii) generate photorealistic output images that preserve the shape and texture details from the input images. In response to these requirements, this thesis presents a novel approach to gaze redirection using a 3D-aware conditional diffusion model that leverages the intrinsic geometric properties of human faces. This approach effectively transforms the task into a conditional image-to-image translation. To embed 3D awareness comprehensively, we adopt a viewpoint-conditioned diffusion model, that can learn the 3D context of the facial geometry. Then, the conditions to this model are unique gaze rotations and latent facial parameters from the face images. These strategies are further reinforced by a novel loss function focused on gaze direction and head orientation, which enhances the model's ability to learn and apply accurate gaze and head adjustments effectively. Together, these elements underscore the potential of our approach to produce high-quality, accurate gaze redirection, fulfilling the complex demands of this sophisticated visual task.