Wrinkles and all: Hi-res eyelid reconstruction makes digital doubles look more realistic

Little details, such as the wrinkling or twitching of an eyelid, can have a big impact on whether a digitally rendered face looks real in a film or videogame. Now scientists at Disney Research have devised the first method to capture these subtleties of the eyelids in detail.

Their method, to be presented at ACM SIGGRAPH 2015, the International Conference on Computer Graphics and Interactive Techniques, in Los Angeles Aug. 9-13, captures detailed, dynamic skin features and creates plausible folding and stretching in areas that aren't directly captured during data collection.

Artists have always recognized that the eyes are a key feature for conveying emotion and even their subtle motions can contribute to the realism of a face. But despite tremendous progress in facial scanning, performance capture and skin rendering in recent years, creating photorealistic actors for films has required highly skilled artists to manually recreate eyelid movements.

"The eyelids are extremely challenging for digital scanning and reconstruction," said Thabo Beeler, a research scientist at Disney Research. "The skin rolls and folds inward when the eye opens and stretches over the eyeball when the eye is shut," he explained. Eyelashes can block large segments of the eyelids or cast shadows. "Even worse, in many facial expressions, a significant part of the eyelid is folded in and not visible at all."

Beeler, along with Amit Bermano, Yeara Kozlov, Derek Bradley, Bernd Bickel and Markus Gross, tackled the problem by first creating a geometric deformation model of the eyelid. Based on the physiological behavior of the eye-skin interface, this model provides plausible motions for the inevitable instances when eyelids are not fully visible on the original human actor.

This model is then personalized with image-based data. The team observed that wrinkles, in particular, can greatly change the folding of the eyelids - resulting in differences not only from person to person, but also between the left and the right eyelids.

"Eyelids are extremely unique and can produce extremely different wrinkles," Bermano noted. Their method therefore focuses on accurately determining the location of wrinkles from images and then predicting how the location and shape of the wrinkles will change based not only on the current state of the eye region, but also its history.

The method can be used with a variety of motion capture techniques, including the Medusa Performance Capture System developed by Disney Research, the researchers said.

"The ability to accurately capture eyelids can be a great addition to Medusa, which already has been used to create realistic digital doubles for several feature films," Bradley said.