For Computer Graphics (CG) film production, animation is a crucial step giving a dynamic representation into the scene. In terms of facial animations, different deformations to a facial model is applied in order to represent expressions following the scripts. Even though this task is more or less mechanical, it requires a huge amount of manipulations on a graphical user interface. In order to facilitate this task, we are developing a semi-automate system for animating a face of a CG character by capturing an actor with a kinect camera. The facial action of an actor directly drives facial animation of a character.
Our system consists of a kinect camera, recording application, CG animation preview monitor, and a post-processing application. The kinect camera inputs; RGB and depth images, are basically processed in two steps. Firstly, the input images are analyzed to obtain a set of levels along facial action units (FACS), which are the fundamental actions of individual muscles or groups of muscles. In other words, we express actions at defined facial parts such as cheeks, lips, and eye blows numerically. Secondly, using this level set, we project the expression of the actor to a character by deforming its 3D shape. The character is deformed by merging its topologically identical 3D shape models at different expressions such as neutral, stretched lip, puff out a cheek, widely open an eye, etc at the maximum. The merging weight to each shape model is computed from the level set of FACS.
Collaborators:
- Luc Petitot (RealReality.com)