Dynamic facial asset and rig generation from a single scan
Abstract
The creation of high-fidelity computer-generated (CG) characters for films and games is tied with intensive manual labor, which involves the creation of comprehensive facial assets that are often captured using complex hardware. To simplify and accelerate this digitization process, we propose a framework for the automatic generation of high-quality dynamic facial models, including rigs which can be readily deployed for artists to polish. Our framework takes a single scan as input to generate a set of personalized blendshapes, dynamic textures, as well as secondary facial components ( e.g. , teeth and eyeballs). Based on a facial database with over 4, 000 scans with pore-level details, varying expressions and identities, we adopt a self-supervised neural network to learn personalized blendshapes from a set of template expressions. We also model the joint distribution between identities and expressions, enabling the inference of a full set of personalized blendshapes with dynamic appearances from a single neutral input scan. Our generated personalized face rig assets are seamlessly compatible with professional production pipelines for facial animation and rendering. We demonstrate a highly robust and effective framework on a wide range of subjects, and showcase high-fidelity facial animations with automatically generated personalized dynamic textures.
Document Details
- Document Type
- Pub Defense Publication
- Publication Date
- Nov 27, 2020
- Source ID
- 10.1145/3414685.3417817
Entities
People
- Hao Li
- Jiaman Li
- Karl Bladin
- Mingming He
- Yajie Zhao
- Zhengfei Kuang
Organizations
- Adobe
- Sony Group
- United States Army Research Laboratory
- University of Southern California