Proposal: Facial Rigging with shape keys

Over the years of animating on numerous films and projects, I’ve worked with different rigging techniques to create expressive faces. Each character required their own personality, style, and shape language, and therefore, their own rigging structure. At Blender, most recent facial rigs use a combination of facial bones, corrective blendshapes, and additional deformers. Bone-based rigs have their advantages but can become complicated when combined with action constraints, shape keys, and other drivers to achieve a specific deformation.

My experience as a lead animator at Sony Imageworks and a teacher of the 'Facial Animation' course at AnimSchool gave me the opportunity to dive into the process behind their facial rigs. I came to the conclusion that, in order to achieve specifically designed shapes, it’s best to minimize computer interpolation and instead have them designed by an artist. Looking at the animation industry over the past 10 years, the overwhelming majority of feature film facial rigs use the same method: a lot (and I mean a really lot) of blendshapes, or what we call shape keys.

I do not claim to be a facial rigger at all. I come from the other side of the fence—as an animator using facial rigs. However, given that the majority of appealing facial rigs I’ve worked with are shape key-based, I propose supporting this rigging workflow in Blender to assist studios that want to adopt this method for their productions.

This is by no means the only way to rig a 3D character’s face, considering the wide range of styles and approaches to facial rigging. However, improving this particular workflow in Blender could greatly benefit many animation studios and riggers. Ultimately, I would like to create a Facial Rigging course on studio.blender.org to provide studios and individuals with a solid foundation to build upon. This would involve improving the current shape key system, enhancing quality of life features, introducing more dynamic operators, and developing new tools to streamline the shape key-based facial rigging workflow.

This is a proposal for Blender Studio, as well as the community to respond to in order to gage the need for such an investment. Therefore we would love to get your input on this topic and hear from you if this would be something that would help your rigging workflow as well. Let your voice be heard by placing a comment at the bottom of this article, we’d love to hear from you!

What is Shape key Based rigging?

Shape Key based facial rigging consist primarily out of multiple connected shape keys and corrective shape keys that neatly blend together resulting in dynamic and designed facial deformations. These shapekeys are manipulated by control bones connected to individual vertices and are linked through drivers (with expressions), triggering one or multiple shape keys at once in order to reach the designated deformation. The animator moving the bone in 3d space is actually an illusion, it is bound to the mesh which is simply triggering shapekeys and seem like it is moving in space. This allows the rigger to combine infinite amount of shapes while the bones are following correctly.

The method of creating shape keys is duplicating the face mesh and creating designed deformations on this. In order to create a smooth transition between the different shapes (for example the mouth corners, a temporary armature is created in order to generate the range of motion on the initial mesh) Parts of the face like the lips, cheeks and brows are using a single deformation bone to create the range of motion (positive and negative in X, Y and Z) for the shape keys. this bone will be neatly weightpainted to the correct deformation area to create an organic and appealing deformation. Once this is established, I extract all key poses from the animated bone. This should get us 85% to the final shapes and can be tweaked, sculpted, masked additionally. Once these are looking good, the shape keys will be split into separate (local) shapes and assigned to its designated control bones. This process can be partially automated and will save a lot of time.

Adjusting shapes can be done rather easily when following this workflow, by adding a new shape key as a layer and tweaking on that layer as desired. This way it is non-destructive and makes it easy to manage iterations. For updating existing shapes on the facial rig, i’ve created a script that automatically updates shape keys within its designated collection. This is a clean way to update vertex data within each shape key without losing its drivers.

Additionally, Bones will be the foundation for the main structure and movement of the skull, jaw, eyes, eyelids/lashes, nose and ears. (and eyelids) as they are more static and in no need of heavily designed or complex deformation. Deformations are pre-designed by the rigger and approved by the supervisor so that the animator has a good foundation of setting up the expressions relatively easily without manipulating too many bones.

Control Bone Attachment to Mesh Vertices To attach control bones to the face, we need to avoid dependency cycles. In order to achieve this, we use the Display Override transform. The hook bones are placed with a location constraint attached to a ‘single-vert’ vertex group. Now the bones are following the designated vertex location. The function of these hook bones are basically to drive the display location of the Control Bones which drive the connected shapekeys.

Proposal for developing a Facial Rigging course

Now, you might think this sounds like an old technique from the 2000s, but it’s still the standard at AAA animation studios. This is because of its artist-oriented workflow and the ability to effectively art-direct it in the early rigging stages. However, Blender currently does not support this workflow very well—at least not to the extent of managing and iterating a facial rig properly in a production environment in a non-destructive way.

I’d like to improve this by developing better tooling, enhancing the quality of life aspects of shape key implementation, and optimizing the way shape keys are stored.

Wow! You made it to the end of the article :D. I’m excited to implement a solid workflow like this for high quality characters in Blender. Would you like to see a course on how to make a shape key based facial rig with this workflow? Let us know in the comments below!