Lip sync animation can be one of the most resource-intensive tasks in the animation workflow. For creators working on low-budget or microbudget projects, this limitation often pits dialogue against all the other creative elements of your production. Incorporating facial motion-capture into your workflow can cut your lip sync time in half (at least!), and save you from having to sacrifice meaningful dialogue in an effort to save time and money.
As we’ve built out our own facial mocap tools, we’ve picked up some tricks along the way to share with our fellow animations who are just getting started with this process.
Film at an extremely high frame rate
We film our actor’s face at 100 frames per second to ensure that there is almost no motion blur of the markers on his/her face. Otherwise, the motion blur prevents your animation program from following the marker from frame to frame.
Add a small dot in the middle of your markers
As you can see in the video above, we add a small dot (just a mark with a pen) to the markers on the actor’s face. This allows for more precise tracking.
Don’t use more facial markers than you really need
More dots are not always better. If you are just using your facial motion capture markers to compute lip syncing and add some basic expressions like raised eyebrows, etc., you only need a small number of markers in key points around the facial features. You can see the points we’ve chosen in the video above.
The reason that you’ll sometimes see behind-the-scenes images with an actor’s entire face covered in markers is that those mocap systems are probably being used to create a character that actually looks just like the actor! So, they need to follow how every muscle and wrinkle in that actor’s face moves. This type of matching is usually irrelevant if you’re just doing lip sync, as in our example. Obviously we don’t need the alpaca to look exactly like our actor.
Your actor’s face has to be completely still
You’ve got two options for keeping your actor’s face completely still.
(1) Literally hold your head still. That is, put your index fingers of each hand up to your temples to help stabilize your head as you speak. Obviously, this will significantly limit the freedom your actor has to perform and be expressive. But in a pinch, this can work to at least allow you to do basic lip sync.
(2) Mount your camera to a helmet system. This can be as simple as mounting a Go-Pro to the front of a bike helmet that can be securely strapped to your actor’s head. This option will give your actor much more freedom of motion while s/he performs.
We hope these tips will help you more easily incorporate facial motion capture into your lip sync animation process. Of course, there’s always trial and error involved with any new set-up. So, our final tip is to enjoy the learning process as much as you can, and remember that all the time you’re investing up front to sharpen your system will pay dividends in the end when you cut hours off your lip sync time!