Producing animations from 3D face scans

In this paper we take our existing research into 3D surface scanning using uncoded structured light, optimized for human faces, and develop a method to record a sequence of face movements and visualise these as an animation in real time; we call this method the 3D Animation Processor (3DAP). The applications for this work include 2D and 3D face recognition, broadcast and feature film animation, and computer games production. It should be stressed that the 3D recording of facial movements in real time is a difficult problem that is attracting considerable research attention; but accurate and appealing results have so far proved elusive. We record the faces of a number of speaking subjects, process the data representing their shape and colour as it changes over time, and visualise the animated face models. We identify a particular problem in franze continuity, whereby unacceptable jumps and jitters occur in both the shape of the face and its colour mapping, and begin to solve this problem using hole-filling and interframe interpolation. We also investigate methods of feature tagging, so that the model can be placed in a fixed coordinate system and thereby incorporated into computer generated animations.