Minor Project- Media Design School
Facial Motion Capture:
Facial Motion Capture is the process of electronically converting the movements of a person's face into a digital database using cameras or laser scanners.
Facial Motion Capture is the process of electronically converting the movements of a person's face into a digital database using cameras or laser scanners.
Shoot a nice footage with a dialog and some facial gestures .
Variety of techniques are used to do motion capture
Motion capture
Production Technique
Production Technique
it depends on how high or low quality Footage u have shoot.
Kinetic is one of the apparatus which is used to motion capture
Kinetic is one of the apparatus which is used to motion capture
http://www.xbox.com/en-NZ/Kinetic
2D and 3D are other two types of tracking by which motion capture.
3D:
To capture 3D motion, we need more then 2 cameras to tracks 3D co-ordinates of one tracking point.
eg: avatar motion capture which tracks human character movement.
specific trackers/ sensors are used to track the actual co-ordinate of the sensor.
Advantage/Disadvantage:
That 3D position of the tracker is perfectly captured captured and it can be played in real time
but can be very expensive and can also need special software to Capture
2D:
Only one camera used in this tracking data,which is Rigged to face with a setup to track head movement
Data can only be captured in X and y,coordinates, no depth is involved .more important to have a stabilized camera
Advantage/Disadvantages:
It is cheaper to do but tracking will have to be done manually with tracking software eg,Nuke,PF Track etc.
disadvantage is that you wont get this in real time .This technique is also used in avatar for facial capture
This setup requires a light weight camera which is mounted on a helmet or a cap attached to head.
If wired then will have to connect that camera to the system.
Camera should have sustainable frame rate- 30FPS- 640*480
Characters head should be very much stable and aligned to camera.
if moved than the animation will give unnecessary movement to the character
Markers:
2D markers should be easy and removable eg, eyeliner.
Light should be proper, not to bright not to dark .avoid highlights and grains on the tracking markers
decide the amount of markers which need to be placed for the character movement
more the markers more the quality of the animation we get from the tracking data
Fewer markers are good for games kind of animation .
The placement of the markers should be according to the bone setup of the character .
meaning they should be in the same general area of the face .
Modeling :
Take Front and Side images of the face which u need to Model ,
Create a face model using Z brush, Maya.
Texturing :
Unwrap the models Uvs and texture
Rigging:
Rig the character according the markers , smooth bind the character's skin and keep ready for transferring Animation.
Tracking :
As the head is stable enough then roto out the outer area where there are no markers .so it becomes much easier to track the points
Import the footage into a tracking software ,Syntheyes , pf track , boujou ,mocha, match move, blender etc
Export the tracking data in form of locators
eg Match move the tracking data is exported in ASCII point track , which when imported in Maya will give locators which can be easily parented with the corresponding joint .
If Maya doesn't support the exported data from the software which u have extracted ,then a 3rd party plug-in is used to import those tracking data into Maya
plug-in name : survey solver 2D
http://minchomarinov.com/SS/
the plug in will allow you to import the tracking data as locators .
constrain the corresponding locators to the respective joint
Note down all the data while recording the video .eg . focal length , FPS , and resolution
Refining Animation :
Refine the animation which is transferred on the Joints through locators.
We need to bake the animation by selecting all joints
edit>keys>bake simulation
and then delete all the constraints, to avoid problems
re tweak the animation in certain necessary area .
final result is motion capture Facial Animation
Lighting :
Create HDRI and throw some lights on the model ,
Render out with Vray with multiple passes
2D and 3D are other two types of tracking by which motion capture.
3D:
To capture 3D motion, we need more then 2 cameras to tracks 3D co-ordinates of one tracking point.
eg: avatar motion capture which tracks human character movement.
specific trackers/ sensors are used to track the actual co-ordinate of the sensor.
Advantage/Disadvantage:
That 3D position of the tracker is perfectly captured captured and it can be played in real time
but can be very expensive and can also need special software to Capture
2D:
Only one camera used in this tracking data,which is Rigged to face with a setup to track head movement
Data can only be captured in X and y,coordinates, no depth is involved .more important to have a stabilized camera
Advantage/Disadvantages:
It is cheaper to do but tracking will have to be done manually with tracking software eg,Nuke,PF Track etc.
disadvantage is that you wont get this in real time .This technique is also used in avatar for facial capture
This setup requires a light weight camera which is mounted on a helmet or a cap attached to head.
If wired then will have to connect that camera to the system.
Camera should have sustainable frame rate- 30FPS- 640*480
Characters head should be very much stable and aligned to camera.
if moved than the animation will give unnecessary movement to the character
Markers:
2D markers should be easy and removable eg, eyeliner.
Light should be proper, not to bright not to dark .avoid highlights and grains on the tracking markers
decide the amount of markers which need to be placed for the character movement
more the markers more the quality of the animation we get from the tracking data
Fewer markers are good for games kind of animation .
The placement of the markers should be according to the bone setup of the character .
meaning they should be in the same general area of the face .
Production Workflow
4Weeks
Analyze the video for tracking purpose.
If the head is moving a little more then expected then the video needs to be stabilized using software's like after effects ,nuke etc.
Modeling :
Take Front and Side images of the face which u need to Model ,
Create a face model using Z brush, Maya.
Texturing :
Unwrap the models Uvs and texture
Rigging:
Rig the character according the markers , smooth bind the character's skin and keep ready for transferring Animation.
Tracking :
As the head is stable enough then roto out the outer area where there are no markers .so it becomes much easier to track the points
Import the footage into a tracking software ,Syntheyes , pf track , boujou ,mocha, match move, blender etc
Export the tracking data in form of locators
eg Match move the tracking data is exported in ASCII point track , which when imported in Maya will give locators which can be easily parented with the corresponding joint .
If Maya doesn't support the exported data from the software which u have extracted ,then a 3rd party plug-in is used to import those tracking data into Maya
plug-in name : survey solver 2D
http://minchomarinov.com/SS/
the plug in will allow you to import the tracking data as locators .
constrain the corresponding locators to the respective joint
Note down all the data while recording the video .eg . focal length , FPS , and resolution
Refining Animation :
Refine the animation which is transferred on the Joints through locators.
We need to bake the animation by selecting all joints
edit>keys>bake simulation
and then delete all the constraints, to avoid problems
re tweak the animation in certain necessary area .
final result is motion capture Facial Animation
Lighting :
Create HDRI and throw some lights on the model ,
Render out with Vray with multiple passes
No comments:
Post a Comment