Jump to content

Sign in to follow this  
youngskywalker

Face mocap data -> custom head model

Recommended Posts

Hi everyone

I'm currently struggling with a project. Let me try to be as clear as possible :

I am basically trying to animate a face on a character that has a Mixamo mocap animation on it.

 

I am using CV-AR to record facial mocap with an iPhone 11, but the C4D plugin gives me a face (not a head) with a bunch of Xpresso premade for me.

 

If I wanted to transfer that mocap data to a custom model of a head, or to the head on the Mixamo character, what would I need to do ? Do I need to rig the face ? If so, how do I do that ? Do I use FACS / blendshapes ? Is there a specific rig I need to use so that it works with the Xpresso I got from the CV-AR data ?

 

I'm very new to mocap and I am stuck with this part of the project right now. Any help is appreciated

 

Please don't hesitate to ask if you need extra info

 

Thank you. I hope someone can help me with this

Share this post


Link to post

CV-AR gives you the vertex movement of the face and it also gives you the blend shape values from ARKit that you can use to drive your own Pose Morphs to control your own model of a face.

 

If you want to have a face talking on your own mesh then you would need to create the blend shapes yourself (using FACS is good) and then drive them using the blend shape parameters from the CV-AR.  The Xpresso setup that CV-AR gives you is a good starting point to see how this can be done, but you don't have to use it if you don't want to.

 

You could also drive a rigged model using the blend shape values as well, but that would need a different xpresso setup. 

 

Hope that helps.

Share this post


Link to post
  • Topic Author
  • 17 hours ago, kbar said:

    CV-AR gives you the vertex movement of the face and it also gives you the blend shape values from ARKit that you can use to drive your own Pose Morphs to control your own model of a face.

     

    If you want to have a face talking on your own mesh then you would need to create the blend shapes yourself (using FACS is good) and then drive them using the blend shape parameters from the CV-AR.  The Xpresso setup that CV-AR gives you is a good starting point to see how this can be done, but you don't have to use it if you don't want to.

     

    You could also drive a rigged model using the blend shape values as well, but that would need a different xpresso setup. 

     

    Hope that helps.

    Thank you so much for your reply, and great work on CV-AR ! I have some follow up questions if that's okay ?

     

    I've never rigged a face, and I never worked with Xpresso. I am doing research right now to understand what FACS is. I am going to have my model's face rigged, and try to have it animated using the Xpresso that CV-AR automatically makes. My question is the following :

     

    Assuming I have a rigged face, how easy is the setup to transfer the CV-AR data to my rigged face ? Especially for someone who's never worked with blend shapes and Xpresso. What do I need to do specifically ?

    Share this post


    Link to post

    I would love to see some sample  CV-AR captures if anyone doesn't mind? 

     

    I think you need similar\exact blend shapes in your target model.  Maximo models have quite a lot of blends shapes. They are the hidden meshes in the model that drive the pose morphs.   I haven't used CV-AR, but an external face capture program. I have found that if the blend shapes are exact or similar. You can past the pose morph data from one one model to the target model. Easier said than done but that's the aim.  Not all blend shapes are constructed the same, from model to model.  In a nut shell you need a target model that is like your capture model. 

    Share this post


    Link to post
  • Topic Author
  • On 1/22/2020 at 6:45 PM, joolsd said:

    I would love to see some sample  CV-AR captures if anyone doesn't mind? 

     

    I think you need similar\exact blend shapes in your target model.  Maximo models have quite a lot of blends shapes. They are the hidden meshes in the model that drive the pose morphs.   I haven't used CV-AR, but an external face capture program. I have found that if the blend shapes are exact or similar. You can past the pose morph data from one one model to the target model. Easier said than done but that's the aim.  Not all blend shapes are constructed the same, from model to model.  In a nut shell you need a target model that is like your capture model. 

     

    I figured out in the meantime that this is the problem and solution... There is no "universal" or "standard" blend shape structure. A blinking left eye can be named differently in your target. So you have to search manually... I managed to connect some blendshapes from my capture to my target, obvious ones like "MouthOpen" and "LeftEyeBlink". That worked, but the target was far from showing the accurate facial expressions that the capture had. It was just named completely differently in the target face rig, which made it a pain and ultimately impossible for me to develop a workflow that would be time efficient.

     

    I can post a CV-AR capture if you want.

    Share this post


    Link to post
    19 hours ago, youngskywalker said:

     

    I figured out in the meantime that this is the problem and solution... There is no "universal" or "standard" blend shape structure. A blinking left eye can be named differently in your target. So you have to search manually... I managed to connect some blendshapes from my capture to my target, obvious ones like "MouthOpen" and "LeftEyeBlink". That worked, but the target was far from showing the accurate facial expressions that the capture had. It was just named completely differently in the target face rig, which made it a pain and ultimately impossible for me to develop a workflow that would be time efficient.

     

    I can post a CV-AR capture if you want.

    Yes please, just from a curiousity point of view, because I do not have a high end phone. Thank you very much.

    Share this post


    Link to post

    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.

    Guest
    Reply to this topic...

    ×   Pasted as rich text.   Paste as plain text instead

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.

    Sign in to follow this  

    • Recently Browsing   0 members

      No registered users viewing this page.

    ×
    ×
    • Create New...