Face tracking / Blender/ all points jump on 0.0.0


#1

I have a strange bug. Or maybe I didn’t used the program right.

I am using a Kinect V2, Ni-Mate 2, and a win 8.1 Pc.
So far everything with the normal motion tracking works. I managed to make a moving Mesh on the screen.

Now I also wanted to try the face tracking. For even a better animation.
So I open a new blender file, delete everything in it, and start the Ni-mate with ONLY the face analysis check box.
in blender I start the tracking with creating the emptys. I was expected to see a bunch of point roughly forming the shape of a face.

Now the problem is that all emptys are instant moving to the 0.0.0. point of my grid. All of the point.
When I move infront of my camera, I see that there are some twitching on the screen. A few points (looked like 7 in number) moved. And I am able to recognise my movement. But they are blinking, becaus ethey always get teleportet back to 0.

Here you can see the programm analyzing the face correctly, but in blender the points are all on 0.0.0. while a few of them move a bit apart.

Don’t get the wrong impression. These points are there only for a frame. I needed a few minutes to catch them with a screenshoot. Else they all are on the 0 point.

I allready love your program but this is an error I can’t handle on my own. I hope for your support.


#2

I can confirm there’s some kind of reset to 0 going on. I’m wondering if this happens when NI mate does not get new data from the sensor, or there’s a low confidence frame in the middle.

Until this can be properly fixed, here’s a version of the Blender plugin that ignores zero data: https://ni-mate.com/release/plugins/blender/animation_delicode_ni_mate_tools_beta.py

Thank you for the report.


#3

Okay I used this beta file (and of course unchecked the old one in the plugin settings of blender).

The results are… different.
Now the jumping to 0 is gone. And I had the chance to analyze what is now on my screen.

There are still a lot emptys in the 0 point. I selected them and noticed that they all have one thing in common.
They are no points on a head, they are sort of integer. 1 or 0 for mouth open or closed. At least that is the impression I get. They are more values, than points you could find on a face.
But these points still twitch the way like before.

ONE change is that the emptys in the lower right corner, hold their position and work like the user wants. They move along the real face movement.

A thing I now noticed is that not all values/emptys from Ni Mate are converted into emptys in blender. As you can see in the object list, several Face points of Ni Mate are missing, like “/Lower_Lip_Center” for example, even thou they are checked in N.M.
I think if these missing points get fixed it would work perfectly.

I hope this response helps you developing N.M.
Thank you for everything!


#4

NI mate with K4W does not provide data for all the points shown in the user interface. They should probably be hidden for K4W to avoid the confusion. Most of those points are values that other sensors supporting more landmarks might output. Looking at it, the values that K4W outputs are: Nose, EyeLeft, EyeRight, MouthLeft, MouthRight.

I’m wondering if this [0,1] integer problem is an issue with the values being between that range and then being casted into ints. What happens if you change the face analysis addresses into: /Eye_Left_Center {X} {Y} {Z} (replacing the name for each address)? These are parameters tags which can be used to customize outgoing OSC in NI mate. In this case, using {} will output the values as floating point values instead of integers.

You can also try: /Eye_Left_Center {X*10} {Y*10} {Z*10} which will multiply all values by 10 before outputting them.