Documentation: Controller tracking

Controlling tracking is used to send out OSC or MIDI messages every frame, computing assorted relations between joints and users.

  • Clicking the component’s checkbox in the page tree toggles between the output modes. It’s possible to output OSC, MIDI or both. Alternatively, this can be done by clicking the enable buttons directly on the Controller Tracker page.

  • IP address & port override: If enabled, these IP address and port are used when outputting OSC data. Otherwise the default address & port on the setup page is used.

  • Smoothing: This slider controls the amount of smoothing applied to the motion ranges. Lower smoothing values may result in erratic values being outputted.

For each of the controller values there are a number of parameters:

  • The motion range specifies the range in which the value is. This is used when interpolating the value into the message range.

  • Message range is used to control the final value sent out from the Controller Tracker. The motion range is interpolated so that it remains within the message range. For example, hand distance with min of 0 and max of 500, and message range of 128 will result the hand distance value of 250 being mapped to the value 64.

  • OSC path is used for defining the OSC address and its parameter tags. The controller values all use the parameter tags [I], [U], {V}, {v}

  • MIDI control CC specifies the CC sent out for each controller value

  • MIDI output is one of the output devices and channels as defined in NI mate preferences.

  • Solo: If enabled, only this value is sent out and all others are ignored.

  • Mute: Don’t send out this value.


The general controller values are used for values not directly related to anything specific.

  • Hand distance: The distance of one hand from the other

  • Hand angle: The angle between two hands

  • Number of people: How many people are currently detected

  • Active users: How many people are currently active

Left and right hand

  • X, Y and Z: The hand’s coordinates

  • V: The hand’s velocity as a vector

  • Vx, Vy and Vz: The components of the velocity vector


  • X, Y and Z: The body’s coordinates

  • V: The velocity vector for the body

  • Twist, Bow and Lean: Angles of rotation for each of the respective values.


  • Left & Right hand: Sends the position vector (3 floats) for the hands

  • Body: Sends the position vector (3 floats) for the body

  • Head: Sends the position vector (3 floats) for the head