What are the .nirec files?


#1

The NI mate Record file (.nirec) is used by Delicode NI mate 2 to playback recorded sensor data. This data contains information about active users and their skeleton tracking. The format may be expanded in the future to include more data. This document explains how the format works and how it can be read by 3rd party applications.

The files are written using:

QDataStream out(&save_file);
out.setVersion(QDataStream::Qt_5_4);
out.setFloatingPointPrecision(QDataStream::SinglePrecision);
  1. The file begins with the format tag that is used to figure out that the file is indeed a proper recording. The tag is is: [NI mate Record File] followed by a single quint32 that gives the format version number. The [] brackets are part of the tag and not to be omitted

  2. After the version file format is the version of NI mate that was used for writing this file. Since other software can also write these files, the version should be a string with the program name in it.

   "NI mate version 2.00"
  1. Then are the [device_family] and [device_type] numbers that tell which sensor device uses this data. These also define how the skeletonjoints section appears as only the supported joints are written. These are quint32.
[device_family]3[device_type]2
  1. Then comes the number of frames and framerate, both given in the format:
[frames]60[framerate]30.0

where the numbers are quint64 and float

  1. Then follows the data header section as a [fields] tag followed by a single quint64 with bitflags that explain what kind of data will follow. The bitflags are:
typedef enum {
    Field_empty = 0,
    Field_userdata = 1,
    Field_skeleton = 2,
    Field_controller_osc = 4,
    Field_controller_midi = 8,
    Field_gestures = 16
} NIRecField;
  1. After this comes the [end_header] tag that tells the data section is next. A full header might look like this:
[NI mate Record File]3[program_version]NI mate version 2.02[frames]60[framerate]30.0[fields]25[endheader]

The [] tags might not be needed if the data is written and read using the QDataStream binary format, but if some other software writes in this format they might not use the Qt format. It’s easier to parse the format when at least the header section can be understood without the Qt binary serialization. If Qt is used for importing the data, then no parsing of the tags is needed - the data is serialized and can be simply filled in the fields using QDataStream >> operator

  1. The data section is binary data serialized using QDataStream. Frames that don’t have data for a specific field will fill this with 0. Every frame begins -without- a frame number, but does contain a quint64 timestamp, quint32 user id and number of active users for that frame. For example:
[endheader]736 2 1 305.1 867.1 759 0 13.5 9 ... ... 2 295.1 285.4 589.2

where the 786 is timestamp, 1 and 2 are user ids, and the first 2 is the number of users active this frame, followed by some data. I’m not sure how to parse this data without Qt’s serialization. I’m expecting it can’t be done unless you write your own importer using the same format.

  1. The frames are written in this sequence:
    1. Timestamp (quint64)
    2. Floor plane (4 floats)
    3. Active users this frame (quint32)
    4. For every user:
      1. userdata[i].id (quint32)
      2. If header specified userdata to be written
        1. User id (quint32)
        2. User center of mass (3 floats)
        3. User skeleton calibrating flag (bool)
        4. User face tracking 2D flag (bool)
        5. User face tracking flag (bool)
      3. If header specified skeleton data to be written
        1. For every joint supported by the sensor family specified in the header
          1. joint type (SkeletonJoint::Type casted into quint32)
          2. joint coordinates (3 floats)
          3. joint orientation (4 floats)
          4. joint position confidence (1 float)
          5. joint orientation confidence (1 float)
          6. joint validity (bool)
      4. If header specified other tags to be written [to be added later]

SkeletonJoints are defined like so:

enum Type {

              Pelvis, Body, Chest, Neck, Head,

              LeftCollar, LeftShoulder, LeftElbow, LeftWrist, LeftHand,
              LeftHandTip, LeftThumb, LeftHip, LeftKnee, LeftAnkle, LeftFoot,

              RightCollar, RightShoulder, RightElbow, RightWrist, RightHand,
              RightHandTip, RightThumb, RightHip, RightKnee, RightAnkle, RightFoot,

              LeftPinkyDistalPhalange, LeftPinkyIntPhalange, LeftPinkyProxiPhalange, LeftPinkyMetacarpal,
              LeftRingDistalPhalange, LeftRingIntPhalange, LeftRingProxiPhalange, LeftRingMetacarpal,
              LeftMiddleDistalPhalange, LeftMiddleIntPhalange, LeftMiddleProxiPhalange, LeftMiddleMetacarpal,
              LeftIndexDistalPhalange, LeftIndexIntPhalange, LeftIndexProxiPhalange, LeftIndexMetacarpal,
              LeftThumbDistalPhalange, LeftThumbIntPhalange, LeftThumbProxiPhalange, LeftThumbMetacarpal,

              RightPinkyDistalPhalange, RightPinkyIntPhalange, RightPinkyProxiPhalange, RightPinkyMetacarpal,
              RightRingDistalPhalange, RightRingIntPhalange, RightRingProxiPhalange, RightRingMetacarpal,
              RightMiddleDistalPhalange, RightMiddleIntPhalange, RightMiddleProxiPhalange, RightMiddleMetacarpal,
              RightIndexDistalPhalange, RightIndexIntPhalange, RightIndexProxiPhalange, RightIndexMetacarpal,
              RightThumbDistalPhalange, RightThumbIntPhalange, RightThumbProxiPhalange, RightThumbMetacarpal,

              LeftHandPalm, RightHandPalm, None,

              NumberOfJoints};

Documentation: User interface