Custom sensors protocol


#1

NI mate has a feature called “custom devices” which allows creating devices of your own liking in the user interface as well as outputting their data. This feature is still WIP and largely undocumented, but it can be used by writing your own websocket application registering your custom device with NI mate and then outputting data over the websocket.

We have a basic skeleton of a websocket device in this open source repository: https://github.com/Delicode/Websocket_Accelerometer

These JSON documents here specify the protocol used for creating and communicating with custom devices.

Basic json commands NI mate can send to a sensor:

{
    "type": "enumerate"
}

{
    "type": "start",
    "value": {
        "depth_width": 640,
        "depth_height": 480,
        "skeleton_tracking": true,
        ...
    }
}



{
    "type": "parameters",
    "value": {
        "depth_width": 640,
        "depth_height": 480,
        "skeleton_tracking": true,
        "led_color": [255, 172, 20],
        "doohickey_0": 5,
        "doohickey_1": 9,
        ...
    }
}


{
    "type": "stop"  // stops the sensor device, but stays running and connected to the websocket server
}

{
    "type": "quit"  // stops the actual sensor process, manual restart required after this!
}

Basic messages sensors can send to NI mate:

Status change: running, stopped, error

    "type": "status",
    "value": {
        "status": "running",
        "device_id": "k4w_5871917",
        "error_msg": "boop"
    }

Generic messages that can be used for debugging

    "type": "message",
    "value": {
        "message": "heloo",
        "device_id": "k4w_5871917"
    }

Response to enumeration, a dynamic UI page is created based on this

{
    "type": "device",
    "value": {
        "device_type": "PS Move",
        "device_name": "PS Move",
        "device_id": "q23phr081gt",
        "values": [
            {
                "name": "Accelerometer",
                "type": "vec",
                "vec_length": 3,
                "datatype": "float",
                "count": 1,
                "min": -1,
                "max": 1,
                "flags": per_user
            },
            {
                "name": "Buttons",
                "datatype": "bool",
                "count": 3,
                "flags": per_user,
                "names": ["Button1", "Button2", "Button3"]
            },
            {
                "name": "Doohickey",
                "type": "vec",
                "vec_length": 1,
                "datatype": "int",
                "count": 3,
                "min": -1,
                "max": 100,
                "flags": per_user
            },
            {
                "name": "Depth",
                "type": "frame",
                "datatype": "ushort",
                "count": 1,
                "modes": [{"w": 640, "h": 480, "fps": 30}, {"w": 320, "h": 240, "fps": 60}]
            },
            {
                "name": "Color",
                "type": "frame",
                "datatype": "uchar",
                "count": 3,
                "modes": [{"w": 640, "h": 480, "fps": 30}, {"w": 320, "h": 240, "fps": 60}]
            },
            {
                "name": "Skeleton",
                "type": "vec",
                "vec_length" : 3,
                "datatype": "float",
                "count": 15,
                "flags": per_user,
                "names": ["Head", "Neck", "Left Shoulder", ...]
            }
        ],
        "parameters": [
            {
                "name": "Led color",
                "type": "vec",
                "datatype": "int",
                "vec_length:" "3",
                "default:" [ 255, 0, 0],
                "min": 0,
                "max": 255,
                "flags": 0
            },
            {
                "name": "beeping",
                "type": "vec",
                "datatype": "bool",
                "vec_length:" "1",
                "default:" false,
                "min": 0,
                "max": 1,
                "flags": 0
            }
        ],
        "commands": [
            {
                "name": "Motor",
                "datatype": "bool"
            }
        ],
        "recording" : [
            {
                "name" : "RecordNirec",
                "format" : "nirec",
                "compression:" : "basic"
            },
            {
                "name" : "RecordRGBD",
                "format" : "rgbd",
                "compression:" : "bgdiff"
            },
            {
                "name" : "RecordOther",
                "format" : "rgbd",
                "compression:" : "none"
            }
        ]
    }
}

// value types: single (default), vec3, rgb, frame
// data types: float (default), bool, char, uchar, short, ushort, int, uint

Data devices can send to NI mate might look like this:

// Data stream
{
    "type": "data",
    "value": {
        "device_id": sensor_xt175175,
        "timestamp_ms": 1068205729,
        "Accelerometer": [ [0.124, 1.125, 0.2345], [0.14, 1.58, 185], [0.15, 0.14, 0.214] ],
        "Buttons": [true, false, false]
    }
}


// Data stream for 2 users
{
    "type": "data",
    "value": {
        "device_id": sensor_xt175175,
        "timestamp_ms": 1068205729,
        "user_1": {
            "Accelerometer": [ [0.124, 1.125, 0.2345], [0.14, 1.58, 185], [0.15, 0.14, 0.214] ],
            "Buttons": [true, false, false]
        },
        "user_2": {
            "Accelerometer": [ [0.124, 1.125, 0.2345], [0.14, 1.58, 185], [0.15, 0.14, 0.214] ],
            "Buttons": [true, false, false]
        }
    }
}

Websocket accelerometer
#2

Thank you for valuable info