Stickman

Back to Tutorials

13. Skeleton Tracking 15. Video Exporter 1 2  3  4  5  6  7  8  9  10  11  12  13  14  15  16



Different from the Skeleton tracking sample, which uses the joints' positions, the stickman example uses the joints' rotation matrixes. The application goal is to draw a stickman that follows the user movement in front of the camera. You may think that we could do it with the skeleton tracking sample, and your right, we could. But the applications one can create with these two methods are completely different. Plus the joints' rotation matrixes allow direct avatar control, although that complexity is reflected in the theory and computation behind this technique.

The stickman is a basic representation of the human body; a collection of lines that connect the joints. Initially the stickman is in a T-Pose (please refer to the Skeleton technical document for complete information about the skeleton unit), but when the user gets detected by the sensor, its joint's rotation matrix are passed down to the stickman joints position. Multiplying the joints' rest position with those rotation matrixes will transform the points on both extremes of the lines that form the stickman and thus it will follow the user's movement.

Before rushing to the code, if you haven't read the other tutorials, we strongly advise you to read at least the tutorial about the ViiM engine, the tutorial about registering a unit and the skeleton example.

Before going for the .cpp file, let's just inform you about the header file where, among the normal initializations, we define the stickman joints' size and create variables for the joints' rotation matrixes, resting position and current position. We have also this method:

where we multiply the user's rotation matrix with the points of the stickman. This will be more detailed in the next part of the tutorial. In the .cpp file we have our regular openframeworks structure plus the aforementioned applyRotMatrix function. Let's leave the latter for when it's used. Firstly in the setup function we prepare the stickman by setting it to the rest position, create the ViiM engin, register the skeleton unit and limit the application to one stickman. In the update function, just like in the skeleton example, we get the pixels of the depth image from this unit. In the draw function is where the magic happens.

In line 108 we get the data from the skeleton joints (rotation matrix, position, quaternions, angles of rotation and confidence) and from line 111 to 120 we copy the rotation matrixes for all the joints to ViiMMatrix3x3 variables, ViiM's type for 3x3 matrixes. These matrixes will be used for the calculus of the new position of each joint in the applyRotMatrix function. This method is called forward kinematics where we apply the rotation matrix of the user joints to the resting point of the stickman, making the joints position change to the desired position. For instance, if the user moves the right elbow, the new position of the stickman's right elbow (lines 132 - 133) will be the right shoulder rest position affected, in order, by the right shoulder's rotation matrix and after the torso's rotation matrix. Don't forget that we have to put the influencing joint in a zero position (remove the offset) before applying its rotation matrix to the influenced joint. We then add that same offset before applying the next rotation matrix.

This will happen to all joints, always depending on the parent's joints. Finally we draw on the screen the lines already affected by the transformations above described in all joints.

Lines 150 and 151 serve the purpose of inverting the Y coordinate and centering the stickman in the scene. From line 155 to 168 we draw the stickman's lines connecting the new positions of the joints.

Check this example on ViiM_OF_Samples, stickman. Hope you enjoy breakdancing or mimic super man in front of the pc :)

13. Skeleton Tracking 15. Video Exporter 1 2  3  4  5  6  7  8  9  10  11  12  13  14  15  16

Back to Tutorials