Simple Gestures

Back to Tutorials

8. Cropping 10. Composed Gestures 1 2  3  4  5  6  7  8  9  10  11  12  13  14  15  16



We provide 3 different samples and respective tutorials for the gestures unit: this one explains the basic operations to work with gesture recognition and allows the recognition of what we consider simple gestures, the second one shows the composed gestures (sequence of simple gestures) and the last one shows the full capabilities of this unit adding progress events and graphical assets to improve the experience. To follow this tutorial you should refer to the simple gestures sample files: SimpleGestures.h and SimpleGestures.cpp. Those files are inside the ViiM SDK, in the open frameworks applications folder. The explanation is focused on class implementation, the “SimpleGestures.cpp” file, thus if nothing is said, the code presented here refers to “SimpleGestures.cpp”, not to “SimpleGestures.h”, the header file. Also, the code is comprehensively commented so you can easily understand what’s going on just by looking at it, but first if you haven't read the other tutorials we strongly advise you to read at least the tutorial about the ViiM engine, the tutorial about registering a unit and the tutorial about ViiM's events.

Let us start with the open frameworks setup phase.

THE SETUP PHASE As explained in previous tutorials, during the setup phase and after starting your ViiM engine you should register the desired unit, in this case a ViiMUnitGesture one. This is performed using:

After this you gain access to the unit methods using the _gesture variable. For example the line

addEventListener is the method that adds the listener, in this case a listener to the hand event, CREATE. Internally, what this methods is doing is telling ViiM to be aware of this type of events, and to inform you whenever they occur. Thus the first parameter of this method is just that: the specific type of event you want to be informed of. The second parameter specifies what ViiM should do when the event occurs, that is, what specific block of code should ViiM run at the hand creation (the beginning of hand tracking). This is accomplished by passing the name of the function you want ViiM to run, in this case “handCreateHandler”. Finally, the third parameter informs ViiM of the object that it should use when running the function. The keyword “this” just refers to the class we are currently in, that is “Gestures”. If the reason for this is not completely clear for you right now, don’t bother with it, you just need to remember to pass this parameter as the third parameter, exactly as you can see in all the 11 calls to addEventListener. You may have noticed that before the CREATE word we write ViiMHandEvent. This way you can use the same method (addEventListener) for different types of events, without confusion. The ViiMHandEvent types are: CREATE, UPDATE and LOST (see lines 29-31). As the name indicates they are used to deal with the hand creation (when tracking starts), the update (at every frame) and the hand disappearance (when tracking is lost). In this example we just flag the hand as visible when created, not visible when lost and update the position at every frame (please see THE EVENTS’ HANDLER FUNCTIONS paragraph).The other two types of the gesture unit's events are “ViiMProgressEvent” and “ViiMGestureEvent”. Progress events are used to signalize evolution, such as: the distance the hand should move so that the Enter gesture can be completed (detected) or the steady frames to detect the steady gesture. Gesture events are events that signalize actual gestures, such as: enter, waving, swipe, etc (ViiMProgressEvent will be explained in detail in the hand interaction tutorial). The following line of code exemplifies adding a listener to the swipe right event.

In this case we want a ViiMGestureEvent, specifically swipe right, and we want to run the swipeRightHandler method when this event occurs. Using ViiM to work with events and using specific gestures to trigger methods is as simple as that. At least during the setup phase, that’s what you have to do. The rest of the setup code is there to load fonts and allocate images (lines 45-53), for the openFrameworks graphics, and control variables (lines 69-75) to trigger them. Before describing the update and draw phase, lets understand how to implement handler functions for ViiM events.

THE EVENTS’ HANDLER FUNCTIONS  The events’ handler functions are the blocks of code that are called when an event occurs. If you remember, we have specified that ViiM should call the “handCreateHandler” function whenever ViiMHandEvent::CREATE happens (i.e. when the hand starts to be tracked), so let’s start with this function.

First take a good look at the header (line 79) because that’s going to serve as prototype for all your ViiM handler functions. You should always have the first parameter declared as a reference to one of the possible event types, in this case a reference to ViiMHandEvent. Please notice the “&” symbol, don’t forget it even if you don’t fully understand the reason. The other event types we use in the gestures unit are “ViiMGestureEvent” (see for example line 117) and “ViiMProgressEvent” (only in the hand interaction tutorial ). The second parameter is always the same, a pointer to void that ViiM declares as VVoidP. Inside the function, if one wants to refer to our class variables, as we do in this case, one should always have a line like this one:

“SimpleGestures” is the name of the class we are in, thus by using this:

pThis will be a pointer to our class. Now one can use “pThis” to refer to the class variables and methods, as we do when setting the hand as visible. Notice that the event also has properties, in this case the hand position (x,y,z) at the time of the event. Now, as an exemple of gesture events we exemplify the code for the “swipeRightHandler” function:

This code just saves the "swipe right" message to draw on the screen in the draw method and sets the flag for existence of a gesture. All other ViiM event handler methods follows the same prototype described above. Easy right? THE UPDATE PHASE There really isn't much to say about this phase, unless that it follows the same logic as described in the other tutorials. There are just two lines of code:

The first uses the _engine to get the depth image, i.e. the one that comes from the sensor, and the other uses _gesture to get the depth image, i.e. the gesture unit's copy of the raw image. In this case, since we didn't apply any settings to this unit's image (like threshold) it's exactly the same as the one from the _engine.

THE DRAW PHASE In the draw phase we display the detected gesture with the string written in the eventHandlers.

Notice that you should have your hand facing the sensor and perform the gestures as clearly as possible. If you move your hand opened you activate the move event. To perform a waving event you should calmly wave your arm from one side to the other. The steady event is, as the work indicates, detected if you don’t move your hand for 2 seconds. By swiping the hand to the right, left, up or down the matching event will be fired. For a complete description of the gestures unit pleaser read the gestures technical documentation.

8. Cropping 10. Composed Gestures 1 2  3  4  5  6  7  8  9  10  11  12  13  14  15  16

Back to Tutorials