#leapjs

This week I added three additional primitive creation  modes (sphere, cube and plane). Switching to primitive creation mode currently isn't detected by the Leap (just using buttons for now). When I figure out a better way to handle switching modes, I'll pull it over to it.

For the time being, I'll be removing panning functionality from the camera controls while I work on creating a less limiting camera rotation function. For pitch and yaw, I'll be using the upward and downward hand movements to independently control each rotation. 

So this isn't really a progress update but I found an interesting article here about how gestural ui's really aren't that intuitive to users due to their inability to convey to the user visually the capabilities of an application. In other words, it's difficult to create a truly natural user interface using gestures simply because it's hard to figure out how to interact with a computer gesturally without the visual cues. 

Gestural interfaces have been historically more difficult to grasp and learn due to the fact that there really are no set discrete motions to accomplish tasks on the computer. The traditional keyboard and mouse method of interaction is relatively straightforward and provides a set of discrete actions a user can provide to the computer. With a more manageable toolset of input methods and good graphical feedback in the form of menus and dashboards, the user can more easily grasp how to use a new application using just the mouse and keyboard to interact.

This really lent some perspective about the direction I'm going with MotionSculpt. Although I have been focusing on gaining a good understanding of the leap javascript library and Three.js WebGL library, I also have spent a good amount of time this week trying to reevaluate the best way to create an intuitive and natural user interface. This article leads me to believe that I need to insert more visual cues for the user to easily pick up the gestures required to manipulate 3D objects in virtual space. I also need to narrow down my gestural library into a set of easy to learn, and relatively intuitive motions, both discrete and mixed.

So far, all my implementation has been focused on getting the two libraries to work together robustly (which needs more work with the camera... - although the camera controls work to a certain, they really aren't user friendly yet). I am going to change (yet again) how mode switching will happen with the camera. Instead of using finger counting - which is still a little bit shaky due to the tendency of the Leap controller to sense extra fingers - I want to create a simple menu that a user can simply point and tap to select a mode. The method I'm using right now is really too finicky for a user to interact with a scene effectively. I've tried different approaches and parameters to make a smoother transition but it still isn't the best due to the controller's limitations. Additionally, the purpose of the project isn't to create a gestural ui for camera, but also for mesh manipulation. So I don't want to focus too much time on that.

That's it for now. Here's a screenshot of how things look:

For this week, I finished getting the camera controls to work with the leap motion controller through a series of simple gestures. Originally I had planned for the workspace to have a series of modes that the user can cycle through using a circular gesture. With a clockwise/counterclockwise gesture, the user can go through rotation, translation, and zoom modes of the camera and then use their hand position to adjust the camera location.

This approach unfortunately was quickly scrapped due to multiple circular motion gestures being registered at once despite the hand gesture. This makes going from one mode to another highly difficult and frustrating. I needed to approach this with a gesture that is more binary in nature, with less noise in the input.

With this in mind, I decided to use a left hand signaling mode: to change between modes of the camera, the user would simply signal with their left - or right - hand to change the camera into either translation, rotation, or zoom mode. I had the choice of creating a very robust method for detecting left/right hand, but after a bit of testing and research I ultimately decided to go with a simple method: given two hands are detected by the leap motion controller, figure out which hand has the smaller/larger x-coordinate. That will dictate the left/right hand. This only works given that the left and right hands do not flip orientations and that the leap motion controller is in the correct orientation. I chose to use this way over wasting too much time being nit-picky over smaller implementation details - especially at this stage of the project.

Been having fun with the gestures library and implementing things like gesture created drawings, etc. I've managed to get my stuff integrated into the main webgl frame. So it draws gestures on the canvas with the cube. Like this:

This week has been mostly about trying to get the camera control working with the Threejs framework, as well as figuring out how to get the 3D locations for fingers and hands. I've also calibrated size of the 3D coordinate space of the leap motion device to match up with the size of the canvas on the window. This allows for more intuitive interaction with the screen.

For camera controls. I've gotten the leap motion to recognize a few gestures, which will allow it to change the mode of camera movement - translation, zoom and rotation. I've also refactored the camera mode to switch according to the gesture that the leap motion device has registered. Next, I will implement actual live finger tracking to move the camera instead of mouse click and drags.

In addition, I've tested and wrote up a small demo HTML page that parses and draws the positions of the fingers and hands as sensed by the leap motion device. This little demo file helped me understand how to visualize what the leap motion is capable of sensing (on a 2D scale - since the camera controls are largely still limited to 2D movement within a 3D interface).