#kinect sdk

Chris Milk + Kinect !!!

##Continued… Right, so - in [the first bit of this post](http://uxgeek.tumblr.com/post/7932902913/kinect-for-windows-sdk-flailing-your-arms-more), we were able to use basic joint tracking with the Kinect SDK to control a UIElement onscreen. We'll spend some time here polishing this functionality, so that the object we're tracking onscreen behaves a bit more like those we're familiar with on the Xbox 360's Dashboard. ###Relatively Speaking The first thing you'll notice using example we put together earlier is that in order to move your object from one side of the screen to the other, you have to move your hand across the entire viewable area that the Kinect camera can sense. Clearly, that's not ideal. In scouring the web for things folks have already done with the Kinect SDK, I've come across a few examples of neat tools that make the mistake of leaving cursor tracking set up like this. While it absolutely does function, it's not convenient for users, and perhaps more importantly, it's far from what they expect. As we covered before, the Kinect sensor gives values from roughly -1 to 1 in both X and Y directions (from my own experimentation, I have seen that the device can output values that fall slightly outside these ranges. I believe this comes from estimated positions for joints after they leave the viewable area). WPF Elements which inherit from the System.Windows.Controls.Panel class use an X, Y coordinate system that varies from this slightly - (0,0) is the top left corner of the element, with X values increasing to the right, and Y values increasing downward. If we're going to mimic the hand tracking system used on the Xbox 360 Kinect Dashboard, we need to track the hand relative to the rest of the body. From a little bit of my own experimentation, it seems that if we're tracking the position of the right hand as our cursor, it is best to use the right shoulder as the relative center of the screen. Doing this is actually pretty simple - mathematically, it's a matter of subtraction to treat the shoulder as a centerpoint in the raw data coming out of the Kinect. This is data can then be sent to the ConvertScaledPosition function (from my previous post) to get a relative screen coordinate. public static Point GetPositionRelativeToSelf(Microsoft.Research.Kinect.Nui.Vector position, double armLength, Panel panel, SkeletonData skeleton) { Microsoft.Research.Kinect.Nui.Vector centeredPoint = new Microsoft.Research.Kinect.Nui.Vector() { X = position.X - skeleton.Joints[JointID.ShoulderRight].Position.X, Y = position.Y - skeleton.Joints[JointID.ShoulderRight].Position.Y }; double maxlength = armLength * 0.75; double xmax = skeleton.Joints[JointID.ShoulderRight].Position.X + maxlength; double ymax = skeleton.Joints[JointID.ShoulderRight].Position.Y + maxlength; centeredPoint.X = (float)((centeredPoint.X / xmax)); centeredPoint.Y = (float)((centeredPoint.Y / xmax)); return ConvertScaledPosition(centeredPoint, panel); } The only added requirement for this function is a measurement of the length of our skeleton's arm. In my own implementation, I track this by keeping tabs on the JointID.ShoulderRight and JointID.ElbowRight joints during each SkeletonFrameReady event - // set the arm length value if both the shoulder and elbow are tracked, // and the distance between them is longer than the previously stored value if (skeleton.Joints[JointID.ShoulderRight].TrackingState == JointTrackingState.Tracked && skeleton.Joints[JointID.ElbowRight].TrackingState == JointTrackingState.Tracked) { double curArmLength = Distance(skeleton.Joints[JointID.ShoulderRight], skeleton.Joints[JointID.ElbowRight]);// +Distance(skeleton.Joints[JointID.ElbowRight], skeleton.Joints[JointID.HandRight]); if (armLength < curArmLength) armLength = curArmLength; } The armLength value is used to determine the distance that our player's hand needs to sweep through in order to go from the centerpoint to any edge of the screen. I added a constant multiplier of 0.75 in this example to shorten the sweep slightly for a more natural feel. In your own applications, you may want to play around with that constant a bit to get things to your liking. In experimentation, I've found that with the "right" settings for the the nui.SkeletonEngine.SmoothParameters (again, these values were determined through experimentation), using my hand to point at elements onscreen feels nearly natural.