#cfcc.

There are several things to consider when thinking about how this experience will exist in a physical space and interact with a user.

window size  —  currently, the window size for the starfield is 600px by 600px. This has worked fine for testing on-screen, but I need to find the optimal dimensions the starfield window when shown by the projector (I might have to make the window bigger). If I have to change the window size, I may also need to change the number of total stars in the array.

projector  —  I have to make sure the placement of the projector won’t be behind the person interacting with the exhibit, or the projection image will be interfered with.

maximum / minimum distance  —  stepping too close to the sensor makes the pplPosition variable go to 0, and when that happens, the program needs to be restarted. I also need to calculate a maximum distance so that the pplPosition variable won’t go over 1000 (the value I’m subtracting the pplPosition variable from to invert the z axis).I could mark the minimum and he maximum distance so people know the boundaries for the exhibit for a good experience. 

My original intention was for the stars to idle when a user was far from Kinect device, and when they stepped closer, the stars would move faster. Currently, because of how the z axis is set up in reference to the Kinect device + how I am using the z axis in Processing, the stars are moving faster the further you are away from the Kinect device, and idling when you are close to it.

I don’t mind this too much, as my main intention was to introduce an interactive aspect to the starfield code, but I believe that the experience could be improved if I could invert the z axis (this would give me an output that followed my original intention). 

On the web, there doesn’t seem to be any other people who are using Processing and are looking to invert the z axis. I looked briefly into the reverse() function for Processing, but that seemed to only work for arrays with a series of information i.e. ‘String name[] = { "OH", "NY", "MA", "CA"};’.

I might settle for the output that I have now, considering that it is already at a satisfactory level, unless I am able to find a way to invert the z axis in time for the hand-in.

Now that I have the code that executes the starfield (starfield base) and the code that retrieves the depth data (interactivity base), I can combine the two. My intention is to give the starfield base an interactive quality, so users are able to alter the starfield by their distance placement in regards to the Kinect device.

The first problem I ran into was trying to use ‘_s.position.z’ in substitute for mouseX in the line of code that controlled the ‘speed’ of the stars. When simply substituting mouseX with  _s.position.z, Processing told me that _s variable did not exist. After some trial and error, I figured out that anything with an ‘_s variable’ had to be placed in a function that belonged to the Skeleton Data i.e. ‘void drawPosition(SkeletonData _s) {’.

However, it wasn’t as simple as assigning the void draw() function with ‘SkeletonData _s’. That rendered the rest of the draw function null. Instead, I decided to create a global variable that would hold the _s.position.z value. This was a guess, but it worked! 

I assigned the _s.position.z value to a global variable called ‘pplPosition’, and replaced mouseX with pplPosition. I then divided the pplPosition integer value by 45 so that the value was in the 0-600 range. 

In preparation for combining the two codes together, I went through trial-and-error to see which parts of the code I needed to sense + calculate the distance of a person with the Kinect device, and which parts of the code I could delete.

Most of the code that I deleted had to do with displaying the RGB/depth/etcetera images in the Processing window, or drawing the lines used for displaying the skeleton data. 

I am using a 64-bit operating system, Windows 10 home OS, connected to the Kinect for Windows sensor device. The Processing Library I am using is Bryan Chung’s Kinect for Processing Library.

01.   Install Kinect for Windows SDK v1.8, Kinect for Windows Runtime v1.8, and the Kinect for Windows Developer Toolkit v1.8; you can download the SDK here, the Runtime here, and the toolkit here. Windows installation instructions warn against having your Kinect device connected while downloading these packages. Make sure your Kinect device is disconnected from your computer before installing. Complete the installation process and close all the windows.

02.   Connect your Kinect for Windows device by the USB port and the power socket. Check that the green light between the cameras is blinking. That will tell you the device is connected to the computer and to power.

03.   Open Device Manager. ‘Kinect for Windows’ should expand to a camera element (and possibly an audio element). If there is a yellow warning sign next to the Kinect for Windows icon, you may have the wrong SDK installed (see previous post for details).

04.   Open Processing and import Bryan Chung’s Kinect for Processing Library (Sketch > Import Library > Add Library > Filter by ‘Kinect’ > Install ‘Kinect4WinSDK’). If the installation seems to be stuck on loading while the library is available, close and open Processing before checking if the library is installed fully in the ‘Import Library’ tab.

Note: this post is friendlier on dash view because of the bullet-point format. Click here to view on the dash.

As per the instructions on the Windows site, I downloaded and ran the v2 SDK and Kinect Studio. However, my Kinect for Windows device failed to connect with my computer, as the Device Manager showed a yellow warning sign alongside the Kinect for Windows icon.

This was solved by uninstalling the v2 SDK + accompany v2 applications, and installing the v1.8 SDK + accompanying v1.8 toolkit. The Device Manager then registered the Kinect for Windows device as working.

Initially, I was also following the tutorial Daniel Shiffman provided, as well as the open source Kinect for Windows library that he recommended in said tutorial. However, I got the error message ‘initDevice() does not exist’, among other things. 

This was solved by instead using Bryan Chung’s Kinect4Win library. By running the provided example program alongside the v.18 SDK, the Kinect sensors were able to provide me with all four views when executed (skeleton + depth + RGB, mask).

Oddly, after a successful connection and execution of the four-view feedback provided by the Kinect for Windows device, it ceased to work again when I got back home and tried running it. I had not altered anything since the successful attempt. When I ran the Kinect4Win example program, it gave me the error message: “(Init) Initialize sensor failed 0” — the 4-view window was also blank (no RGB / depth / infrared / skeleton view).

I uninstalled all the Kinect for Windows programs (v1.8 + v2, SDK, Runtime, Kinect Studio, Toolkit, etc) and removed the Kinect libraries from Processing (Bryan Chung’s Kinect4Win, Daniel Shiffman + Thomas Sanchez Lengeling’s Open Kinect for Processing). Then I reinstalled the v1.8 SDK and Runtime, as well as Bryan Chung’s Kinect4Win library in Processing. 

In Device Manager, I also enabled the hidden devices (View > ‘Show hidden devices’) to double-check that the other elements (Audio Array, Security Control, etc) was available - just not connected.

I then rebooted my computer and ran the Kinect4Win example code. This sometimes gave me the RGB and depth, and sometimes it did not. Neither the skeleton + depth view showed. I also noticed the infrared camera on the Kinect device itself would flare with infrared light for a few seconds at the start before turning off. Note: for this attempt, the Kinect device is plugged directly into the wall socket.

After disconnecting the device and reconnecting again (I had to move to another workspace), it reverted back to only being connected to the Kinect for Windows Camera. Note: for this attempt, I am using an extension cord rather than plugging Kinect device directly into the wall socket - unsure if that is a variable?

Rebooted computer, disconnected + reconnected device, ran the example code despite only the Kinect camera showing. Started working anyways???? Don’t know how Kinect works but I’m too afraid to ask.

Skeleton and mask views weren’t showing up for some reason

Moved at a distance in an attempted to activate the sensor. It worked! Skeleton and mask views became live.

Depth data would revert back to 0 when I got to close to the Kinect device, and would not register my depth even if I moved back to a distance that it detected before.

ENTOURAGE   —  click for better view

STARFIELD 02 — VERSION 4 (pastebin code)