Assignment #6: Make a processing app that you interact with via a depth-sensing camera (Kinect)

• This assignment may be done alone or in pairs.
• You are to take input from a text file that contains the output of a Kinect. The output was collected using kinectable_pipe
• You are to parse the file line by line to create an array of JSON objects.
• Each JSON Object is a represenation of one skeleton. Here is an example of one line:

  {"skeletons":[{"userid":0,"joints":[{"joint":"head","X":-702.131,"Y":486.367,"Z":1579.783},{"joint":"neck","X":-676.385,"Y":273.182,"Z":1570.143},{"joint":"neck","X":-676.385,"Y":273.182,"Z":1570.143},{"joint":"l_shoulder","X":-645.730,"Y":268.028,"Z":1768.476},{"joint":"l_elbow","X":-645.831,"Y":-27.049,"Z":1768.808},{"joint":"l_elbow","X":-645.831,"Y":-27.049,"Z":1768.808},{"joint":"l_hand","X":-647.107,"Y":-322.086,"Z":1764.067},{"joint":"l_hand","X":-647.107,"Y":-322.086,"Z":1764.067},{"joint":"l_hand","X":-647.107,"Y":-322.086,"Z":1764.067},{"joint":"r_shoulder","X":-707.041,"Y":278.335,"Z":1371.810},{"joint":"r_elbow","X":-714.375,"Y":-16.645,"Z":1369.905},{"joint":"r_elbow","X":-714.375,"Y":-16.645,"Z":1369.905},{"joint":"r_hand","X":-755.104,"Y":-308.767,"Z":1378.805},{"joint":"r_hand","X":-755.104,"Y":-308.767,"Z":1378.805},{"joint":"torso","X":-647.342,"Y":50.508,"Z":1559.868},{"joint":"torso","X":-647.342,"Y":50.508,"Z":1559.868},{"joint":"l_hip","X":-603.110,"Y":-174.720,"Z":1647.858},{"joint":"l_knee","X":-816.152,"Y":-553.847,"Z":1768.455},{"joint":"l_knee","X":-816.152,"Y":-553.847,"Z":1768.455},{"joint":"l_foot","X":-796.903,"Y":-987.331,"Z":1763.658},{"joint":"r_hip","X":-633.487,"Y":-169.613,"Z":1451.327},{"joint":"r_knee","X":-776.181,"Y":-590.984,"Z":1527.173},{"joint":"r_knee","X":-776.181,"Y":-590.984,"Z":1527.173},{"joint":"r_foot","X":-775.487,"Y":-1024.921,"Z":1527.667}]}],"elapsed":8.181}

• Using what you know about JSON and processing, create a character that interacts with you by mimicing the skeleton data that was collected from the Kinect
• For extra credit, add a physics system that is keyed to the skeleton. In the video below I have simulated a virtual bull-fighting cape that the user is manipulating by waving her arms.
• Here are some sample input files:
  • output1
  • output2
  • output3
  • output4
  • output5
  • outputErick
• To get this to work you will need to import the JSON for processing library manually and possibly a physics library
• Here is some code you can put in your setup function to import the files. You will need to make sure the files are in your sketch's "data" folder for this to work:

  List data = new ArrayList();
  					
  String[] lines = loadStrings("output.txt");
    for(int i = 0; i< lines.length; i++){
      if((lines[i].length() > 0) && (lines[i].charAt(0) == '{')){
        try{
          data.add(new JSONObject(lines[i]));
        }
        catch(Exception e){
          println("error"+i);
        }
        finally{}
      }
    }

• Finally, you will need to transform the coordinates from Kinect into something that processing can plot. The z values seem to vary from about 500 to 3000. The x and y positions seem to vary from -1000 to 1000, roughly. Transforming involves moving the 0 axis in the skeleton data to some other position on the Processing canvas. These functions will take skeleton points as input and output Processing points. You need to set the 0 axis using the "offset" variables. You need to scale the points using the "scale" variable so that the full range of Kinect positions will map to your window. Then you can see that the y position is flipped so that up in Kinect world is up in Processing world.:

  float myX(float x){
    return (_offsetx+x*_scalex);
  }
  
  float myY(float y){
    return (_offsety + (-1.0*y*_scaley));
  }
  

• Please turn in your Processing code as a text file. It should be a single file. Please put a comment at the top indicating what needs to be done to run it (if anything special). Tell us about libraries required and where to find them in particular and how to set the file name for the input. Here is a link to the dropbox to turn it in: https://eee.uci.edu/toolbox/dropbox/index.php?op=openfolder&folder=309443
• You will be graded on whether or not you were able to represent the skeleton accurately, your creativity in drawing your character, and your effective use of the processing animation loop. Additional credit will be given for doing something clever or unique. Finally, bonus points will be given for people who incorporate a virtual physics system into their sketch.