Introduction To Computational Media (ICM) : Week 9

Loose Ends

Video Tracking

One interesting thing that can be done with video capture is to use it as a means to track obects through space.

Here is an example (courtesy of DanO):
            import processing.video.*; 
            
            Capture video; 
            
            float targetRed = 255.0; //set some numbers for the target you are chasing
            float targetGreen = 0.0;
            float targetBlue = 0.0;
            
            void setup()
            {
              video = new Capture(this, 320, 240, 15); //initiate the video, resolution and frame rate
              size(320, 240); //give you Processing window a size
            }
            
            void captureEvent(Capture camera) 
            { 
              camera.read(); 
            } 
            
            void draw()
            {
              float worldRecord = 1000.0; //intialize the worldrecord
              int xFound = 0; // initialize the location of the red tracking ball
              int yFound = 0;
            
              for(int j=0; j < video.height; j=j+1) { //for each row
                for(int i=0; i < video.width; i=i+1) { //for each column
                  //get the color of this pixels
                  //find pixel in linear array using formula: pos = row*rowWidth+column
                  color pix = video.pixels[j*video.width+i]; 
                  //find the difference
                  float diff = abs(targetRed - red(pix)) + abs(targetGreen - green(pix)) + abs(targetBlue - blue(pix));
                  if (diff < worldRecord){ // if this is closest to our target color
                    worldRecord = diff;
                    yFound = j; //mark the spot for drawing it later
                    xFound = i;
                  }
                }
              }
              image(video,0,0); //draw the video, this might be optional
              //after all the pixels have been tested, draw the winner
              fill(255,0,0);
              ellipse(xFound, yFound, 10, 10);            
            }
            
            void mousePressed(){
              //allow the target color to be changed 
              color pix = video.pixels[mouseY*video.width+mouseX];
              targetRed = red(pix); //get the color of the pixel they clicked on
              targetGreen = green(pix);
              targetBlue = blue(pix);
            }
        
More examples

Pulling back the curtain

Processing is really Java

In fact the code you write in Processing, goes through a minor conversion to straight Java before it is compiled.

Here is a very simple Processing example:
            int x = 0;
            int y = 0;
            int xDirection = 1;
            int yDirection = 1;
            
            void setup()
            {
              size(500,500);
            }
            
            void draw()
            {
              background(255);
              
              if (x > width || x < 0)
              {
                xDirection = xDirection * -1;
              }
              x += xDirection;
              
              if (y > height || y < 0)
              {
                yDirection = yDirection * -1;
              }
              y += yDirection;
              
              ellipseMode(CENTER);
              ellipse(x,y,50,50);
            }
        
To illustrate what this looks like after Processing converts it to straight Java, we can Export and take a look at the .java file that is created.

Here is the .java file for the above code.

Not much is different!

Processing added a bunch of import statements automatically which tells the Java Compiler and the Java Virtual Machine what is needed in the way of external libraries and from the Java API.

More interesting is that it wrapped all of our code inside a class:

        public class SimpleEllipse extends PApplet 
        {
            // Our code...
        }
        
Low and behold, everything is in a class. This is because, Java is an Object Orientated language. Almost everything in Java is a class and correspondingly your Processing code gets instantiated as an Object.

Let's deconstruct the line:
public class SimpleEllipse extends PApplet {
The name of the class, SimpleEllipse is what I saved the example as. The rest of the items on that line are as follows:

public: This tells Java who has access to this class in code. public means that other Java programs can encorporate and use this class and make objects of it.

extends PApplet: This tells Java that our class, SimpleEllipse is a child of PApplet. PApplet is the class that allows us to use all of the Processing methods, variables and constructs that we are used to, such as size(), mousePressed(), mouseX and so on. More about this later...

Java Mode

We can actually write our code in Processing in straight Java by simply including the same type of class definition that Processing does for us. This allows us to use the full Java API by simply using the imports that are required for each class that we want to use.

We don't need to include the ones that Processing does for us:

            import processing.core.*; 
            import java.applet.*; 
            import java.awt.*; 
            import java.awt.image.*; 
            import java.awt.event.*; 
            import java.io.*; 
            import java.net.*; 
            import java.text.*; 
            import java.util.*; 
            import java.util.zip.*; 
        
Of course, the J2SE 1.4 Javadocs are our best friend when working like this.

All of the possible classes we can use (without resorting to external libraries) are listed in the frame to the left.

Clicking on the name of a class, such as ArrayList will bring up the documentation for that particular class. It will show you on the very top what the import statement will need to include. (For ArrayList we don't need to import anything as Processing already does it for us.) The other items that it shows us are what it is a child of (extends), the constructors and the methods that are available with it.

Using Java Classes

ArrayList happens to be a nice one for us to learn. It basically allows us to create Arrays with the ability to grow and shrink.

Here is an example (using our trusty Ball class from earlier in the year):
import java.util.*;  // We don't actually need to do this..

public class ArrayListExample extends PApplet 
{
  //declaring a global variable of type ArrayList
  ArrayList a;

  void setup() 
  {
    size(400,300);
    colorMode(RGB,255,255,255,100);
  
    // Creating an instance of ArrayList and storing it in our global variable
    a = new ArrayList();
  }

  void draw() 
  {
    background(0);

    if (mousePressed) 
    {
      // Create a new ball
      Ball newBall = new Ball(width,height,10,mouseX,mouseY);
      
      // Add the ball to the ArrayList
      a.add(newBall);
    } 

    // Iterate through the ArraList and get each Ball
    for (int i = 0; i < a.size(); i++) 
    {
      // when something comes out of the arraylist via "get" we have to
      // remind ourselves what type it is.  In this case, it's an instance of "Ball"
      Ball b = (Ball) a.get(i);
      b.compute();
      b.display();
    }
  }

    /* 
    We are including our Ball class here instead of in a different tab as when we enter "Java mode" we loose the default imports and can't use normal methods like ellipse() and ellipseMode().  By including it inside the main class we get those back.
    */
    class Ball
    {
      // our Class variables.  
      int wWidth;
      int wHeight;
      int bSize;
      
      int x, y;
      int xDirection, yDirection;
      
      // Constructor.  This gets called when we create a "new Ball"
      Ball(int windowWidth, int windowHeight, int ballSize)
      {
        wWidth = windowWidth;
        wHeight = windowHeight;
        bSize = ballSize;
        x = 1;
        y = 1;
        xDirection = 1;
        yDirection = 1;
      }
      
      // This is an Overloaded constructor
      Ball(int windowWidth, int windowHeight, int ballSize, int xPosition, int yPosition)
      {
        this(windowWidth, windowHeight, ballSize);
        x = xPosition;
        y = yPosition;
      }
      
      // Our method to determine current position of the ball.
      void compute()
      {
        if (x < wWidth && x > 0)
        {
          // Move along x axis
          x += xDirection;
        }
        else
        {
          // Change direction, from positive to negative and vice versa
          xDirection = xDirection * -1;
          x += xDirection;
        }
        
        if (y < wHeight && y > 0)
        {
          y += yDirection;    
        }
        else
        {
          yDirection = yDirection * -1;
          y += yDirection;
        }
      }
      
      // This actually displays our ball, it gets called in the main draw function
      void display()
      {
        compute();  // first we run the computation
        ellipseMode(CENTER);
        ellipse(x,y,bSize,bSize);   
      }
    }  // Close our Ball class
}  // Close our PApplet derived class
        
You will notice that this program starts to slow down as we keep adding Ball's to our ArrayList. We can rectify this by removing Balls from the ArrayList as well by using the ArrayList remove() method.
            if (mousePressed) 
            {
              // Create a new ball
              Ball newBall = new Ball(width,height,10,mouseX,mouseY);
              
              // Add the ball to the ArrayList
              a.add(newBall);
              
              // If we have 100 balls or more, remove the first Ball
              if (a.size() > 100)
              {
                  a.remove(0);
              }
            }         
        
Here it is running

Another useful Java class is Rectangle. It provides us with some more functionality than the rect() method that is built into Processing because it is an class/object.

Here are is the Javadoc: http://java.sun.com/j2se/1.4.2/docs/api/java/awt/Rectangle.html. You should notice, that it has a nice contains method built in.

Here is an example

Other examples that might interest you:
  • DanO's Google Parsing with ArrayList example
  • DanO's TextField GUI example
  • DanO's Rectangle Shoot'em example
  • Handling Errors

    By now you have all probably run into code that might give an error when running it but might not at other times. The loadStrings method for instance will throw an error if a network connection is unavailable or if the server being called is down.

    Fortunately, Java has a means to deal with this and allows you to make your program still function even if there is an error.

    It is called, try, catch. You simply enclose the error prone code in a try block and if an error is thrown your catch block will be called. In the catch block you need to include the type of Exception that is thrown. The easiest thing to do is to catch Exception itself.
                void setup()
                {
                  String[] htmlSource;
                  
                  try
                  {
                    htmlSource = loadStrings("http://blablah blah.com");
                  }
                  catch (Exception e)
                  {
                    htmlSource = new String[1];
                    htmlSource[0] = "Couldn't Load Page";
                  }
                  
                  println(htmlSource);
                }        
            

    Not quite there

    Even though we are writing straight Java in Processing, we are still using the Processing libraries which are nice. If we had to do all of this from scratch it would look like the Java code on this page: https://itp.nyu.edu/~sve204/cgi-bin/wiki/wiki.pl?EmulatingProcessing

    Compiling Java by hand isn't terribly difficult but you do need to have a JDK installed and your environment setup correctly.

    Here is how to compile the Java code in the example above:

  • Create a text file and copy in the code.
  • Save it with the name of the class followed by ".java"
  • Open up a terminal or command line program, change directories to the directory containing the .java file and run the following commands (Mac or Windows):
  •         cd /to/directory/containing/file/ 
            javac EmulateProcessing.java 
            java EmulateProcessing 
            
    "javac" compiles the .java file, "java" runs it.

    Books and Tutorials

    Java is a big (and flexible) language and there are some very good tutorials and books available.

    Here is a list of some that I like http://stage.itp.nyu.edu/~sve204/cgi-bin/wiki/wiki.pl?JavaTutorials

    DanO recommends, Head First Java.