Why do we need messaging?

This page will cover the basics of the current messaging system in the MPE library. Because MPE assumes that every frame is computed and rendered at precisely the same time, if any external information is introduced into the system, that information MUST be retrieved by each client during the exact same frame. The MPE messaging system ensures that this is the case.

For example, if you have a microphone hooked up to one client computer, it should not use the sound level from the microphone directly in the code. Rather, it should read broadcast the sound level to all clients and use the value when retrieved via the messaging system. Following is a brief outline of all the available messaging functions in the client class, as well as an outline of three scenarios (sound input, video input, data input).

All of the code is on the CVS under:

/home/dts204/bigscreens/examples/ PROJECT: Week4Examples

Broadcasting

(If you need more, let me know and we can integrate them into the library!)

The method we care about are:

broadcast()

The simple broadcast() method takes a String and sends it out to all clients.

String s = "A message I want to send.";
client.broadcast(s);

It can also be used to send a single number, by converting it to a String first.

int val = 100;
client.broadcast("" + val);

You can also design your own protocol, for example sending a list of numbers separated by commas.

int val = 100;
int val2 = 20;
client.broadcast(val + "," + val2);

Never broadcast a message with a ":" in it. The MPE server/client framework uses colon as a delimiter in its internal code so extraneous colons can mess stuff up. If you think you might have a color by accident, you can always replace it with a space.

String msg = . ..  some code that gets the message . . .
msg = msg.replaceAll(":", " "); // replacing all colons with a space

Also, in many cases you only want one client to be the "messenger", in which case you'll want to enclose the broadcast statement in a conditional.

if (client.getClientID() == 0) {
  client.broadcast("message here");  
}

Receiving

The client that broadcasts the message should never use the data directly. All external data must be used during the identical frame and this can only be achieved via the frameEvent() method. In that method, you can check and see if a message is available with three methods:

messageAvailable()

The above method will return true or false, based on whether or not the type of requested data is available. Once you know a message is available, it can be retrieved with the three following method:

getDataMessage()

String messages come in as an array. In most cases, the array is of length one, however, if multiple clients broadcast a message, the array will have an element for each message. Here is some code that assumes a single message (an integer encoded as a String).

	// This is triggered by the client whenever a new frame should be rendered
	public void frameEvent(Client c){
		if (c.messageAvailable()) {
			String[] msg = c.getDataMessage();
			int val = Integer.parseInt(msg[0]);
		}
		redraw();
	}

Let's say you are broadcasting coordinates, separated by a comma. Here, we loop through all messages in the array, split based on the comma, and create an object with the resulting values. It assumes the existence of a Ball class, as well as an ArrayList named balls.

    // This is triggered by the client whenever a new frame should be rendered
    public void frameEvent(Client c){
        if (c.messageAvailable()) {
            String[] msg = c.getDataMessage();
            for (int i = 0; i < msg.length; i++) {
               String[] xy = msg[i].split(",");
               float x = Integer.parseInt(xy[0]);
               float y = Integer.parseInt(xy[1]);
               balls.add(new Ball(this,x,y));
            }
        }
        redraw();
    }

Scenario: Sound Input

Let's assume that you are using the volume from a microphone (obtained via http://sonia.pitaru.com/?) to drive the behavior of a Processing sketch. You'll have the following code in setup():

   Sonia.start(this);
   LiveInput.start();

And later, you'll ask for the level from the microphone, and use that level:

  float level = LiveInput.getLevel();
  rect(x,y,level*100,level*100);

The conceptual leap we need to make is that the level variable should no longer be used directly. Instead, the value will be broadcast (only if you are a particular client):

    if (ID == 0) {
      float level = LiveInput.getLevel();
      client.broadcast("" + level);  // quick way to convert to String
    }

Then, the value can be retrieved and stored in a global variable:

  // This is triggered by the client whenever a new frame should be rendered
  public void frameEvent(Client c){
    if (c.messageAvailable()) {
      String[] msg = c.getDataMessage();
      volume = Float.parseFloat(msg[0]);
    }
    redraw();
  }

and used later on, perhaps in draw():

  rect(x,y,volume*100,volume*100);

Scenario: Video / Motion Detection

The simplest motion detection "hello world" example is to simply analyze a video frame for overall motion, accomplished by taking a copy of the previous frame, and looking at the average pixel difference across the entire image:

  if (video.available()) { 
    // Save previous frame for motion detection!! 
    prevFrame.copy(video,0,0,video.width,video.height,0,0,video.width,video.height); 
    prevFrame.updatePixels(); 
    video.read(); 
  } 

  video.loadPixels(); 
  prevFrame.loadPixels(); 

  // Begin loop to walk through every pixel 
  // Start with a total of 0 
  float totalMotion = 0; 

  // Sum the brightness of each pixel 
  for (int i = 0; i < video.pixels.length; i++) { 
    int current = video.pixels[i]; // Step 2, what is the current color 
    int previous = prevFrame.pixels[i]; // Step 3, what is the previous color 
    // Step 4, compare colors (previous vs. current) 
    float r1 = red(current); float g1 = green(current); float b1 = blue(current); 
    float r2 = red(previous); float g2 = green(previous); float b2 = blue(previous); 
    float diff = dist(r1,g1,b1,r2,g2,b2); 
    totalMotion += diff; 
  } 

  float avgMotion = totalMotion / video.pixels.length;

Once you've got that value, you can broadcast it and receive it, just as we did with sound volume:

Broadcasting:

  client.broadcast("" + (int) avgMotion);

Receiving:

public void frameEvent(Client c){
  if (c.messageAvailable()) {
    String[] msg = c.getDataMessage();
    motion = (float) Integer.parseInt(msg[0]);
  }
  redraw();
}

This example could be extended to making a map of motion around the image and sending those values as a String.

Scenario: XML Feed

Grabbing data from the internet whether from a database, XML feed, php script, etc. can be accomplished with the messaging framework. In certain cases, you might not even need the framework. For example, if each application only needs to load the data once when it starts up, each client could do this individually before the frame syncing kicks in. However, to ensure that all clients are operating with the same data at precisely the same time, using MPE's protocol is advisable.

There are many different methods for retrieving information asynchrnonously with Processing. One quick way is with the SimpleML library. You can also read this tutorial from my A2Z class for ideas, and take a look at Processing's XML library and proXML.

For now, let's say that you just want to grab a random headline from Yahoo's News Feed, using simpleML.

  if (ID == 0) {
    xmlRequest = new XMLRequest(this,"http://rss.news.yahoo.com/rss/topstories"); 
    xmlRequest.makeRequest(); 
  }

Once the request is made the results will arrive in the callback function, netEvent():

public void netEvent(XMLRequest ml) { 
  // Retrieving an array of all XML elements inside "<title>" tags 
  yahoos = ml.getElementArray("title"); 
  int index = (int) random(0,yahoos.length);
  String headline = yahoos[index].replaceAll(":"," ");
  client.broadcast(headline);
}

It's pretty straightforward, with one funny thing. The MPE messaging system using a colon (:) as a special character, so if you send String messages, you'll need to make sure you avoid having any stray colons by accident. That's what the replaceAll() method above is doing. Once the message is broadcast, all the clients will receive it in frameEvent().

public void frameEvent(Client c){
  if (c.messageAvailable()) {
    String[] msg = c.getDataMessage();
    for (int i = 0; i < msg.length; i++) {
      Headline headline = new Headline(this,client,msg[i], random(client.getMWidth()),client.getMHeight());
      headlines.add(headline);
    }
  }
  redraw();
}

The above code will work with any number of headlines that are broadcast, looping through the entire String message array.

Scenario: Serial Input (xBee)

http://itp.nyu.edu/varwiki/BigScreens/XBee