Getting Started with TUIO, Reactivision, AS3, and PD

IMG_5332

Recently, a bunch of us were at Universidad Autónoma del Estado de México (Facultad del Artes) in Toluca, teaching a workshop and having an expocision there! Luis gave a class on Blender, Anne-marie taught about Gamemaker, Andreas taught about Pure Data, and I attempted to teach about TUIO/Kinect/Reactivision.

I say attempted, because I and Andreas did not speak much spanish (or rather we did not actually speak ANY spanish on arrival), and as I am a particularly longwinded and wordy person (with some incredibly longwinded and complicated ideas to communicate about the purpose and meaning behind interaction or programming), there were many things lost in translation due to the language barrier, despite the very kind attempts by some of the MA students such as Julio and Blanca (who tried to bridge the language and learning gaps for us by teaching the younger students a little bit more about flash and programming before we got there). But I hope everyone there still learnt something from my murmurings about tangible touch (and thanks to Janitzio for having us all there to teach as well!).

P5254497

IMG_5440

P5254505

P5244488

Some pictures from the part of the workshop that Andreas and I were teaching

The class was actually very sweet and it was lovely to go there and teach them all for a week, although there were truly some moments of utter exasperation of not being able to communicate an idea better as I simply did not have any common words to do so. In the end I also had to improvise and turn the class into a reactivation focused lesson because there were too few kinect sets in the classroom (I had only brought two from Singapore). I suppose I too have learnt a lot of things from this experience:

  1. it is truly very difficult to really teach programming to people who speak a different language to one’s own (especially when simple pseudocode examples do not make any sense to them)
  2. its easier to teach a short workshop with opensource tools like pure data, rather than a program like flash that requires a licence, or with external hardware like kinect, because one really needs to have a day in advance to do a pre-setup session or installfest. i had not catered for this earlier unfortunately. also, that part about Flash files not being readable by more than the version before it (ie: Flash CS5 files are only editable in CS4 and CS5, but not CS3) really bothers me. (caveat: despite this drawback, i feel Flash is a pretty practical tool with its own commercial uses so i would still recommend learning Flash although its a little more bothersome…)
  3. it is very difficult to learn how to make a tangible touch setup AND to write the program if one does not have any basic understanding or grounding in programming. IN FACT LET’S JUST SAY ITS IMPOSSIBLE. this is not going to be the easiest “MY FIRST FLASH APP” to start with. perhaps i could eventually find a way to write a simpler tutorial, but at this point i suppose i don’t have enough teaching experience to know how to make it understandable to someone with TOTALLY ZERO programming experience. it would take much longer. but i suppose maybe as i slowly accrue more teaching experience, i might also learn the mystical way of magically teaching anything to people who have zero knowledge of the thing that you are teaching about……

P5254502

Julio and Blanca in a funny moment. The MA students really helped translate and communicate some of the things I could not say in Spanish to the rest of the class. Thank you so much guys!


P5244490

at the workshop i tried to show examples in both as3 and pure data. having played a little bit with max and pd before, it was straightforward for me to figure out how i could start using pd and tuio from the examples provided although this was also my first time really using PD.

I am by no means some sort of ultimate expert on learning about TUIO (as these were things I had slowly figured out by myself on my own from reading on the internet and posting on forums) but I suppose one thing I’ve appreciated about this in the few years that I have been trying to figure it out is how TUIO is easy and consistent so that when you code up something, say for Flash AS3, you would then have a consistent way to refer to things and a flash app could be built for a touch table OR also used for kinect and it would still work immediately both ways for both setups (assuming you used the term TUIOCursor and TUIOObject consistently)

The difference between TUIOCursor is that a cursor is like a dot on the screen that is tracked and does not have rotational angle. There can be many many Cursors as well, whereas an TUIOObject is something that is already defined an object number and it is an object which has an “Up” or “Down” while on screen and can be rotated around and around. (There is also a TUIOBlob, but suffice to say since its been all pretty homebrew I rarely get around to calibrating my blobs)


Click here to view the full
TUIO + Kinect/Reactivision + AS3 Workshop Guide
http://dreamsyntax.org/tuioworkshop

– This guide is for Mac Users who use 10.5 and above.
– Windows users can also use the guide but some instructions may differ (such as the way one uses the Terminal/Console)
– Example start file available at http://dreamsyntax.org/tuioworkshop/tuioclientexample.zip
– Solution file available at http://dreamsyntax.org/tuioworkshop/tuioclientexample_solution.zip
– Presentation built with slidedown


BONUS! – Getting started with TUIO and PD

To get started with TUIO and PD, download PD Extended from the puredata websitew.

Download the TUIO Client Reference Implementations for PD

Download reactivision and print out some fiducial markers to wave at your camera to test it

Open the programs in this order: reactivision > pure data > the tuio pd patch of your choice, and voila it should work out of the box when you wave a symbol at your webcam!

Screen shot 2012-05-25 at PM 01

Creating a Flash app with a PHP script to upload webcam images to WordPress

This month’s challenge is to build a flash application that is able to take a photo, upload it, and insert it into a wordpress post so that posts can be easily edited later. (AND ALL WITHIN ONE OR TWO WEEKS!) I am building this for a museum exhibition that has just opened and it is going to be a kiosk with a large touchscreen that will be moved around from venue to venue, collecting photos of people’s pockets and stuff, and then digitally archiving them, while allowing one to also tag it collaboratively after that.

I am not a PHP developer although I understand PHP sufficiently well in order to make it do what I need to do, so if anyone more technically inclined can tell me if this is an efficient way to proceed with this, I would be glad to hear if there are other ways to do this!

1. Uploading Image to server

I used php’s mktime to generate a unique number for each photo’s filename. Each photo was encoded using JPGEncoder in flash, and then using urlLoader I sent it to my php file, which saved it onto my server with a filename derived from the mktime number.

After that, the posts were pushed to wordpress with the help of Incutio XML-RPC Library for PHP. You can post to wordpress with a php script with the help of xml-rpc. The following examples are my own mashup of bits and pieces found online, the most instructive guide being this entry: WordPress XMLRPC – Posting Content From Outside WordPress Admin Panel

<?php
if(isset($GLOBALS["HTTP_RAW_POST_DATA"])){

$jpg = $GLOBALS[“HTTP_RAW_POST_DATA”];
$img = $_GET[“img”];
$filename = “images/booth_” . mktime(). “.jpg”;
file_put_contents($filename, $jpg);

require_once(“IXR_Library.php.inc”);

$client->debug = true; //Set it to false in Production Environment
$title=’Pocket No. ‘.mktime(); // $title variable will insert your blog title
$body='<img src=”http://yourdomain.org/’.$filename.'”>’; // $body will insert your blog content (article content)
$category=”pocket”; // Comma seperated pre existing categories. Ensure that these categories exists in your blog.
$keywords=”keyword1, keyword2, keyword3″; // Comma Seperated keywords
$customfields=array(‘key’=>’Author-bio’, ‘value’=>’Author Bio Here’); // Insert your custom values like this in Key, Value format

$title = htmlentities($title,ENT_NOQUOTES,$encoding);
$keywords = htmlentities($keywords,ENT_NOQUOTES,$encoding);

$content = array(
‘title’=>$title,
‘description’=>$body,
‘mt_allow_comments’=>0, // 1 to allow comments
‘mt_allow_pings’=>0, // 1 to allow trackbacks
‘post_type’=>’post’,
‘mt_keywords’=>$keywords,
‘categories’=>array($category),
‘custom_fields’ => array($customfields)
);

// Create the client object
$client = new IXR_Client(‘http://yourdomain.org/xmlrpc.php’);
$username = “admin”;
$password = “password”;
$params = array(0,$username,$password,$content,true); // Last parameter ‘true’ means post immediately, to save as draft set it as ‘false’

// Run a query for PHP
if (!$client->query(‘metaWeblog.newPost’, $params)) {
die(‘Something went wrong – ‘.$client->getErrorCode().’ : ‘.$client->getErrorMessage());
}
else
echo “Article Posted Successfully”;

} else{
echo “Encoded JPEG information not received.”;
}
?>

 

2. Retrieving images to put into gallery

This php script will retrieve the last 10 entries in your wordpress and help format them into an xml sheet that you can use in flash to display wordpress entries and images in a flash gallery.

<?php
set_time_limit(0);
require_once("IXR_Library.php.inc"); 
// Echo the header (XML)
header("Content-Type: application/xml;charset=ISO-8859-1");
echo '<?xml version="1.0" encoding="ISO-8859-1"?>' . "rn";
echo "<GALLERY>rn";

// Prepare the XML file
$client->debug = true; // Set it to false in Production Environment

// Create the client object
$client = new IXR_Client(‘http://artsciencepocketbooth.org/xmlrpc.php’);
$username = “admin”;
$password = “password”;
$params = array(0,$username,$password,10); // Last Parameter tells how many posts to fetch

// Run a query To Read Posts From WordPress
if (!$client->query(‘metaWeblog.getRecentPosts’, $params)) {
die(‘Something went wrong – ‘.$client->getErrorCode().’ : ‘.$client->getErrorMessage());
}
$myresponse = $client->getResponse();
$i=0; ?>

<?php
//foreach ($myresponse as $key => $value)
foreach ($myresponse as $res)
{

if($res[‘post_status’]!=”draft”){
//$times = new IXR_Date(); ?>
<image id ='<?php echo $i+1; ?>’ postid='<?php echo $res[‘postid’]; ?>’ title='<?php echo $res[‘title’]; ?>’ keywords='<?php echo $res[‘mt_keywords’]; ?>’/>
<?php $i++; } } ?>
<?php echo “</GALLERY>”; ?>


The resulting XML file will look like this:
Screen shot 2012-03-21 at PM 08

3. Tagging images in gallery

 


 

PHP: Creating automatic XML Sheet from ALL images within a folder

If you need to create an XML sheet from all the photos within one folder, this is a script you can use to create one! It will include any image with a image-related file extension. This is my script which I have customised for my own needs based on notes and tutorials found online.

<?php
// Set which extensions should be approved in the XML file
$extensions = array
(
  'jpg', 'JPG',
  'png', 'PNG',
  'gif', 'GIF',
  'bmp', 'BMP'
);

// Echo the header (XML)
header(“Content-Type: application/xml;charset=ISO-8859-1″);

// Prepare the XML file
echo ‘<?xml version=”1.0″ encoding=”ISO-8859-1”?>’ . “rn”;
echo “<GALLERY>rn”;

// Get the files from the current directory
if (file_exists (“./”))
{
if (is_dir (“./”))
{
$dh = opendir (“./”) or die (” Directory Open failed !”);

// Prepare the images array
$imgs = array();
while ($file = readdir ($dh))
{
// Only get the files ending with the correct extension
if ( in_array( substr($file, -3), $extensions ) )
{
array_push($imgs, $file);
}
}
Closedir ($dh);
}

// Sort the array
sort($imgs);

foreach ($imgs as $img)
{
// Return all images in XML format
echo (‘ <image FULL=”/images/’ . $img . ‘” THUMB=”/images/’ . $img . ‘” />’);
echo “rn”;
}
}
echo “</GALLERY>”;
?>

 


 

OOP & PHP?

there is an interesting article here that argues for OOP in PHP. i admit even in flash sometimes when doing something quick dirty for example i would use global code instead instead of encapsulation. but i guess i find the analogies arguing in favour of OOP rather interesting.

as3isolib, regular expressions

Today’s roundup of experiments and snippets:

as3isolib

I downloaded as3isolib some time ago but never had time to play around with it. Today I went back to their site after a long while and realised that all their example swfs on their documentation site were broken. Most of the examples were originally for Flex but I sometimes like to work in Flash IDE itself so this is an example how to use it with as3iso. Credit goes to the documentation wiki as well as various commenters correcting each other on the wiki.

As3isolib is an open-source ActionScript 3.0 Isometric Library developed to assist in creating isometrically projected content (such as games and graphics) targeted for the Flash player platform.”

This is a simple example of an IsoScene with an IsoBox. It uses classFactory to create the shadow below itself, and TweenMax to handle the animation.

import as3isolib.core.ClassFactory;
import as3isolib.core.IFactory;
import as3isolib.core.IIsoDisplayObject;
import as3isolib.display.IsoView;
import as3isolib.display.primitive.IsoBox;
import as3isolib.display.renderers.DefaultShadowRenderer;
import as3isolib.display.scene.IsoGrid;
import as3isolib.display.scene.IsoScene;
import as3isolib.geom.IsoMath;
import as3isolib.geom.Pt;

import eDpLib.events.ProxyEvent;
import com.greensock.TweenMax;

var box:IIsoDisplayObject;
var scene:IsoScene;
var tm:TweenMax;

scene = new IsoScene();

var g:IsoGrid = new IsoGrid();
g.showOrigin = false;
g.addEventListener(MouseEvent.CLICK, grid_mouseHandler);
scene.addChild(g);

box = new IsoBox();
box.setSize(25, 25, 25);
box.moveBy(0, 0, 20);

scene.addChild(box);

var factory:as3isolib.core.ClassFactory = new as3isolib.core.ClassFactory(DefaultShadowRenderer);
factory.properties = {shadowColor:0x000000,shadowAlpha:0.15,drawAll:false};
scene.styleRenderers = [factory];

var view:IsoView = new IsoView();
view.clipContent = true;
view.setSize(stage.stageWidth, stage.stageHeight);
view.addScene(scene);
addChild(view);

scene.render();

function grid_mouseHandler(evt:ProxyEvent):void
{
var mEvt:MouseEvent = MouseEvent(evt.targetEvent);
var pt:Pt = new Pt(mEvt.localX,mEvt.localY);
IsoMath.screenToIso(pt);
var squareSize:int = 50;//
var gridX:int = Math.floor(pt.x / squareSize);
var gridY:int = Math.floor(pt.y / squareSize);
TweenMax.to(box, 0.5, {x:gridX * squareSize, y:gridY * squareSize, onComplete:completeCallback});

if (! hasEventListener(Event.ENTER_FRAME))
{
this.addEventListener(Event.ENTER_FRAME, enterFrameHandler);
}
}

function completeCallback():void
{
this.removeEventListener(Event.ENTER_FRAME, enterFrameHandler);
}

function enterFrameHandler(evt:Event):void
{
scene.render();
}

Using regular expressions to parse loaderInfo url name:

I have a folder full of swfs which have an external file / xml entry where they must get their information from. Each swf file is named in the format xxx_0.swf where xxx is a 3 letter suffix and 0 is any number (can be any number of digits). The issue is that when you try to return the url name you will actually get the full directory path. Don’t know if this is the most efficient way, but the solution I eventually went with involves putting each URL segment (divided by a slash) into an seperate array item and then going to the last one. Finally we slice off the front 4 characters (xxx_) and last 4 characters off (.swf):

var myFileNameArray:Array = new Array();
var num:Number;
var myFileNumber:String;
var myFileNum:Number;

var myFileName:String = this.loaderInfo.url.split(“/”).pop().replace(/%5F/g, “_”).replace(/%2D/gi,”-“);
myFileName = this.loaderInfo.url.split(“/”).pop().replace(/%5F/g, “_”).replace(/%2D/gi,”-“);
myFileNameArray = myFileName.split(“/”);
num = myFileNameArray.length;
myFileNumber = myFileNameArray[num-1].slice(4,-4);
myFileNum = Number(myFileNumber);

JPEG Glitch Generators

Found this AS3 BitmapData Glitch Generator. Reading the code to find out how it works. Here is some of the output from this:

Picture 4
Picture 5 Picture 7 Picture 6 Picture 10 Picture 9 Picture 8

The opensource example from soulwire works by altering a few bytes in the ByteArray of a JPEG file, which is loaded back into a DisplayObject using Loader.loadBytes(). You will notice that the corruption starts slightly further down – because the code is trying to avoid the important JPEG headers. It is able to do the same for PNG and GIFs by first converting them to JPEGs then doing the same (PNG and GIF are not saved in the same way and only JPEG’s format is suitable for this method of corruption – although I wonder how else we could corrupt PNG or GIF?)

A JPEG image consists of segments and markers. Some of them are important and specify important things like the width, height, subsampling, and quantisation tables. These parts are not to be edited because because this needs to be more or less intact for the JPG to still sorta display, and the part that is altered is the portion from SOS (start of scan) and EOI (End of Image). – See more on JPEG syntax and structure)

EXAMPLE: MANUALLY CORRUPTING JPEGS

The same can be done with any text editor. Here is an example using a random cat picture.

Picture 12
All those characters are storing the information for this picture:

sadcat

I opened up a duplicate of this JPEG file and pasted in the above paragraph on JPEGs into one of the lines.

Picture 14

This is the effect on the JPEG image:

sadcat_edit

VERDICT: 400% MORE ADORABLE?

Traversing XML structures – Loops

I am working on a AS3 project with an XML data sheet and this week I figured out another solution to traversing the structure in a better fashion…

<?xml version="1.0" encoding="utf-8"?>
<TIMELINE>
 <YEAR NAME="2000">
  <ARTICLE>
   <MONTH>January</MONTH>
   <NAME>Lorem Ipsum</NAME>
   <DESCRIPTION>Lorem Ipsum is simply dummy text.</DESCRIPTION>
   <IMAGE>image1.jpg</IMAGE>
  </ARTICLE>
  <ARTICLE>
   <MONTH>December</MONTH>
   <NAME>Ipsum Lorem</NAME>
   <DESCRIPTION>Lorem Ipsum is simply dummy text.</DESCRIPTION>
   <IMAGE>image2.jpg</IMAGE>
  </ARTICLE>  
 </YEAR>
 <YEAR NAME="2003">
  <ARTICLE>
   <MONTH>April</MONTH>
   <NAME>Ipsum Lorem Ipsum</NAME>
   <DESCRIPTION>Lorem Ipsum is simply dummy text.</DESCRIPTION>
   <IMAGE>image3.jpg</IMAGE>
  </ARTICLE>  
 </YEAR>
</TIMELINE>

Basically, in my XML sheet, there were years, and within those years there could be more than one article. Some years had numerous articles. Some years had only one article.

myXML.*.*.length();

When I traced something like this, I would get the total number of articles. I could also traverse all the articles as if the year was not of consequence and all the articles were just being counted and assigned numbers in order from 0 onwards.

myXML.*.ARTICLE[articleNum].DESCRIPTION.toString();

The quandrary was that I needed to be able to scroll through all the articles chronologically but I needed to be able to tell what year it was from, according to the article number (which was now effectively independent from the year), and also be able to find out the article number within that year alone, rather than as a whole.

The solution I came up with: Use a loop to check for number of articles in all previous years, add them up so i can subtract it from the full number of articles. This gives you the article number within that year alone, instead of the article number within the entire “timeline” sheet.

var g:Number = 0;
//myXML.*.ARTICLE[articleNum].parent().childIndex() is year index for that article
while (g < myXML.*.ARTICLE[articleNum].parent().childIndex())
   {
        //myXML.YEAR[g].*.length() is the number of articles in g year
        articleNumContainer += myXML.YEAR[g].*.length();
        g += 1;
   }
// one is added to articleNum cos it actually starts at zero
articleNumberWithinYear = (articleNum + 1) - articleNumContainer;

Flash AS3 and SoThink SWF Decompiler

While saving my work, I had an error and lost the FLA while fortunately, having first published a perfect SWF. If you are saving your work in Flash and Flash crashes before it saves, it often causes your FLA to completely disappear. Always backup your work periodically. Thus I was forced to decompile my own code with SoThink SWF Decompiler to recover my work into an FLA file (for speed) but I also realised that it had decompiled it in a different way from how I had originally coded it. It was easy to figure out and I recovered my entire file within an hour to almost the exact way I had originally coded it. Here are some notes on what I noticed about how it had changed my code in the process of decompiling from swf to fla.

  1. All variables that are randomly declared all over the code are collected and put at the top of the code.
  2. New variables are given names like “_loc_2” or “_loc_3”. Anything with many brackets is also assigned various variable names for different parts of the equation, and then a few of these temporary functions are added up together instead.
  3. All for loops are turned into while loops.
  4. “this.” is added to any objects referred to on the main timeline
  5. It also makes use of the rather uncommon addFrameScript, which adds script to movieclips in AS3. If you leave bits of code outside functions by mistake (eg stage display items), it might group them together in a function called frame1.
  6. It adds the “end function” comment at the end of functions.

Example of typical while loop as decompiled in SoThink:

while (t < max_T_Number)
            {
                // do this
                t = (t + 1);
            }

Example of typical addFrameScript function as decompiled in SoThink:

public function MainTimeline()
        {
            addFrameScript(0, this.frame1);
            return;
        }// end function

function frame1()
{
stage.scaleMode = StageScaleMode.SHOW_ALL;
stage.align = StageAlign.TOP_LEFT;
stage.displayState = StageDisplayState.FULL_SCREEN;
}// end function
}

Although the decompiler is perhaps not the most ethical way of accessing other people’s source code, I think the decompiler still makes an excellent learning tool for flash. When I have more time I will download more swfs or analyse more of my old swfs to see if the conditions listed above are consistently true.

Traversing XML structures in AS3

There may come a time when one has to traverse some XML data….

Alright, so maybe never. But let’s just pretend that the ability to sort XML data might be of some use to someone out there one day! After working on a project this week, this is what I figured out about traversing XML structures.

// load XML file into Flash
var myNewXML:XML;
var myNewXMLLength:Number;
var myNewLoader:URLLoader = new URLLoader();
myNewLoader.load(new URLRequest("xml/singapore.xml"));
myNewLoader.addEventListener(Event.COMPLETE, processmyNewXML);
function processmyNewXML(e:Event):void{
 myNewXML = new XML(e.target.data);
 myNewXMLLength = myNewXML.*.length();
 //trace(myNewXML); // traces the ENTIRE xml sheet
 //trace(myNewXMLLength); // traces the number of items in xml sheet
 //trace(myNewXML.row[0]); // traces the first row
 //trace(myNewXML.row[1]); // traces the second row
 //trace(myNewXML.row[0].name); // traces the first row's name field 
}

// creating a new XMLList to sort and loop through specific category filters
var myNewXMLList:XMLList = myNewXML.row.(category == “Ind” && type == “Pte”);
for (var i:int = 0; i < myNewXMLList.length(); i++)
{
var myNewXMLElement:XML = myNewXMLList[i];
//trace(myNewXMLElement) // traces xml list of only the filtered items
//trace(i + “: ” + myNewXMLElement.name); // trace the number and name
}

// checking for a value
if (“myValue” in myNewXMLList) {
trace(“myValue exists in my XMLList”);
}

/*
Other useful operators to use to traverse XML:
.parent();
.childIndex();
.length();
.toString();
.firstChild();
.nextChild();

You can also use a combination in a chain to find specific fields
*/

Q: Should you put the XML data into an Array before sorting it?

A: There isn’t a real need to put XML into an Array for handling in AS3. Apparently ECMAScript (which includes Actionscript, Javascript, and JScript) now has native XML support (E4X), allowing XML to be treated as its various primitive parts, rather than a seperate object on its own.

I also wondered if I should put it in an array and sort it, as I initially didnt know how to sort for combinations of categories on different “levels”. But in the end, it seems that it is not necessary to put your values into an array because there are certainly ways to sort and loop it through in the form of an XMLList. You could potentially create different XMLLists from the same XML file that has been loaded.

Q: How does one export Excel to XML on Microsoft Excel for Mac?

A: WELL, YOU CAN’T. It seems as if every single version of Microsoft Excel for Mac (including Excel 2011) is tragically unable to export XLS files into XML directly. It is said online that Windows Excel has both the Visual Basic editor and the magical XML function that Mac Excel is inexplicably bereft of. Excel 2007 for example, has no macros or VB editor at all, rendering it completely pointless. Why was such a simple and basic function omitted from all the Mac versions of Excel? As for alternatives like OpenOffice and NeoOffice, let’s not even go there since they always crash on my Mac.

So my final solution? I saved all my XLS files to CSV (comma-seperated values), and then I used an opensource online tool called Mr Data Convertor to convert it to XML. OPENSOURCE WIN.

Q: How do I install an open source Syntax Highlighter?

A: You only need to add a little snippet to get a syntax highlighter running on your page. For the benefit of this post I also installed Alex Gorbatchev’s syntax highlighter, an open source code syntax highlighter which can autoformat nearly any common programming language in blogger or your own site. If you are in the habit of posting code then this highlighter and HTML encoder may come in handy for converting all your pointy brackets into their HTML tags.