Getting Started with TUIO, Reactivision, AS3, and PD

IMG_5332

Recently, a bunch of us were at Universidad Autónoma del Estado de México (Facultad del Artes) in Toluca, teaching a workshop and having an expocision there! Luis gave a class on Blender, Anne-marie taught about Gamemaker, Andreas taught about Pure Data, and I attempted to teach about TUIO/Kinect/Reactivision.

I say attempted, because I and Andreas did not speak much spanish (or rather we did not actually speak ANY spanish on arrival), and as I am a particularly longwinded and wordy person (with some incredibly longwinded and complicated ideas to communicate about the purpose and meaning behind interaction or programming), there were many things lost in translation due to the language barrier, despite the very kind attempts by some of the MA students such as Julio and Blanca (who tried to bridge the language and learning gaps for us by teaching the younger students a little bit more about flash and programming before we got there). But I hope everyone there still learnt something from my murmurings about tangible touch (and thanks to Janitzio for having us all there to teach as well!).

P5254497

IMG_5440

P5254505

P5244488

Some pictures from the part of the workshop that Andreas and I were teaching

The class was actually very sweet and it was lovely to go there and teach them all for a week, although there were truly some moments of utter exasperation of not being able to communicate an idea better as I simply did not have any common words to do so. In the end I also had to improvise and turn the class into a reactivation focused lesson because there were too few kinect sets in the classroom (I had only brought two from Singapore). I suppose I too have learnt a lot of things from this experience:

  1. it is truly very difficult to really teach programming to people who speak a different language to one’s own (especially when simple pseudocode examples do not make any sense to them)
  2. its easier to teach a short workshop with opensource tools like pure data, rather than a program like flash that requires a licence, or with external hardware like kinect, because one really needs to have a day in advance to do a pre-setup session or installfest. i had not catered for this earlier unfortunately. also, that part about Flash files not being readable by more than the version before it (ie: Flash CS5 files are only editable in CS4 and CS5, but not CS3) really bothers me. (caveat: despite this drawback, i feel Flash is a pretty practical tool with its own commercial uses so i would still recommend learning Flash although its a little more bothersome…)
  3. it is very difficult to learn how to make a tangible touch setup AND to write the program if one does not have any basic understanding or grounding in programming. IN FACT LET’S JUST SAY ITS IMPOSSIBLE. this is not going to be the easiest “MY FIRST FLASH APP” to start with. perhaps i could eventually find a way to write a simpler tutorial, but at this point i suppose i don’t have enough teaching experience to know how to make it understandable to someone with TOTALLY ZERO programming experience. it would take much longer. but i suppose maybe as i slowly accrue more teaching experience, i might also learn the mystical way of magically teaching anything to people who have zero knowledge of the thing that you are teaching about……

P5254502

Julio and Blanca in a funny moment. The MA students really helped translate and communicate some of the things I could not say in Spanish to the rest of the class. Thank you so much guys!


P5244490

at the workshop i tried to show examples in both as3 and pure data. having played a little bit with max and pd before, it was straightforward for me to figure out how i could start using pd and tuio from the examples provided although this was also my first time really using PD.

I am by no means some sort of ultimate expert on learning about TUIO (as these were things I had slowly figured out by myself on my own from reading on the internet and posting on forums) but I suppose one thing I’ve appreciated about this in the few years that I have been trying to figure it out is how TUIO is easy and consistent so that when you code up something, say for Flash AS3, you would then have a consistent way to refer to things and a flash app could be built for a touch table OR also used for kinect and it would still work immediately both ways for both setups (assuming you used the term TUIOCursor and TUIOObject consistently)

The difference between TUIOCursor is that a cursor is like a dot on the screen that is tracked and does not have rotational angle. There can be many many Cursors as well, whereas an TUIOObject is something that is already defined an object number and it is an object which has an “Up” or “Down” while on screen and can be rotated around and around. (There is also a TUIOBlob, but suffice to say since its been all pretty homebrew I rarely get around to calibrating my blobs)


Click here to view the full
TUIO + Kinect/Reactivision + AS3 Workshop Guide
http://dreamsyntax.org/tuioworkshop

– This guide is for Mac Users who use 10.5 and above.
– Windows users can also use the guide but some instructions may differ (such as the way one uses the Terminal/Console)
– Example start file available at http://dreamsyntax.org/tuioworkshop/tuioclientexample.zip
– Solution file available at http://dreamsyntax.org/tuioworkshop/tuioclientexample_solution.zip
– Presentation built with slidedown


BONUS! – Getting started with TUIO and PD

To get started with TUIO and PD, download PD Extended from the puredata websitew.

Download the TUIO Client Reference Implementations for PD

Download reactivision and print out some fiducial markers to wave at your camera to test it

Open the programs in this order: reactivision > pure data > the tuio pd patch of your choice, and voila it should work out of the box when you wave a symbol at your webcam!

Screen shot 2012-05-25 at PM 01