Eightlines Creations » ActionScript

Augmented Reality

Brent — Sun, 18 Jan 2009 19:00:45 +0000

Recently Saqoosha’s port of ARToolkit from C to Flash has taken the Flash world by storm. The biggest hurdle for me was that the comments in the source were all in Japanese. That was until Mikko Haapoja published a getting started in FLARToolkit entry. Now we’re witnessing an onslaught of Augmented Reality projections available right in our browsers.

My initial experiments with the toolkit are really no different from anyone else’s. I had Neil send me a 3D model which we skinned and dropped into Papervision. Immediately the clients at work started to pay attention. But aside from seeing a thousand advertising campaigns six months from now all with their own 3D model that spins around, what can we do with this technology?

Most computers outfitted with webcams have the camera’s targetted at the user’s face. This means the marker used to identify the placement of the model should be placed in that zone in order for a 3D model to appear. On my development system I’m using a PS2 Eyetoy, which is a camera on a USB cable enabling it to be moved around. In many user situations this will not be the case — so orient your models with the marker standing up.
Through the input I’ve received markers with thin lines in them tend not to be recognized at long distances. This could be due to the resolution of my camera, but even with a high resolution web cam it will cause a problem at longer distances. Make your marker files bold without fine detail and they should work better in different environments (poor lighting, greater distance, oblique angles).
Print your marker on card stock. If the marker bends it won’t register. I’ve found smaller markers won’t distort as much.

Now currently I’m working on adding some physics to the 3D scene. I’m using a mixture of Papervision and WOW-Engine. One of the difficulties I’m encountering is determining the rotation of the marker. From what I’ve determined, the Papervision camera is static. The camera is your eye which looks at the marker. As the marker changes orientation the camera does not move. Within FLARToolkit the scene includes a class named FLARBaseNode which extends DisplayObject3D. Tracing out rotationX/Y/Z results in 0/0/0 as the marker rotates. But what is shifting is the rotation matrix of the FLARBaseNode. So essentially if we wish to derive the rotationX/Y/Z we need to retrieve the Matrix3D of the FLARBaseNode and translate the coordinates back into degrees. From what I’ve been told, Papervision includes a static method Matrix3D.matrix2Euler() which should provide this data returning radians.

And I’ve been at this off and on for three days now but haven’t quite found the results I’m looking for. I’ll be sure to post more (plus examples) once I get everything working. In the meantime, this document does a brilliant job explaining Matrix3D.

Thinking out loud — Arduino as Webcam

Brent — Tue, 09 Dec 2008 19:51:55 +0000

Lately much of my work has been dealing with getting Flash to talk to the Arduino. There’s no real functionality to do this on its own, so you have to rely on some other components to host the conversation. There’s a whole host of methods you can use here:

Socket Connection - This requires a socket policy to be set up on the server or else you’ll get a nasty SecurityError. I used Processing to open the connection, and Flash connects to that socket. Check out this version by Mikko Haapoja which uses Processing to generate OpenGL and pass it to Flash as a ByteArray. You pass bytes from here. Also check out AS3Glue which extends the Socket Connection.
Merapi - This bridges Java and Flash.
Serproxy - Included with Arduino, different versions for Mac/Win/Linux. I had difficulty with Serproxy on a Mac and had to switch to a Java version.

All of these work, but the issue is that it still requires a second piece of software to be installed to mediate the connection. In an ideal world it would be nice if Flash could handle the Serial connection itself. This would probably fall under the AIR umbrella in order to properly handle the security permissions.

From what I understand this is a common request to Adobe, and there’s no date that has been set for the feature. This has got me thinking if there’s a way to do it ourselves. (Note: I have no experience writing drivers, so I’m just speculating at this point.)

The general idea is to write a driver that would fool the OS into thinking the Arduino is a webcam. With Flash having native support for the webcam, content sent from the Arduino would be received through the Camera class (which is Final, so we can’t extend it). The video object would connect to Camera and interpret the transmission as a ByteArray. Theoretically, this would only give us Read ability.

This still leaves us with another piece of software to install, so in the end we’re no further ahead than we were using a proxy connection.

Before I dig in and figure out how to build a webcam driver, anyone have any input? Has anyone produced a simillar project? I’ve been searching for a while now and haven’t come up with any useful information.

Mapping GPS Data

Brent — Sun, 16 Nov 2008 20:35:33 +0000

When I last left the data logger project off I was having quite a few issues with managing the size of the data in the allotted EEPROM space on board the Arduino. I built up an SDCard writer based on several specs I found in the messageboard and it worked flawlessly. I successfully captured RPM data from my bicycle commute. The problem was, I had no way to correlate one collection of data to the other. Different routes, days, time, weather could all affect the results of the dataset.

Note: Requires Flash Player 10

It was never on my list of things to do since I knew the battery usage scenario would be so high, but I figured I’d bite the bullet and go the GPS route anyways. I picked up LadyAda’s GPS Sheild and have had a wonderful time working with it ever since.

I had been working on a way to parse the NMEA string data from the GPS device delivers when I figured it would be a perfect way to try out the Google Maps Flash API (my previous experience had only been using the JavaScript API, and Yahoo Maps API) and Flash Player 10. Flash Player 10 offers a number of new features, one of which being the FileReference API which allows you to browse for a file on your local machine and load it into the Flash Player without making a round trip to the server. Once loaded I split the text file based on the carriage return and cast all of the GPS Points to a Vector (another new feature — typecast Array. The speed difference is quite staggering!).

The end result is plotted on a map and colour coded based on velocity, which is recorded in Knots and translated to Km/h. The end result is a colourful chart of the route I commuted that particular day.

In the screenshot above, note the big red blotch at Allen Gardens — flat tire. The other red marks are stop lights or streetcars.

An example of the map in action requires some coordinates, so save these text files to your machine first (contains a couple hundred lines of NMEA data), and hit the load button in the map application. Once loaded the map should recentre on the coordinate boundary.

I’m currently working on some new modifications for the map so I should have more available soon. But first I’m putting the finishing touches on a wooden case I’m hand crafting as a shell for the GPS device. I’ll have photos up from that project soon.

Conversation Clouds with Physics

Brent — Mon, 03 Nov 2008 03:42:15 +0000

So after posting the previous entry I went to sleep, only to find myself thinking about improving the visual aspect of the Conversation Cloud. I decided that adding physics would be the way to go, it would allow words to deflect off of each other, grow and gain mass, and constrain themselves to a container.

I’ve used APE for the physics engine, and Tweener for some of the elasticity when the mass increases.

Conversation Clouds

Brent — Sun, 02 Nov 2008 02:51:27 +0000

Word of mouth is one of those impossible things people like to track. The best people seem to be able to do is to ask the question on a registration form, “Where did you hear about us?”

Is it possible to track word of mouth any better? This is the premise of my latest project, Conversation Clouds.

Imagine a conference on multiple floors, with thousands of people. You stand at a screen visualizing all the conversations going on simultaneously. And there on the screen is the general location of a topic you’re interested in, so you venture down a floor and join in.

Now, privacy concerns aside, the system works as such:

Microphones are placed around a physical space. Each is marked with an id which relates to its geographic location.
The audio is transmitted to a server which performs voice to text translations in small batches.
As each sample is translated it pushes to the visualization script which then displays on a central monitor.

In this early proof of concept I’m using recorded conversations left through answering machines which are transcribed into textual form. Once a new message arrives the script parses it through a custom UIComponent in Flex. Words are added to a hashmap and as repetition occurs the visualization increases the scale and opacity. With future versions I hope to add in a decay parameter as well, so as the conversation flows from one topic to the next so does the visualization. (In the demo above I’m using the content from a previous post.)

Presenting at the Toronto Flex User Group

Brent — Fri, 26 Sep 2008 14:43:45 +0000

Just a quick note that I will be presenting at the Toronto Flex User Group on Thursday October 23rd, 2008 at the Oakham House on the Ryerson Campus, starting at 6:30pm.

Oakham House
Room: Oakham Lounge
55 Gould Street
Toronto, Ontario
M5B 1E9

I’ll be presenting: ActionScript or Flex Project? Development lessons learned building client projects.