Although I created a standalone Windows based (.dll) webcam plugin for Unity3d in May, 2009 utilising Theodore Watson’s – VideoInput library along with some Open CV functions, trying to get a working cross platform Unity3d solution for augmented reality was still heavy on my mind. Knowing of the many existing ARToolkit based examples, working flash AS3 libraries and the various SIFT and SURF algorithms scattered across the web, I really wanted to create a Unity based non platform specific solution, where I could obtain the webcam feed directly into unity and start to utilise C# code for the algorithms.
When a collegue came across the Honeyway Train augmented reality game from Boffswana for Cheerios, I was blown away with the fact that they had appeared to take a webcam directly into Unity utilising either Adobe Flash Player and/or the Unity3d Webplayer itself. Although how they achieved this is still not quite clear, Unity has disclosed that Boffswana made a custom web player based upon agreements made with Unity itself (more info here.)
So after realising no further information on this would be disclosed, I thought I’d have a stab at getting a webcam image from flash to unity3d myself, this seemed to be a valid method/idea at the moment as most people have flash player installed, this accesses a webcam on all platforms, and the Unity3d webplayer is also multiplatform but does not support hardware access to any platforms webcam and will not allow the embedding custom DLL’s (by default).
Ideally I would have liked to get the memory address/pointer of the webcam image/bitmap from flash, access the pointer in unity3d and copy across the bytes to my Unity Texture Pointer, this would have been as fast as any .DLL and exactly how I had done it in C++ however with sandboxes and webplayer securities to deal with I never found a method of making this possible.
Realising that I needed to send a valid policy response to flash to unblock the port I continued and finally got a workable first solution, the project is now something that needs to grow and obviously everything at the moment is at an embryonic stage, but I intend to create both a Unity and Flash Class based library for this for release later in the year, unless beaten to it.
The Initial Demo
Although I intend to move the demo over to this site later next week you can find a link to the demo below. You will need both Flash Player and Unity3D Player installed, also a Webcam plugged into your machine. Please bear in mind that this is a very user unfriendly release and that the demo may not work if using a proxy server (not sure about this, please let me know of your experience as feedback is what we require). The demo does not include any augmented reality as of yet, but serves as a proof of concept awaiting feedback. There may be a slight delay of around 8 seconds whilst Unity resolves Flash connection.
Please give positive or constructive feedback below, many thanks Jon.
This entry was posted on Thursday, June 24th, 2010 at 1:04 am and is filed under Development.