Although I can’t disclose the actual code i’m using right now, as I wish to release some class packages for unity and flash later this year which should be available for all and in the interim want to use it for commecial purposes, however I can point you in the right direction in terms of structure and reference code material if that would be of help?

Congratulations man !!
I tried the javascript and sockets route myself, but got stumped on the flash policy stuff.
Hopefully your development will progress smoothly.
Looking forward very much to your planned release.

Ciao, Patrick

Thankyou for the feedback, most appreciative, as for the policy stuff, I can share this part of the code which should hopefully help others, although I am looking into making this more efficient and into a class. This should make sense to those in the know about threads and listeners:-

In Flash send webcam data using localhost, I used port 8080, in Unity use TCP listener localhost same port, in your thread in the while listening loop,

Code:

mySocket = tcp_Listener.AcceptTcpClient();
TcpClient(Host,Port);
stream = mySocket.GetStream();
bytesReceived = 0;
receivedBytes =0;
if (setpolicy) {
  byte[] flashpolicy = System.Text.Encoding.ASCII.GetBytes(“<cross-domain-policy><allow-access-from domain=\”*\” to-ports=\”*\”/></cross-domain-policy>”);
   while(shouldRun) {
        byte[] bytes = new byte[mySocket.ReceiveBufferSize];
       receivedBytes = 0;
       receivedBytes = stream.Read (bytes, 0, (int mySocket.ReceiveBufferSize);
       if (receivedBytes>2) {
           stream.Write(flashpolicy, 0, flashpolicy.Length);
           setpolicy=false;
           break;
       }
       if(receivedBytes == 0) {
           break;
       }
   }
} else{
   // Read your image bytes and process into texture
}

Essentially flash’s first communication to unity will be a string asking for a policy ‘<policy -file-request/>’, it first asks on port 843 then the port you’ve tried to open, if you reply with the string above, it starts to send your data across otherwise blocks flash from communicating.

Hope this helps, Jon

Thanks for the info.
I must say that my video transfer/augmented reality project is on the backburner at the moment.
But I’ll give you a shout when I make progress again.

You did not mention anything about compression soo.. afaik you get raw bitmap data from the webcam. Before you send it over the socket, compress the data using something “fast” (perhaps run-length encoding?) as3corelib (http://code.google.com/p/as3corelib/) includes jpeg and png encoders, but i think it might be slower than sending compressed raw data since there is also the overhead of decoding.

Very interesting, I look forward to seeing Unity being used for AR!

On this version, no compression is included JPG/PNG encode it tended to slow things down. The image transmitted is unfortunatly low-resolution currently in byte format, and I haven’t tried pushing up the resolution yet due to other work commitments. If higher resolutions fail performance speed some kind of vector math based compression could be utilised, this works quickly in as3, and some of the routines I wish to push into the AS3 class such as background removal, motion detection and OpenSURF may take away some of the needs of transmitting all colour image data.

This all remains to be seen although one thing I am sure of, is that when I release the first draft i’m hoping the community will overhaul and help with optimisation. What would the feelings be on setting up a SVN Depository with GoogleCode in the near future?

This is a pretty brilliant idea… I’ve done a lot of AR and would love to marry Unity’s capabilities with that. Flash is so limited it’s painful. I definitely need to look into this more. Thanks for sharing some ideas and techniques.

Hello Jon,

Well done!

I’m trying very hard to understand what’s going on here. Could you point me in the right direction as you offered to Daniel above? Any help would be hugely appreciated!

Many thanks in advance,

-r

Although I have carried out further experimentation, at higher resolutions and with splitting data over many sockets as opposed to one, the best results still seem to come from the example shown, although I have further tests to do and scripts to ammend including a flash control script this is still ongoing in my spare time.

Both flash SWF and Unity3d objects must be loaded in browser on webpage or (possibly both pages active at same time, this I haven’t tried)

The described process below is ideal for taking a snapshot of image data from flash and sending it to unity, currently this works best at sending 160×120 pixels RGBA which is very low res, but probably ok for motion tracking and some AR techniques in terms of speed, even for simple facial texture mapping for game characters, but there is no reason this couldn’t be scaled, however as it is web-based it is important to consider the speed of users machines.

The Current Working ActionScript code looks something like (pseudo-code):

SetupCamera, Bitmap (320×240)
Set up socket (localhost,port 8080)
Set up Connect timer,started
Set up SendData Timer,stopped
Connect Timer Function {Connect}
On Connect Init Listener{Send Data Timer Start, Stop Connect Timer}
On Close Connect Listener{Reset Timers, Re-Connect}
On Errors Listener{try..Catch then Close Connect(null) – Re-Connect}
Send Data Timer Function{if not connected connect, get Image from cam, scale using matrix (160×120), get bytes of bitmapdata, send data }

The current Unity.cs script looks like (pseudo-code):

Declare vars: texture2d,pixels color array, material, thread, TcpClient, is running bool, sent policy bool etc
Start: Set up thread function, Put pixel data to chosen material, thread function starter
LateUpdate: Copy any pixelsdata[] to texture, Update
Thread (see forums on thread scripts): {
Try Setup 8080 listener, Try GetClient Get Stream, While !Pending Sleep else
if (connected and data, haven’t sent policy), send policy(see above), set policy sent bool true,
if (connected and data, have sent policy), Read Bytes from stream (160×120x4) if complete process bytes to pixeldata
Catch all errors, solve problems.
OnDisable: (important otherwise web-browser crash when leave site): Close Connection, Close Thread, Reset Vars
OnAppQuit: Close Connection, Close Thread, Reset Vars

Thats about as far as I can go for now, and this should have given enough information, including the posts above for coders to get this working, Hope this helps.

By the way, I stumbled across a cunning way to get the memory address of flash bitmap data, but due to sandboxing and security, no way of passing this to unity in webplayer plus its probably local memory adress so would require base address, if any devs have any ideas or indeed anyway of accessing the browser clipboard from unity or accesing unlocal memory address in webplayer, I would be keen to hear from you, although to my knowledge for security reasons I understand why this should not be possible.