wiimote I2C camera

Graham Stabler · 2008-06-25 07:57

I recently discovered that as well as containing some accelerometers the Nintendo wiimote contains a camera. This camera has 1024x768 resolution and in built hardware to track up to 4 points at 100hz. The camera sits behind an IR filter so it can be used to track IR leds or IR light reflected off retro-reflective targets.

So far this has been used for:

Interactive white boards
Stereo tracking (using two remotes)
Head tracking
Laser point tracking (taking off IR filter)
And others?

You can see some demo videos here:

www.cs.cmu.edu/~johnny/projects/wii/

He has also done a good TED talk on it

www.youtube.com/watch?v=QgKCrGvShZs

What I have since discovered is that the camera because it is designed specifically for point tracking outputs point data via I2C, this should make it ideal for using with the propeller. And when in I2C mode you get 200Hz operation!!

Graham

Keith D · 2008-06-25 09:00

Hi Graham

Just so you are aware the WiiMote camera only has a resolution of 128x96 but because it only tracks ir points the resolution of the points can be given at up to 1024x768.

The only downside with trying to use the camera directly on the propeller is having to desolder it (I am half way through doing this but don't have the time to complete it at the moment!)

I did put a post on the forum some time ago asking if anyone knew of any modules to allow the propeller to access HID devices but no one came up with any answers (this would allow the WiiMote to interface directly to the propeller without needing a PC to proxy the data). I have also made enquiries with several companies to see if I can get a BT module to do this, unfortunatly it looks like I would have to spend a lot of money in order to do it (and have to write the code myself!!)

If you do get any further with connecting one up, I'd be really interested in hearing about it!

Cheers
Keith

Rayman · 2008-06-25 09:23

This sounds interesting!
But, I don't get this resolution thing:

Just so you are aware the WiiMote camera only has a resolution of 128x96 but because it only tracks ir points the resolution of the points can be given at up to 1024x768.

···· Is this massive interpolation or what?

Graham Stabler · 2008-06-25 10:25

Thanks Keith, sorry I missed your post. Where do you get you info on resolution from? Still it makes no real difference as I don't think the image data is available.

As long as you are only looking at round blobs it should be very accurate assuming sufficient gray scales are available, PSDs have only two analogue signals but are highly accurate in blob positioning.

Graham

Rayman · 2008-06-25 12:27

I check out those videos and am very impressed! I would definitely be fun to figure out how to make this work with Propeller...

Maybe there's some way to use one of Sparkfun's bluetooth modules with this...

Keith D · 2008-06-25 14:30

Hi Rayman

I think the additional resolution is calculated using the number of pixels used to make up each dot, and it is possible to change the sensitivity to it's lowest sensitivity which will reduce the resolution to the cameras 128x96 resolution. As for the algorithm I am not sure about how it is done, but it would be a pretty cool project for someone to track more than 4 light sources using a custom camera on the propeller (one of the things I have heard discussed is to track many light sources and be able to identify each one - not a simple task without loosing data, but could then be used for motion tracking etc.)

Unfortunatly Sparkfun's modules does not handle the Bluetooth HID profile (I got an answer back yesterday to this) and other devices such as mice are slaves so cannot act as a master device (and cannot connect to a wiimote). I am still looking for alternatives though, and I don't consider myself to be an expert in Bluetooth so I'd love someone to prove me wrong!!!

Hi Graham

You are correct, the image data is not available only the 4 bright light sources (you can remove the IR filter and use bright light sources such as laser pens, though I don't know how well this works!)

Some interesting WiiMote links for you are:

Johnny Chung Lee's pages:
procrastineering.blogspot.com/
www.cs.cmu.edu/~johnny/projects/wii/

Information on talking to the WiiMote:
www.wiili.org/index.php/Wiimote
wiibrew.org/wiki/Wiimote (mentions the actual camera resolution)

General information:
en.wikipedia.org/wiki/Wiimote

There are other pages on the net that give other details but your best bet is to google for those.

Cheers
Keith

Post Edited (Keith D) : 6/25/2008 2:45:24 PM GMT

Graham Stabler · 2008-06-25 14:34

Thanks Keith

Graham

jazzed · 2008-06-25 23:43

Very creative. Johnny's head tracking demo is great!

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔

stevenmess2004 · 2008-06-26 00:52

Presumably the remote uses a seperate bluetooth chip/module. It probably uses spi or uart to communicate with the bluetooth module. Can you intercept the signal there? If you can you would get all the other buttons and accelerometers as well.

Keith D · 2008-06-26 08:20

stevenmess2004 said...
Presumably the remote uses a seperate bluetooth chip/module. It probably uses spi or uart to communicate with the bluetooth module. Can you intercept the signal there? If you can you would get all the other buttons and accelerometers as well.

Although this would solve half my problem (as I do want to be able to connect such as the camera up to the propeller) part of my issue is to be able to wirelessly control a Propeller based robot wirelessly (without having to carry a laptop around!) I know people already control devices with a WiiMote (one person controls an iRobot roomba with one) but these all use a PC to read the HID data then pass the data serially to the microcontroller.

Cheers
Keith

Graham Stabler · 2008-06-26 11:08

Steve, as I said in my first post it uses an I2C interface.

simonl · 2008-06-27 14:13

@Graham: I had to pick my jaw up off the floor! This opens-up loads of possibilities

I particularly liked the fact that Johnny thought outside the box (gees I hate that term!) and put the IR on his head, not the wiimote (much less geeky!). Simple, but effective

Thanks for sharing.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Cheers,

Simon

www.norfolkhelicopterclub.co.uk
You'll always have as many take-offs as landings, the trick is to be sure you can take-off again ;-)
BTW: I type as I'm thinking, so please don't take any offense at my writing style

Graham Stabler · 2008-06-27 18:22

It's cool stuff; essentially Nintendo have done all the hard work on the camera and loads of other researchers over the years have done the hard work on these interface concepts. Johnny is is joining the dots, but at the end of the day if someone has to. I'm not sure Nintendo ever thought of doing head tracking or what that might mean to games.

Graham

Jay Kickliter · 2009-01-07 21:27

I just read the elektor article about this. Does anyone have any Spin sitting around to use this camera?

Also, can a counter be used to as a substitute for the 25 MHz crystal/CLK?

Post Edited (Jay Kickliter) : 1/7/2009 9:39:23 PM GMT

Jay Kickliter · 2009-01-09 20:42

I couldn't find any propeller code for this, so I translated Johnny Chung Lee's C code found here. I don't know if it will work, since I don't have one yet (waiting to get a used Wiimote off eBay.) But it looks like a useful thing to have a cheap IR tracking camera (useful in the sense that it helps you do cool things that aren't particularly useful.)

Not sure yet what to do with the data once I get it.

I still haven't found an answer to whether a Propeller counter can output a 25 MHz clock that the camera can use. If not, maybe run the Prop and camera off a 20 MHz oscillator.

CON
  ACK = 0
  NAK = 1

OBJ
   i2cObject      : "i2cObject"
   synth          : "Synth"

VAR
   long _1mS, _10mS, _30mS, _100mS
   word x, y
   byte pixartData[noparse][[/noparse]16]

DAT
  initData byte $30, $01, $30, $08, $06, $90, $08, $C0, $1A, $40, $33, $33     'Elektor example, Wii default

PUB start(_sda, _scl, _cameraClockPin)
  synth.synth("A", _cameraClockPin, 25_000_000)
  i2cObject.Init(_sda, _scl, false)
  _1mS   := clkfreq / 1_000
  _10mS  := clkfreq / 100
  _30mS  := clkfreq / 30_000
  _100mS := clkfreq / 10
  initPixartCamera

PUB initPixartCamera | index
  index~
  repeat 6
    i2cObject.start
    i2cObject.i2cWrite($B0, 8)
    i2cObject.i2cWrite(initData[noparse][[/noparse]index++], 8)
    i2cObject.i2cWrite(initData[noparse][[/noparse]index++], 8)
    i2cObject.stop
    waitcnt(_10mS + cnt)
  waitcnt(_100mS + cnt)

PUB readPixartData | index
  index~
  i2cObject.start
  i2cObject.i2cWrite($B0, 8)
  i2cObject.i2cWrite($36, 8)
  i2cObject.stop
  waitcnt(_1mS + cnt)
  i2cObject.start
  i2cObject.i2cWrite($B1, 8)
  repeat 15
    pixartData[noparse][[/noparse]index++] := i2cObject.i2cRead(ACK)
  pixartData[noparse][[/noparse]index++] := i2cObject.i2cRead(NAK)
  i2cObject.stop
  x[noparse][[/noparse]0] := (((pixartData & $30) << 4) & pixartData)
  y[noparse][[/noparse]0] := (((pixartData & $C0) << 2) & pixartData)
  x := (((pixartData[noparse][[/noparse]6] & $30) << 4) & pixartData)
  y := (((pixartData[noparse][[/noparse]6] & $C0) << 2) & pixartData)
  x := (((pixartData[noparse][[/noparse]9] & $30) << 4) & pixartData[noparse][[/noparse]7])
  y := (((pixartData[noparse][[/noparse]9] & $C0) << 2) & pixartData[noparse][[/noparse]8])
  x := (((pixartData[noparse][[/noparse]12] & $30) << 4) & pixartData[noparse][[/noparse]10])
  y := (((pixartData[noparse][[/noparse]12] & $C0) << 2) & pixartData[noparse][[/noparse]11])
  waitcnt(_30mS + cnt)

PUB blobX(i)
  return x[noparse][[/noparse]i-1]

PUB blobY(i)
  return x[noparse][[/noparse]i-1]

Post Edited (Jay Kickliter) : 1/11/2009 4:06:43 AM GMT

Harrison. · 2009-01-09 21:53

Take a look at Synth.spin included with the Propeller Tool. I've used it to generate a 25MHz clock for a enc28j60 since I didn't have a 25MHz crystal available.

Photronix · 2010-03-18 19:36

Did anyone get any further on interfacing the Propeller to one of the pixart IR cameras?

The above code doesn't seem to work and there are several init variations out there.

Thanx

Graham Stabler · 2010-03-18 21:57

I was hoping to announce this with some some cool videos etc but seems silly not to mention it now, I have written a fully featured object for the wii camera.

I just need to polish a few details before publication. I wrote it before Christmas but have not been able to work on it for quite a while because of some other commitments. One of those was 700kg of pick and place machine that was sitting in my garage, I finally managed to sell it and it went to its new home today. I guess that is a good excuse to get my act together and finish the last details.

Graham

jazzed · 2010-03-18 23:28

Woo hoo! Sounds great! Can't wait to try it out.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Short answers?
Not available since you deserve more information than you requested.

May the road rise to meet you; may the sun shine on your back.
May you create something useful, instead of just another hack.

Graham Stabler · 2010-03-18 23:44

OK just to wet your appetites and to force me to finish it off this weekend I have posted a video:

www.youtube.com/watch?v=TOoXNzTXhe4

This is just one application, I have a list as long as my arm for other applications, this is one of the more mundane [noparse];)[/noparse]

Time to get some sleep, might not get it at the weekend [noparse];)[/noparse]

Graham

p.s. I have two versions of the object, one is spin only based on Mikes I2C engine, the other is in assembly, also based on Mikes I2C code but translated. Pasm version needed for the 200hz read rate that they say is possible (I need to test the limits).

Photronix · 2010-03-19 03:14

Got it working!

Attached is my hack job code.· Hope someone finds it useful.

Baggers · 2010-03-19 10:40

Great news Graham [noparse]:D[/noparse] will have to try and get hold of a unit [noparse]:D[/noparse]

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
http://www.propgfx.co.uk/forum/·home of the PropGFX Lite

·

Graham Stabler · 2010-03-21 00:30

Posted my object in a new thread.

markaeric · 2010-03-21 01:51

Graham, how about attaching one of these cameras to the nose of a dummy bomb, and drop it from your quadcopter when it's done using a laser to identify the target

I'm not really implying that you do it, but it would be pretty slick!

Toby Seckshund · 2010-03-21 10:16

I wonder how many goverment agencies woke up to that post ????

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Style and grace : Nil point

markaeric · 2010-03-21 23:12

lol.. Oh geez. Hold on, I'll be right back - someone is pounding at the front door.

Graham Stabler · 2010-03-22 16:06

I put the camera on the shaft of a stepper motor and made it follow an LED, not much different to that but I think there are better apps.

Air hockey versus computer for example, you only need a single camera to track position on a surface.

Graham

wiimote I2C camera

Comments