Parallax CMUcam4

Kye · 2012-08-02 08:25

Hey all,

I'm going to be working on a CMUcam4 board design for Parallax soon. I wanted to ask if anyone has any suggestions for a form factor for the camera board.

Please visit www.cmucam.org for more information. The pin usage and firmware build for the platform is locked down right now, but the layout of ports for the camera board and attachments to other micro controllers is not. For example, I've already made a CMUcam4 form factor that is an Arduino shield.

Any suggestions are appreciated.

Thank you,

rjo__ · 2012-08-02 10:15

Hi

I looked at the CMUcam4 when it first came out, I have been waiting to see Parallax's take on the general design, and I will be buying it when it becomes available. I would prefer the camera board to be self contained and detachable.

But if you can't do this, please provide access to the path between the camera and the Prop... with either a connector ($12) or simple through holes. I would be interested in future use of the camera with a direct to memory option... by connecting a multiple chip SRAM board. Also there is the possibility of connecting a board that not only has memory but also drives a display... as Rayman has discussed..

The competition at this point is the Kinect... which can be had for about $140 and gives you full depth imaging. There are two problems with the Kinect. First, you need a fairly powerful processor in between the Propeller and the Kinect. Second, it is slow for many routine tasks. BUT if I can get it running... anyone with half a brain can do it. The Kinect essentially becomes a vision coprocessor, with its own storage and with a serial link to the Propeller... very neat and works incredibly well.

A Propeller based camera has the advantage of being potentially faster and more flexible for many routine tasks... but only it you have a design that is open to expansion.

Rich

Ken Gracey · 2012-08-02 10:45

Hey Kwabena,

Since we'll be manufacturing it, here's what we'd like to see:

Ability to detach camera
SIP module format with right-angle header or straight pins included (customer can choose best setup) for compatibility with all kinds of microcontrollers
More cost-optimized BOM using components we currently stock (e-mail me for our current inventory list, especially of the passives)
Eagle is probably okay, but we'll have to get it into Altium at some point for production and diverge our files (our open-source files are released in DipTrace)
Improvement of any issues you've encountered on the Arduino version related to layout, power supply, etc.
Blue PCB, like the other Parallax products
Adherence to our PCB layout guidelines. E-mail Thomas for a copy (tbauer@parallax.com).

I'm not trying to raise the bar on you, but to ensure that we produce the most popular, most successful CMUCAM4 available.

And most importantly, thank you for taking this project on. We will do our part to make it a success!

Sincerely,

Ken Gracey

Daniel Harris · 2012-08-02 12:05

Might I make a firmware update request? I've checked your RAM usage, you easily have room for what I am asking. This update would benefit all platforms.

When outputting a T-string, I had to write some pretty specialized code for my host Propeller to both receive and parse the T-string as the CMUcam outputs results as ASCII data. My request is a raw mode where it simply sends a byte for each value in the T-string. The first "T" byte can be ASCII, but the data should be represented as the raw value.

Thanks!

Kye · 2012-08-02 15:02

Actually, I am at 99.999% memory usage. The free space you saw was the stack (needed mainly for the dump frame command). There is no way to fit raw mode.

---

I have an interface library in C that handles all of the communication, it makes interfacing with the CMUcam4 trivial. Its targeted for the Arduino but can be ported to work on the prop chip easily. I'll port it when I start on the Parallax platform in the future.

Thanks,

Sariel · 2012-08-09 11:23

Daniel Harris:

code for my host Propeller to ... parse the T-string

Any chance you are willing to share the method(s) you used to parse the string then convert to dec? I am working on exactly that, and I'm curious as to how you are attempting that feat.

Anyone else come up with a good way of doing just this? Here is a bit of what I have going on:

Using a flavour of "FullDuplex", my CMU Driver is constantly checking for a non-zero byte to come in, then storing it to a 256 byte buffer, and null terminating it when if finds a CR. This buffer address is passed on to the parent file so that responses can be easilly displayed by a generic TV.text clone. because of this, I do not have "direct" access to the buffer... only through methods that accept the address, and not the actual buffer. When the TC command is issued to the CMUCam either by the user, or as a part of an automated process, I would like to glance at that buffer and extract the space delimited values to seperate variables so that decisions can be made depending on the individual values.

I read around, and was looking into using the "SubStr" method from strings2.spin, but the problem is in the fact that the data is not of a fixed length, therefore I cannot just tell it "go here, do this". some of the info can come back with anywhere from 1 to 3 characters, so it is completely dependant upon the space delimiter. Here is an example of the data I am seeing come back to me:

T 66 52 50 48 98 55 1 12

Any good suggestions out there?

Daniel Harris · 2012-08-09 12:13

Hi Sariel,

I'll post the important bits this evening. I am about to hop a plane and don't have time at the moment.

From reading about your method, it looks like I have my code set up to do nearly the same thing.

Cheers!
Daniel

Kye · 2012-08-09 17:09

The way you process the strings is to use some type of string tokenization routines. Basically, you break the string into tokens, where each token is delineated by the space between characters. Then you just call a dec2int conversion function to convert each token to a number. The way you process input is to just look for a carriage return that marks the end of the string. Then load the string into a buffer without the carriage return. Tokenize the string and then grab the numerical results.

When I get home I'll post some SPIN code to do this. It's very easy if you have some good string manipulation routines. The CMUcam4 source code actually has very good ones in it and implements string processing like I've just talked about above.

In C, a single call to sscanf() is all that is needed to processes the string. The CMUcam interface was designed with this in mind.

Thanks,

Kye · 2012-08-09 18:47

Okay the code is attached

.

Thanks,

Paul · 2012-08-09 18:54

Anybody notice that over on the CMUcam4 <wiki> page, about half way down, that Kwabena is in 3D? (cross your eyes to see the effect)

Internet serendipity at it's best!.

</hijack thread>

John Board · 2012-08-09 20:18

G'day,

Quick question, what will the FPS of the CMUCAM be? I've never used it before, and am looking into it. I've heard 30FPS somewhere, is this correct?

I'm wanting to use this on a line following robot, would this be practical? I am thinking of possibly having an embedded computer as the 'brain'.

Thanks,

John

Kye · 2012-08-09 22:31

Yes, it can do line following. The frame rate is 30 FPS. If you Google "CMUcam4" we have a website with all the information.

You'll have to use the advance "Line Mode" feature the CMUcam4 has to do line following. You'll need a rather fast processor to handle the "Line Mode" feature. Basically the CMUcam4 sends you 80x60 binary bitmaps of whatever color it is tracking at 30 FPS.

To line follow you then take 80x1 slices in a few different places in the binary bitmap. You compute the center of mass of the pixels in the slices and then that gives you a sequence of points which you can tell your robot to drive to.

Apply some filtering and integration to the sequence of points you generate at 30 FPS and your robot will follow the line like magic.

Thanks,

Sariel · 2012-08-10 04:05

@Kye,
Excellent. Thanks for the code and advice. Better yet, it does not consume another cog, and that is glorious news since I am all out of them on my current project. It has become quite hungry for processing power, and every little bit helps.

@Daniel Harris
I am still interested in looking at your code, if you are willing to share. I am always looking at the different ways of doing things, and would love to see your implementation.

Hopefully I can get some time today to work on this a bit more. It is currently my favorite project at work, but we have a lot of stuff coming in the door and the looming deadlines that go with them, and I have to deal with them first.

Daniel Harris · 2012-08-10 15:38

Alrighty, here is some test code. This version is not my most recent for the robot, but I have not changed the T string parser any in my later versions - it seems to work pretty well.

You can strip away the rest of my code if you like - the important methods to keep are "receive_string_packet" and "Fill_T_String". Receive_String_Packet fills a buffer with whatever is received from the CMUcam, up to RX_BUFF_SIZE bytes. This is a safe method that does its best not to mangle any memory - there are plenty of checks to make sure it doesn't. Fill_T_String pulls all 9 ASCII bytes out of the receive buffer and puts them into a byte array called t_string[]. Once the t_string array has been filled, you can reference the value with named constants:

' named packet index constants
#0
T_PACKET
X_AVG_TRACKED
Y_AVG_TRACKED
X1_WINDOW
Y1_WINDOW
X2_WINDOW
Y2_WINDOW
TRACKED_PERCENTAGE
CONFIDENCE

ex: if(t_string[TRACKED_PERCENTAGE] > 50) ...

The code is reasonably well commented, especially the two methods you are interested in. There is a lot of code that may not relate to your particular project, specifically the movement state machine. This particular version of code has the beginnings of my movement state machine and may not work correctly. For now, leave the drive_mode variable set to NORMAL - the other drive modes may not work.

Lemme know if you have any other questions.

Thanks!

Sariel · 2012-08-13 09:27

Thanks to both of you. Between the 2 methods, not only have I learned something new, but I have the confidence levels stored in an easy to access variable. Thanks again guys.

jazzed · 2012-09-03 16:43

Kye, I have a stupid question: Is it possible to detect infrared light with the CMUcam4 camera (or some other camera)?

Tubular · 2012-09-03 16:51

Also Kye is there any way to get faster than 30 lines/second out in the line mode?

thanks
tubular

Kye · 2012-09-03 17:25

@jazzed - The OV9665 FSL camera has an IR coating on it so that it can work out doors. So, it can't detect IR light. If rebuild the CMUcam4 circuit, get a OV9665 camera with a changeable lens then sure. It wouldn't be hard to re-purpose what I've done with another camera too.

@tubular - If you use a 6.25 MHz crystal then you'll be able to get a frame rate of ~31.25 FPS. The camera won't go any faster...

Um, why do you need 80x60 binary images faster than 30 FPS? Can you process them that fast with the propeller chip?

jazzed · 2012-09-03 17:51

Kye wrote: »

@jazzed - The OV9665 FSL camera has an IR coating on it so that it can work out doors. So, it can't detect IR light. If rebuild the CMUcam4 circuit, get a OV9665 camera with a changeable lens then sure. It wouldn't be hard to re-purpose what I've done with another camera too.

Thanks Kye. Would an easy board hack make it possible to use the changeable lens camera?
A laser would probably work better, but I'm very concerned about eye damage even with low power.

Tubular · 2012-09-03 18:50

Kye wrote: »

@tubular - If you use a 6.25 MHz crystal then you'll be able to get a frame rate of ~31.25 FPS. The camera won't go any faster...
Um, why do you need 80x60 binary images faster than 30 FPS? Can you process them that fast with the propeller chip?

Hi Kye

My application is a distance measuring one. It theoretically needs a single line of pixels (Phil's TSL1401, or TSL3301 work), but there are some benefits in using a 2D rather than 1D array, for seeing where the reflection from the object is.

I was hoping there may be some way to abort the 'scan' of the camera and start afresh, but perhaps it just spews out composite continously.

I like what you've done with the CMUcam4 and think I'll just order one of that + Joe Grand's laser range finder and see how we go

cheers
tubular

Phil Pilgrim (PhiPi) · 2012-09-03 19:11

Tubular,

With a linescan sensor, the best way to locate the laser "spot" is to use a laser line generator, oriented perpendicular to the linear array axis. That way your alignment is much less critical.

-Phil

Tubular · 2012-09-03 19:38

Thanks Phil, that's good advice

Kye · 2012-09-04 08:16

You can always reprogram the CMUcam4 to do whatever you want. The code is pretty clean.

You could capture a single line of pixels (640). But, this would require 4 processors at once to do it at 30 FPS.

Thanks,

John Board · 2012-10-06 23:46

Kye wrote: »

Yes, it can do line following. The frame rate is 30 FPS. If you Google "CMUcam4" we have a website with all the information.

You'll have to use the advance "Line Mode" feature the CMUcam4 has to do line following. You'll need a rather fast processor to handle the "Line Mode" feature. Basically the CMUcam4 sends you 80x60 binary bitmaps of whatever color it is tracking at 30 FPS.

To line follow you then take 80x1 slices in a few different places in the binary bitmap. You compute the center of mass of the pixels in the slices and then that gives you a sequence of points which you can tell your robot to drive to.

Apply some filtering and integration to the sequence of points you generate at 30 FPS and your robot will follow the line like magic.

Thanks,

G'day,

Thanks for this information - really helpful... This robot is getting designed for International RoboCup Junior competition... Whether it goes or not is another question

-John

Mickster · 2012-12-01 06:40

Very interesting product!

Here is my application:

I need to check the orientation of an object which will be in one of two positions, one is correct, the other is not.

I propose to use the "structured light" approach rather than overall image analysis and therefore plan to project a horizontal line on the object using a laser line-generator for high contrast.

The reflected line profile will be quite different between the two possible orientations.

In a nutshell, is the CMUcam4 able to perform this comparison as a standalone device and output a go/no-go or does it simply output the x,y locations of the light pixels via the serial interface and rely on something else making the decision?

Thanks

Mickster

Kye · 2012-12-01 10:12

Hi,

The CMUcam4 is a coprocessor, you'll need to connect it to another device to make the decision.

As for how to do this using the CMucam4, please look through the CMUcam4 website to see about what the system can do.

Thanks,

Duane Degn · 2012-12-01 11:07

Any word on when the Parallax CMUcam4 will be available?

I'm tempted to get the one from SparkFun but I'd rather get one from Parallax.

Kye · 2012-12-01 11:47

Hi, the Parallax one is not going to be manufactured anymore, me and Parallax decided that the best time to bring the Parallax CMUcam4 device to market has already passed given ramp up times, long term costs, etc.

You should go ahead and buy a CMUcam4 from SparkFun.

Thanks,

Duane Degn · 2012-12-01 12:20

Kye wrote: »

Hi, the Parallax one is not going to be manufactured anymore, me and Parallax decided that the best time to bring the Parallax CMUcam4 device to market has already passed given ramp up times, long term costs, etc.

You should go ahead and buy a CMUcam4 from SparkFun.

Thanks,

Good to know.

Thanks,

Kye · 2012-12-01 13:05

There will be plenty of SparkFun CMUcam4s and Lextronics CMUcam4s made however.

Thanks,

Parallax CMUcam4

Comments