Best choice for machine vision, Cmu4 Cam, Pi, Beagle Bone, Android phone...
rwgast_logicdesign
Posts: 1,464
So Im interested in implementing machine vision, Im not so intrested i writing it all from the ground up im looking for an easy to use library like OpenCV or something. I did some reading on the CMU4 Cam last night and its not cost effective at all 99 dollars is alot when theres more powerfull solutions, but looking at the board it looks like I could build my own for the cost of the camera, this makes it very cheap im not sure how much the camera is but id guess not more than 10 or 15 bucks. This way is also somewhat of pain due to having to connect the camera to the board, im thinking id actually have to lay out a PCB, couldn't just protboard a cam together. It seems decent for a beginner does color and motion detection along with shapes at 30fps, the thing is it will be outside alot and in the desseret all the colors blend so this may be a huge show stopper! Also spark fun doesnt stock the cmos cameras anymore, does anyone know where to get them, maybe even on a standard through hole breakout?
Now I am helping a friend whos a coder and likes to build things. Hes building a big robot running off a laptop and using my boards to interface I/O for sensors and stuff to the laptop. His mane reason I can tell for needing a PC is to run OpenCV he wants to do some crazy six webcam set up. When I told him about the PI and the Beagle Bone we both thought those may be a killer solution for machine vision, especially if you clustered a few! Well the sad truth is at 320x240 OpenCV runs at 1-3 fps with no processing from what I can tell on the PI forums. I thought originally this was due to OpenCV not utilizing the PI's DSP, it turns out that OpenCV can use its DSP the problem is the closed source video drivers, and it appears to be the same issue on the beagle Bone. If someone on her is doing vision with a PI and it works well id be happy to be proven wrong.
The next choice would be android, I have plenty of scrap phones around. They already have a camera and are ready to go! Plus they have other sensors!! Now the issue I see here is in order to read the phones GPS I have to use Bluetooth since theres no way to do a hard connectin to an android unless you have ICS and one of those open hardware kits, but as far as any other micro than arduino using the hardware featues via USB I havent seen it and I have no idea hw to use the ADK. Plus all the android software is no where near OpenCV or even CMU4 polished if im correct.
And lastly is a PC, which is not really an option my bot is going to be 4x6 inches. It would be doable on some micro atx atom board im sure but thats tooo pricey and im not sure how id even power something like that very well. Right now my bot is looking at 4 hours run time on NiCAD, and its only 9.6v. I plan a 12v lithium upgrade but that is exspensive too.
So does anyone have any suggestions, basically im hoping to maybe do a stereo set up and analyze pictures for path finding, I also would like to detect animals and motion, I probably need something comparable to OpenCV for this stuff idk
Now I am helping a friend whos a coder and likes to build things. Hes building a big robot running off a laptop and using my boards to interface I/O for sensors and stuff to the laptop. His mane reason I can tell for needing a PC is to run OpenCV he wants to do some crazy six webcam set up. When I told him about the PI and the Beagle Bone we both thought those may be a killer solution for machine vision, especially if you clustered a few! Well the sad truth is at 320x240 OpenCV runs at 1-3 fps with no processing from what I can tell on the PI forums. I thought originally this was due to OpenCV not utilizing the PI's DSP, it turns out that OpenCV can use its DSP the problem is the closed source video drivers, and it appears to be the same issue on the beagle Bone. If someone on her is doing vision with a PI and it works well id be happy to be proven wrong.
The next choice would be android, I have plenty of scrap phones around. They already have a camera and are ready to go! Plus they have other sensors!! Now the issue I see here is in order to read the phones GPS I have to use Bluetooth since theres no way to do a hard connectin to an android unless you have ICS and one of those open hardware kits, but as far as any other micro than arduino using the hardware featues via USB I havent seen it and I have no idea hw to use the ADK. Plus all the android software is no where near OpenCV or even CMU4 polished if im correct.
And lastly is a PC, which is not really an option my bot is going to be 4x6 inches. It would be doable on some micro atx atom board im sure but thats tooo pricey and im not sure how id even power something like that very well. Right now my bot is looking at 4 hours run time on NiCAD, and its only 9.6v. I plan a 12v lithium upgrade but that is exspensive too.
So does anyone have any suggestions, basically im hoping to maybe do a stereo set up and analyze pictures for path finding, I also would like to detect animals and motion, I probably need something comparable to OpenCV for this stuff idk
Comments
The RPi might be able to run OpenCV, but it might be slow. For me the consideration is the entry level price point, so slow is ok for now.
Lots of use the cheap bluetooth modules as a cable replacement between androind and prop, we have suport in propforth and its pretty easy.
I've seen guys with apps running on android phone that do image recognition 1-2 FPS, it looks really cool but I haven't done it yet.
http://buildsmartrobots.ning.com/group/usingopencvforrobotics?
http://buildsmartrobots.ning.com/group/androidbasedrobotics
Were also talking about adding a customized processor for the vision stream but this is still in the noodling phase.
Kye's prop based CMUcam4 is also a fascinating proposition. It may be possible to stack multiple prop processor boards off the same camera bus - that way each prop could be tasked with its own recognition job.
i would rather have the limited abilites of the cmu4 with its 30fps than i would all the stuff opencv can do on a pi at 30 frames per minute. as someone who knows nothing about machine vision im sure the cmu is a great place to start. once i knew what i was doing im sure i could throw another propeller and some quad spi sram on the cmu board to get more power.
my issue with the cmu route is dont know if id ever get a second prop or sram working, as im sure id have to be pretty into machine vision algorythms to get that working. but my biggest issue is that they state it has a hard time with simalar colors. so id never even notice the coyote in the desert.... i really wanted to go the pi route with a usb cam but i just dont think less than 1 fps is worth it
Sorry I said that wrong. The ANDROID was slow. It was giving results (locking on to an object) a every second or so.
As far as I can tell, The RPi camera can shoot full HD at 30 frames per second.
http://www.raspberrypi.org/phpBB3/viewtopic.php?t=6120&p=171661
The guys are talking about writing optimizations for the GPU, but I don't know any details, I'm going off their previous track record (of astounding every time).
ive got an account on the pi forums ill make a post and get a definate answer.
I understand your thought process becuase i figured if the pi could do 1080p its run opencv well enough. a friend doing an opencv bot using a p4 took my advice to check into the pi. when he reported his 1fps finding i didnt belive him so i googled rasberry pi and opencv to find that his research seemed correct! i couldnt belive it the pi can do monster media stuff!
heres what the problem is from my best guess knowing what i do about the pi. i do know that the gpu drivers are only available as binarys and the binaries are crippled i belive so bad open gl sucks in x.
i think the issue with opencv is it needs all the power it can get to do image processing and capture and im pretty sure it needs lower level register access to the gpu which broadcom wont give out withoit an nda and you cant just be some guy. the gpu is totally locked unless your a miilion dollar dev team.
so its probably the video stuff in opencv is ran with a software driver comparable to running quake or quake2 without a voodoo card in the day. opencv is using 100% cpu power to cruch numbers and process images no gpu assistance. the project has stated the 600mhz arm is on par with a p200mhz.. so u can imagine opencv on a pentium 200 without a gpu.
i could be wrong as i said this my best guess based on what i know about broadcoms gpu in the pi. i have no direct experince with opencv so ill make a post over there and figure out whats up, then report back
Yes the Raspberry Pi GPU is driven by a closed source binary blob. Sad but true. But then this is true of a lot of graphics hardware on the PC and elsewhere not to mention true of many other devices on PC's and elsewhere.
That binary GPU driver is not crippled. If you know any reason why it might be please post a link as I have not read such anywhere on the Pi forums.
Accelerated graphics on the Pi works very well. For example I can play video at 1600 x 900 resolution easily. Things like Quake 3 and OpenGL stuff runs just fine.
There is an issue with hardware acceleration in X Windows. Namely that X needs adapting to use that GPU binary blob. This is coming soon only this morning I read on the this thread http://www.raspberrypi.org/phpBB3/viewtopic.php?f=63&t=4649&start=200 that an experimental accelerated X is getting up to 5 or 10 times faster. No doubt opengl in X is just around the corner.
I suspect you do not need X to do what you want anyway.
As for OpenCV, as far as I can tell from the OpenCV web site it only gets GPU support with NVIDEA GPU's, quote:
"The OpenCV GPU module is a set of classes and functions to utilize GPU computational capabilities. It is implemented using NVIDIA* CUDA* Runtime API and supports only NVIDIA GPUs."
It is possible some of the slowness of OpenCV on the Pi is due to limited RAM. Newer Pi's have twice as much RAM, 512MB, so that may help.
I made a post on the PI forums still waiting for an answer, Im thinking I should just go get a webcam and openCV mess with it on a PC with no graphics acceleration and see what the results are, Im really thinking the TM might be the best choice, you keep mentioning the mk802, thats just an a10 corte 9 module with hdmi correct? I know they come with android on them but are they hackable do people have linux up and going on these things? I think the atom board would be the ideal amount of power but I also think its alot bigger than id like.
I just found the cmos cameras for 7.50 im thinking maybe ill buy a few and set up a CMU im not sure, Id like to know if theres a supplier for these camera that have a male header break out board attached
You'd probably want to use the Overo EarthStorm module, if you do go with Gumstix, but if webcams work, the MK802 is a more powerful processor.
The CMUcam4 is really a simple vision processing platform. Its distinctly "low end" computer vision. However, it uses very little power, is small, and works.
Computer vision is about what camera you have and what processor you have. You may find yourself deep in a rabbit hole if you aren't careful.
Thanks,
Oh wow thank you I had thought you guys were using the TCM8230. So there is no way I can pick up a few of the cameras used on the CMU4 Cam? Its an open hardware project, using a part average people can get in an open hardware project geared twords hobbyist, makes the open hardware part of the project kind of useless, I mean even you buy a real cmu4 and the camera breaks you cant repair it. I wouldn't mind buying cams directly from the project, you are the designer correct?
Im really intrested in the CMU i dont need the power of openCV, im basically the market for this thing, a beginner with machine vision who doesnt need alot of power. Im not sure if it the camera will even work for me though based on the scenes id be working with, so I figured if I built my own no problem If it wont work outside ill end up using it for something inside later and I hadnt spent alot. At 99 dollars I could buy the mini atom and a decent webcam and run openCV which has more power than I need but isnt a hard thing to set up. I yhink theres a market for this thing I just think 99 is a bit much for a prop board with a camera, when the firmware and schematics are open.. I also think you might not sell alot of these at that price point becuase of the PI.
First of all the PI has just got a 512mb ram upgrade, so that should help a bit, but heres the answer I recived about openCVs speed. Once the these guys launch the camera module with drivers the pi may be in much better shape to do more advanced open cv stuff..
Just because the Raspberry PI combination costs less does not mean that you will spend less time working to get the system operational. The CMUcam4 is for people who are comfortable using an Arduino who want to add simple color tracking to their project.
If your goal is to track colors as quickly as possible then the CMUcam4 saves you time, which is worth money.
---
We are using the OV9665 because it won't get EOL'ed for a while by Omnivision. We tried to design the CMUcam4 to have somewhat of a life span. The CMUcam4 is a university project. As it looks right now, the market for such type of cameras like the CMUcam4 is slowly disappearing. Back in 2002 the CMUcam was more revolutionary. Now, there's a lot of competition. However, we still offer the CMUcam because no simple computer vision system is available on the market today. Maybe the Raspberry PI CV board will change that.
It would be cool to be able to use a camera available on the open market. But, the decision to use the OV9655 was done by our manufacturer before I started on the project.
You also have to be careful with the cameras you buy. Some are not able to take a lot of abuse... We've found the OV9655 to be really good and it has a lot of features in its hardware that made development easier.
---
The CMUcam4 isn't really for serious computer vision applications. Its for students and hobbyist who just want to get something working quickly. If you want to go the route of using the Raspberry PI then do so. If you regret it, try out a CMUcam4 and see how you like it.
I don't see what open source has to do with the price of the system. I could have easily just not posted all the source code and what not for it. I did post all that information for the system because I wanted the project to be open... I don't understand you argument about that.
We've already moved plenty of CMUcam4's without any advertising (Depends of what you define as plenty for a small university project). It would be really awesome if our university group had the resources and time of the Raspberry PI group to do more. Unfortunately, the CMUcam4 development team is really just me and my managing university professor. I don't have a lot of time for the project right now because I have a full course load at school. Its amazing to me in hind sight that I got the system to go on sale averaging 4 classes each semester.
(I spent my Christmas breaks working on the system... That's how it got done).
Thanks,
The price has absoulutely nothing to do with it being open source that's not why Im frustrated with the CMU4 Cam. The reason Im saying 99 seems a but hi becuase its basically a stock prop board with certain IOs broke out for pan/tilt and such, it also has the camera. Judging the price of other propeller systems it seems as though $50/60 dollars may a more reasonable price, There arw boards out there by Bill and others in the 100 dollar price range with tons more hardware the the CMU4. Im not saying you wont sell any or no one will buy it or really complaining about the price, as Im planning a board and the price is my buisness. Im just observing the market and also stating that if the PI and camera module work well with openCV it may really kill you market. I have not used openCV but im sure setting up software on a linux box and maybe sending serial data to a micro is easier than the CMU4 and propeller/arduino combo. I have a feeling it when the combo comes it will really effect the CMU4s sales, people dont even have to know hardware or embedded programing to slide a camera on a pcb and install linux drivers.
Now I will also say I love the idea, like I said I just want to get a grip on machine vision basics i dont need openCV type power. Im developing a bot mostly to larn about radar and utilize the MIT open courseware for a continous wave radar system with hi res imaging. The machine vision is just a toy for me to help chase my dog. I would no doubt love to use a CMU4 cam but as the site states it has a hard time in similarly colored terrain. A fawn dog in the the brown dirt is probably not going to happen. I had considered building the CMU just as an intro to machine vision and using it inside till I found a better solution, as I will need a main CPU for the SAR imaging algorithms anyways.
This is why Im frustrated with the open hardware/open source aspect. I am on a shoe string budget at the best, I cant be spending 99 dollars on a cmu4 at the moment, if i had the money I would no problem I like supporting projects and giving back when I can, I know you put alot of work into the thing. Im frustrated because I know I could build one if I had a camera module for it, if I built one I cold easily see if it will work for my purposes and get stared with machine vision as I am mostly your target audience. If I picked it up enough I could also try to upgrad the thing with sram and a second prop to maybe make it more capable, which would be giving back to the community. But the fact you chose a camera that cant be purchased makes the whole open source point moot for me, the only option I have is to buy your full board, and if my camera broke well id have to replace the board not just the camera. Having to use your pcb also means I cant very well work to improve the hardware for it, in the end the open hardware benefits are almost trivial, not the open source I believe that's excellent. It would be like qualcomm open sourcing my phones chipset, what do I care I cant buy that stuff unless I sign an NDA and buy large quantitys. I just wish you could find a way to get cameras to people at the very least or maybe list some options for cameras which may be compatible with very little work
The camera is actually more expensive than you think... The camera cost is a bit less than double of everything else on the board combined. We could hit the $50-$60 dollar or less price point if it weren't for the cost of the camera.
In retrospect, the choice of the OV9665 isn't that great because of the NDA.
I have plans for when the Prop2 comes out to do something that pushes the limit of the silicon. I don't want to say to much yet, but, I plan to use a really high FPS camera.
---
As for usability, the CMUcam4 has a complete Arduino Interface Library. You can get up and running in less than 20 lines of code. It is very easy. We specifically designed this to be easy. Otherwise the product would have no point
http://cmucam.org/docs/cmucam4/arduino_api/
Thanks,
i am not sure you guys are still interesting in this old Post
Your discussion is very interested, i would like to ask some things (after searching long i coudnt get my ans) I am software professional interest in image processing, i would like to design my own camera i mean buy a sensor/module and then design it (designing board ,writing driver etc)
Please can you all advice me how to start with it
This is not something I would consider doing. It will take a good deal of time and effort and the results will be poor.
Especially as now you can by a Raspberry Pi and a camera module for it for 50 dollars.
Would it not be better to apply your software skills to making applications of the vision technology available?
Of course if it is something you have to do, just because. Like the reason people build their own CPU's out of relays or transistors or get CP/M running on Propellers then all power to you.
Sadly I have no constructive advice there.
If you want to get started, I would suggest looking at what folks are doing already, and determine your direction based on what you can build on.
In Post 3, there are links to a guy doing a LOT of work on vision. They demonstrated both a laptop based robot (ridiculously expensive if you ask me) and an android phone based robot (expensive at the time but more readily availabel these days). Those could indentify an object and output a command (turn right, turn left, go forward) a couple times a second, or every couple seconds, depending on the image.
We're looking at openCV to see how it will work on a donated Rovio. If that works out, we would move to RPi + camera.
We have a fairly relatively way for the control application to talk to the vision system and the control system. PM me if you are interested. Be warned that it uses FORTH, and is very addictive.
It runs at 16GHz, so it's good for all your computer vision needs!
Nothing runs at 16GHz. If we had a core with FPU that ran at that speed we would not need 16 FPU cores running in parallel and porting code like existing vision algorithms to it would be a lot easier.
I think it means its the equivalent of a single core processor running at 16Ghz. Although the docs say the ARMs run at 700Mhz or 800Mhz, so a 16 core board would be 11.2Ghz or 12.8Ghz equivalent, if you go for that kind of comparision.
Since the vision processing software does not exist, we should find someone smart like heater to start in on it.
Problem is, as you probably know, an algorithm that runs in some time, T, will not run in time T/16 just because you have 16 cores available.
If you work very hard on parallelizing that algorithm, spreading it over 16 cores, you might get closer to T/16.
But often in turns out you cannot get the speed up you expect. See Amdahl's law: http://en.wikipedia.org/wiki/Amdahl's_law
There is a reason why the world record for calculating the digits of Pi has not been held by a gazzillion processor super computer since 2009. They just cannot do it.
Sadly you are going to need someone a lot smarter than me.
This is "talking dog" novelty. We don't mind that the dog has improper diction, just talking is interesting enough.
I still think we need our best minds on this.
From the little bit of image processing code I've written, I can see how multiple floating point units could help. You could either process multiple frames simultaneously or sub-divide a single frame into regions or color layers for separate analysis.
I only mention the Pi thing as an extreme example of what I was talking about.
The gist of that was to correct the statement that any part of the parallella board runs at 16GHz and that sixteen 1GHz processors quite likely don't give you the performance of one 16GHz processor. I thought I said it quite clearly.
Now my knowledge of machine vision algorithms is almost zero so I prepared to accept the idea that an array of floating point units or CPU's with floating point could help a lot. Sounds reasonable.
I share your interests and have looked at the same sorts of issues. For your small bot, you might consider using the Pi camera and stream the video to a nearby computer and then do the processing there.
Then wifi your answer back to the Pi and have it tell your Propeller what to do over a serial line.
I have no experience with OpenCV but what I have read seems very impressive. I personally like ImageJ, which is a general purpose image processing software, written by Wayne Rasband at NIH. ImageJ can be programmed (java), using its own macro programming language, or javascript. ImageJ is cross-platform
can accept all sorts of camera and video feeds, but I'm not sure about wifi. ImageJ can import a named image from a URL. So, if you can figure out how to get the PI to export the camera image to a computer and have the computer then put it somewhere with a fixed URL, you are in business.
For stereo work, you could use two Pi camera with streaming images. But camera alignment is probably going to be a problem. It's a 3d alignment, not for the faint of heart, but if you fiddle enough, it should work.
I have been looking at using a Bloggie 3D, which has a "webcam " mode. So, you can plug it into a computer and then use Processing.org to view it and process it. I have a program that works fine, let me know it you want it. This means that U can mount the camera on a Mac Book Air, wifi it to an Apple tv and get 3D telepresence. Bloggies can be found reconditioned for less than $100. You can use OpenCV within Processing.org and it's speed is limited by what you are trying to do. How to get streaming data into Processing.org is not in my experience, but I think that puts the computer back on the robot.
For image processing, integer performance is king. It doesn't matter how many floating point operations you can do in a second.
To get both image data and 3d data on a limited budge, using a Propeller today, get a tsl1401 line scanner http://www.parallax.com/Portals/0/Images/Prod/2/283/28317-M.jpg add a laser line...$20 on a stepper or servo motor to to move the laser perpendicular to your beam.
Then put the whole thing on a controlled gimbal and you are done... except for the software:)
I'm happy to see Kye is already planning a camera for the P2. And in fact I'm in a little bit of a stall myself until the P2 arrives.
Rich
http://www.kickstarter.com/projects/254449872/pixy-cmucam5-a-fast-easy-to-use-vision-sensor
I've used the 204 MHz dual core LPC4330, it is definitely a great price performer for less than $10
Just to be clear the $10 price you mention is for the processor used by the Pixy not the camera. (The Kickstarter page has a $10 level which gets your name listed in a text file of the source code.)
The camera costs $59.
It looks like the Pixy will be the new CMUcam5.