Machine Vision path finding experiment bot, is this idea even possible?

rwgast_logicdesign

Ok so I scanned through this thread a bit

http://forums.parallax.com/showthread.php?118634-Autonomous-robot-s-navigation

Very cool stuff but a bit over my head at the moment, I was also reding up on how curosity works, it uses stero vision takes a picture and runs a bunch of algorythms to help find the shortest safest path. All of this kind of stuff would be great in my final bot but im sure its going to take alot of hardwork and learning to get there even if im not writing the actual vision algorithms myself, im hoping theres software out there to do this for me, My bigger bot will be navigating the same kind of terrain found on mars. There are all kinds of ways to navigate from point A to point B and on earth we have GPS yay, but there are still alot of big obsticales out here to navigate around and I dont thing a ping sensor and laser is going to be enough.

This robot is really an inspiration to this project too http://www.robotroom.com/StreamHawk-Wireless-Explorer-Robot.html as far as it freely exploring the house. I was thinking I need to start small on this, I also want the bot to be very quiet and roam at night. Im waiting on my motors in the mail currently, I got four electric screw drivers for the rover and then two of these http://www.goldmine-elec-products.com/prodinfo.asp?number=G16279 I have no idea how much load these things are going to carry I doubt theyll carry a can this size of the robot above though, especially with all those electronics packed in.

So my idea is to use an Attiny, or even a propeller if n Attiny wont cut it. Get a cheap cmos camera and some wireless connectivity. The bot would traverese the house sending pictures back to my computer wirelessly the computer would run the images through robo realm or whatever software I need to do path finding and then the computer would tell the robot where to drive without hitting things. My biggest concerns here are will an attiny be able to handel this amount of work while controlling the motors etc, a have no problem with a prop its just bigger, I want to run on liPo button cells, if these motors are as weak as i think. And secondly how fast would the wireless be whats a good option, I would like to use the cheap bluetooth dongles but considering there serial connections im not sure how fast this would be. Im thinking the cmos images have to be fairly small though. I could go wifi with either a wiznet or ENC chip and a dlink wifi to ethernet bridge deal for the xbox but once again that kind of small.

So all in all the thing needs to be capable of motor control using a feedback system PID algorythms etc, it needs to be able to take pictures and read a few other sensors. It needs to be able to perform all those functions while sending the pictures out and excuting commands recived from the PC based on the data it sent.

Martin_H

I don't know much about that sort of machine vision. But I've seen some impressive videos in this thread:

http://forums.parallax.com/showthread.php?134199-Skynet-is-Tracking-16-Bogeys&highlight=ExDiv

Take a look and consider PM-ing ExDiv with questions.

GordonMcComb

Except for basic blob/color analysis, it's hard to do machine vision on an 8-bit controller. This is the domain of the larger 32-bit ARM controllers and similar, or full-blown microprocessors (the Prop can be used for some machine vision tasks, but it doesn't have enough memory for the larger jobs). Machine vision is quite possible on laptops, netbooks, and other computers that have sufficient RAM and processing power. For example, the project below (these are friends of mine) uses machine vision to detect faces, people, furniture, door frames, and many other types of objects:

http://www.leafproject.org/

Options for the vision libraries include OpenCV and RoboRealm, and if you're familiar with it, you can use a low-level framework like Windows DirectShow to sample frames as they come in from a camera. The frames are converted to conventional bitmaps, where the pixels are in rows and columns. You then use various algorithms (many in open source land, and detailed in places like CodeProject) to detect outlines, track colors, create maps, and so on. While the larger computer does the image crunching, a microcontroller operates the robot.

-- Gordon

NWCCTV

GordonMcComb wrote: »

While the larger computer does the image crunching, a microcontroller operates the robot.

-- Gordon

This is exactly what I am working on. I have an Intel Atom Board/CPU based system that will run MSRS for image crunching and other CPU intensive apps while the Stamps will control motors, led's, etc. The problem is that there is no feasable way of doing this with button cell batteries. Is the 32 bit ARM going to be able to run with that type of batteries?

rwgast_logicdesign

funny did alot of reading about leaf before this post.

i know a ucontroller isnt going to get me far prcessing images. what im trying to figure out is what would be an optimal set up to have the controller send the data to a pc for processing, and the pc send back navigation data to the ucontroller. im definately no looking to write direct draw based vision i want something thats easy to use but with advanced capabilities. all the micro is there for is motor control and to wirelessly send and recive data, including sending video

NWCCTV

OK, But I have a feeling sending large amounts of video data may cause some issues. Video is bloated and you may end up losing some in the process since it is going to be xmitted wirelessly. I am not 100% sure but I do not think the solution would be Bluetooth given its distance limitations. Is 802.11 N an option? You could possibly get by with a 900 MHz wireless camera to send back to wireless receiver then to computer to process. I just think trying to pull the video in to a device and then sending the data out may be an issue.

prof_braino

I think its possible. Our project is starting with a single prop to control sensors and actuators, and we're considering adding a RPi to do the vision processing. RPi is scheduled to have a fancy camera soon. If THAT works out, we have musings of adding one of the recently discussed multicore parts as an image co-processor. But, its a lots of big math, and we (at least me) are going to look into borrowing from other projects out there. It should be a fun winter!

NWCCTV

prof_braino wrote: »

I think its possible.

Which part?

rwgast_logicdesign

well bt may not be a good idea but i think wirelessn is overkill i mean people stream 1080p over n. All i wanna do is send single images every few seconds to a pc, and im thinking i may be using a low res cmos camera. no reason to stream video if the pc u analyze shows a clear path for 10ft you probably should take another pc at 8ft. im guessing max speed for thisv experiment is 5 pics a second, there will be sonar and liidar to help correct problems with havimg a slow frame rate.

as i said this is a small indoors guy no bigger thqn 3 by 4 inches, and and i view it as a precouser to the vision nav id like to put on my 4 wheel outdoors rover wich will have an onboard atom or pi. What i really need to know is if a small cmos cam would work on the little guy like the one on parallax, hanno suggests a 10 dollar black and white cam, in his simpler tutorials, but i think i will need two color cams to do depth perception and navigation. the next challenge is finding out what kind of ucontroller power is needed to take 5 to 10 pics a second and then send them wirelessly, while dealing with sonar, lidar, accelarometer, and clodsed feedback motors. the more i think about it the more i think ill need a prop with an sram chip.

NWCCTV

Oh, my bad. I thought this was for the outdoor one that was not going to have the atom or pi. So, is the prop capable of handling the color cam by just adding more sram?

rwgast_logicdesign

well i think the cmos cams are small like 160x160 or something so if you just want to snap a pic then send it to a ram chip used as a buffer before it gets sent out the wireless im sure a prop is fine if not an attiny. As fas as full on hi speed streaming im not sure. i just saw a video on dr_aculas site with the prop playing a full on video i didnt even know that was possible

prof_braino

Dudes! you're thinking small! THINK BIG! (Then scale back drastically when it doesn't work!)

We're going to start with something simple, but our initial assessment is that the prop alone is a wee bit too small for a vision system on a bot. So were going to leave the prop to control the bot, and off load the vision to another part. If you have success on the prop alone, I will surely try to duplicate the result, and save myself the hassle. It will be interesting to see the progress on both options.

rwgast_logicdesign

it surely will, but remmber that im not having the prop/attiny do anything other than send the pictures to a pc for full blown navigation. Id like to use webcams so i can acually watch a true video stream on the pc but i just dont see a way of doing this without usb. maybe the prop could stream a full 30fps from a cmos cam, im just not sure where to start i have never used a cmos cam and im sure ill buy some nutty camera that is almost impossible to get working.

ive been looking at duanes index lately and saw a few projects using a wii mote ir cam, some of this stuff looks like it may be a really good option for distance/depth measurement and object detection.

id really like the bot to be able to see "dogs" im thinking itd have to detect motion if there is motion then measure tempature, if its warm enough then try to grab some demensions and see if there pet size, if so its an animal and not a human

Duane Degn

rwgast_logicdesign wrote: »

well bt may not be a good idea but i think wirelessn is overkill i mean people stream 1080p over n.

They do but the video is very compressed and it takes a lot of computer power to compress it.

Even with a 160 x 160 image, that's 25,600 pixels. I assume you want more than 8-bits per pixel which very quickly puts you past the memory of a uC.

You still have the issue of capturing the image. I have a "Machine Vision" section in my index with some of the ways a Propeller can capture and image.

I think the easiest way of getting an image to a PC would be to use a wireless IP camera that takes care of the image capture, compression and transmission for you. Then just use the uC to control motors to drive the wheels and maybe servos to point the camera.

NWCCTV

@Duane, I agree, Wireless IIP would be the best way to go. However, if you want a good quality one they are not cheap. The ones on EBay for under $40.00 are basically, IMHO, the "you get what you pay for" type. I have seen these in action and there is not much that does NOT cause interference with them. Microwaves, cell phones and even vehicles driving by with crazy sounding stereoes all cause issues. The best I have used are from ACTI and Vivotek. Vivotek does have one for $50.00 on EBay. I have never used that one but suspect it is the same as the other cheap off brand type. The best wireless I have set up are devices that are seperate from the cameras. However, there again they are not cheap and @rwgast is looking for inexpensive solutions.

Duane Degn

NWCCTV wrote: »

However, there again they are not cheap and @rwgast is looking for inexpensive solutions.

I think a uC based video capture and transmission project would be very cool. I toyed with a few methods myself. I don't know of any wireless solutions of getting video from a camera sent to a PC using a uC. There are several wired solutions but I think they are all for black and white images (Phil has a cool color capture process but it's slow).

I think the suggestion of using a 900MHz video link with the video capturing being done by the PC might be a good compromise solution.