Following a Giant Line

crgwbr

In a contest I'm planning to compete in next year, the robot must follow a along a blacktop path. I attached a picture of part of the path below. Does anyone know of a good way to follow this path? I thought of something like a CMUcam or an AVRcam, but I wasn't sure if the contrast would be high enough. Anyone had experience with something like this?

Thanks,
Craig

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning.

Microsoft: "You've got questions. We've got dancing paper clips."

John Abshier

You may want to check out the CMU Cam3. Its ARM7 gives you more processing power. The camera is the same as the CMU Cam2 and the AVRcam.

crgwbr

ooh, but a bit pricey at $240. What do you think about the CMUcam2+. It's slightly cheaper ($170 at Acroname Robotics).

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning.

Microsoft: "You've got questions. We've got dancing paper clips."

Phil Pilgrim (PhiPi)

Just hope for a cloudy day! The dappled sunlight through the tree branches is going to cause no end of trouble for any camera. I think the best approach would be to use a camera from which pixels can be extracted in the HSB (hue, saturation, brightness) color space. The blacktop will have a low saturation value compared to the surrounding bark mulch, vegetation and occasional traffic cone.

Good luck!
-Phil

crgwbr

It's Pennsylvania, what day isn't cloudy? lol. Thanks for the tip, I'm in the process of reading the CMUcam2+ manual, hopefully it will have the HSB extraction feature. One though I had, would putting a polorized lens (like Sunglasses) in front of the camera help to deal with excess sunlight?

Thanks,
Craig

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning.

Microsoft: "You've got questions. We've got dancing paper clips."

Ken Gracey

How about differential GPS? If you're within 200 nautical miles of a station this could be a good solution if you can determine points on the entire path ahead of time.

http://en.wikipedia.org/wiki/Differential_GPS

Ken Gracey
Parallax, Inc.

Ken Gracey

I know an easier way. . . maybe.

Yesterday the guys from RoboRealm came to Parallax. Their free software (runs on a PC, is that allowed in your competition) can identify the extent and location of the black path with no problem and send serial messages of your choice. You'd need an RF transmitter/receiver (our Embedded Blue devices) between the robot and PC, and to stay within range of the PC the whole time.

Take a look at what they offer, if your contest allows a remote PC.

Ken Gracey
Parallax, Inc.

crgwbr

I'm only allowed to walk 25% of the corse ahead of time, so thats no good. The is going to be GPS on board though. The goal is to navigate to different GPS locations throughout the PennState Albington collage campus. So I will have the GPS coordinates of these points, just not the coordinates of every turn in the road (and yes, you must stay on the path for most of the course). I'll post the link to the contest below. Nothing is posted for the 08 contest yet, but it should be in several monthes.

http://www.ecsel.psu.edu/~avanzato/robots/contests/outdoor/contest05.htm

Thanks,
Craig

Ken Gracey said...
I know an easier way. . . maybe.

Yesterday the guys from RoboRealm came to Parallax. Their free software (runs on a PC, is that allowed in your competition) can identify the extent and location of the black path with no problem and send serial messages of your choice. You'd need an RF transmitter/receiver (our Embedded Blue devices) between the robot and PC, and to stay within range of the PC the whole time.

Take a look at what they offer, if your contest allows a remote PC.

Ken Gracey
Parallax, Inc.

Better Yet, I'll just put a laptop on the robot, it's already 100+ pounds, 20 more won't hurt.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning.

Microsoft: "You've got questions. We've got dancing paper clips."

Post Edited (crgwbr) : 4/10/2007 6:40:58 PM GMT

kelvin james

Maybe an idea to consider, the possibility of the intensity of reflected light to guide the robot. Different colors and terrain have different reflective qualities for light. Lasers projected ahead and sensed on each side might work, and a main photo sensor to adjust the readings for amount of changing ambient light.

crgwbr

That would probable work too. Last night I downloaded RoboRealme, and have been experimenting with it. So far, it looks like that will be the software I use.

Thanks,
Craig

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning.

Microsoft: "You've got questions. We've got dancing paper clips."

RoboRealm

Craig,

We played around with your two images a bit and here is what we were able to come up with in terms of finding the route. Note that we used the floor finder module for both images with the same configuration.

The biggest problem will be dealing with shadows. You can see that if a shadow occupies the lower part of the image then it will be counted as part of the road (due to how the floor finder module works). But in the other image you can clearly see the shadow being eliminated from the detected route .. that's something we will have to work on eliminating. It is a form of the "fill highlight" option that we have in the floor finder module .. but instead is a "fill lowlight" type function. We'll see what we can do.

STeven.
http://www.roborealm.com/

crgwbr

Thanks Steven,
From the functions I've used o far, RoboRealm has proved to be really great software. Could you post the list of objects you were using? For example, was it just the floor finder object, or did it have the normalize object too.

Thanks Again,
Craig

Post Edit: I just tried the floor finder, with a normalize and Raise to the power of 2 before it, then a Center of Gravity after it. It works great.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning.

Microsoft: "You've got questions. We've got dancing paper clips."

Post Edited (crgwbr) : 4/11/2007 7:22:04 PM GMT

Phil Pilgrim (PhiPi)

Here's a similar treatment, using the saturation data I mentioned above. This was done in Corel PhotoPaint, using an HSB separation and then thresholding the saturation channel. No other massaging was necessary. If you're using a color NTSC TV camera, the saturation data should be easy to separate out with hardware by means of a high-pass filter and peak detector.

-Phil

DiablodeMorte

Wait, I Guess I don't completely understand, if you are given GPS data of where to go why not go on Google maps and route the turns yourself? Or is there a chance there is fork and only one way is allow?

crgwbr

If you noticed the traffic Cones in the photo I posted, those are to prevent the robot from going down the wrong side of a fork. In other words, there is only one way to go through the course, and that way is unknown to you.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning.

Microsoft: "You've got questions. We've got dancing paper clips."

Dave Hein

One attribute of the walking path is that it has little or no color. It consist mostly of shades of gray. I wrote a program to extract the chroma and set the gray areas to a value of 128. I found that the bright areas of the walking path have a blue-ish tint, so I also added a test for that.

I processed the two images with the program, and I then filled the area containing the 128's with yellow. The results are shown below. Note that the shadows do not have much effect on locating the path using this method.

I would also use an ultrasonic range finder to determine if the path ahead is clear. You could also shine a laser in front of the robot and mount a camera a few feet to the side looking at the line on the path. If the line appears straight then there are no bumps or valleys in front of the robot. If the line deviates up and down then there are bumps ahead. If the line is broken in places then the bumps and valleys are large enough to hide part of the line.

You won't be able to use the imaging technique for the off-road portion of the contest. A combination of using GPS, ultasonic rangefinder and the laser technique will probably work OK.

The image processing code that I used is attached below. Good luck on your project.

Dave Hein

Dave Hein

The images didn't get attached to my previous message. Here are the images.

crgwbr

Thanks Dave,
I don't understand C, so your code is Greek to me. Anyway, will the program do the processing in real time? Also, once it finds the yellow path, is there anyway to find the center?

Thanks,
Craig

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning.

Microsoft: "You've got questions. We've got dancing paper clips."

Dave Hein

Sorry for the triple-post, but I just wanted to make it clear that the method I suggested is essentially the same as the one proposed by Phil Pilgrim. Hope you find the code useful.

Dave Hein

Dave Hein

Craig,

You could do a center of mass calculation to locate the center of the path. Or you could compute the midpoint between the left and right edges.

Dave

Dave Hein

crgwbr said...

Anyway, will the program do the processing in real time? Also, once it finds the yellow path, is there anyway to find the center?

Craig,

I modified my program to draw red lines on the sides and middle of the path. The pictures are attached below. Of course, all you really need is a single value that tells the robot how far to turn to the right or left. You would only need to process a few lines of the image that are looking a few feet ahead of the robot. An average value computed over 10 or 20 lines is probably sufficient. In theory you could get by with a single scan line, but this would give you a noisy result.

This technique should be able to run in real time if you process only a few lines per frame, and you capture a few frames per second.

Dave

crgwbr

Thank Again Dave,
Once again, I can't read C, so I really don't understand your code. I'm amazed though, it seems to work very well for such a short program. Because of the fact I don't understand C, I'll probable stick with RoboRealm.

Once Again Thanks,
Craig

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning.

Microsoft: "You've got questions. We've got dancing paper clips."

DiablodeMorte

crgwbr, don't worry, I don't think his code is "finished" in the sense that it seems to be a "sample" and not a finished product. I looked at it, very interesting but I contains no ways to open the jpg or actually derive any of the specified values which led to believe that It wasn't complete.

Dave Hein

The code I posted·was only a piece of the whole program.· It only shows the processing for a single pixel.· This code sits within a double loop that scans the rows and columns of the image.· My intent was to describe the algorithm, and not to provide a finished product.· I felt that the details of how the image was read and written were not important.

By the way, I used PaintShop Pro to convert from JPG to BMP and back to JPG.· I also used PaintShop Pro to do the fill with yellow.· My program reads and writes BMP images.· It's easier than decoding and encoding JPG images, which I have done in the past.· My goal was to write a quick and dirty program to demonstrate the algorithm.

Dave
·

Paul Baker

Craig,

The code Dave has written is pretty straightforward, and written in such a way that it is quite similar to PBASIC. The only thing of any complexity·are his formulas, but thats math, not C. What about his code do you find "Greek"?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Paul Baker
Propeller Applications Engineer

Parallax, Inc.

Post Edited (Paul Baker (Parallax)) : 4/18/2007 5:00:42 AM GMT

crgwbr

Thanks Guys, I understand it a little more now. Apart from the math being complex, I couldn't find where he loaded or captured the image. I figured that it must be there, an I was just not understanding it correctly. Now that I know it's not there, I can read most of it fine.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning.

Microsoft: "You've got questions. We've got dancing paper clips."

RoboRealm

crgwbr,

Do you have any more images from the course? We'd like to work up a tutorial surrounding all the great stuff that has been discussed here and offer more insight into what the problems you will have with this task. The color saturation, YCrCb processing techniques are very simple and will work great as long as the path remains black/grey. We can post the configuration for all the techniques discussed in this forum which will allow everyone to experiment more interactively with those algorithms. Note that for this kind of a contest you will need many more example images. Specifically look for areas along the course where the road color/intensity changes (i.e. like between old and new road), areas with gravel or dirt, areas of sharp turns, areas with pot holes or broken asphalt, areas with any colored markings like arrows or signs on the road, areas with obstacles like a black trash can, etc. Also take a couple images from non-road surfaces.

When we did a field trip a couple years ago for research for the first Darpa Grand Challenge contest we spent a day out in the California dessert and took about 300 photos. After selection of the 'hard' images we ended up with about 5 images. Unfortunately, we are *still* working on those images from time to time even today ... so yes, this can be a hard problem but perhaps we can share what we've learned so far.

Dave,

We were also a little confused by your technique until you mentioned that you used PSP to fill in the 128 grey area with yellow. You made a manual tweak to the image that should probably be automated. I assume you simply used the flood fill by clicking in the lower center of the image within PSP? The floor area to the left of the orange cones is also 128 and by was not highlighted due to the selection of the larger/center area for the flood fill. Automating this selection would require some simplistic blob processing where either the larger blob or the one more in the image center is selected. It wasn't clear from your code that this action is needed so we thought we'd try to suggest how this could be done. This is just our guess ... perhaps you can explain more about your procedure.

Thanks,
STeven.
www.roborealm.com

crgwbr

I'll post as many pictures as I can. The website of the contest (with all the photos) is located here:www.ecsel.psu.edu/~avanzato/robots/contests/outdoor/index.htm

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning.

Microsoft: "You've got questions. We've got dancing paper clips."

crgwbr

Here's some more photos.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning.

Microsoft: "You've got questions. We've got dancing paper clips."

crgwbr

Here's the last of the photos.

Thanks for all the help,
Craig

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning.

Microsoft: "You've got questions. We've got dancing paper clips."

Paul Baker

The bridge in 6 & 7 will be an interesting part of the navigation software, seeing how the non-path portion of 9 is similar in color.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Paul Baker
Propeller Applications Engineer

Parallax, Inc.