Vision Systems - For automated airsoft gun turret/sentry

Shane M

I may be stubborn but I'm looking to reproduce the same result as http://www.paintballsentry.com/multimedia.html

I say I may be stubborn because I could either A) Buy their software or Use OpenCV and do it myself.·

I just would love to do the same WITHOUT a dedicated computer.· I'd like to use one or more propellers to do it.

The project above (I'll call it PaintabllSentry) -- is pretty nice.· It uses a camera system to identify objects.· Largest, fastest, etc.· Even can identify WHAT KIND of object.

I'll settle for any moving object > x size and <·x size.· I would even START with a heat sensor instead.

I can't find a narrow heat sensor that has a range larger than 12'.· And I think it will be quite a feat to write a vision system/filter for the rest.· But I STILL am convinced there must be a way.

I have even considered using OPENCV and trying to complile it on a small linux embedded device and then communicating with the propeller.

I am looking for advice from anyone that will read this.

I want to find the hardware/software that will allow me to create this type of unmanned sentry without the use of a full PC.

Shane Merem
www.websiteforge.com
www.magnusoft.com

Shane M

Oh yeah and there is the whole "Lead Calculation" to determine WHERE to shoot when the object is in motion. My first revision I would point at the object and accellerate in the same direction until it passes the "sweet spot" and ultimately hits the target. But eventually it should actually lead and shoot in the right spot from the start.

Shane

Ale

Even if I know I'd NOT help you with something like that (What part of "you shall not kill" you did not understand ?)

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Visit the home of pPropQL: propeller.wikispaces.com/pPropQL
pPropQL020: propeller.wikispaces.com/pPropQL
OMU for the pPropQL/020 propeller.wikispaces.com/OMU

Shane M

Ha. A tree hugger. Thats OK. I still like you (I am totally just making a friendly joke.. Don't get upset).

It's for a AIRSOFT gun. We play friendly games. Even my 6 year old plays. I don't want it for a real gun.

We want to make it more interesting with a robot that shoots people from both sides of the field. Just to make it interesting. Not killing anything.

Airsoft is just like paintball. Just a sport. Safer than hockey, football, etc. [noparse]:)[/noparse] [noparse]:)[/noparse]

Look for yourself:

Just a fun game...

Thanks, Shane

kwinn

The first thing that comes to mind is using the TSL1401 Linescan Imaging Sensor Daughterboard. By mounting it on a stepper or servo motor you could have it scan up to 360 degrees of the horizon to produce an image and look for changes/movement by comparing the new image to the old one. Of course you could also use a standard spy cam to cover a narrower arc without moving it , but most of them have video out which would be more difficult to convert to a digital image than the TSL1401.

heater

Ale's point maybe that anyone could take the system developed for an airsoft gun and bolt on something a bit more spiteful.

I might caution against this kind of project in case you end up like this guy:

www.interestingprojects.com/cruisemissile/

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.

lonesock

Hi.

OK, to see the level of work you'd have to do on the prop, check out this link:
www.merl.com/projects/fasthumandetection/

As to the morality of the project, I propose an altogether more selfish project:
* targeted water sprinkler system
* operates as usual
* unless it detects a pooping dog or cat

[noparse][[/noparse]8^)

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
lonesock
Piranha are people too.

Shane M

LOL. If that were my goal I'd just buy the software I showed you and it would be already be running.

Come on guys. I can't believe that my request is getting this kind of attention.

I'm looking for advice/assistance on getting this done. I am not interested in political discussions. Here is a large group of smart and helpful people. I am just a fellow "developer" looking for some development advice.

If I somehow accidentially posted in a liberal "feelings" forum. Just tell me and I'll look for help elsewhere. I thought this was the "Parallax Forums > Public Forums > Propeller Chip > " forum. Thats the help I am looking for.

Shane Merem

Shane M

Thanks!· I'll check that link out now!

And the sprinkler system is cool. If/when I ever·get this done.· I'll share everything with anyone that wants it.

Shane Merem

lonesock said...
Hi.

OK, to see the level of work you'd have to do on the prop, check out this link:
www.merl.com/projects/fasthumandetection/

As to the morality of the project, I propose an altogether more selfish project:
* targeted water sprinkler system
* operates as usual
* unless it detects a pooping dog or cat

[noparse][[/noparse]8^)

Shane M

··· Hmm.· Good idea.· I'll look it up.· Thank you!

kwinn said...
The first thing that comes to mind is using the TSL1401 Linescan Imaging Sensor Daughterboard. By mounting it on a stepper or servo motor you could have it scan up to 360 degrees of the horizon to produce an image and look for changes/movement by comparing the new image to the old one. Of course you could also use a standard spy cam to cover a narrower arc without moving it , but most of them have video out which would be more difficult to convert to a digital image than the TSL1401.

heater

Well, you might be right. Reflecting on it, when it comes to worrying about home made cruise missiles, weapon aiming systems, ED209's etc. we have to face the fact that the technology required is surely out of the bag now.

However given the current state of, shall we say, paranoia in the US and other places re: terrorist threat I would be a little nervous about designing and publishing such a system on the net.

Fear 1) The bad guys use your design to do bad stuff.
Fear 2) The "good" guys are watching what you do.
Fear 3) The remote possibility that you are fibbing to us about the airsoft gun (well you never know on the net)
Fear 4) Could be it's me who's paranoid[noparse]:)[/noparse]

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.

Post Edited (heater) : 6/29/2009 8:11:06 PM GMT

Shane M

I agree heater. This isn't new stuff or classified stuff. And I DID provide a video of us playing airsoft [noparse]:)[/noparse] Thats my front yard!

Thanks for the perspective.


_______________________________________________________________________
"People kill people - Not guns"

Shane M

I ordered one.· Even if it's not the ultimate solution.· It will force me to learn some image processing and how to handle it with the propeller.· Very much appreciated.

Shane

kwinn said...
The first thing that comes to mind is using the TSL1401 Linescan Imaging Sensor Daughterboard. By mounting it on a stepper or servo motor you could have it scan up to 360 degrees of the horizon to produce an image and look for changes/movement by comparing the new image to the old one. Of course you could also use a standard spy cam to cover a narrower arc without moving it , but most of them have video out which would be more difficult to convert to a digital image than the TSL1401.

Kye

Doing any advanced stuff with the propeller chip for object detection will be very difficult. First you will mostlikely want to use more than one chip. That said, one chip should take care of all the physical stuff including moving the turrent, firing, etc. And one chip should do the vision processing.

I think the software package viewPort has a vision processing system that can detect colors. Using that you could track players wearing a certaint colored vest like hunters use.

As for finding sensors to use, well thats something you need to do.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Nyamekye,

SciNemo

I have an idea!!

Since you can't do the level of video processing you want with the propeller, and you don't want a dedicated computer, maybe you could consider using a different type of sensing entirely.

I propose using several microphones spaced out at very specific intervals in order to triangulate the location of sounds. You could probably do it with a prop (as long as the mics were far enough apart to increase the time delay), and the only other thing you would need is a turret.

I was bitten by this bug a while ago, and am still trying to reach that holy grail, so good luck to ya.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Not the fish.
sites.google.com/site/bitwinproject/

Shane M

So you mean send a signal out.· Kinda like a sonar?· Then monitor the mics for the bounce back?

SciNemo said...
I have an idea!!

Since you can't do the level of video processing you want with the propeller, and you don't want a dedicated computer, maybe you could consider using a different type of sensing entirely.

I propose using several microphones spaced out at very specific intervals in order to triangulate the location of sounds. You could probably do it with a prop (as long as the mics were far enough apart to increase the time delay), and the only other thing you would need is a turret.

I was bitten by this bug a while ago, and am still trying to reach that holy grail, so good luck to ya.

Shane M

Hi.· I'm ok with using one propellor for image processing.· I figured that was what I would need.· May need more memory too if the filter requires comparing several screenshots, etc.
I was trying to stay away from the 'color detection' for now.· But I appreciate the advice.
I agree on your points.· This will be a challenge.
Shane

Kye said...
Doing any advanced stuff with the propeller chip for object detection will be very difficult. First you will mostlikely want to use more than one chip. That said, one chip should take care of all the physical stuff including moving the turrent, firing, etc. And one chip should do the vision processing.

I think the software package viewPort has a vision processing system that can detect colors. Using that you could track players wearing a certaint colored vest like hunters use.

As for finding sensors to use, well thats something you need to do.

mctrivia

SciNemo is sugesting a location device based on sound output by the object you want to hit. The US military has a device that using several microphones can tell in what direction a shot was fired from.

If something is making sound the sound will travel at rufly 278m/s if you had microphones place in a triangle 1 m apart you should be able to compute what direction the sound is coming from by how long it takes to get to each microphone.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
propmod_us and propmod_1x1 are in stock. Only $30. PCB available for $5

Want to make projects and have Gadget Gangster sell them for you? propmod-us_ps_sd and propmod-1x1 are now available for use in your Gadget Gangster Projects.

Need to upload large images or movies for use in the forum. you can do so at uploader.propmodule.com for free.

mctrivia

If you get this working i will be very interested. I built a powerful turet capable of moving my camera around. would be cool if I could set up in woods and have take pictures of animals.

Made it for taking 10GPixel paneramas but never got around to building the control system.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
propmod_us and propmod_1x1 are in stock. Only $30. PCB available for $5

Want to make projects and have Gadget Gangster sell them for you? propmod-us_ps_sd and propmod-1x1 are now available for use in your Gadget Gangster Projects.

Need to upload large images or movies for use in the forum. you can do so at uploader.propmodule.com for free.

Shane M

I can't see this tracking a moving object/person in the 50 foot range.· There will be alot of noise, etc.· I can see it determining exactly where a sound came from...· Did you watch the video I posted originally.. That paintballsentry site.· That has a good example of what I am trying to accomplish.
Thanks for the help.· That microphone concept may be helpful to orient the sentry to an approaching target.

Shane

mctrivia said...
SciNemo is sugesting a location device based on sound output by the object you want to hit. The US military has a device that using several microphones can tell in what direction a shot was fired from.

If something is making sound the sound will travel at rufly 278m/s if you had microphones place in a triangle 1 m apart you should be able to compute what direction the sound is coming from by how long it takes to get to each microphone.

Shane M

Very nice!· If I could just find a longer range thermal sensor I could do a decent version with it.· It would work for animals.· The longest range heat sensor I could find was about 12 feet.· Its on the parallax site.· I need it for at least 50'.

Shane

mctrivia said...
If you get this working i will be very interested. I built a powerful turet capable of moving my camera around. would be cool if I could set up in woods and have take pictures of animals.

Made it for taking 10GPixel paneramas but never got around to building the control system.

WNed

What scares me is that the same people who make Roomba are also producing robots for the military. I may have to change my sig line to "They may have Roombas and other weapons of mass destruction."

@Shane - I somehow doubt you are going to create something more sophisticated than could be thought up by the wonderful people who brought us IED's and programmable fanatic humans, so good luck with your project. I've been an EMT, but still like paintball and understand that you can want to shoot at people without ever really wanting to hurt them.
ViewPort can be used to do what you want - the download has a video of ViewPort tracking a face - but I think the PC is doing the video processing and sending tracking data to the Prop, so it's not that useful for what you have in mind.

@kwinn - The TSL1401 looks like a nifty device, but when I looked it up, one source had it tagged as obsolete, and TI lists it as no longer available... Shane's comment that he ordered one puts that somewhat in question, though, doesn't it? As a heads up - that device by itself is not going to make an adequate sensor, is it? Won't it need at least one lens to focus the field of view on the array?

A Ping sensor on a servo might be useful. It would not track motion, as such, but could be used to store an array of sector/distances. Break up the field of view into, say, five sectors, then have the Prop point the sensor at the center of each sector, take a reading, store it, move to the next sector, etc. Once you have readings for the "normal" range in each sector, set the scanner to monitor mode. While monitoring, the scanner points the Ping sensor at the center of each sector, compares the reading to the "normal" value, and if the reading is smaller than normal, something (someone) new is in that sector. The weapon could then fire in a "cover pattern"... one center of target, one left, one right... that way you don't need geometric perfection in aiming. when you want to step up the level of sophistication, you can get into looking at the actual range value and calculating the angle of left-right variation you want based on distance...

Ned

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
"They may have computers, and other weapons of mass destruction." - Janet Reno

Shane M

This sounds doable!· I had thought of something similar.· I don't mind using cover patterns for now.· It's just for fun.· And frankly if I could achieve the same accuracy of the paintball sentry -- I would end up programming in a "miss" function so that it's more realistic with some programmed "misses".· So a cover pattern is a· good idea even if a little less accurate on fast/far leads.

On the ping servo.· I bet I have a better chance finding a long range sonar/ping than the infrared device.·· Who knows maybe one slow servo to identify possible targets and a couple real fast ones to narrow down the object and speed?· Maybe narrow down left/right edge of an object?· Possible?··· Then top/bottom of it?· (or just top)

I even thought of a large board of ping sensors to draw a rough picture too.· But seems bulky.· Just snowballing --

Shane

WNed said...
What scares me is that the same people who make Roomba are also producing robots for the military. I may have to change my sig line to "They may have Roombas and other weapons of mass destruction."

@Shane - I somehow doubt you are going to create something more sophisticated than could be thought up by the wonderful people who brought us IED's and programmable fanatic humans, so good luck with your project. I've been an EMT, but still like paintball and understand that you can want to shoot at people without ever really wanting to hurt them.
ViewPort can be used to do what you want - the download has a video of ViewPort tracking a face - but I think the PC is doing the video processing and sending tracking data to the Prop, so it's not that useful for what you have in mind.

@kwinn - The TSL1401 looks like a nifty device, but when I looked it up, one source had it tagged as obsolete, and TI lists it as no longer available... Shane's comment that he ordered one puts that somewhat in question, though, doesn't it? As a heads up - that device by itself is not going to make an adequate sensor, is it? Won't it need at least one lens to focus the field of view on the array?

A Ping sensor on a servo might be useful. It would not track motion, as such, but could be used to store an array of sector/distances. Break up the field of view into, say, five sectors, then have the Prop point the sensor at the center of each sector, take a reading, store it, move to the next sector, etc. Once you have readings for the "normal" range in each sector, set the scanner to monitor mode. While monitoring, the scanner points the Ping sensor at the center of each sector, compares the reading to the "normal" value, and if the reading is smaller than normal, something (someone) new is in that sector. The weapon could then fire in a "cover pattern"... one center of target, one left, one right... that way you don't need geometric perfection in aiming. when you want to step up the level of sophistication, you can get into looking at the actual range value and calculating the angle of left-right variation you want based on distance...

Ned

Shane M

There are alot of software packages called "Viewport" out there?· Anyone have a link?

mctrivia

http://www.parallax.com/Portals/0/Downloads/docs/prod/soft/ViewPort_setup414.zip

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
propmod_us and propmod_1x1 are in stock. Only $30. PCB available for $5

Want to make projects and have Gadget Gangster sell them for you? propmod-us_ps_sd and propmod-1x1 are now available for use in your Gadget Gangster Projects.

Need to upload large images or movies for use in the forum. you can do so at uploader.propmodule.com for free.

modkil

http://www.cmucam.org/

cmucam3 supports custom code, would be a bit of work, but with the right lens, it could work

GreyBox Tim

Shane M said...
Hi.· I'm ok with using one propellor for image processing.· I figured that was what I would need.· May need more memory too if the filter requires comparing several screenshots, etc.
I was trying to stay away from the 'color detection' for now.· But I appreciate the advice.
I agree on your points.· This will be a challenge.
Shane


Shane,
I was recently working at a company called Anchor Bay Technologies, now I'm contracting at another company· that does computer stuff.· What you are attempting to do is not going to work very well with a propeller doing the processing.
Example:· If you are using a 320x240 narrow angle B&W camera (with 8-bit luma) mounted on the airsoft, in order to "find motion" by comparing pixel differences between frames, you will need to look at (320x240x8-bit) at least·76.8KB of data per frame.· Multiply that by the number of frames your filter will be looking at and the rate at which your frames are coming out of the camera and you will begin to grasp the seriousness of the data-rate here.

The minimum you will need to look at is two frames, but I'd recommend at least four frames.· For the output, you can do a simple (i.e. "dumb") motion detection by looking at the difference of a specifically addressed pixel between one frame and the next - this will only tell you that something has changed, not what, which leaves a LOT of filtering to do in post-processing.· What you will most likely need for tracking is some form of comparison of pixel data from line-to-line and column-to-column with multiple instances operating at the same time.

For example, to see where the pixel at "X" below ended up, you will need to look for it at all the locations marked by "O":
··OOO
OOOOO
OOXOO
OOOOO
··OOO

This means you will need 21 simultaneous data streams comparing against the new stream (20 of movement, 1 of stationary).· This will give you·a simple X/Y motion reading of that pixel (which is itself another output data "stream", with two new output variables that have to be processed).· Of course, this only allows for a small ammount of movement before the tracking loses the target completely - which means you have to do one of two things - 1) user more streams to expand the movement range, or 2) up the framerate so that the target can't move more than a few pixels per frame.· Either one is darn near impossible for the Propeller...

My advice for you is to use an FPGA with DSP signal processing optimization, like a Xilinx Spartan-3A.· A standard-definition video camera shoots 720x480 pixels at 30/60fps (depending on the technology, WDR cameras are slower due to pre-processing).· You will likely need a ton of memory - to look at 2-frames of SD in B&W at 30 fps·will take about 20.7MB/s (that's mega-BYTES) of bandwidth.· Using a wide memory bus on an FPGA will allow you to load five or more lines of video simultaneously, and use delay buffes to feed the other detection channels (without going off-chip).· You will also need to process your "hits" (when a motion matches the detection stream) to memory for further post processing, which adds to your overall bandwidth requirement.
If you generate a "cloud" of pixels that have a specific motion, you can use simple math to locate the center of the cloud (both horizontally and vertically) and calculate the error from the camera's field-of-view (FOV)·center to the cloud's center.· This error value can be used to move steppers to position the camera (incidentally the airsoft would be mounted to it, so whould be "aimed").· Once the ceners are aligned to within an error, it can send an output to a fire-controll circuit.· For calculating the "lead" you will need at least 3-frames of data to generate a trend of movement from the cloud data, and you will need some distance data from the camera to the cloud to calculate time-of-flight (TOF) for the projectile (either through laser bounce or sonar estimation, or by using very precis zoom and focus coupled with edge detection to ensuer you have nice sharp edges in the point cloud [noparse][[/noparse]i.e. it is "in-focus"]).· Note that for close fast moving targets, the lead may be "off-camera"...
Above was mentioned "triangulation", using microphones.· If you use very wide angle (180degx180deg) motion detection from several cameras, you can deduce (via triangulation, and camera FOV mapping) where the cloud is in 3D space.· This data can be fed into a calibrated turret, based on a table of which turret is best positioned to hit that XYZ location, and whether it is assigned a "target task" already (in which case the target aquisition network [noparse][[/noparse]we'll call it a TAN]·can assign it to the next best resource - think of what happens if a friendly runs into the line of fire...).· Once a task has been assigned, the turret can confirm the motion it spots with its on-board camera with the TAN and take over the target tracking and tell the network when it has engaged the target, and fired a few airsoft pellets (or a butt-load, whatever the TAN thinks it need to lob towards that target)...

This is all academic - I don't think even ten Propellers could do all of this, I think you need a real serial signal processing solution for this problem.· I also agree with others here, this is probably the kind of thing that will get you in trouble with local, state, and federal law enforcement (or a nice cushy desk-job at Boeing or Lockheed-Martin...).

I can also imagine what would happen to you if you "forgot" to turn it off one day and it put some kid's eye out...· just seems like a liability.

-Tim

Post Edited (GreyBox Tim) : 6/29/2009 10:46:16 PM GMT

mctrivia

You can buy fanless embeded PCs that are very small

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
propmod_us and propmod_1x1 are in stock. Only $30. PCB available for $5

Want to make projects and have Gadget Gangster sell them for you? propmod-us_ps_sd and propmod-1x1 are now available for use in your Gadget Gangster Projects.

Need to upload large images or movies for use in the forum. you can do so at uploader.propmodule.com for free.

Hanno

Hi,
It's easy to use ViewPort to track a face, a color, or a circle. Just download, make sure your Prop and Webcam are connected, then run tutorial #15. In ViewPort you can click on what type of object you want to track. The scaled x,y coordinate will be continually sent to your spin program, where you can control your servos. Good luck!
Hanno

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Download a free trial of ViewPort- the premier visual debugger for the Propeller
Includes full debugger, simulated instruments, fuzzy logic, and OpenCV for computer vision. Now a Parallax Product!

Microcontrolled

To decrease liability make it human-controlled. That would be doible with 1 prop, some servos, and a keypad.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Toys are microcontroled.
Robots are microcontroled.
I am microcontroled.


If it's not Parallax then don't even bother. :-)
·

Mini-Din/PS2 connectors are for sale! 5 for $1! PM me if you wish to make an order.
Cheap·shipping unless specified!· ···· Order 100 get 25 FREE!····· About 300 left!!··

Shane M

YEs.· That just might be the final solution after listening to everyone [noparse]:)[/noparse]

mctrivia said...
You can buy fanless embeded PCs that are very small