PDA

View Full Version : Prop with high speed camera array??



Graham Stabler
01-30-2007, 09:20 PM
I've just started a new job where I am working on design of insect inspired hovering micro air vehicles. Something I have been asked to look at is in-flight tracking of the insects along with high speed imaging. What is needed is a high speed camera that can be moved at high speeds and does not break the bank (and they can).

Cypress do a sensor array that works at 250fps for about $250, my hope is that this array from micron that works at 500fps full frame will not cost too many orders of magnitude more. Even if it does if I can create or at least sketch out a low cost electronic back end for it then we might be in business.

This is the chip:

http://www.micron.com/products/partdetail?part=MT9M413C36STM

It uses a TrueSNAP shutter, which is basically just like Phil's propcam has, that means it exposes all the pixels simultaneously and captures a true freeze frame. But unlike Phil's camera it does 1.3Mpix at 500fps. By reducing the number of rows used the speed can be increased greatly for example 100 rows gives 5000fps. Typically for a hoverfly that is the sort of frame rate required to give about 20 frames per flap.

The chip has a 10X10bit output port so it can read out 10 pixels simultaneously, the control of exposure and reading in and out is done by an external FPGA/CPLD/ASIC but I was wondering if the prop could cut it??

The second problem is what to do with the data, I would be happy if it could be written to some RAM and then later transfered to a mass storage device. But I don't know anything about using RAM so I'd really appreciate some advice there. In terms of transfer to a mass storage device the Vinculum chip could do that. I should note that typically 1 second of capture would be more than enough (that's about 200 flaps of a hoverfly).

Really just asking for a reality check and some ideas.

Graham

Lawson
01-30-2007, 11:47 PM
From what I understand from reading up in the Prop forums, the data rate these sensors can generate is WAY beyond what the current Prop can handle.·(10x10bit @ 66MHz YIKES! that chip really gets the data moving!)··I'm guessing a fairly large and fast FPGA is going to be needed to keep up with this sensor's data generation.· (a·128bit·digital I/O card for a PC might also be able to keep up)· Asside from that the Prop should work just fine as a supervisor and control processor.· (what the FPGA is shown doing in the data-sheet)·

I'd add up what this project is going to cost·in·lost time and material costs.· (figure at least one revision of the·electronics and several·fried chips)· I bet that once all of that is added up the cost of a commercial high-speed camera is not as extreeme.

Good luck,
Marty·

Graham Stabler
01-31-2007, 01:04 AM
The point was to whack that data straight into some external ram, the propeller would not see it at all.

The problem with commercial cameras is not just the cost but the fact that when goverment bodies give money to buy cameras they don't expect you to put them on a stick and move them around at brake neck speed. Plus many of them are just too bulky. Oh and multiple cameras might be prefered.

Graham

BTX
01-31-2007, 03:21 AM
Graham.
It is not cheap but...why don't you think in a FIFO memory instead a RAM, you could get it more easily to do, and faster.



▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Regards.

Alberto.

Phil Pilgrim (PhiPi)
01-31-2007, 03:36 AM
Graham,

Wow! That's a really high data rate and a lot of pixels! Instead of rolling your own acquisition system, why not consider an off-the-shelf high-speed frame-grabber card from either Dalsa (http://www.dalsa.com) or Matrox (http://www.matrox.com)? You could still build the camera using the Micron chip to interface with one of their offerings.

-Phil

Lawson
01-31-2007, 09:59 AM
just a thought. A fixed camera with a galvo might be better able to follow the movements of the insect. If depth of focus is really small a high speed focusing lense could be added. (shifting the camera lense sidways relative to the sensor will also steer the camera axis, but it can cause more distortion.)

I also wonder if anybody makes a 128x128 pixel image sensor with that can do 5000fps? This would cut the data rates by about 10x from the sensor you mentioned up top. The only disadvantage of the smaller sensor I can see is that with the above 1.3Mpix sensor the camera could be electronically "steered" by selecting a different pixel window for each frame.

Wow, good luck with this,
Marty

Graham Stabler
01-31-2007, 05:13 PM
BTX, this is my problem having never used memory except in a PC or in the prop I did not even know FIFO memory existed. But I definately don't need random access so something I could just stuff the data in and get out bit by bit would be much easier. Have you or anyone got any suggestions for sources or devices?

Phil, its quite an array although it is not the fastest, some cameras are doing 4000fps at a megapixel! Using an off the shelf frame grabber is an option and probably a good one. Having said that one thing that will be required is multiple cameras so that the kinematics of the wings can be determined, this will mean multiple cards and possibly worse multiple cables and as you can imagine they are rather thick. But its not ruled out!

Lawson, nice idea on the galvos but I don't see it working at the optical resolution required and the size of flight arena we are looking at. To my mind the tracking part is easy, just use a low res camera and a delta robot, of course ensuring the insect does not mind being followed around by a camera is another thing.

great idea on the steering by row address! In my last job-but-one I was using a region of interest camera by COHU, it was doing 32khz frame rate on 32X32 pixels but the thing would take a fornight to change ROI so I had never even considered it.

thanks all,

Graham

p.s. The camera array is 1300 euro which is not too bad on a research budget, its also 3.3v so will fit well with the prop.

Graham Stabler
01-31-2007, 10:18 PM
Just as an aside the Micron MT9v022 is available from Digikey for $30, it is wide vga, 750X480 but can also be used in window mode and binning modes. In binning mode 4, each pixel output is the averaged value from 16 pixels from 4 rows and 4 columns and it multiples the frame rate by 4 giving 240fps! At a resolution of 187X120! The binning also is good in terms of required light levels and noise!

Its a BGA unfortunately but it does have the truesnap facility but the price is amazing and considering the ADCs are built in you would not need much electronics to make something pretty interesting! Certainly worth looking into Micron's arrays for potential propeller fun!

Graham

Graham Stabler
01-31-2007, 10:54 PM
I just found a better bet, an CLLCC packaged version:

http://www.micron.com/products/partdetail?part=MT9V032C12STM

Just bought two for 16 euros each (from framos) however postage will probably be that much on top. No use for the project but much fun for me!

BTX
02-01-2007, 12:40 AM
Hi Graham.
I was looking for a bit, for FIFO memory, I found these interesting for you, in the following order.

SAA4955 2.9 Mbit field memory easily cascadable. Its in 12 bit organization !!
MSM514223B (256k x 4 bit field memory) you'll need at least three., ans it's cascadable too.
CY7C4291V it's 128k x 9 bit FIFO memory... (you need two in paralel to get 18 bits.... if i'm not wrong you need 10 bits wide) and is also cascadable.

I think that could be not so hard, trying with the SAA4955 from Philips (NXP).
The datasheets of them are all in: www.alldatasheet.com (http://www.alldatasheet.com)
Take· a look.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Regards.

Alberto.

Graham Stabler
02-01-2007, 04:49 AM
Thanks Alberto but apart from the fact they all seem to be discontinued the storage capacity will not be big enough.

If running at 1280X128 and 4000fps then a second's worth of footage would be:

1280X128X4000X10 bits or 6553Mbits

Or 11bit wide and 600Mbit deep if I could find the right memory.

Graham

BTX
02-01-2007, 05:22 AM
Oh...
Seems to be impossible to get, such a large memory capacity (easily).
Correct me, if I am wrong..........and you need to transfer 655_360_000 words of 10 bit / Second ?? Thus 6.5536 Ghz of pixel clock or 655.36 Mhz/word ??
Combining both, capacity and speed you'll create a chip-bomb...http://forums.parallax.com/images/smilies/freaked.gif
If it is correct...sorry I'm out of this discussion.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Regards.

Alberto.

Graham Stabler
02-01-2007, 05:38 AM
No the bit rate per pin is 66Mb/s because the clock is 66Mhz but there are 100 pins in parallel so it is 6600Mb/second bandwidth.

Its confusing, the datasheet gets it wrong even!

Actually things are not as bad. Reducing vertical resolution increases frame rate but reducing horizontal resolution does nothing but reduce the overall bandwidth or more importantly the amount of data that has to be stored.

So if I go for a 4:3 aspect ratio that is 170X128 pixels and ONLY :) 870Mb to store per second of footage or 108Mbytes roughly.

It seems the solution is pretty much available off the shelf:

www.epixinc.com/products/pixci_cl3sd.htm (http://www.epixinc.com/products/pixci_cl3sd.htm)

I am 99% certain the Basler camera has a Micron array and the image capture card is using DIMMs for the storage, they are going to be 64bit wide so they will use two 512MB ones to store a 1GB of image data. The price is actually better than expected though from past experince flexibility will be greatly reduced by the off the shelf solution.

Ethernet is pretty fast these days, could I just serialize the data and send it over the network?

Graham

BTX
02-01-2007, 09:26 AM
Sorry I can not say anything about ethernet, I never try it. (but serializing you must increment the clock speed)
Talking about DIMMs, do you think that you'll need to refresh the RAM ?, it takes a lot of work for the micro.
Perhaps that task is better for a SX48 than a Propeller.
Some in my mind is comming to think in a PCI bridge ? (after look at the Basler card), but I have not experience with that. Sending the captured data to a PC using a Vinculum (like you comment in the 1st post) will take so much time to do it.

So ..perhaps to get the data with a 10Mbit FIFO (as temp buffer, independent RD/WR clock) while using a PCI bridge (plus necessary hard) sending it directly to the PC Hard disk ????
Graham...you're in a serious problem...too difficult project, that's why FPGA, etc, are better for this kind of applications.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Regards.

Alberto.

BTX
02-01-2007, 09:47 AM
Graham.
Did you see the NAND Flash at the micron.com site.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Regards.

Alberto.

Graham Stabler
02-01-2007, 06:17 PM
Yes I have a feeling I may have reached the end of this thought experiment.

The nand flash looks very good but like most memory it will need to be to be controlled at high speed, I then need a way of getting the data out and that means multiplexing and lots of messing around.

What I need doesn't really exist, 100 shift registers in parallel 66Mbits deep, then I could shift them in and later clock them out of the other end.

Graham

BTX
02-01-2007, 09:27 PM
Graham said...
What I need doesn't really exist, 100 shift registers in parallel 66Mbits deep, then I could shift them in and later clock them out of the other end.

·So, I was thinking in FIFO memory, that's the nearly to that, let's look for the latest FIFO in the market, I don't know which is the capacity of them today, but I'm sure that could be so expensive...., many of them in cascade, could be the most simple solution.


▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Regards.

Alberto.

Lawson
02-01-2007, 11:07 PM
One bonus of using a FPGA is that it should be able to include a simple blob-tracker to track the brightest object in the video frame. This would give a nice output to steer the camera's pixel window, and this signal could be pulled out to drive the the Delta robot.

my 2 cents,
Marty

Graham Stabler
02-02-2007, 08:41 AM
The movement tracking could be done with a lower res low speed camera as the movement of the fly in the air is much slower than the movement of its wings.

Graham