Prop with high speed camera array??
Graham Stabler
Posts: 2,507
I've just started a new job where I am working on design of insect inspired hovering micro air vehicles. Something I have been asked to look at is in-flight tracking of the insects along with high speed imaging. What is needed is a high speed camera that can be moved at high speeds and does not break the bank (and they can).
Cypress do a sensor array that works at 250fps for about $250, my hope is that this array from micron that works at 500fps full frame will not cost too many orders of magnitude more. Even if it does if I can create or at least sketch out a low cost electronic back end for it then we might be in business.
This is the chip:
http://www.micron.com/products/partdetail?part=MT9M413C36STM
It uses a TrueSNAP shutter, which is basically just like Phil's propcam has, that means it exposes all the pixels simultaneously and captures a true freeze frame. But unlike Phil's camera it does 1.3Mpix at 500fps. By reducing the number of rows used the speed can be increased greatly for example 100 rows gives 5000fps. Typically for a hoverfly that is the sort of frame rate required to give about 20 frames per flap.
The chip has a 10X10bit output port so it can read out 10 pixels simultaneously, the control of exposure and reading in and out is done by an external FPGA/CPLD/ASIC but I was wondering if the prop could cut it??
The second problem is what to do with the data, I would be happy if it could be written to some RAM and then later transfered to a mass storage device. But I don't know anything about using RAM so I'd really appreciate some advice there. In terms of transfer to a mass storage device the Vinculum chip could do that. I should note that typically 1 second of capture would be more than enough (that's about 200 flaps of a hoverfly).
Really just asking for a reality check and some ideas.
Graham
Cypress do a sensor array that works at 250fps for about $250, my hope is that this array from micron that works at 500fps full frame will not cost too many orders of magnitude more. Even if it does if I can create or at least sketch out a low cost electronic back end for it then we might be in business.
This is the chip:
http://www.micron.com/products/partdetail?part=MT9M413C36STM
It uses a TrueSNAP shutter, which is basically just like Phil's propcam has, that means it exposes all the pixels simultaneously and captures a true freeze frame. But unlike Phil's camera it does 1.3Mpix at 500fps. By reducing the number of rows used the speed can be increased greatly for example 100 rows gives 5000fps. Typically for a hoverfly that is the sort of frame rate required to give about 20 frames per flap.
The chip has a 10X10bit output port so it can read out 10 pixels simultaneously, the control of exposure and reading in and out is done by an external FPGA/CPLD/ASIC but I was wondering if the prop could cut it??
The second problem is what to do with the data, I would be happy if it could be written to some RAM and then later transfered to a mass storage device. But I don't know anything about using RAM so I'd really appreciate some advice there. In terms of transfer to a mass storage device the Vinculum chip could do that. I should note that typically 1 second of capture would be more than enough (that's about 200 flaps of a hoverfly).
Really just asking for a reality check and some ideas.
Graham
Comments
I'd add up what this project is going to cost·in·lost time and material costs.· (figure at least one revision of the·electronics and several·fried chips)· I bet that once all of that is added up the cost of a commercial high-speed camera is not as extreeme.
Good luck,
Marty·
The problem with commercial cameras is not just the cost but the fact that when goverment bodies give money to buy cameras they don't expect you to put them on a stick and move them around at brake neck speed. Plus many of them are just too bulky. Oh and multiple cameras might be prefered.
Graham
It is not cheap but...why don't you think in a FIFO memory instead a RAM, you could get it more easily to do, and faster.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Regards.
Alberto.
Wow! That's a really high data rate and a lot of pixels! Instead of rolling your own acquisition system, why not consider an off-the-shelf high-speed frame-grabber card from either Dalsa or Matrox? You could still build the camera using the Micron chip to interface with one of their offerings.
-Phil
I also wonder if anybody makes a 128x128 pixel image sensor with that can do 5000fps? This would cut the data rates by about 10x from the sensor you mentioned up top. The only disadvantage of the smaller sensor I can see is that with the above 1.3Mpix sensor the camera could be electronically "steered" by selecting a different pixel window for each frame.
Wow, good luck with this,
Marty
Phil, its quite an array although it is not the fastest, some cameras are doing 4000fps at a megapixel! Using an off the shelf frame grabber is an option and probably a good one. Having said that one thing that will be required is multiple cameras so that the kinematics of the wings can be determined, this will mean multiple cards and possibly worse multiple cables and as you can imagine they are rather thick. But its not ruled out!
Lawson, nice idea on the galvos but I don't see it working at the optical resolution required and the size of flight arena we are looking at. To my mind the tracking part is easy, just use a low res camera and a delta robot, of course ensuring the insect does not mind being followed around by a camera is another thing.
great idea on the steering by row address! In my last job-but-one I was using a region of interest camera by COHU, it was doing 32khz frame rate on 32X32 pixels but the thing would take a fornight to change ROI so I had never even considered it.
thanks all,
Graham
p.s. The camera array is 1300 euro which is not too bad on a research budget, its also 3.3v so will fit well with the prop.
Its a BGA unfortunately but it does have the truesnap facility but the price is amazing and considering the ADCs are built in you would not need much electronics to make something pretty interesting! Certainly worth looking into Micron's arrays for potential propeller fun!
Graham
http://www.micron.com/products/partdetail?part=MT9V032C12STM
Just bought two for 16 euros each (from framos) however postage will probably be that much on top. No use for the project but much fun for me!
I was looking for a bit, for FIFO memory, I found these interesting for you, in the following order.
SAA4955 2.9 Mbit field memory easily cascadable. Its in 12 bit organization !!
MSM514223B (256k x 4 bit field memory) you'll need at least three., ans it's cascadable too.
CY7C4291V it's 128k x 9 bit FIFO memory... (you need two in paralel to get 18 bits.... if i'm not wrong you need 10 bits wide) and is also cascadable.
I think that could be not so hard, trying with the SAA4955 from Philips (NXP).
The datasheets of them are all in: www.alldatasheet.com
Take· a look.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Regards.
Alberto.
If running at 1280X128 and 4000fps then a second's worth of footage would be:
1280X128X4000X10 bits or 6553Mbits
Or 11bit wide and 600Mbit deep if I could find the right memory.
Graham
Seems to be impossible to get, such a large memory capacity (easily).
Correct me, if I am wrong..........and you need to transfer 655_360_000 words of 10 bit / Second ?? Thus 6.5536 Ghz of pixel clock or 655.36 Mhz/word ??
Combining both, capacity and speed you'll create a chip-bomb...
If it is correct...sorry I'm out of this discussion.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Regards.
Alberto.
Its confusing, the datasheet gets it wrong even!
Actually things are not as bad. Reducing vertical resolution increases frame rate but reducing horizontal resolution does nothing but reduce the overall bandwidth or more importantly the amount of data that has to be stored.
So if I go for a 4:3 aspect ratio that is 170X128 pixels and ONLY [noparse]:)[/noparse] 870Mb to store per second of footage or 108Mbytes roughly.
It seems the solution is pretty much available off the shelf:
www.epixinc.com/products/pixci_cl3sd.htm
I am 99% certain the Basler camera has a Micron array and the image capture card is using DIMMs for the storage, they are going to be 64bit wide so they will use two 512MB ones to store a 1GB of image data. The price is actually better than expected though from past experince flexibility will be greatly reduced by the off the shelf solution.
Ethernet is pretty fast these days, could I just serialize the data and send it over the network?
Graham
Talking about DIMMs, do you think that you'll need to refresh the RAM ?, it takes a lot of work for the micro.
Perhaps that task is better for a SX48 than a Propeller.
Some in my mind is comming to think in a PCI bridge ? (after look at the Basler card), but I have not experience with that. Sending the captured data to a PC using a Vinculum (like you comment in the 1st post) will take so much time to do it.
So ..perhaps to get the data with a 10Mbit FIFO (as temp buffer, independent RD/WR clock) while using a PCI bridge (plus necessary hard) sending it directly to the PC Hard disk ????
Graham...you're in a serious problem...too difficult project, that's why FPGA, etc, are better for this kind of applications.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Regards.
Alberto.
Did you see the NAND Flash at the micron.com site.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Regards.
Alberto.
The nand flash looks very good but like most memory it will need to be to be controlled at high speed, I then need a way of getting the data out and that means multiplexing and lots of messing around.
What I need doesn't really exist, 100 shift registers in parallel 66Mbits deep, then I could shift them in and later clock them out of the other end.
Graham
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Regards.
Alberto.
my 2 cents,
Marty
Graham