How is this project coming? Is there anything I can do to help?
Jonathan
Also watching this.
It has been briefly mentioned already, but you need to be ready for that odd
time when the card will suddenly be *very* slow (usually for a write) for a
single block. This does happen periodically (when the card decides it
needs to erase a new page, or do some housekeeping).
So even with all the magic available, at some point the card *can* just
stall, for a good fraction of a second.
There is a lot of intelligence inside the card to manage wear leveling and
the like, and that does show up as occasional delays.
But let us know how it is going; we're all keen to hear.
Sorry I haven't updated this lately. We've been traveling for the holidays and although I brought a small Prop platform I haven't had any time to work on this. I've had a hard time even keeping up with the forum. I'm back home now and hope to find time to work on this. I'll post any results I get.
Tom, we've had similar discussions before and I recall needing about 1/3 of a second buffer to handle those occasions when the card just went off to do its own thing. If that's the case, I have no chance of buffering things in the prop. If I find the same things now (I can't remember if I was using the FAT layer at that time or not), this discussion will turn to external memory solutions.
I had a chance to return to this problem. I switched SD cards. The first was a 4GB SanDisk micro SDHC class 4 card. Data presented so far have been from that card. The graphs below were created using a 2GB PNY MicroSD. There are some other numbers on the bottom of the card, but clearly no SDHC and no class rating. This smaller "off-brand" card has much better read performance. In fact, the read is much faster than the write.
In the attached plots, each point is represents the max, min, average of 10,000 512-byte blocks. The amount of data I need to store is 30 seconds at a 160KBytes/sec which works out a bit less than 5.12Mbytes over 30 seconds. The code used to produce these plots, starts at block zero of the card. It writes one 512-byte block, releases the card then reads one 512-byte block and releases the card, repeating this 10,000 times. For each individual 512-byte block, I time how long it takes to both read and write this block and capture the maximum and minimum times over these 10,000 blocks and print out the results. I then started at block 10,000 and do this again, etc until I had tested nearly the entire card. Here's a sample of the output. The times listed below are microseconds except for the "time(ms)" column which is clearly in milliseconds:
Main Cog: 0
Serial ports running in cog: 1
Mount Command Returned: 2
Cardsize: 2
Blocks to write: 4193280
Blocks in buffer: 10000
Expected Iterations: 419
Running speed tests.
start time(ms) sumWrite avgWrite maxWrite maxWLoc minWrite minWLoc sumRead avgRead maxRead maxRLoc minRead minRLoc
039173255613302556157436815225064105072191050117616908141000039252256383232563121071478322501008010511105105111761000190816494200003958425971361259737793724576218924607105107141051117620001908245903000038786251737392517111193791621893279910511094105111763000090832846400003873825127028251211217406262189409911050973010501176400009084099150000387462513446725131114456495218957375105101111051117650016908513276000038740251278772512112786513921896556710510224105111766000090861535700003875625143314251410997733962189737911051094710511176700009087023980000387552514383925141133981137218981951105090381050117680001908819519000038764251529242515112539752221899014310508984105011769000090890111100000389152530330525301105810526322011000631051008410511176100001908102623
The plots make things a bit easier to see over the entire capture. Keep in mind the y-scale is logarithmic. One plot is for the write measurements; the other for read measurements. In both cases blue is total time for 10,000 operations, red is the max single time, green is the min single time, and yellow is the average (total/10,000). A few things stood out to me. First the reads are faster and more stable than the writes. This is the opposite of my 4GB SanDisk card. The range between max and min on the read times are very small compared to the same measurement on the writes. The min and avg write times are stable at about 3x the read time. However the max write is more than 100 times as long as the max read. Because the average is so close to the minimum it appears that the max value only happens once. In fact it appears to me that this one max value is FAR outside the normal, but it does happen at least once every 10,000 blocks, which is all I need to know.
I hope to run this same program on my 4GB SanDisk card. It will probably be tonight because it takes hours to run. Each of these 10,000 block read write tests takes ~40 seconds. Doing several hundred in the 2GB test took ~4 hours. Moving to a 4GB card will presumably double that. Then again, maybe not. Clearly there's a big difference between cards.
Thoughts, observations, and suggestions are welcome and appreciated.
Peter
Just a thought... I had trouble with my 3D camera cutting out after 20 seconds of video...
Did some googling and found out that I needed a class 10 card to be fast enough for the camera.
So, I'd recommend getting a class 10 card. They're fairly cheap on Amazon...
BTW: I did upgrade my camera to a class 10 and it fixed the problem. So, I think there's something to it.
So, I'd recommend getting a class 10 card. They're fairly cheap on Amazon...
Hi Rayman- I need to get a few different cards and see how the performance varies. I'll probably try a range of classes and a range of sizes. But I still think this is headed toward an external memory solution.
p
Comments
How is this project coming? Is there anything I can do to help?
Jonathan
Also watching this.
It has been briefly mentioned already, but you need to be ready for that odd
time when the card will suddenly be *very* slow (usually for a write) for a
single block. This does happen periodically (when the card decides it
needs to erase a new page, or do some housekeeping).
So even with all the magic available, at some point the card *can* just
stall, for a good fraction of a second.
There is a lot of intelligence inside the card to manage wear leveling and
the like, and that does show up as occasional delays.
But let us know how it is going; we're all keen to hear.
-tom
Sorry I haven't updated this lately. We've been traveling for the holidays and although I brought a small Prop platform I haven't had any time to work on this. I've had a hard time even keeping up with the forum. I'm back home now and hope to find time to work on this. I'll post any results I get.
Tom, we've had similar discussions before and I recall needing about 1/3 of a second buffer to handle those occasions when the card just went off to do its own thing. If that's the case, I have no chance of buffering things in the prop. If I find the same things now (I can't remember if I was using the FAT layer at that time or not), this discussion will turn to external memory solutions.
Thanks.
Peter
I had a chance to return to this problem. I switched SD cards. The first was a 4GB SanDisk micro SDHC class 4 card. Data presented so far have been from that card. The graphs below were created using a 2GB PNY MicroSD. There are some other numbers on the bottom of the card, but clearly no SDHC and no class rating. This smaller "off-brand" card has much better read performance. In fact, the read is much faster than the write.
In the attached plots, each point is represents the max, min, average of 10,000 512-byte blocks. The amount of data I need to store is 30 seconds at a 160KBytes/sec which works out a bit less than 5.12Mbytes over 30 seconds. The code used to produce these plots, starts at block zero of the card. It writes one 512-byte block, releases the card then reads one 512-byte block and releases the card, repeating this 10,000 times. For each individual 512-byte block, I time how long it takes to both read and write this block and capture the maximum and minimum times over these 10,000 blocks and print out the results. I then started at block 10,000 and do this again, etc until I had tested nearly the entire card. Here's a sample of the output. The times listed below are microseconds except for the "time(ms)" column which is clearly in milliseconds:
Main Cog: 0 Serial ports running in cog: 1 Mount Command Returned: 2 Cardsize: 2 Blocks to write: 4193280 Blocks in buffer: 10000 Expected Iterations: 419 Running speed tests. start time(ms) sumWrite avgWrite maxWrite maxWLoc minWrite minWLoc sumRead avgRead maxRead maxRLoc minRead minRLoc 0 39173 25561330 2556 15743 6815 2250 64 10507219 1050 1176 16 908 14 10000 39252 25638323 2563 12107 14783 2250 10080 10511105 1051 1176 10001 908 16494 20000 39584 25971361 2597 377937 24576 2189 24607 10510714 1051 1176 20001 908 24590 30000 38786 25173739 2517 11119 37916 2189 32799 10511094 1051 1176 30000 908 32846 40000 38738 25127028 2512 11217 40626 2189 40991 10509730 1050 1176 40000 908 40991 50000 38746 25134467 2513 11144 56495 2189 57375 10510111 1051 1176 50016 908 51327 60000 38740 25127877 2512 11278 65139 2189 65567 10510224 1051 1176 60000 908 61535 70000 38756 25143314 2514 10997 73396 2189 73791 10510947 1051 1176 70000 908 70239 80000 38755 25143839 2514 11339 81137 2189 81951 10509038 1050 1176 80001 908 81951 90000 38764 25152924 2515 11253 97522 2189 90143 10508984 1050 1176 90000 908 90111 100000 38915 25303305 2530 11058 105263 2201 100063 10510084 1051 1176 100001 908 102623The plots make things a bit easier to see over the entire capture. Keep in mind the y-scale is logarithmic. One plot is for the write measurements; the other for read measurements. In both cases blue is total time for 10,000 operations, red is the max single time, green is the min single time, and yellow is the average (total/10,000). A few things stood out to me. First the reads are faster and more stable than the writes. This is the opposite of my 4GB SanDisk card. The range between max and min on the read times are very small compared to the same measurement on the writes. The min and avg write times are stable at about 3x the read time. However the max write is more than 100 times as long as the max read. Because the average is so close to the minimum it appears that the max value only happens once. In fact it appears to me that this one max value is FAR outside the normal, but it does happen at least once every 10,000 blocks, which is all I need to know.
I hope to run this same program on my 4GB SanDisk card. It will probably be tonight because it takes hours to run. Each of these 10,000 block read write tests takes ~40 seconds. Doing several hundred in the 2GB test took ~4 hours. Moving to a 4GB card will presumably double that. Then again, maybe not. Clearly there's a big difference between cards.
Thoughts, observations, and suggestions are welcome and appreciated.
Peter
Did some googling and found out that I needed a class 10 card to be fast enough for the camera.
So, I'd recommend getting a class 10 card. They're fairly cheap on Amazon...
BTW: I did upgrade my camera to a class 10 and it fixed the problem. So, I think there's something to it.
Hi Rayman- I need to get a few different cards and see how the performance varies. I'll probably try a range of classes and a range of sizes. But I still think this is headed toward an external memory solution.
p