FlexProp and SD Card Driver
iseries
Posts: 1,492
Just got done updating the FlexProp SD card driver to use multiple volumes. This allows two SD cards to be accessed at the same time from a C program.
The FatFS library allows multiple volumes and just needed to be turned on as well as update the backend driver to except drive number or volume number to access.
With single volume access the library runs at about 950KB per second.
Now adding the drive number and having to pass this number around to multiple low level functions costed about 250KB per second with a recorded speed of about 620KB per second.
For testing I used a 16Meg SD card and a 8Meg file that I read from this card.
Here is the program I used to run the test:
#include <stdio.h>
#include <fcntl.h>
#include <propeller.h>
#include <sys/vfs.h>
#define PIN_SS 23
#define PIN_MISO 20
#define PIN_CLK 21
#define PIN_MOSI 22
char filename[] = "/ffs/1:BigFileName.bin";
char Buffer[4096];
int main(int argc, char** argv)
{
int fd;
int i;
int total;
int t;
printf("Starting\n");
mount("/ffs", _vfs_open_sd(1, PIN_SS, PIN_CLK, PIN_MOSI, PIN_MISO));
fd = open(filename, O_RDONLY, 0);
if (fd < 0)
{
printf("File Not Found!\n");
while (1)
_waitms(1000);
}
i = 1;
total = 0;
t = _getms();
while (i > 0)
{
i = read(fd, Buffer, 4096);
total++;
}
close(fd);
printf("Total: %d\n", total);
t = _getms() - t;
printf("Time: %d ms\n", t);
while (1)
{
_waitms(1000);
}
}
If speed is important then using a single volume library makes sense.
At this point don't know why you would need to access two different SD cards from the P2.
The library also requires about 100k of memory in addition. There is also a petite library that I have not looked at may be faster and lighter than this library. Also might only support short file names as this also adds some size to the library.
Mike
Comments
You can also run a second instance of FatFs. I was looking into this so could do Plan9 and uSD access at the same time. Also, the second instance could be adapted to flash or eMMC service in the future...
Interesting. I wonder how often people will want to use multiple SD cards? If it's a common use case scenario perhaps we can find a way to avoid that overhead. Actually, I'm curious: isn't the drive number already being passed around internally? I guess it isn't at the very lowest level (sector accesses). But it seems like that shouldn't have to be a bottleneck, given that the pin numbers are already being read from memory.
You can already do Plan9 and uSD access at the same time -- see for example the "shell" sample code in the latest FlexProp release, which mounts both the SD and the host file system to allowing copying between them.
@ersmith thats great news
@ersmith ,
I also looked at the numbers on the FatFs site and see they have better read performance than I'm getting from the P2 code. They're using much slower CPUs's so the problem may be waiting too long to access the SD card?
Also porting the Petit version may also be an option for higher speed and use on the P1?
Mike
@iseries it could be you need 300 MHz for best performance....
There’s a delay set to be safe up to around there. This delay setting could probably be lowered for lower clock speed..
@iseries
I'm generally unhappy with the way file systems are structured right now in the FlexProp languages. Really what would be ideal would be to separate the file system layer from the underlying device layer, so we could for example layer FatFs on top of any other device (SD card, flash, memory or even another FatFs file system). Performance isn't great either, and needs some looking at.
You posted some Fat and sd_mmc driver code earlier which I think is faster than what FlexProp already has. What license did you intend for that to have? Is it something we could incorporate into FlexSpin at some point?
Thanks,
Eric
@iseries
Wow that is quite a big drop in performance for a feature that probably should not normally be a huge bottleneck. I can understand there will be some impact at the higher levels to now track extra per volume state, but transfer rate at the lower level shouldn't be impacted much if the pins can be read/setup at the start of each 512byte sector transfer requested. Is it the case that the higher level component is responsible for so much total execution time/latency? Lots of computation overheads and/or additional directory entry sectors read per real transfer? I've not taken a look at it but perhaps some benchmarking could be done to see where the bottlenecks are and what might be possible to be sped up. Perhaps any sector caching may need to be done per volume for example.
Seems large, so a big chunk of the P2 memory will be consumed which is not ideal. Was this mainly because of LFN support?
@ersmith
It's great you are considering the file system layer could be decoupled from underlying device. If this was done it would be quite easy to put fast filesystems in different devices such as HyperRAM, HyperFlash (initially RO, hopefully eventually also R/W), SPI Flash and SD cards. I'm working on a PSRAM driver right now by altering my HyperRAM one, but once that is done I am also considering extending my memory driver to support SPI flash and SD card transfers in another third variant. This would allow a global external memory space and software interface to access all these devices (though the SD would need to use slightly different addressing probably with sectors instead of bytes to expand it to far more than the 256MB limit per bus). We'll see how far that goes. It does burn a COG for each bus but should let multiple COGs share the devices such as SPI flash+SD on the same bus and handle the pin arbitration. It would be nice for the P2 to be able to easily access SPI Flash, HyperRAM/Flash, and PSRAM all with a common address space, all devices running independently and accessible by any COG as needed using a simple mailbox scheme per bus/driver COG. That's the general plan anyway.
@ersmith ,
Eric, the code is free to use and just an example of what can be done. I was surprised that my code was faster than the bit bagging that you used. I was also surprised that my assembly code which I thought was good got better because the compiled version used REP instruction instead of DJNZ.
It should be possible to rewrite the DISKIO and SDMMC layer to adapt it to Flash memory. This would make it possible to use other devices and still use the same POSIX interface to access them.
Also, the way it is set up we could use other VFS commands to access them.
I did do some speed testing of FlexProp code compared to mine and at first, FlexProp code was faster. I then changed some timing code that was different from FlexProp code and was able to get the code to run 3 times faster. Went from 58 seconds to 9 seconds to read 8Meg. Still, this is slower than the bench marks they showed.
Let me post both versions of the code and maybe we can make it work better.
Mike
Here are the two drivers, sdx for single drive, and sdm for multi drive.
To use them you need to add these lines to vfs.h
struct vfs *_vfs_open_sd(int cs=60, int clk=61, int mosi=59, int miso=58) _IMPL("filesys/fatfsx/filesystem.c"); struct vfs *_vfs_open_sdm(int drv=0, int cs=60, int clk=61, int mosi=59, int miso=58) _IMPL("filesys/fatfsm/filesystem.c");Test program here:
#include <stdio.h> #include <fcntl.h> #include <propeller.h> #include <sys/vfs.h> #define PIN_SS 23 #define PIN_MISO 20 #define PIN_CLK 21 #define PIN_MOSI 22 char filename[] = "/ffs/BigFileName.bin"; char Buffer[4096]; struct vfs *fs; int main(int argc, char** argv) { int fd; int i; int total; int t; printf("Starting\n"); //fs = _vfs_open_sdm(0, PIN_SS, PIN_CLK, PIN_MOSI, PIN_MISO); fs = _vfs_open_sd(PIN_SS, PIN_CLK, PIN_MOSI, PIN_MISO); mount("/ffs", fs); fd = open(filename, O_RDONLY, 0); if (fd < 0) { printf("File Not Found!\n"); while (1) _waitms(1000); } i = 1; total = 0; t = _getms(); while (i > 0) { i = read(fd, Buffer, 4096); total++; } close(fd); printf("Total: %d\n", total); t = _getms() - t; printf("Time: %d ms\n", t); while (1) { _waitms(1000); } }BigFileName is just a binary file with 0x00 through 0xff write to 256 bytes by 32,768. (8,388,608).
Mike
@iseries : Thanks, Mike!