File system driver of choice?

ErNa

With the dual propeller quickstart compatible cpu I will have a micro-sd card and the question arises: what is the preferred driver. Priority: make use of large capacity cards and 4 bit (I think, that is called quad speed). Long filenames are preferred, code compactness should be a minor issue, as beginners propeller capacity has doubled ;-)
In the future all my applications will use this driver to reduce varieties.

Cluso99

Erna,
I use Kye's FATEngine as it covers the SDHC cards and FAT16+32.
Long filenames are not supported.
Why don't you take a look at my PropOS (link in signature)? I am working on making the pasm driver stay-resident, and getting Michael Parks' sphinx compiler working too.

BTW I don't know of any drivers that use the 4bit SD interface.

Peter Jakacki

LFN is a kludge of a kludge and besides the long names require big buffers which is incongruent with the Prop chip and most embedded applications. Even cameras only use 8.3 names. As for FAT16 it is now totally obsolete since it's hard to buy less than 4GB cards and even those are getting rare.

That's why my FS supports FAT32 with 8.3 names, paths, and multiple cards and open files along with 32-bit virtual memory addressing of files and cards.
While most embedded FSs don't need to be fast some of them are downright slow which isn't a problem for casual access but most of my applications need to be able to write large volumes of data as quick as it comes since we don't have the luxury of gigabytes of memory, or even just megabytes.

The reason we all use simple SPI mode rather than 4-bit SD mode is that the SD mode requires $$$$$ licensing and the SD mode documentation is not released publicly either otherwise it wouldn't be "secure digital".

DavidZemon

Well, here's a vote for PropWare's suite of filesystem classes. Easy, flexible, supports sdhc, fat 16/32, hooks into PropWare:: Printer and PropWare::Scanner for easy and consistent formatted printing/reading across lots of different I/O devices. Allows for multiple files opened simultaneously with a choice between a single shared buffer, unique buffers per file, or any combination.

Support for long filenames hasn't been started, but I could accelerate that feature if you're interested. Files in non-root folders is one of the main features I aim to complete before a v2.1 release, along with multiple partitions.

A rarely used feature is that the spi bus can be shared with other devices. There's no lock implementation, but it is at least accessible so you can share the three main pins with other devices of you so choose. If you choose not to, creating a second instance of PropWare::SPI is not difficult.

Not as fast as fsrw. I know how to increase the speed significantly (in the spi driver), but I don't currently have plans to implement the read ahead/write behind feature of fsrw, so that does put a cap on speed.

I'm not sure what you mean by quad speed. I've heard of quad spi with relation to sram chips, but not with sd cards. If your referring to the SD protocol - as opposed to SPI - I believe that requires licensing fees.

DavidZemon

Oh, and PropWare's filesystem classes don't require a dedicated cog. I know you're not tight on cogs but I thought it was a cool feature with mentioning

Cluso99

PropOS is actually a DOS like Operating System that physically runs on a propeller chip.

These are the supported commands...

*** Cluso's Propeller SD card Operating System v0.94 ***
-------------------------------------------------------------------------------
clear                                     ' Clear screen
cmd     ---                               ' Internal use (command interpreter)
copy    <source_file> <destination_file>  ' Copy a file
del     <file>                            ' Delete a file
diff    <file1> <file2>                   ' Display file differences
dir     [mask]                            ' Display directory of files
dircpm  <cpm_disk>                        ' Display directory of files on CPM
dnload                                    ' Download code Ctl-F10/F11 w/o reset
echo    <text>                            ' Echo the line of text
free                                      ' Display FAT16/32 used/free space
getcpm  <cpm_disk> <cpm_file> <fat_file> -T/B ' Get (copy) CPM file to FAT16/32
getfat  <fat_file>                        ' Get (copy) FAT16/32 file to PC
help                                      ' Display help info
hubdump <addr> <lines>                    ' Dump hub memory addr(hex) n lines
lf                                        ' Toggles LF on/off
ls      [mask] [A]                        ' Display directory of files
mapcpm  <cpm_disk> [<cpm_file> [-d]]      ' Maps info about the CPM filesystem
os      ---                               ' PropOS binary file (runs PropOS)
program <source_file> [{-WL}{-WU}]        ' Program to EEPROM L=Lower/U=Upper
putcpm  <fat_file> <cpm_disk> <cpm_file>  ' Put (copy) FAT16/32 file to CPM
putfat  <fat_file>                        ' Put (copy) FAT16/32 file from PC
reboot                                    ' Reboot the Propeller Chip
ren     <source_file> <destination_file>  ' Rename a file
run     <file>                            ' Run a ".BIN" file (kills PropOS)
testsd                                    ' Tests the SD card (writes to SD)
type    <file> [-Hn]                      ' Display the contents of a file
used                                      ' Display FAT16/32 used/free space 
ver                                       ' Display info about PropOS 
-------------------------------------------------------------------------------

Other OS commands can be added simply to the OS, utilising the StdIn and StdOut routines which provide a HAL.

Binary files can be run from the OS (replaces the OS).

I am well on the way to supporting self-hosted (on the propeller) spin & pasm compiling using a modified Sphinx (by Michael Park).

The PASM section (the SPI driver and basic read/write SD routines) of Kye's FATEngine resides in its own cog.

David Betz

DavidZemon wrote: »

Oh, and PropWare's filesystem classes don't require a dedicated cog. I know you're not tight on cogs but I thought it was a cool feature with mentioning

I find it amusing that people taut their no-COG implementations of various drivers as an advantage. I think Peter's Tachyon system runs in a single COG as well. I thought we all liked multi-core processors? Why are we so anxious to avoid using COGs? I suppose it is so we can leave them to jobs that can only be handled by a separate processor but sometimes some of the applications seem like they could be run on an ARM equally well as a Propeller. :-)

Peter Jakacki

One of the problems with running in another cog is that the main cog no longer has immediate and direct control. Instead it must pass commands and parameters and thus this introduces extra delays although there are advantages as well which could include writing a buffer in the background. However it is easy for tachyon to access the SD directly rather then dedicating a whole cog to this task although the code is structured so that it could use a cog if so desired. Easyfile as it is called is also very fast even though it uses bit bashed spi whereas fsrw can use the counter yet is still slower because of the Spin overhead.

DavidZemon

Peter Jakacki wrote: »

One of the problems with running in another cog is that the main cog no longer has immediate and direct control. Instead it must pass commands and parameters and thus this introduces extra delays although there are advantages as well which could include writing a buffer in the background. However it is easy for tachyon to access the SD directly rather then dedicating a whole cog to this task although the code is structured so that it could use a cog if so desired. Easyfile as it is called is also very fast even though it uses bit bashed spi whereas fsrw can use the counter yet is still slower because of the Spin overhead.

I know some part of tachyon doesn't use fat, but the blocks directly. Is that Easyfile or something else? Removing the fat logic would certainly make things faster :P

DavidZemon

David Betz wrote: »

DavidZemon wrote: »

Oh, and PropWare's filesystem classes don't require a dedicated cog. I know you're not tight on cogs but I thought it was a cool feature with mentioning

I find it amusing that people taut their no-COG implementations of various drivers as an advantage. I think Peter's Tachyon system runs in a single COG as well. I thought we all liked multi-core processors? Why are we so anxious to avoid using COGs? I suppose it is so we can leave them to jobs that can only be handled by a separate processor but sometimes some of the applications seem like they could be run on an ARM equally well as a Propeller. :-)

The last time you brought this up, it sunk in very deep. I have plans for PropWare v3.x to be a significant overhaul, and that will include the ability to easily choose between a parallel or sequential approach for drivers. I think it is important for libraries to allow or even encourage parallelism, but never force it.

If I'm feeling up to it, I may even take the assembly driver from fsrw and wrap it with a PropWare-compatible class so that you get the speed of fsrw's read ahead/write behind routines and all the other features of PropWare's higher level filesystem classes.

kwinn

David Betz wrote: »

DavidZemon wrote: »

Oh, and PropWare's filesystem classes don't require a dedicated cog. I know you're not tight on cogs but I thought it was a cool feature with mentioning

I find it amusing that people taut their no-COG implementations of various drivers as an advantage. I think Peter's Tachyon system runs in a single COG as well. I thought we all liked multi-core processors? Why are we so anxious to avoid using COGs? I suppose it is so we can leave them to jobs that can only be handled by a separate processor but sometimes some of the applications seem like they could be run on an ARM equally well as a Propeller. :-)

Not "wasting" cogs seems to be a similar psychology to the "not wasting" computer cycles back in the time sharing days. Understandable if your business was renting out computer time, but not so much when the computer replaced a number of employees in the business at a much lower cost.

I recall having that discussion with a very ocd manager, and pointed out that 100% utilization is not needed to make something worthwhile. After all we have millions of cars that are parked far more often than they are driven and they are still considered very useful or even essential.

ErNa

I have to analyse these answers and understand the implications. In doing so I will revise some of the cmaps I have created and move them to a new server, we manage on our own. This is a link to the main entry page: https://submesh.de:8080/rid=1PV5B78DS-1CL7WHH-1PFP/PropellerRoot.cmap As over the last decade a lot has changed, the links are currently not working, I will update them continuously

potatohead

I recall having that discussion with a very ocd manager, and pointed out that 100% utilization is not needed to make something worthwhile

Had a few of those too. And when it was my turn, I chose to remember the outcome of that conversation. Well said.

As for the COGS, I prefer stuffing something into a COG, until it's a problem. Then, it's about optimizing the use of everything. There is no "wasting" a COG. The things are there, used or not, and that can be optimized, and it might not need to be.

David Betz

DavidZemon wrote: »

David Betz wrote: »

DavidZemon wrote: »

Oh, and PropWare's filesystem classes don't require a dedicated cog. I know you're not tight on cogs but I thought it was a cool feature with mentioning

I find it amusing that people taut their no-COG implementations of various drivers as an advantage. I think Peter's Tachyon system runs in a single COG as well. I thought we all liked multi-core processors? Why are we so anxious to avoid using COGs? I suppose it is so we can leave them to jobs that can only be handled by a separate processor but sometimes some of the applications seem like they could be run on an ARM equally well as a Propeller. :-)

The last time you brought this up, it sunk in very deep.

Oops! Sorry for repeating myself. Anyway, it isn't really a criticism. What I'm really looking for is real parallel uses of the COGs rather than just offloading sequential code. The read-ahead/write-behind support certainly qualifies.

DavidZemon

David Betz wrote: »

What I'm really looking for is real parallel uses of the COGs rather than just offloading sequential code.

I'm very much in agreement. Most of PropWare is not designed with parallelism in mind (with notable exceptions being PropWare::Runnable and PropWare::SynchronizedPrinter). My first goal with 3.x will be to separate the transmit/receive routines into different classes and then offer wrappers with buffers for async communication. For instance, SPI will become SPITransmit and SPIRecieve. SPITransmit will have a sister class such as AsyncSPITransmit, which invokes an SPITransmit instance in a new cog and communicates asynchronously via a buffer. SPIReceive will not have an async sister class. The new UART classes will be split up similarly, but have async sister classes for both transmit and receive.

I'm very excited to finish up v2.1 so I can start these massive changes on v3.x

ErNa

I have to make a decision. To start, I will use Cluso99's Prop-OS and try to run it on Raymans PSB Board, I have available. After gaining such experience, I will have a try with the new board. The goal I have is to create a system that runs different services where any client, using such a service subscribes to it without knowing where the service resides. Quality of service must be given and every service will, when acknowledging an request, provide information about what it is able to deliver. Somehow utopic, foggy, but I'm full of hope, this will work in the end. Now, with all the technology we have. Maybe forth is a language to drive such a system, but up to now, I do not see how all this works together. OCCAM, I miss you so!

David Betz

ErNa wrote: »

I have to make a decision. To start, I will use Cluso99's Prop-OS and try to run it on Raymans PSB Board, I have available. After gaining such experience, I will have a try with the new board. The goal I have is to create a system that runs different services where any client, using such a service subscribes to it without knowing where the service resides. Quality of service must be given and every service will, when acknowledging an request, provide information about what it is able to deliver. Somehow utopic, foggy, but I'm full of hope, this will work in the end. Now, with all the technology we have. Maybe forth is a language to drive such a system, but up to now, I do not see how all this works together. OCCOM, I miss you so!

This sounds very exciting! I'll be interested to hear of your progress.

DavidZemon

ErNa wrote: »

The goal I have is to create a system that runs different services where any client, using such a service subscribes to it without knowing where the service resides. Quality of service must be given and every service will, when acknowledging an request, provide information about what it is able to deliver.

Wow! Sounds very enterprisy. This will be very cool when complete.

ErNa

Back to the roots! I gave Clusos Prop-OS a try (first on Raymans PSB) and: It works, thanks to PidZero's perfect guide. Just to throw in the right pins, and: here we go! Now I have to do the same procedure on the new hardware, that means: The Bottom Propeller runs the OS, the top propeller first has to route through the PST and later can work as a terminal itself. Very nice, indeed.