Shop OBEX P1 Docs P2 Docs Learn Events
Use FSRW on flash? — Parallax Forums

Use FSRW on flash?

RaymanRayman Posts: 14,646
edited 2020-04-28 18:12 in Propeller 2
I'm thinking about trying to use FSRW on the flash chip on the P2 eval board...
I have no idea if it will actually work though.

I think Chip suggested that we can use addresses above 1 kB freely.

So, I'm going to see about copying the contents of a newly formatted uSD card onto flash and then adapting FSRW to work with it...

Anybody know if this will or won't work?

Comments

  • ElectrodudeElectrodude Posts: 1,657
    edited 2020-04-28 18:56
    FAT is bad for flash memory because it doesn't do any wear-leveling. Also, since the flash chip isn't easily removable, you can't just reformat it with a computer if you turn off power at the wrong time. (I'd argue that use of ancient filesystems such as FAT, if it's actually used as a writable filesystem, and not just as a read-only or modify-in-place thing, is always totally inappropriate for use with something like the Propeller, because it can easily get corrupted if power is lost at the wrong time.)

    A better alternative would be littlefs, which Tubular also suggested a few months ago. It's is a simple filesystem designed for constrained systems using flash memory. It does wear leveling, and it never causes filesystem corruption if power is lost at the wrong time.
  • JonnyMacJonnyMac Posts: 9,104
    edited 2020-04-28 19:41
    It would be nice to have a generic file system object that would work with and open and use files on an SD card, and also on the flash -- even if the flash was limited to a single file.
  • pmrobertpmrobert Posts: 673
    JonnyMac wrote: »
    It would be nice to have a generic file system object that would work with and open and use files on an SD card, and also on the flash -- even if the flash was limited to a single file.

    YES. That would be extremely useful.
  • RaymanRayman Posts: 14,646
    I just made this low level processor of SD sector info.
    Think I can use info from this to format flash in a way that makes sense...

    zip
    zip
    fsrw_test4.zip
    18K
  • Ahle2Ahle2 Posts: 1,179
    edited 2020-04-29 08:06
    I vote for littleFS as as the new standard for the Propeller. FAT32 is an old, slow limited and not failsafe FS that only lives on because it has become a standard. I have already corrupted my SD card on the P2 numerous times using fsrw (which is a fantastic FAT32 implementation, no shadow there) because of hangings and power losses. Rayman, you seem to have the time and energy of 3 normal people, why not use it to make an implementation of littleFS för the P2? ;)
    I would use it in a heartbeat!
  • roglohrogloh Posts: 5,787
    It would be great to try out a LittleFs implementation with the P2. Looks like the implementation of it is in C (C99). I wonder if a standalone self contained common driver object / COG could somehow be generated that would then be shareable in different environments/languages such as from Chip's SPIN2, PASM, Fastspin, C, etc, rather than just one tool like Fastspin (which is great don't get me wrong). I guess one issue is some people may not want to dedicate a self-contained COG just for managing a flash file system, and would prefer just calling it inline from C.
  • RaymanRayman Posts: 14,646
    Just thinking through the math here...
    The flash is 128 Mbit or 16,777,216 bytes.

    This gives 32,768 sectors of the standard 512 bytes each.
    But, we need to reserve 1 MB (or so) for boot from flash data. That is 2,000 sectors.
    We are left with 30,768 sectors.

    Looks like FAT16 with 4 sectors per cluster makes most sense. Then, we have 7,692 clusters.
  • RaymanRayman Posts: 14,646
    It works!
    At least, my first test works... Here's the output: 
    Attempting to Mount SD.
SD Mounted.
SD Root Directory:
TEST.TXT
End of SD Root Directory.
Filesystem= 0
First Partition at 00000085
Filesystem = 1
Sector Size [B] = 512
Sectors Per Cluster = 4
Cluster Size [B]= 2048
#Reserved sectors= 6
# FATs = 2
>total # sectors = 30843
FAT#1 sector start = 139
#rootentries = 512
sectors/fat = 237
FAT#2 sector = 376
root dir sector = 613
root dir end sector = 645
data region start sector= 637
n
Tn
Tn
Tn
Tn
TEST    TXT
filesize= 46
data first sector = 657

Trying to mount, result = 0
Trying to open file test.txt, result = 0
filesize = 46
file content =T h i s   i s   a   t e s t   f i l e   f o r   F S R W   o n   P 2   b o o t   f l a s h .
  • roglohrogloh Posts: 5,787
    Good stuff. Even just a really basic read only FAT overlayed in the flash could be a useful way to get applications bundled with multiple file resources working with a P2 as long as there is tool to create and write it initially. Performance might take a slight hit with the extra FAT traversal overhead etc vs raw access but could be good enough in many cases.
  • Cluso99Cluso99 Posts: 18,069
    Nice work Ray.
    How much memory does it use?
  • RaymanRayman Posts: 14,646
    Was able to write to and read back from a new file!
    But, there is an issue that I may or may not have resolved...

    Turns out you can't just add to the file directory without erasing it first.
    And, the smallest amount you can erase is a 4kB block.

    What seems like it might work is only doing this for directory. It seems like the FAT chain and the new data might be OK.
    Right now, I only erase for the first sector in the FAT directory, #613. This may work for some small number of files.
    But, will eventually have to make for entire FAT directory...

    @Cluso99 FSRW is tiny...
  • RaymanRayman Posts: 14,646
    I think I have it now... Almost too easy...
    Might start new thread with code and instructions...

    Was able to flash this bitmap file and then show it using FRSW.
    IMG_0457.JPG
    4032 x 3024 - 2M
  • RaymanRayman Posts: 14,646
    Flash driver is actually in Spin (Spin1 actually), but seems to work well enough under Fastspin.
    Speed is decent, but obviously not assembly speed.
  • Cluso99Cluso99 Posts: 18,069
    Ray,
    From what I understand, you are using C to access the fsrw pasm driver.
    If so, what is the space used for a minimal implementation in KB please?

    We all know how bloated C is with its' minimum requirements. I want to know how bloated this minimum requirement is please. If it's only a few KB then great, but if it's 10's-100's KB then it's a no-go for me.
  • Wuerfel_21Wuerfel_21 Posts: 5,053
    edited 2020-04-30 22:28
    using FAT directly on a flash chip, aside from the erase block size issue, is data corruption waiting to happen... there's a reason SD cards, SSDs, USB flash drives and basically anything else storing files on flash is either using a block remapping layer or a flash-aware filesystem to avoid excessive erasing and perform wear leveling
  • RaymanRayman Posts: 14,646
    edited 2020-04-30 23:51
    @Cluso99 There’s no c here. All spin … @cheezus did most of the work making FSRW work for P2 with Fastspin...
  • RaymanRayman Posts: 14,646
    edited 2020-04-30 23:51
    @Wuerfel_21 I think it depends on how you use it. For example, if you write only once and then just read after that, there certainly cannot be an issue.
  • RaymanRayman Posts: 14,646
    edited 2020-05-01 00:09
    BTW: The datasheet says this: 
    Min. 100K Program-Erase cycles per sector

    I can't envision any scenario where I'd have to erase/write more than 100k times...
  • ElectrodudeElectrodude Posts: 1,657
    edited 2020-05-01 01:47
    Rayman wrote: »
    BTW: The datasheet says this: 
    Min. 100K Program-Erase cycles per sector

    I can't envision any scenario where I'd have to erase/write more than 100k times...

    You could easily hit that limit if you were to continually log data to the flash chip for long periods of time. If you used an appropriate filesystem for the application, all the sectors would wear out at around the same time, but if you use FAT, you'd destroy the FAT table long before any of the data sectors wore out, seriously shortening the usable lifetime of the chip.

    Yes, of course you could avoid this by pre-allocating each file when you create it, but that defeats the whole purpose of having a file system to begin with - that of automatically dealing with allocations and hierarchy for you - effectively building your own filesystem on top of the underlying one. Why bother with a filesystem at all if all you need is a circular buffer?
  • Roy ElthamRoy Eltham Posts: 3,000
    Using FAT file system on a normal flash memory chip is a really bad idea.

    SDcards and SSDs have controller chips that manage wear leveling for you. The bare flash chip isn't going to do that, so you are going to wear out the blocks with the FAT table and directory way before any other blocks.
  • AribaAriba Posts: 2,690
    Also my first thought was that this will not work for very long.

    But if I think about it again, it's not so bad:
    The FAT only changes when a new cluster is allocated, which is not so often the case.
    Most sensitive is the directory, but neither name nor time will change with FSRW. Actually only the filesize has to be rewritten. Maybe you can save the size at the beginning of the file instead of in the directory.

    Of course you should not use this for logging, but if you just want to save modified, or newly created images or text, I don't see a big problem.

    If you save a file 10 times per day, you can do this for 27 years to reach 100'000 write operations.
    Until then we already work with the P3 ;)

    Andy
  • roglohrogloh Posts: 5,787
    I would certainly consider using this FAT filesystem overlay over flash for read-only/bootup cases if the performance is good enough and you already have applications that work well with FAT file access for reading their data. For read/write use it is far less clear. If it was just for occasional software bundle updates I guess I could risk it, if it was for logging or regular file I/O use (e.g. built files in dev system) no way would I consider to do that. Either an SD card or alternatively a proper flash file system would be the way to go then.
  • iseriesiseries Posts: 1,492
    What about using SPIFF's which is open source and works with memory.

    Mike
  • RaymanRayman Posts: 14,646
    continuous data logging with this is a bad idea...

    Don’t do that and should be fine.
  • roglohrogloh Posts: 5,787
    edited 2020-05-01 10:23
    Rayman if you abstract the sector read/write routines then your code can probably be used a front end for a temp filesystem in HyperRAM. It would still need the ability to dynamically construct an initial filesystem in memory though when the volume is created at initialization time. I've not had time to go through your examples but with any luck something like this might be possible.

    If we have all 3 SD/Flash/HyperRAM interfaces to some common upper layer tracking each mounted or active volume that could be quite good and versatile from a software application standpoint.
  • RaymanRayman Posts: 14,646
    Hmm... I think it could work on HyperRAM ...
    don’t know if it makes any sense, but should work
  • roglohrogloh Posts: 5,787
    edited 2020-05-01 10:52
    It could make sense for a temporary filesystem or a (volatile) one copied out of flash after bootup for much faster raw access speeds (like 200-300MB/s). For readonly use this might be very useful. It could also be useful for streaming playback of multichannel audio data/files perhaps vs over a slower SPI interface.
  • RaymanRayman Posts: 14,646
    Hyperflash might be interesting too
  • roglohrogloh Posts: 5,787
    edited 2020-05-01 10:55
    Rayman wrote: »
    Hyperflash might be interesting too

    Yes for reads. For writes it seems like it will be a dog from what erase speeds numbers I found.

    The HyperRAM reads and HyperFlash reads will have the same driver interface, just a different "bank" field of an address which would represent the device.
  • ElectrodudeElectrodude Posts: 1,657
    rogloh wrote: »
    Rayman wrote: »
    Hyperflash might be interesting too

    Yes for reads. For writes it seems like it will be a dog from what erase speeds numbers I found.

    The HyperRAM reads and HyperFlash reads will have the same driver interface, just a different "bank" field of an address which would represent the device.

    Considering that LittleFS tries to avoid needing to do erases by only writing partial blocks until the block is full (at least with metadata blocks), I imagine that the slow erase speeds of HyperFlash won't be so bad if you're modifying lots of small files.
Sign In or Register to comment.