Use boot Flash just like an SD card

Rayman

Here are some instructions and example code for how to test the use the spare space on your boot flash like an SD card.
This is with the regular FSRW sd-card interface, but with a special, low-level driver.

Note: For the P2-ES board, you need to have the "Flash" jumper in the "ON" position for this to work...

The boot flash chip on P2-ES board is 16 MB, but 1 MB needed for booting, leaving 15 MB free for use storing files.

The posts below outline some steps for testing this out.

These files compile with the latest FastSpin, version 4.1.7. But, there are also binary versions that can be loaded without compiling first.

Known Limitations:
1. FSRW only accesses the root directory and all files have to be there.
2. Current low level driver is in Spin and therefore not as fast as could be if were in assembly, but still can load VGA bitmap image 4 seconds.

Rayman

Step1: Format the Flash

Warning: This will completely erase the flash chip.
Maybe I didn't have to do this, but this is easier...

Use the attached binary or compile the source to erase and then reprogram the flash chip.
After this it will seem to FSRW to be a blank disk, except for one small text file.

Note: It takes about a minute to format, so be patient...

Use a serial terminal to see the output, should look like this:

3
2
1

First Partition at sector # 133
Sector Size [B] = 512
Old Cluster Size [B]= 32768
Old sectors/cluster = 64
New sectors/cluster = 4
New Cluster Size [B]= 2048
Old total sectors = 3868539
New total sectors = 30843
#reserved sectors = 6
#root entries = 512
Sectors per fat = 237
FAT#1 sector start = 139
sectors/fat = 237
root dir sector = 613
#rootentries = 512
root dir end sector = 645
data region start sector= 637
filesize= 46
data first sector = 657
Flash responded with correct JEDEC: 00EF7018
Erasing Entire Flash. Please Wait, this will take about a minute to complete

Writing copied sectors to flash.

Copy complete.

Rayman

Step2: See if you can then mount and open and read from the text file included in the format.

After formatting, see if you can open the text file and display it.
Output in serial terminal should look like this:

3
2
1
Attempting to Mount SD, result = 0
Trying to open file test.txt, result = 0
filesize = 46
file content =This is a test file for FSRW on P2 boot flash.

Rayman

Step3: Save new files to flash

This code writes one small text file and then one big VGA bitmap file to flash and then displays directory content.

It takes a few seconds to write and you should see four of the P2-ES board leds lit up while it works...

Output on serial terminal should look like this:

3
2
1

Trying to mount, result = 0
Trying to open file test.txt, result = 0
filesize = 46
file content =This is a test file for FSRW on P2 boot flash.
Trying to open file test.txt, result = 0
filesize = 46
file content =This is a test file for FSRW on P2 boot flash.
Trying to open new file test2.txt for writing, result = 0
Preparing to write this many bytes to flash: 56
Writing new data to flash, result = 56
Trying to re-open new file test2.txt for reading, result = 0
filesize = 56
file content =This is a test of writing to a file using FSRW on Flash
Trying to open new file, bitmap2.bmp, for writing, result = 0
Preparing to write this many bytes to flash: 308279
Writing new data to flash, result = 308279
SD Root Directory:
TEST.TXT
TEST2.TXT
BITMAP2.BMP
End of SD Root Directory.

All done...

Rayman

Step4: Show a bitmap file that was written to flash in last step.

Note: You will need the A/V accessory attached to P0..P7 with a VGA monitor connected to see the bitmap.

This example loads a bitmap file from flash and then shows it over VGA.
It takes just a couple seconds to read it in.
You can see 3 of the P2-ES board LEDS lighting up while reading.

Output on serial terminal should look like this:

3
2
1

Trying to mount, result = 0
SD Root Directory:
TEST.TXT
TEST2.TXT
BITMAP2.BMP
End of SD Root Directory.

Trying to open big file bitmap2.bmp for reading, result = 0
filesize = 308279
Reading in file
Closing big file.

All done, showing bitmap now.ÿ

Output on VGA monitor should be as shown below.

Publison

Great stuff Ray! All my electronics is 1000 miles away.

Rayman

That's sad to hear... My P2 work might be the only thing keeping me sane...

cgracey

Ray, this is a great idea. We've got 16MB of flash and we only use <512KB for booting. Think back to the days of 20MB Seagate hard drives. I couldn't believe it when they started making PC's with 40MB drives, as standard. In the context of P2, 16MB is huge.

Rayman

There area lot of things that could be done with this... Store several programs... Fonts, images, sounds used by your programs.
Could save you from needing an SD card or other storage for your application.

Just don't use it as a data logger. You could write a new file every hour, no problem. But, writing every minute might lead to wear out as it's only spec'd at 100,000 write cycles per sector.

You can read as much as you like. That doesn't cause any wear...

iseries

So putting the Spin interpreter there along with several standard fonts would work then.

Mike

Rayman

I thought maybe the spin interpreter was going to be put somewhere below the 1 MB line...

Not really sure though. But, RAM is only 512 kB and Chip suggested reserving the lower 1 MB.

Rayman

I looked at applying this scheme to hyperflash and big problem is the min erase size of 256kB.

But, perhaps I can keep the FAT and directory info on the onboard flash instead of on the hyperflash

Think can just steal 512 kB back from the flash drive...

Electrodude

LittleFS should work just fine with 256kB erase blocks, since it hardly ever performs erase operations on metadata blocks anyway...

rogloh

Or to simplify just limit HyperFlash to be a read only FAT volume perhaps, and use other tools to create/update it.

HyperFlash could even contain multiple volumes (some FAT, some LittleFs etc), but that needs a workable partitioning scheme which only adds complexity. I doubt we get any agreement there.

Electrodude

You could get wear levelling for your FAT partitions by nesting them inside of LittleFS files

I get that FAT's good for now because it's what's already implemented, but would it still have any advantage for non-removable flash devices if someone got LittleFS to work on the P2 with the same interface with a reasonable memory footprint?

rogloh

Electrodude wrote: »

I get that FAT's good for now because it's what's already implemented, but would it still have any advantage for non-removable flash devices if someone got LittleFS to work on the P2 with the same interface with a reasonable memory footprint?

Probably not apart from familiarity perhaps and Spin2 just being smaller than the C. I wonder how much room it would take if LittleFs could somehow be converted into SPIN2 or PASM to shrink it and make it more portable across tools. For people already using C it may be a no-brainer to just try to use LittleFs directly though we don't yet have a gauge of how big it is unless somehow has already compiled it with P2GCC perhaps. LittleFs seems to be around 5800 lines of source code in C (headers+implementation).

Rayman

Now that I'm getting comfortable with FlexSpin C, I think I'll take this low level Spin2 driver and attach it to FatFs.
That will allow long filenames and directory support on the flash.

Then, could have simultaneous access to host, uSD, and flash file systems...

Also, I think FatFs has code to format drives. That could make this process a lot simpler.

iseries

@Rayman ,

What about just using SPIFFS. It's not as if were going to attach this to my PC or anything. It maybe simpler to port.

Mike

Rayman

That does look good, with the wear levelling.

But, I've already got FatFs working, thanks to @ersmith. It should be very easy to adapt to use flash...

If I thought I might need wear levelling, that would be worth exploring though...

iseries

@Rayman ,

I know that doing FatFs is already there but in the case of the flash, using SPIFFS makes it possible to point it at an address and also preload that area with files if need be. FatFs wants to use the hole card which leads to issues with using it to save programs.

Mike

Rayman

Yes, saving things routinely might well be an issue without wear leveling...

But, I'm mostly interested in reading from it.
Preloading is an interesting idea though. Perhaps one could load the files needed onto a freshly formatted uSD and then copy it's image to the flash.
That is what I did earlier in this thread, but with an empty drive...

Wuerfel_21

If you're just reading prewritten files (or writing to fixed size pre-allocated files), you could save some flash space by bypassing the whole FAT lookup business, since the files will all be linear, anyways. Should not be a difficult modification to make. Writing a script to create such images shouldn't be, either.
I've been thinking for a while about doing something like that, but inside of SD files, like a nested filesystem. Would allow bundling application binaries + data files + metadata all into one file.

Cluso99

Wuerfel_21 wrote: »

If you're just reading prewritten files (or writing to fixed size pre-allocated files), you could save some flash space by bypassing the whole FAT lookup business, since the files will all be linear, anyways. Should not be a difficult modification to make. Writing a script to create such images shouldn't be, either.
I've been thinking for a while about doing something like that, but inside of SD files, like a nested filesystem. Would allow bundling application binaries + data files + metadata all into one file.

That is precisely what i've done with CPM. There are 8 x 8MB files which represent 8 HDDs in CPM. They must be contiguous files and all I do is locate the filename and its' first data sector for each HDD file when starting CPM. The each file is accessed by physical sector address on the SD card. My PASM SD Driver supports physical sector access. All higher level access is done via spin now although there is PASM code to initialise and locate the file too.

Wuerfel_21

Cluso99 wrote: »

Wuerfel_21 wrote: »

If you're just reading prewritten files (or writing to fixed size pre-allocated files), you could save some flash space by bypassing the whole FAT lookup business, since the files will all be linear, anyways. Should not be a difficult modification to make. Writing a script to create such images shouldn't be, either.
I've been thinking for a while about doing something like that, but inside of SD files, like a nested filesystem. Would allow bundling application binaries + data files + metadata all into one file.

That is precisely what i've done with CPM. There are 8 x 8MB files which represent 8 HDDs in CPM. They must be contiguous files and all I do is locate the filename and its' first data sector for each HDD file when starting CPM. The each file is accessed by physical sector address on the SD card. My PASM SD Driver supports physical sector access. All higher level access is done via spin now although there is PASM code to initialise and locate the file too.

Well, imagine that, but instead of CPM files it's a _BOOT_P2.BIX (or a P1 equivalent), a manifest of sorts and whatever other files are needed. The manifest would contain info about the program, like what board it's compiled for. Or maybe allow giving addresses where the loader is to fill in the pin numbers, making the package portable.
That's the idea, anyways.

Rayman

If you just want to preload flash with a bunch of files, one could use the "Eve Asset Builder".
It combines all the files you drop into it into a single output file and gives you a list of offsets to all the files.

Only minor issue is that it reserves the first 4k for EVE's own use. But, you can just skip over writing that as you don't want to mess with P2 boot code anyway...