I've also tested the GD25Q80 and the IS25LP080, now. Both show the same results, dir after demo shows the files but format hangs. If I mount again after a reboot the files are still there so format seems to do nothing at all. I thought I also bought some XT25F08 from XTX but I can't find them anymore. Time to clean up the workshop, again ...
Anyway, all chips except for the Zeta are working. I have to debug what's wrong with the format.
Hmm, if I comment out if (are_you_sure() == true) the flash.format() runs without problems and finishes within 2 seconds. This seems to be an issue with FlexSpin and the way are_you_sure() handles the input buffer.
What serial terminal are you using that sends a lone CR? I thought everyone was finally moving from CRLF to LF, and now I'm hearing about lone CRs and getting demo files from Parallax with CR-only line endings...
@JonnyMac said:
Make sure you have Options/Use Internal PST Terminal selected. This is what I get on Windows.
Errr, where is that option? I'm using VSC which calls loadp2.exe with the terminal optionj "-t". I think, I don't have much choice which control character it uses.
Sorry, @ManAtWork, I didn't realize you were using VSC. To Evan's point, the -T option with the loader should give you the PST-compatible terminal. Hopefully, that sorts it out
FWIW, I don't like the PST adds LF to every CR, but there is no chance of ever getting that fixed. Thankfully, other terminals (e.g., PuTTY) seem to have the ability to add LF to CR so code I've written for PST will work with them.
Jon, not your fault. I should have read the docs...
[ -t ] enter terminal mode after running the program
[ -T ] enter PST-compatible terminal mode
The problem is that there is so much to learn. In the good old days there was only the P1 with excellent manuals an Propeller Tool that was very easy to operate. Just write your code and press F10 or F11... Now there are so many tools with different compilers, terminals and no single manual you can read. You have to search the forum, github and other sources to get information. I'm still struggeling with VSC and the json scripts.
But I don't like the editor of FlexProp and I really depend on the graphical debug windows (especially the scope display) when debugging real time signals. The PASM level debugger is also nice to have. It is VERY rudimentary but handy to find typical assembler errors like messing up the flags and branching to the wrong address.
How do I know when/if I need to format? I found out that the FFs also works if I don't format at all. But is that safe? I have an "embedded" application where the user never has to deal with the file system and doesn't even know that there is one. So everything has to work automatically. So I think the best idea is to do a format only once as part of the production programming and then never again.
I think it would be a good idea to check the flash ROM size during mount. That way it would be possible to auto-detect smaller chips <16MB. Most of the KISS boards are equiped with only 4MB, for example. Chips become cheaper and cheaper, so one day it's probably cheaper to mount a 32MB flash or bigger onto the Parallax boards. Auto-detection is not necessary for my own code as it always knows what chip is to be expected. But it would allow to write an universal bootloader which works on all boards without requiring modifications to the code.
A size check could be done easily by checking if the first valid block found repeats after 1MB, 2MB, 4MB and so on.
Even if you back it up by a nuclear battery, customers still manage to wreck it. Their power is greater than yours. I had one that accidentally clicked on the update button, realised his mistake and in panic pulled out the power cable to "undo" his error. Guess what, the update was aborted and he had to return the board for hardware programming. If he had just relaxed and waited nothing harmful would have happened.
That's why it's so valuable if the file system is fail-safe.
Lately, I've been focused on Spin2 Language Server Extension work while Chip is working on the read/modify/write feature of the flash filesystem. I missed seeing some of your questions. This one caught my eye so I thought I'd introduce some concepts that will be appearing in the Theops.
@ManAtWork said:
Hmm, I doubt that. AFAIK, only one block is buffered. So if I write multiple blocks and the power failure happens between writing the 1st and the 2nd block then I assume the old file is already deleteted and only the first block of the new file is valid. Or is the old file really preserved until I close the new file?
I know I'm late to this discussion of how files are re-written, but here is some info based on my current study and understanding:
When writing changes to a file, all writes are placed in a "commit chain of blocks." Each of the blocks is written to the flash as it is filled. The last (possibly partially filled) block is written when the file is closed/flushed. The final flash write during closing then marks the splicing block as a higher priority than the one it replaces. Prior to this marking if the power failed, this marking would be finished at the next mount. The original file stands untouched until the file close(or flush) occurs along with this marking.
Does this help clarify? I'll be adding more commit chain details to the Theops document likely when I do the updates for the read-modify-write addition, as that code Chip is working on is included in this Flash File system.
Yes, thanks, that helps to clarify. I wish all P2 library code was documented like this!
BTW, remider for unanswered questions:
1. Is there a simple was to check if the flash chip already contains valid files or is formatted at all?
2. Would it make sense to automatically detect the memory size and adapt LAST_BLOCK?
Is there a simple was to check if the flash chip already contains valid files or is formatted at all?
After mounting you'll know. Speaking with Chip on Saturday he told me that in effect, the mount process will do a defacto format, that is, any invalid blocks will be touched up and marked as unused.
Would it make sense to automatically detect the memory size and adapt LAST_BLOCK?
This might be a case where overriding a constant on the object declaration line is useful. I just did this to make my 16M flash on an Eval look like a 4M flash.
flash : "flash_fs" | LAST_BLOCK = $3FF
The LAST_BLOCK value is used for setting the size of the blocks table in the program, so reading the flash size on-the-fly might not be the best way to go. It can be done, though. I have this method in a simple flash demo program.
pub jedec_info() : result
pinl(SF_CS)
shiftout($9F, 8)
result := shiftin(24)
pinh(SF_CS)
The flash size is 1 << result.byte[0]. You can divide that by the block size (4096) and subtract one for the last block.
@ManAtWork said:
Yes, thanks, that helps to clarify. I wish all P2 library code was documented like this!
BTW, remider for unanswered questions:
1. Is there a simple way to check if the flash chip already contains valid files or is formatted at all?
2. Would it make sense to automatically detect the memory size and adapt LAST_BLOCK?
@JonnyMac, thanks for your answers. I have additional info as well.
Re documenting: thanks I do pay attention to trying to do this in a useful manner. ;-)
Re checking: In the next release, I'll have a non-destructive filesystem formatted check (as mount can possibly damage a non flash_fs image)... so, coming soon!
I can see a technical way to during mount detect the partial use of a flash chip as flash_fs and re-mount only that portion leaving the rest of this chip for other use. So after formatting this is easy. But what are you suggesting for auto-detect of chip size? Is there a means to know the size? Cuz if we do then using that size is possible.
@JonnyMac I just saw your flash-size detect. What are you doing in that code? Why does that work? (I wanna learn! ;-)
@JonnyMac said:
This might be a case where overriding a constant on the object declaration line is useful....
flash : "flash_fs" | LAST_BLOCK = $3FF
The LAST_BLOCK value is used for setting the size of the blocks table in the program, so reading the flash size on-the-fly might not be the best way to go.
Ok, I see. On-the-fly adjusting to the actual flash chip capacity would make the code much more complex and requires allocating the buffer memory dynamically. It's surely much easier to patch the constant and should be no problem as every software somehow has to know on which hardware it runs. I just had that idea to write an universal boot loader that has not to be patched for different board versions.
I also have a code that copies from uSD to Flash. I thought that was somehow actually working with my Eval at work.
But, can't make it work with home eval board for some reason.
@Rayman said:
Ok, I tried this again and guess it's just not possible to transfer files to/from uSD and Flash on Eval board.
(Or, maybe it is?)
It's not possible to be using both the uSD and Flash at the same time. You can mount one, copy the file to RAM, unmount and then mount the other and copy the file from RAM to the other. But you can't copy the file directly between uSD and Flash without buffering in RAM, because both the uSD and Flash want to use the same pins (on the Eval board).
I updated my demo program (see post #63) to match the video Ken posted. As a reminder, the video is a very quick intro -- the goal is to get people exploring.
All methods are backed by a regression test suite. At each release, this one and all upcoming releases, the test suite will be run before the release is posted so that we can be sure future updates will work when they are released. When a release is posted, the updated regression test code and the logs of the passing runs are also versioned.
Comments
I've also tested the GD25Q80 and the IS25LP080, now. Both show the same results, dir after demo shows the files but format hangs. If I mount again after a reboot the files are still there so format seems to do nothing at all. I thought I also bought some XT25F08 from XTX but I can't find them anymore. Time to clean up the workshop, again ...
Anyway, all chips except for the Zeta are working. I have to debug what's wrong with the format.
Hmm, if I comment out
if (are_you_sure() == true)theflash.format()runs without problems and finishes within 2 seconds. This seems to be an issue with FlexSpin and the wayare_you_sure()handles the input buffer.AAaarrggghhh!!! It doesn't crash or hang. It just doesn't exit the loop because it expects a LF instead of a CR.
What serial terminal are you using that sends a lone CR? I thought everyone was finally moving from CRLF to LF, and now I'm hearing about lone CRs and getting demo files from Parallax with CR-only line endings...
PST does that. A lot of Prop1 code is littered with it.
Errr, where is that option? I'm using VSC which calls loadp2.exe with the terminal optionj "-t". I think, I don't have much choice which control character it uses.
loadp2 -Tis supposedly PST compatible. I've never tried it myself.Sorry, @ManAtWork, I didn't realize you were using VSC. To Evan's point, the -T option with the loader should give you the PST-compatible terminal. Hopefully, that sorts it out
FWIW, I don't like the PST adds LF to every CR, but there is no chance of ever getting that fixed. Thankfully, other terminals (e.g., PuTTY) seem to have the ability to add LF to CR so code I've written for PST will work with them.
Jon, not your fault. I should have read the docs...
The problem is that there is so much to learn. In the good old days there was only the P1 with excellent manuals an Propeller Tool that was very easy to operate. Just write your code and press F10 or F11... Now there are so many tools with different compilers, terminals and no single manual you can read. You have to search the forum, github and other sources to get information. I'm still struggeling with VSC and the json scripts.
That's why I don't use this. Flexprop itself maybe is limited, but is enough to write what I want and it works.
But I don't like the editor of FlexProp and I really depend on the graphical debug windows (especially the scope display) when debugging real time signals. The PASM level debugger is also nice to have. It is VERY rudimentary but handy to find typical assembler errors like messing up the flags and branching to the wrong address.
How do I know when/if I need to format? I found out that the FFs also works if I don't format at all. But is that safe? I have an "embedded" application where the user never has to deal with the file system and doesn't even know that there is one. So everything has to work automatically. So I think the best idea is to do a format only once as part of the production programming and then never again.
I think it would be a good idea to check the flash ROM size during mount. That way it would be possible to auto-detect smaller chips <16MB. Most of the KISS boards are equiped with only 4MB, for example. Chips become cheaper and cheaper, so one day it's probably cheaper to mount a 32MB flash or bigger onto the Parallax boards. Auto-detection is not necessary for my own code as it always knows what chip is to be expected. But it would allow to write an universal bootloader which works on all boards without requiring modifications to the code.
A size check could be done easily by checking if the first valid block found repeats after 1MB, 2MB, 4MB and so on.
Always a concern of mine, so:
Link to product page
Even if you back it up by a nuclear battery, customers still manage to wreck it. Their power is greater than yours. I had one that accidentally clicked on the update button, realised his mistake and in panic pulled out the power cable to "undo" his error. Guess what, the update was aborted and he had to return the board for hardware programming. If he had just relaxed and waited nothing harmful would have happened.
That's why it's so valuable if the file system is fail-safe.
BTW, I've added the compatibility test results of flash chips vs. flash file system in this thread.
Lately, I've been focused on Spin2 Language Server Extension work while Chip is working on the read/modify/write feature of the flash filesystem. I missed seeing some of your questions. This one caught my eye so I thought I'd introduce some concepts that will be appearing in the Theops.
Hmm.. I need to bolster the theory of operations if I don't cover this! Flash F/S theory of operations
I know I'm late to this discussion of how files are re-written, but here is some info based on my current study and understanding:
When writing changes to a file, all writes are placed in a "commit chain of blocks." Each of the blocks is written to the flash as it is filled. The last (possibly partially filled) block is written when the file is closed/flushed. The final flash write during closing then marks the splicing block as a higher priority than the one it replaces. Prior to this marking if the power failed, this marking would be finished at the next mount. The original file stands untouched until the file close(or flush) occurs along with this marking.
Does this help clarify? I'll be adding more commit chain details to the Theops document likely when I do the updates for the read-modify-write addition, as that code Chip is working on is included in this Flash File system.
Yes, thanks, that helps to clarify. I wish all P2 library code was documented like this!
BTW, remider for unanswered questions:
1. Is there a simple was to check if the flash chip already contains valid files or is formatted at all?
2. Would it make sense to automatically detect the memory size and adapt LAST_BLOCK?
After mounting you'll know. Speaking with Chip on Saturday he told me that in effect, the mount process will do a defacto format, that is, any invalid blocks will be touched up and marked as unused.
This might be a case where overriding a constant on the object declaration line is useful. I just did this to make my 16M flash on an Eval look like a 4M flash.
The LAST_BLOCK value is used for setting the size of the blocks table in the program, so reading the flash size on-the-fly might not be the best way to go. It can be done, though. I have this method in a simple flash demo program.
The flash size is 1 << result.byte[0]. You can divide that by the block size (4096) and subtract one for the last block.
@JonnyMac, thanks for your answers. I have additional info as well.
Re documenting: thanks I do pay attention to trying to do this in a useful manner. ;-)
Re checking: In the next release, I'll have a non-destructive filesystem formatted check (as mount can possibly damage a non flash_fs image)... so, coming soon!
I can see a technical way to during mount detect the partial use of a flash chip as flash_fs and re-mount only that portion leaving the rest of this chip for other use. So after formatting this is easy. But what are you suggesting for auto-detect of chip size? Is there a means to know the size? Cuz if we do then using that size is possible.
@JonnyMac I just saw your flash-size detect. What are you doing in that code? Why does that work? (I wanna learn! ;-)
The $9F command returns the 24 bits of JEDEC ID information. Here's a snippet from my flash demo that calls that method.
jedec := flash.jedec_info() term.fstr1(string("Manufacturer... %2.2x\r"), jedec.byte[2]) term.fstr1(string("Mem Type....... %2.2x\r"), jedec.byte[1]) term.fstr2(string("Capacity....... %2.2x (%d bytes)\r\r"), jedec.byte[0], 1 << jedec.byte[0])From the docs
Ok, I see. On-the-fly adjusting to the actual flash chip capacity would make the code much more complex and requires allocating the buffer memory dynamically. It's surely much easier to patch the constant and should be no problem as every software somehow has to know on which hardware it runs. I just had that idea to write an universal boot loader that has not to be patched for different board versions.
Playing with this today...
One code here copies embedded wav file (or any file, I guess) from HUB RAM to Flash.
Other one plays .wav files from Flash.
Flash driver is too slow to play the stereo, high bitrate, files. But, mono, low bitrate works fine.
I also have a code that copies from uSD to Flash. I thought that was somehow actually working with my Eval at work.
But, can't make it work with home eval board for some reason.
Ok, I tried this again and guess it's just not possible to transfer files to/from uSD and Flash on Eval board.
(Or, maybe it is?)
This works on my boards because I have uSD on different pins...
It's not possible to be using both the uSD and Flash at the same time. You can mount one, copy the file to RAM, unmount and then mount the other and copy the file from RAM to the other. But you can't copy the file directly between uSD and Flash without buffering in RAM, because both the uSD and Flash want to use the same pins (on the Eval board).
I updated my demo program (see post #63) to match the video Ken posted. As a reminder, the video is a very quick intro -- the goal is to get people exploring.
NEWS
Just released a major update. P2-FLASH-FS GitHub repository:
Installation instructions are found on the top page of the Repo. See: Adding flash_fs to your own project
All methods are backed by a regression test suite. At each release, this one and all upcoming releases, the test suite will be run before the release is posted so that we can be sure future updates will work when they are released. When a release is posted, the updated regression test code and the logs of the passing runs are also versioned.
If you find something not working as you expect, then please file an issue at my GitHub repository issues page
What's next?
I'm also planning on full directory functionality to follow in an upcoming release.
Enjoy!
Stephen