Well, I can afford to go along with the existing PropForth solutions and see if my fear of failure is imagined or not.
From what I read, it seems that by using PropForth, I will have only 32mbyte of SPI storage regardless of the card being 2gbytes.
It would be more interesting if files could be directly moved from a text editor on a PC to the PropForth platform via SDcard and you might appeal to more users with a greater diversity of Forth applications. I've already located at least one example of a simple text editior in Forth, but without writing to a file I don't see much need for it.
Since the card that I have actually cost about $10USD and I have the reader for it, it would be silly to order another card reader and pay the price of that plus shipping to Taiwan at this point in time. I can't seem to buy the card reader hardware locally.
Meanwhile, I will get yet another Propeller Proto Board up and running using Parallax's FAT16/32 solution, maybe with a real time clock included and see what SPIN has to offer.
I know there is a processor in the SDcard that does indeed rotate wear. But past comments on the web have said that using Linux Ext2 or Ext3 will not support it, nor will raw SPI written to blocks without the factory formatting in FAT16 or FAT32. So it seems that the File Allocation Table is involved in keeping file names while rotating block. I am not sure what role the Master Boot Record plays - maybe none. But up to 4 partitions may be made. Linux's Ext2 and Ext3 is actually more complex with Unix group and user rights imposed and that would take even more code than a minimal single user FAT16/32 solution (even LInux just accepts SDcard with factory format of FAT16/32 and doesn't recommend reformat). And I do have an SD document that mentions clearly that not using the 'factory format' will impare performance (but it is not explict on how).
I have lots to do and learn in PropForth as I still have not finished "Starting Forth" or all the PropForth material.
I am also running some examples in GForth when I don't have the PropForth available. In other words, I am learning two different solutions at the same time. In order to avoid more confusion, I am minimally involved with learning TachyonForth as I'd likely just never get anything right.
I see these little SPI 32Mbyte SRAM chips and have to wonder if they are yet another alternative. They may equal the SDcard in utility and speed.
I know there is a processor in the SDcard that does indeed rotate wear. But past comments on the web have said that using Linux Ext2 or Ext3 will not support it, nor will raw SPI written to blocks without the factory formatting in FAT16 or FAT32. So it seems that the File Allocation Table is involved in keeping file names while rotating block. I am not sure what role the Master Boot Record plays - maybe none. But up to 4 partitions may be made. Linux's Ext2 and Ext3 is actually more complex with Unix group and user rights imposed and that would take even more code than a minimal single user FAT16/32 solution
I see these little SPI 32Mbyte SRAM chips and have to wonder if they are yet another alternative. They may equal the SDcard in utility and speed.
No card reader required and they are cheaper than SDcards.
That's 32Mbit or 4M bytes and they are sector Flash (not static RAM) meaning that the smallest unit that you can erase in this case is 4K bytes.
I don't believe that what you heard about SD level wear requiring FAT is correct. The SD card is not that smart to discern the information and format of the card but it can remap sectors (512 bytes) much like a hard drive can. This can slow things down when the card starts wearing but it's all covered in the specification so you don't have to worry about it.
BTW, Solid State Drives have TRIM commands but the closest that SD cards have is the ERASE command (CMD38).
I gave PropForth a try, and I am also having some problems with it. Assuming I have Forth source files on an SD card how do I compile them?
It seems that PropForth just exploits the SDcard as an SPI SRAM without the FAT16/32 support you are hoping for.
In PropForth, one creates files on a desktop with a 'fl' command at the beginning of the ASCII text document. And then with whole file is opened in a text editior which can copy and paste it to the terminal screen which will execute it as if you typed it all it.
The 'fl' removes comments and extra white space -- just accepts what would normally be seen within a colon and a semi-colon. But multiple words are accepted in one go.
After you have your file loading into PropForth and active, you can use 'saveforth' to transfer an image to the SDcard. I am not sure about how naming or multiple images get managed.
I don't believe that what you heard about SD level wear requiring FAT is correct. The SD card is not that smart to discern the information and format of the card but it can remap sectors (512 bytes) much like a hard drive can. This can slow things down when the card starts wearing but it's all covered in the specification so you don't have to worry about it.
Thanks, happy to hear from you. As this topic came up a long time ago, it may all be a tempest in a teapot. I can't find the info I imagined I read more than a year ago.
I remain very excited about Forth just because it is so interactive. For many learners, this is much more engaging that writing pages of code, compiling, loading and debugging.
This can slow things down when the card starts wearing but it's all covered in the specification so you don't have to worry about it.BTW, Solid State Drives have TRIM commands but the closest that SD cards have is the ERASE command (CMD38).
I'm still not sure about all this. The SD card knows nothing about your file system. Therefore when you think you have deleted a file and freed all it's blocks the card does not know that. Hence I would guess it cannot map other blocks to those "deleted" blocks and it's wear levelling capability is reduced.
No idea about low level TRIM or ERASE commands but there is an fstrim command in Linux that works on SD cards and reports back that it has done something useful. Just been trying it out on my Raspberry Pi board.
One last mention. The excerpt below is from the Secure Digital entry in Wikipedia. It does indeed say that 'some wear leveling' is designed to access the FAT.
"Risks of reformatting
Reformatting an SD card with a different file system, or even with the same one, may make the card slower, or shorten its lifespan. Some cards use wear leveling, in which frequently modified blocks are mapped to different portions of memory at different times, and some wear-leveling algorithms are designed for the access patterns typical of the file allocation table on a FAT16 or FAT32 device.[61] In addition, the preformatted file system may use a cluster size that matches the erase region of the physical memory on the card; reformatting may change the cluster size and make writes less efficient. The SD Association provides free formatting software to overcome these problems.[62]"
I also realize that reported incidents may not be accurate as hobbyist can create all sorts of problems with SDcards.
Parallax has mentioned that pullups should be on just about all pins used or not to prevent the cards from running hotter. Mistakes of wrong voltage, reverse polarity, or brownouts can all play havoc with these devices when used on a breadboard. If fact, I went with the Parallax micro SDcard reader because it has complete pullup resistors.
The cards will work with a lot of formats that are not standard, including Ext2, Ext3, and Ext4 and even Apple formats and NTFS. But I can't not find SD Association formating software for anything besides FAT16/32 and maybe FAT12.
"..and some wear-leveling algorithms are designed for the access patterns typical of the file allocation table on a FAT16 or FAT32 device"
Looks like the key thing here is "access patterns typical of" which does not mean actual knowledge of what is where in your file system. Hence I'm still inclined to believe that blocks you have freed in your fs are still not necessarily free as far as the SD card knows. How would it know?
Hence that fstrim command in Linux.
Heater,
I am not up to speed on what the 'fstrim' command does or doesn't do. I will try to understand it and what you are trying to say.
Okay, a computer doesn't know that MicroSoft exists or what a file allocation table is. But if the wear leveling expects that methodology or something similar, and if you don't provide it - what is the result?
Since that comment in Wikipedia was in an SD Association document discussing SDcard formatting and dated 2011, the 'sense' I get is that it would be more prudent to comply if at all possible.
Yesterday, I saw SDcards for sale on the web that are specifically formatted to Ext2 or Ext3 for the Raspberry Pi.
That doesn't mean that they will hold up well. As I understand it, FAT16/32 are not controlled by MS, just longer than the 8 character names - but the SDcard Association doesn't want to explain too much of what makes its card so durable or everyone will clone the technology. It is apparently a 'trade secret' approach to control of their intellectual property.
I will use what is available for PropForth or TachyonForth (learning Forth on the Propeller is a very good way to quickly understand the Propeller) , but I may eventually figure out how to adapt to what I wold prefer - a completely functional SDcard in FAT16/32. If I have to run a two Propeller system to do so, that is okay.
Each to his own.
BTW, since Ext3 is a journalizing file system - it and NTFS may wear out SDcards faster due to that process alone.
The point I was trying to make all along is that SDcards were developed for FAT16/32 as their optimal performance (no journalizing, maybe no user designation), AND the SDcard Association still allowed anyone to format in other ways without the full feature set of wear leveling because they might have to evolve into different markets at some time in the future..
When I purchase my EEEpc 701 with solid-state hard disk, everyone was chatting about proper wear leveling. The simple fact was that nobody came up with a scheme to do it any better than Xandros. And Xandros' scheme was to make half the storage Write Only.
Does your Raspberry Pi have a Swap disk partition on an SDcard? That may be an invitation to failure in its own right. SDcards were never meant to heavily cycle read/write/erase. But it is quite easy to do in Linux.
Here's a Forth program to determine the number of ANS Forth core words that are supported. I would be interested to hear how many words are supported by PropForth and Tachyon. The pfth kernel currently supports 90 core words. It supports all 133 core words by loading an external Forth program.
Here's a Forth program to determine the number of ANS Forth core words that are supported. I would be interested to hear how many words are supported by PropForth and Tachyon. The pfth kernel currently supports 90 core words. It supports all 133 core words by loading an external Forth program.
You keep mentioning that ANS word all the time (tut tut), will it make my Propeller software run faster, leaner, meaner? I don't think so. There are words in Tachyon which have been renamed or at least aliased but may also work a little differently. For instance I don't agree with some names and I prefer to use ones that are familiar to Spin/PASM programmers. An example is SHR instead of RSHIFT, not a big deal but totally familiar to me. I/O words are specific to the Prop and not some general-purpose PC so that I end up with words that can deal with more than one bit at a time and set direction registers automatically etc. It wasn't possible to have a deep stack unless I moved it into the hub and then that would have slowed down most of the words most of the time so this also meant that most recursion is out. I'm looking at how I can make my data push and pop faster to improve performance in that department.
Design goals drive the choice of most MCU Forths to be less standard as opposed to the ease which ANS can be implemented on larger systems. Besides there are a lot of things in ANS that don't make sense on anything but a PC.
Now, how is your pfth performing? BTW, congratulations on getting a kernel up and running
Writing an ANS Forth for the Propeller has never been my goal, it's been my goal to have a mean and lean Forth for the Propeller and one that can run code that normally would require assembler for speed.
PropForth makes it VERY CLEAR in thier .Spin image files that it is NOT an ANSI Forth.
Some of this is a 'necessary evil' of building a Forth on a Propeller, some of it is the personal preferences of the originator.
I can live with the lack of conformity, just because the PropForth people have done a very good job of providing materials for slow learners, like myself.
TachyonForth is also NOT an ANSI Forth and has another different approach.
I am sure that there are those that will grumble, but those are the realities. I still am learning ANSI Forth in GForth on a netbook. It offers me a wider range of study material and may inspire new and better projects.
Here's a Forth program to determine the number of ANS Forth core words that are supported. I would be interested to hear how many words are supported by PropForth and Tachyon.
In propforth, the words "words" displays all the words in the dictionary. "free" tells you how much hub and core memory you have left.
The relevance of ANSI is when you want to do things the ANSI words support. We usually don't want to do many of those things on the prop, since they won't work as intended due to prop archticture. It sw a microcontrol, rather than a workstation processor.
Propforth works WITH what a prop can do, rather than fighting it to make it be something it isn't.
Yikes! I can't believe there's so much resistance to running a small program. This thread is about the ANS Forth 94 dictionary, which is why I posted the checker here. Any Forth should be capable of supporting any dictionary just by implementing the words in Forth. It just have to provide a complete set of of primitive words to do it. I would assume that words like "type" and "source" could be easily implemented in PropForth.
I'm not resisting, I ran it and it bombed on 'source' and 'type' I need to find out what those do and recreate them or else rewrite your test program. No big deal, just time consuming.If push comes to shove, I'll find another way to compare the lists.....but then I wouldn't learn more about Forths.
I do like how Mr. Byteloose jumped to the defense! The Forth is strong in this one, Master Yoda!!
: type ( addr n ... ) 0 do dup c@ emit 1 + loop drop ;
Source puts the address and size of the input source string on the stack. There should be other primitive words in PropForth and Tachyon that provide this functionality.
The relevance of ANSI is when you want to do things the ANSI words support. We usually don't want to do many of those things on the prop, since they won't work as intended due to prop archticture. It sw a microcontrol, rather than a workstation processor.
Propforth works WITH what a prop can do, rather than fighting it to make it be something it isn't.
Another great advantage of ANS Forth is it is a basis to start teaching from. When you have beginners, some conventions make teaching much easier. Forth without standards can be quite a moving target. And that can be far too daunting for the new learner.
An established reference results in a lot of authors willing to write about what it does and how best to use it. With little non-standard Forths, you just have to use what you know and learn as you go - not much reading material or mentoring.
I DO like the idea of an automated comparison to ANS Forth inside Forth. It is an added educational value to the whole scheme of learning Forth on the Propeller.
But we do have a rather restricted space for a dictionary and that is the bugaboo that has intially argued against ANS Forth from the beginning.
Dave, don't give up. Even if you end up with having to impliment a slower Forth than othes due to having to use some sort of Large Memory Model, it will be a reliable teaching tool.
I suppose somebody should write another smaller Forth for the Propeller and just call it Haiku. It may be small and abstrate, but entertaining.
Words, words, words - they certainly get in the way.
Comments
From what I read, it seems that by using PropForth, I will have only 32mbyte of SPI storage regardless of the card being 2gbytes.
It would be more interesting if files could be directly moved from a text editor on a PC to the PropForth platform via SDcard and you might appeal to more users with a greater diversity of Forth applications. I've already located at least one example of a simple text editior in Forth, but without writing to a file I don't see much need for it.
Since the card that I have actually cost about $10USD and I have the reader for it, it would be silly to order another card reader and pay the price of that plus shipping to Taiwan at this point in time. I can't seem to buy the card reader hardware locally.
Meanwhile, I will get yet another Propeller Proto Board up and running using Parallax's FAT16/32 solution, maybe with a real time clock included and see what SPIN has to offer.
I know there is a processor in the SDcard that does indeed rotate wear. But past comments on the web have said that using Linux Ext2 or Ext3 will not support it, nor will raw SPI written to blocks without the factory formatting in FAT16 or FAT32. So it seems that the File Allocation Table is involved in keeping file names while rotating block. I am not sure what role the Master Boot Record plays - maybe none. But up to 4 partitions may be made. Linux's Ext2 and Ext3 is actually more complex with Unix group and user rights imposed and that would take even more code than a minimal single user FAT16/32 solution (even LInux just accepts SDcard with factory format of FAT16/32 and doesn't recommend reformat). And I do have an SD document that mentions clearly that not using the 'factory format' will impare performance (but it is not explict on how).
I have lots to do and learn in PropForth as I still have not finished "Starting Forth" or all the PropForth material.
I am also running some examples in GForth when I don't have the PropForth available. In other words, I am learning two different solutions at the same time. In order to avoid more confusion, I am minimally involved with learning TachyonForth as I'd likely just never get anything right.
I see these little SPI 32Mbyte SRAM chips and have to wonder if they are yet another alternative. They may equal the SDcard in utility and speed.
http://uk.farnell.com/spansion/s25fl032p0xmfi011/memory-flash-32mb-3v-spi-8soic/dp/1972442
No card reader required and they are cheaper than SDcards.
I don't believe that what you heard about SD level wear requiring FAT is correct. The SD card is not that smart to discern the information and format of the card but it can remap sectors (512 bytes) much like a hard drive can. This can slow things down when the card starts wearing but it's all covered in the specification so you don't have to worry about it.
BTW, Solid State Drives have TRIM commands but the closest that SD cards have is the ERASE command (CMD38).
It seems that PropForth just exploits the SDcard as an SPI SRAM without the FAT16/32 support you are hoping for.
In PropForth, one creates files on a desktop with a 'fl' command at the beginning of the ASCII text document. And then with whole file is opened in a text editior which can copy and paste it to the terminal screen which will execute it as if you typed it all it.
The 'fl' removes comments and extra white space -- just accepts what would normally be seen within a colon and a semi-colon. But multiple words are accepted in one go.
After you have your file loading into PropForth and active, you can use 'saveforth' to transfer an image to the SDcard. I am not sure about how naming or multiple images get managed.
Thanks, happy to hear from you. As this topic came up a long time ago, it may all be a tempest in a teapot. I can't find the info I imagined I read more than a year ago.
I remain very excited about Forth just because it is so interactive. For many learners, this is much more engaging that writing pages of code, compiling, loading and debugging.
I'm still not sure about all this. The SD card knows nothing about your file system. Therefore when you think you have deleted a file and freed all it's blocks the card does not know that. Hence I would guess it cannot map other blocks to those "deleted" blocks and it's wear levelling capability is reduced.
No idea about low level TRIM or ERASE commands but there is an fstrim command in Linux that works on SD cards and reports back that it has done something useful. Just been trying it out on my Raspberry Pi board.
"Risks of reformatting
Reformatting an SD card with a different file system, or even with the same one, may make the card slower, or shorten its lifespan. Some cards use wear leveling, in which frequently modified blocks are mapped to different portions of memory at different times, and some wear-leveling algorithms are designed for the access patterns typical of the file allocation table on a FAT16 or FAT32 device.[61] In addition, the preformatted file system may use a cluster size that matches the erase region of the physical memory on the card; reformatting may change the cluster size and make writes less efficient. The SD Association provides free formatting software to overcome these problems.[62]"
I also realize that reported incidents may not be accurate as hobbyist can create all sorts of problems with SDcards.
Parallax has mentioned that pullups should be on just about all pins used or not to prevent the cards from running hotter. Mistakes of wrong voltage, reverse polarity, or brownouts can all play havoc with these devices when used on a breadboard. If fact, I went with the Parallax micro SDcard reader because it has complete pullup resistors.
The cards will work with a lot of formats that are not standard, including Ext2, Ext3, and Ext4 and even Apple formats and NTFS. But I can't not find SD Association formating software for anything besides FAT16/32 and maybe FAT12.
"..and some wear-leveling algorithms are designed for the access patterns typical of the file allocation table on a FAT16 or FAT32 device"
Looks like the key thing here is "access patterns typical of" which does not mean actual knowledge of what is where in your file system.
Hence I'm still inclined to believe that blocks you have freed in your fs are still not necessarily free as far as the SD card knows. How would it know?
Hence that fstrim command in Linux.
You might want to read this thread for a discussion about the need for fstrim on sdcards :http://www.raspberrypi.org/phpBB3/viewtopic.php?f=63&t=19554
This is what happens when I run it on my Raspberry Pi:
Note: the trimmed bytes after the file temp was deleted.
I am not up to speed on what the 'fstrim' command does or doesn't do. I will try to understand it and what you are trying to say.
Okay, a computer doesn't know that MicroSoft exists or what a file allocation table is. But if the wear leveling expects that methodology or something similar, and if you don't provide it - what is the result?
Since that comment in Wikipedia was in an SD Association document discussing SDcard formatting and dated 2011, the 'sense' I get is that it would be more prudent to comply if at all possible.
Yesterday, I saw SDcards for sale on the web that are specifically formatted to Ext2 or Ext3 for the Raspberry Pi.
That doesn't mean that they will hold up well. As I understand it, FAT16/32 are not controlled by MS, just longer than the 8 character names - but the SDcard Association doesn't want to explain too much of what makes its card so durable or everyone will clone the technology. It is apparently a 'trade secret' approach to control of their intellectual property.
I will use what is available for PropForth or TachyonForth (learning Forth on the Propeller is a very good way to quickly understand the Propeller) , but I may eventually figure out how to adapt to what I wold prefer - a completely functional SDcard in FAT16/32. If I have to run a two Propeller system to do so, that is okay.
Each to his own.
BTW, since Ext3 is a journalizing file system - it and NTFS may wear out SDcards faster due to that process alone.
The point I was trying to make all along is that SDcards were developed for FAT16/32 as their optimal performance (no journalizing, maybe no user designation), AND the SDcard Association still allowed anyone to format in other ways without the full feature set of wear leveling because they might have to evolve into different markets at some time in the future..
When I purchase my EEEpc 701 with solid-state hard disk, everyone was chatting about proper wear leveling. The simple fact was that nobody came up with a scheme to do it any better than Xandros. And Xandros' scheme was to make half the storage Write Only.
Does your Raspberry Pi have a Swap disk partition on an SDcard? That may be an invitation to failure in its own right. SDcards were never meant to heavily cycle read/write/erase. But it is quite easy to do in Linux.
it didn't like the word 'source'
it's not going to like 'type' either.
looks like will need to port the standards checker!
You keep mentioning that ANS word all the time (tut tut), will it make my Propeller software run faster, leaner, meaner? I don't think so. There are words in Tachyon which have been renamed or at least aliased but may also work a little differently. For instance I don't agree with some names and I prefer to use ones that are familiar to Spin/PASM programmers. An example is SHR instead of RSHIFT, not a big deal but totally familiar to me. I/O words are specific to the Prop and not some general-purpose PC so that I end up with words that can deal with more than one bit at a time and set direction registers automatically etc. It wasn't possible to have a deep stack unless I moved it into the hub and then that would have slowed down most of the words most of the time so this also meant that most recursion is out. I'm looking at how I can make my data push and pop faster to improve performance in that department.
Design goals drive the choice of most MCU Forths to be less standard as opposed to the ease which ANS can be implemented on larger systems. Besides there are a lot of things in ANS that don't make sense on anything but a PC.
Now, how is your pfth performing? BTW, congratulations on getting a kernel up and running
Writing an ANS Forth for the Propeller has never been my goal, it's been my goal to have a mean and lean Forth for the Propeller and one that can run code that normally would require assembler for speed.
Some of this is a 'necessary evil' of building a Forth on a Propeller, some of it is the personal preferences of the originator.
I can live with the lack of conformity, just because the PropForth people have done a very good job of providing materials for slow learners, like myself.
TachyonForth is also NOT an ANSI Forth and has another different approach.
I am sure that there are those that will grumble, but those are the realities. I still am learning ANSI Forth in GForth on a netbook. It offers me a wider range of study material and may inspire new and better projects.
In propforth, the words "words" displays all the words in the dictionary. "free" tells you how much hub and core memory you have left.
Propforth works WITH what a prop can do, rather than fighting it to make it be something it isn't.
I do like how Mr. Byteloose jumped to the defense! The Forth is strong in this one, Master Yoda!!
: type ( addr n ... ) 0 do dup c@ emit 1 + loop drop ;
Source puts the address and size of the input source string on the stack. There should be other primitive words in PropForth and Tachyon that provide this functionality.
Another great advantage of ANS Forth is it is a basis to start teaching from. When you have beginners, some conventions make teaching much easier. Forth without standards can be quite a moving target. And that can be far too daunting for the new learner.
An established reference results in a lot of authors willing to write about what it does and how best to use it. With little non-standard Forths, you just have to use what you know and learn as you go - not much reading material or mentoring.
But we do have a rather restricted space for a dictionary and that is the bugaboo that has intially argued against ANS Forth from the beginning.
Dave, don't give up. Even if you end up with having to impliment a slower Forth than othes due to having to use some sort of Large Memory Model, it will be a reliable teaching tool.
I suppose somebody should write another smaller Forth for the Propeller and just call it Haiku. It may be small and abstrate, but entertaining.
Words, words, words - they certainly get in the way.