Better than sliced bread! SD3.01 FAT16/32 Driver Updated!
Kye
Posts: 2,200
Well, its a new year, and here's a new driver!
Now available on the OBEX is the (http://obex.parallax.com/objects/619/) is the SD3.01_FATEngine.
There's ALOT of new functionality, and alot of stuff is fixed. Stop using the SD2.0_FATEngine if you can and switch to the SD3.01_FATEngine. (I found ALOT of problems with the SD2.0_FATEngine while updating it... you won't run into the problems easily however... but they are there).
Anyway, new features (Please read the bottom of my post if nothing else):
The file system is locked now for any access from any cog. This ensures that the FAT structure and directory structure are kept intact.
Card detect and write protect are now natively supported. The file system driver will abort and throw errors if the card is not in its socket and if you try to write to a write protected card.
The block driver now is faster and much more robust. It supported a 20Mhz write clock and a 10Mhz read clock. The mounting code in the block driver is much better now and should allow more cards to work with the propeller chip. The rebooting code in the block driver is also much better now and relfects the propeller chip's bootloader 100%.
The file system driver SPIN code is now easier to read and follows the FAT standard to the letter. Previous hacks I used in the SD2.0 version have been removed. The file system code now inspects data on disk more carefully.
File system paths are now also supported. So... you can now do stuff like "mv /prop/code.txt /code.txt" and etc. With that in mind, the rename feature has been replaced with the move feature.
Litsting also works much better now. Now, you can list the next file in the current directory, open it, close it, and then list the next file and repeat.
In general alot of stuff works great now. I worked very hard to fix all the problems that were under the hood previously. Stuff is faster and more robust now.
...
And, on the flip side. There are a few things I don't like. Files are not locked when opened for reading and writing. This means you can delete an open file... open a file for writing more than once... etc. You can also create circular directories with the move function. And, the mount function may let in corrupted or unusable FAT file systems.
But, if you aren't trying to break the file system or the driver then you won't experience any problems, enjoy!
---
Please test out my driver and post speed test results from running the builtin "test" command. I would like to know the average speed and what cards work with the file system.
Thanks,
Now available on the OBEX is the (http://obex.parallax.com/objects/619/) is the SD3.01_FATEngine.
There's ALOT of new functionality, and alot of stuff is fixed. Stop using the SD2.0_FATEngine if you can and switch to the SD3.01_FATEngine. (I found ALOT of problems with the SD2.0_FATEngine while updating it... you won't run into the problems easily however... but they are there).
Anyway, new features (Please read the bottom of my post if nothing else):
The file system is locked now for any access from any cog. This ensures that the FAT structure and directory structure are kept intact.
Card detect and write protect are now natively supported. The file system driver will abort and throw errors if the card is not in its socket and if you try to write to a write protected card.
The block driver now is faster and much more robust. It supported a 20Mhz write clock and a 10Mhz read clock. The mounting code in the block driver is much better now and should allow more cards to work with the propeller chip. The rebooting code in the block driver is also much better now and relfects the propeller chip's bootloader 100%.
The file system driver SPIN code is now easier to read and follows the FAT standard to the letter. Previous hacks I used in the SD2.0 version have been removed. The file system code now inspects data on disk more carefully.
File system paths are now also supported. So... you can now do stuff like "mv /prop/code.txt /code.txt" and etc. With that in mind, the rename feature has been replaced with the move feature.
Litsting also works much better now. Now, you can list the next file in the current directory, open it, close it, and then list the next file and repeat.
In general alot of stuff works great now. I worked very hard to fix all the problems that were under the hood previously. Stuff is faster and more robust now.
...
And, on the flip side. There are a few things I don't like. Files are not locked when opened for reading and writing. This means you can delete an open file... open a file for writing more than once... etc. You can also create circular directories with the move function. And, the mount function may let in corrupted or unusable FAT file systems.
But, if you aren't trying to break the file system or the driver then you won't experience any problems, enjoy!
---
Please test out my driver and post speed test results from running the builtin "test" command. I would like to know the average speed and what cards work with the file system.
Thanks,
Comments
SanDisk 2GB uSD
>_ test
Running command: test
Byte write test... Writ 032 KBs at 003 KBs per second
Byte read test... Read 032 KBs at 003 KBs per second
Word write test... Writ 032 KBs at 006 KBs per second
Word read test... Read 032 KBs at 006 KBs per second
Long write test... Writ 032 KBs at 011 KBs per second
Long read test... Read 032 KBs at 013 KBs per second
Sector write test... Writ 032 KBs at 091 KBs per second
Sector read test... Read 032 KBs at 221 KBs per second
Memory write test... Writ 032 KBs at 093 KBs per second
Memory read test... Read 032 KBs at 234 KBs per second
Block append test... Writ 032 KBs at 100 KBs per second
Reverse seek test... Read 032 KBs at 209 KBs per second
>_ test
Running command: test
Byte write test... Writ 032 KBs at 003 KBs per second
Byte read test... Read 032 KBs at 003 KBs per second
Word write test... Writ 032 KBs at 006 KBs per second
Word read test... Read 032 KBs at 006 KBs per second
Long write test... Writ 032 KBs at 011 KBs per second
Long read test... Read 032 KBs at 012 KBs per second
Sector write test... Writ 032 KBs at 105 KBs per second
Sector read test... Read 032 KBs at 188 KBs per second
Memory write test... Writ 032 KBs at 109 KBs per second
Memory read test... Read 032 KBs at 198 KBs per second
Block append test... Writ 032 KBs at 122 KBs per second
Reverse seek test... Read 032 KBs at 194 KBs per second
>_ test
Running command: test
Byte write test... Writ 032 KBs at 003 KBs per second
Byte read test... Read 032 KBs at 003 KBs per second
Word write test... Writ 032 KBs at 005 KBs per second
Word read test... Read 032 KBs at 006 KBs per second
Long write test... Writ 032 KBs at 011 KBs per second
Long read test... Read 032 KBs at 013 KBs per second
Sector write test... Writ 032 KBs at 090 KBs per second
Sector read test... Read 032 KBs at 224 KBs per second
Memory write test... Writ 032 KBs at 093 KBs per second
Memory read test... Read 032 KBs at 236 KBs per second
Block append test... Writ 032 KBs at 124 KBs per second
Reverse seek test... Read 032 KBs at 267 KBs per second
But no prop code time to test
Minor problem is that I can't seem to get an SD card to mount. 'tis after midnight so I'm probably missing something simple. Need to swap back and forth to the v2 code to find the bug. fatenginestart passes (it would reboot if failed that?).
In the old code it was something like mountPartition(partition, "C")
The new code has no parameters - is that correct? (old code commented out)
Addit: I think mounting needs to be done manually - I added it to the original kyedos automatically on bootup as I figured you would always do this.
So - the command
what value do I use for partition? - a string or a number?
The new code always sets the check disk flags now so I removed the "C" option of setting them. Good luck!
But what if I use 2 SD cards AT same time on same Propeller on that system without Drive letter handler?
Drive letter handling is easy, just a bit of string processing to handle.
and it sends the message "Card Not Mounted" at startup.
There is no real time clock. If I disable the real time clock
Then no messages at all appear.
I wonder if it has something to do with not having a clock? On KyeDO2 the code for the RTC is there but it does not do anything as nothing is connected to pin 28 and 29. However, maybe your new code needs the clock?
Or maybe it is something else? eg WP and CD not being connected?
Would you be able to please test this with the above settings?
---
The "card not mounted" message does not mean the card is not inserted. It just means that you haven't typed "mount" in yet. Its a warning to tell you that the rest of the file system functions will do nothing until the card is mounted.
If you have the card detect lines enabled the driver then will say "card not detected" after you call the mount function if no card is found.
I still can't get it working.
That last line is the first thing being displayed, so do I type something to mount it?
So I need to narrow down the possibilities of why it is not working.
1) Do I need to mount the card? Do I type "mount"? Nothing else, no parameters?
2) Can I automate mounting? I say that because when I boot up with the default settings, on a terminal program I get no carriage return and no echo of characters so it is difficult to debug as lines keep being overlaid on top of each other. In KyeDOS2, one of the things I added was a routine to print carriage return and line feed, so if you changed it from CR only (or LF only) to CRLF it was only one line to change in the program. Different terminal programs have different requirements. If I add in a keyboard object like in KyeDOS 2, it gets complicated. So I need to simplify things.
In the mount PUB there is this line:
What is 'stringpointer'? Can I call this after starting the sd routine from within code rather than typing something in?
3) Can you explain this line of code more? Similar lines appear in other parts of the program
string.compareci to me implies a comparison of strings. It seems to be comparing "mount" with either "mount", in which case it is always true, or with a parameter, eg zero, in which case it is always false. If I wanted to call programmount(stringpointer) from within code, what would I put in "stringpointer"
Help here would be most appreciated!
The little program code you see there just takes a string as an argument. As in "mount 0". Its tokenizing the string however though. So you'll need to change the calls to str.tokenizeString in there.
Did you try:
It boots with "card not mounted" and when you type "mount" then enter, it replies with the same message "card not mounted"
Tried a few different combinations at startup - eg commenting out the ADC and DAC engine, but no matter, it won't mount the card. Error coming back is "card not mounted".
Any help here would be most appreciated. Hardware works fine with version 2. I'm 99% sure I have the same pin settings as in version 2.
and for reference, version 2 code:
I'm wondering about the RTC. Kye, would you be able to test your code with RTC = -1, and also physically pull the RTC chip out of its socket?
You should see an error llike "file system corrupted", "file system unsupported" or "disk I/O error". Not nothing...
In 3.0, you entered:
i've been trying to run the demo without much luck. all i get on my monitor is a thick green horizontal line in the center of the screen.
i checked that VGA was enabled and pin group is 2 (demoboard settings). am i missing something obvious?
should i rely on the serial interface alone?
blake
That fixes it and now it works! How terribly embarrassing for me that I missed this typo.
Well I guess I need to repay the favor by helping someone else.
@b.p.m - yes the green line is correct. I don't think Kye has quite glued it all together yet. I suspect it is "an exercise left to the reader". Addit: I think it might be an oscilloscope to display the microphone sound.
There are a few more bits and pieces to get a display going. You need a simple terminal, and one that behaves like a proper old fashioned typewriter where "line feed" moves the line up, and "carriage return" moves the carriage to the left (and dings a little bell). This version 3.0 does not behave like this because it is designed to talk to the propeller terminal program, so the first thing is to go through all the code and convert "CR" to "CRLF". Then you need a text driver, and a clear screen routine and the delete key needs to delete a character, and a local keyboard, and an interface that can accept input from both local keyboard and the serial port, and sends output to both the local display and the serial port.
KyeDOS v2.0 can do all this. I just need to port the code over...
Running command: test
Byte write test... Writ 032 KBs at 003 KBs per second
Byte read test... Read 032 KBs at 003 KBs per second
Word write test... Writ 032 KBs at 006 KBs per second
Word read test... Read 032 KBs at 006 KBs per second
Long write test... Writ 032 KBs at 011 KBs per second
Long read test... Read 032 KBs at 013 KBs per second
Sector write test... Writ 032 KBs at 093 KBs per second
Sector read test... Read 032 KBs at 281 KBs per second
Memory write test... Writ 032 KBs at 095 KBs per second
Memory read test... Read 032 KBs at 302 KBs per second
Block append test... Writ 032 KBs at 112 KBs per second
Reverse seek test... Read 032 KBs at 375 KBs per second
not sure what all those numbers mean but from looking at it im assuming writing to sd cards is much more efficient if i write bigger chunks instead a variable at a time. anyways cool object
The big big improvement with version 3 is the amazingly fast "boot" command. Kye has done a superb job here. This truly changes the way to use a propeller. In some ways, external memory may not be so important any more as one could potentially chain .bin files one after the other. Each can talk to the next via files written to the sd card. One could even consider a batch file system. Or do crazy things like run a Basic program than then runs a C program. Or even Shells a C program.
Things for Kye:
1) In the new string engine I could not find PUB endsWithString(stringToSearch, stringToFind) '' 12 Stack Longs . Is this in the new string library under a diff name?
2) I seem to have a very strange bug where LS works for about 10 lines then it hangs the VT100 terminal and indeed, the entire program. I suspect there is no range test and it is writing off the end of the allocated hub buffer and overwriting code. Solutions are to either fix the VT100 code, or to print out the files in columns using spaces rather than tabs. For the latter, is there a string function that returns the length of a string? Then one could print 20-strings.len to make up the column.
3) I've had to comment quite a few things out to make things fit, mainly the audio. Not ideal, but then again, using the new way of thinking, one would put that audio demo into a new program and do a 'boot' to it.
4) Regarding the 'boot' command, I am wondering if we can replicate something in CP/M, where if you typed a command, it would first look through the list of commands but if not found, it would look through the list of .COM files (in our case, .BIN files) and treat those as commands as well. So to run "movie.bin", instead of typing "boot movie.bin" I would just type "movie". I think it would be a matter of storing the name of the command, add .BIN to the end then running through the list of all the directory entries looking for a match.
All great fun!
modifications by J. Moxham Dr_Acula to create KyeDOS v3. Pins setup for the Dracblade (similar to the Propeller Demo Board)
- name this file KyeDOS3
- changed sd pins to 12,13,14,15. WP and CD = -1
- RTC DAT and CLK -1
- Audio and Microphone pins all set to -1
comment out 'adc: "PCM2.1_ADCEngine.spin" and compile to find all PUBs referenced by this
comment out adcengine start and PRI programTapeWavFile and dac: "PCM2.1_DACEngine.spin" and PRI programPlayWavFile
comment out programPlayWavFile(stringPointer) and programTapeWavFile(stringPointer) and PRI scope
comment out if(_VGA_Enabled) group
startup is ifnot( fat.FATEngineStart(_SD_DO, _SD_CLK, _SD_DI, _SD_CS, _SD_WP, _SD_CD, _RTC_DAT, _RTC_CLK, -1) and {
} com.COMEngineStart(_receiverPin, _transmitterPin, _baudRateSpeed))
comment out 'bmp: "VGA64_BMPEngine.spin" _VGA_BG_Color = bmp#black _VGA_FG_Color = bmp#green
add OBJ VT100: "VGA_1024_VT100" ' VGA VT100 Terminal Driver
copied VGA_1024_VT100.spin and VGA_80x40.spin to the working folder
added VT100 startup code - now have a blinking white cursor on a blue screen. 80x40 text.
commented out ' long scopeSwtich, scopeStack[17], scopeHeap[((_VGA_H_Res * _VGA_V_Res) * 2) / 32]
(need to watch memory F8 here while adding more objects to the OBJ section)
added key : "Pocketerm_Keyboard" ' pocketerm keyboard
added lcd : "DracLCD" ' 20x4 LCD driver for dracblade
added delay: "Timing" ' for millisecond delays
added the keyboard startup key.startx(26, 27, %100, 40) 'Start Keyboard Driver pin,pin,num,repeatrate
added lcd.start - this is dracblade specific with a 20x4 local keyboard. You can leave this out if not used
commented out the existing .com driver. There is a bug with xmodem with corrupted bytes above 100 in the buffer, this does not show up with typing as the buffer never gets this full
added 2 port serial driver (variation on Tim Moore's code). Copied object to current directory sio : "pcFullDuplexSerial2FC"
F8 now fails with all instances of .com
Added some printstring routines that send data to VGA and to the serial port and the LCD display and do CRLF properly
added sio.addport. removed startup for the .com
global search and replace "com.transmitstring" to "printstring"
global search and replace "com.transmitbyte" to "printchar"
edited a line with #comquotation marks. Keep doing F8 to find errors
edited PRI shellline so it can accept Backspace and can read from keyboard and serial port
found that the string engine syntax has changed. putcharacter is now buildstring
found that the string engine syntax has chnaged. getcharacters is now builtstring
rewrote PRI shelldecision
replaced instances of quotationmarks from the serial driver to a local variable quote (ascii 34)
added local contstants Horizontal_tab, Linefeed, CarriageReturn. Did global find/replace
F8 finally compiles with no errors!
added sio.start, print signon message
added PRI DisplayBinaryFiles. Changed to the new syntax for reading entries
** edited the string engine - added PUB endsWithString(stringToSearch, stringToFind) '' 12 Stack Longs
baud rate 38400 for testing
comment out ' _newLineCharacter = 13 , replace with crlf
go through the DAT shellProgramHelpStrings and replace _newlinecharacter with byte 13 and byte 10
commented out the wav file commands in the help section (can always uncomment them later) amd the PRI
replace instances of PrintChar(_newLineCharacter) with crlf or with printstringcr. Multiple instances
removed "are you sure" from the boot command
edited the 'help' commands so they each fit on one line
moved the check for mounted card after every command to above the loop so only checks once.
working on a very strange bug in programDIR - if send more than a certain number of tabs to the VT100 the whole spin program shuts down. Possibly overrunning a buffer and overwriting code. Need a better solution to line up files in columns
also shows up with the LS command.
---
@BPM -The green line you see is the data coming back from the audio ADC. It's "scoping" the audio input for the prop chip (think propeller demo board microphone). If you disabled the audio adc then you would get just a green line. Use the "play" command to see the audio ouput of a wav file. This should work always... even if the DAC is disabled.
I have a uSD setup on a breadboard. It works fine with version SD2.0 but not with SD3.01.
When I type "mount" or "mount 0" into the shell, I get a message "Card Not Detected."
If I set _SD_CD to -1 I get the error "File System Corrupted," even after formatting the card with version 2.0's demo.
Here are my modifications to your demo
Thanks for sharing your amazing work.
Duane
Do you have RTC on your bb? Have you tried:
The new code checks alot more stuff when mounting the SD card. If you get that warning then truely your file system has been compromised somewhere.
Just copy all the files off of it, reformat, then copy the files on and you'll be good to go.
-- Note: my format is just a mass delete of everything on the file system. It does not fix the patition format information.
1) added a true/false constant for the LCD 20x4 display. This is because the 20x4 display driver is unique to the dracblade, whereas much of the other code is portable to many other propeller platforms. All character input/output on Kyedos, including serial, goes through a couple of PUBs so it is easy to add/subtract display options in a few lines of code. It makes the code fractionally slower but I think it does add better flexibility to modify things, potentially with just one line in the CON section. One might, for instance, want to disable the serial port and just display to the VGA display.
2) Added a 6 column directory display.
3) Kye, can you please explain something. Regarding strings, I see some code that does this
Now, I gather you have a string, and now the str.tokenizestring points to this string.
Let's say I want to work with two strings, and compare them. In a local PUB, do I need to define an array for that second string? If I did, and say it was a small string, could I make the length 80, instead of 255? (so long as the string is always kept under 80).
Please forgive my strange ways of working with strings. I started with mbasic, where you never worried about length of strings. I then moved to sbasic, and pretty much all strings were 80 characters and the way you worked with them was closer to the metal, a bit like C.
I've also written a lot of string handling in assembly, which is even closer to the metal.
In many string handling routines, I found I could save memory by defining a number of generic strings at the beginning of the program, maybe call them string1, string2 etc, and each one is 80 bytes (20 longs), and I set myself a rule that each string stayed local in a function (copying strings between functions was done with pointers a bit like C). Then you could save memory by only having string1 to string7, and reusing them over and over.
I've kept a similar concept in more recent vb.net coding - over and over I use one string, "LineOfText" and it is reused by each function. Saves memory to make it a global string rather than define it for many individual functions.
Hub memory is starting to run out in Kyedos, so, can I define "string1", "string2" etc in the VAR section of the main program, and point to those strings using the Spin string object?
Or maybe you can suggest another way of working with strings that might be better?
Where is this heading?
I'd like a new PRI, which the program falls through to if it can't match any of the commands. Its simply then looks for a match with a .bin file on the sd card, and runs that if it exists. It would only be a few lines of vb.net code but I'm very confused translating this to spin. In vb.net pseudo code:
please forgive my mixing of languages. I see that there is a command
PUB stringCompareCI(characters, otherCharacters) '' 9 Stack Longs
maybe I just need to see a simple example of how to use this, as it clearly uses two strings.
Maybe I'm missing something (very likely!), but it would make the code more readable and I would think tighter as you eliminate all the extraneous calls to both the routines and the stringcompare functions.
Thanks for a great routine, its becoming the heart of my new project!
BruceP