Correct, on the Eval boards, there is a separate regulator powering the EEPROM and SD slot. It is enabled via the external reset circuit from USB and push button.
@RossH said:
... then sending the uSD some clocks (how many, to be safe?) before executing the HUBSET.
Eight is considered enough.
Still testing this - it will take me more time (using different uSD cards) to get more conclusive results, but adding 8 clocks after every uSD access appears to have improved things. I can now reliably read and write to cards that periodically had issues because the card became unresponsive. Fingers crossed!
However, I would still like to see a list of uSD card types that are considered reliable, and those that are best avoided.
@RossH said:
However, I would still like to see a list of uSD card types that are considered reliable, and those that are best avoided.
to quote my earlier self:
@Wuerfel_21 said:
However, such information is worthless, because these days honesty is dead and they'll just change out the internals of the same SKU without notice. And not to mention the many counterfeit cards in circulation.
It'll be SLC Nand Flash. 16 GB could be the very largest capacity available due to reduced layers compared to TLC/QLC products. Capacity takes a back seat to ruggedness.
PS: I'd have thought a video grade SD card would be the better suited though. It's not like the card has to hold critical system information for the car to operate.
Testing multiple software changes on multiple uSD cards takes a loooong time, especially when the errors are rare and unrepeatable. Here is what I have found so far:
Adding 8 clocks after every uSD operation is absolutely required. Thanks to those who recommended that. This seemed to solve several problems, including the problem of not being able to soft boot the Sandisk Ultra card using HUBSET. However, this card still has other problems (see point 3).
I have added a check before every uSD operation to ensure the card is in the "ready" state before proceeding. To do this, I use CMD1, retrying the command until the card says it is "ready". This technique was recommended in older SD specifications, but it is no longer recommended in current ones. However, it seems to improve the overall reliability of several of my cards (including one new card that failed only occasionally - I have not had a failure since implementing this change). And before someone posts that CMD1 is not recommended any more and that I should be using ACMD41, please note that CMD1 is still ok to use AFTER the initialization phase on both MMC and SD cards, whereas ACMD41 is not guaranteed to be supported by MMC cards. I suppose I could try ACMD41 first and then fall back to CMD1, but that seems overly cumbersome if you have to do it before every operation.
Even with both those changes, while the Sandisk Ultra card can now be rebooted via HUBSET, it still eventually fails (usually during a sector write operation). It is also extremely slow to respond even when it appears to be working ok. No idea why. It may simply be a one-off dud or a cheap no-name card in Sandisk packaging, but the same card does work (and works fast) in every other device I have tried it in, so it is still possible there is some basic incompatibility between the P2 and this type of card. However, Sandisk Ultra cards are literally cheap as chips, so when I get a chance I will buy another one just to check.
I will also see if I can track down one of those "industrial grade" uSD cards!
Flexspin's SD driver code doesn't use any commands for the ready check. It just keeps clocking more batches of 8 clocks, waiting for a 0xFF response (MISO pin returned high due to the pull-up resistor).
@evanh said:
Flexspin's SD driver code doesn't use any commands for the ready check. It just keeps clocking more batches of 8 clocks, waiting for a 0xFF response (MISO pin returned high due to the pull-up resistor).
I went to buy another Sandisk Ultra uSD card, but the store didn't have any. They did however have a 32GB Sandisk Extreme, so I bought one to replace my old one. I also bought a Verbatim 16GB card that was so cheap I just thought "why not?". That card turned out to be a real keeper!. Fast and reliable, and works with both the old software and the new! I will be going back to buy any they have left.
But the new Sandisk Extreme did not work at all - even the old software that worked very reliably on my previous Sandisk Extreme didn't work on this one. Clearly, as @Wuerfel_21 posted, you cannot rely on SD cards to remain the same even when they are sold as the same brand, class and size.
Worse, trying to make it work put the card into a state where even a Windows format would not make it work on the P2 - but luckily using the SD Association format program did. Whew! Repeated this a couple of times to make sure, and it happened every time. No idea why, but I have had previous issues using Windows format, and knew enough to try the SD Association one before concluding I had ruined the card (and how is that even possible???).
However, I did eventually manage to get this card working. Took out the CMD1 code and reinstated code similar to the "wait for 0xFF" code posted by @evanh - which I had already tried but which didn't seem quite as reliable as the CMD1 method. I think Sandisk must have updated their FLASH controller and the new one does not implement CMD1 correctly (which is still supposed to be mandatory, even though it is no longer used by modern cards).
More testing required, but I will issue an updated SD card plugin in the Catalina thread once I have re-tested all the cards I have. My key test is a 2-hour self-hosted Catalina C compilation that really thrashes the SD card, so this testing process can take a looong time!
I just finished testing a new SD card plugin with all the uSD cards I have, and the results are that it works with every card EXCEPT my Sandisk Ultra 32Gb, which I now think may simply be faulty (can't confirm that until I can find another one).
Here are all the cards that DO work:
Sandisk Extreme 32GB
Kingston HC 32GB
Verbatim Premium 16GB (tested two of these)
RiDATA HC 8GB
Sandisk 8GB (see note 1)
Sandisk 2GB (see notes 1 & 2)
Sandisk 1GB (see notes 1 & 2)
Here is the only one that still does NOT work:
Sandisk Ultra 32GB
Note 1: These are very old cards I use in various Propeller 1 boards, and any details other than the brand name are worn off.
Note 2: Windows won't format these cards as FAT32 with a 32k cluster size, and my test needs 32k clusters due to the size of the Catalina executables (e.g. the main compiler executable - rcc.bin - is 2.8MB). So instead I format them as FAT16. But the P2 cannot boot from a FAT16 uSD card so I test them by programming Catalyst into FLASH and booting from that instead of from the uSD. Once booted, Catalyst can use both FAT16 and FAT32 volumes, so it all works.
I will post the new SD card plugin on the Catalina thread (here).
@RossH said: Note 2: Windows won't format these cards as FAT32 with a 32k cluster size, and my test needs 32k clusters due to the size of the Catalina executables (e.g. the main compiler executable - rcc.bin - is 2.8MB). So instead I format them as FAT16. But the P2 cannot boot from a FAT16 uSD card so I test them by programming Catalyst into FLASH and booting from that instead of from the uSD. Once booted, Catalyst can use both FAT16 and FAT32 volumes, so it all works.
>
I am very sure I posted the guiformat tool earlier in this very thread, but here it is again: http://ridgecrop.co.uk/index.htm?guiformat.htm Y'all should stop letting microsoft make your life worse...
(if it throws funny errors, use the stock windows format dialog once and then run guiformat)
@Wuerfel_21 said:
I am very sure I posted the guiformat tool earlier in this very thread, but here it is again: http://ridgecrop.co.uk/index.htm?guiformat.htm Y'all should stop letting microsoft make your life worse...
(if it throws funny errors, use the stock windows format dialog once and then run guiformat)
These SD cards are too small for FAT32. Here is what I get from guiformat when I try to use 32k clusters ...
Oh, I never realized that's an extant limitation...
Though if it's about random access to a program file, my usual approach is to require the file to be in one contiguous fragment. That's almost guaranteed when copying a fresh file to a card.
@Wuerfel_21 said:
Oh, I never realized that's an extant limitation...
Though if it's about random access to a program file, my usual approach is to require the file to be in one contiguous fragment. That's almost guaranteed when copying a fresh file to a card.
Requiring programs files to be contiguous on the disk would be a bit restrictive - it would mean (for example) that you could not simply plug the SD card into your PC and read and write it. It would also increase the complexity of the Propeller file system when used for self-hosted development.
On the P1, the use of 32k clusters (the largest supported by FAT16) maximized the size of the program file that could be loaded because it minimized the size of the cluster list required to load the program, which was important given the very limited Hub RAM available on the P1. It would be less necessary on the P2 where there is much more Hub RAM available - but I took the easy option and adapted my P1 code for the P2 - and the load process is so complex that I can't face rewriting it yet again - every time I have to go back to work on it I start to sweat and my hands begin to shake!
@Wuerfel_21 said:
Oh, I never realized that's an extant limitation...
Though if it's about random access to a program file, my usual approach is to require the file to be in one contiguous fragment. That's almost guaranteed when copying a fresh file to a card.
Requiring programs files to be contiguous on the disk would be a bit restrictive - it would mean (for example) that you could not simply plug the SD card into your PC and read and write it.
It's really not. When copying a file, the PC always uses a contiguous area when a available. You only get problems when overwriting a file that already exists with a larger one or if there just isn't enough contiguous free space. Perfectly fine for the normal use case of the program file being read-only except for when it's updated on a PC. I know the FAT library that flexC's libc uses internally has a pre-allocate function for writing contiguous files, though I haven't used it yet.
Perhaps I should repeat that testing with an Ultra 32GB rather than Ultra 16GB. Are these from officeworks?
I've also ordered a 512GB Team SDXC to pick up from the wholesaler perhaps today
Thanks. I'll have another go at getting Taqoz working.
I am sure my Sandisk Ultra 32Gb has a problem, but I am not sure exactly what it is, or if it is just a one-off problem with this card. Yes, it was from Officeworks. But the nearest one is 100km away, and I have not been back again yet and I can't find one anywhere locally. If I don't get a chance to go back soon then I'll try ordering another one on-line to compare it with.
By adding the 8 extra clocks I was able to make this card work for a while ... but never long term. It sometimes works for hours but it always eventually locks up and becomes unresponsive. When it does so, it does so in such a way that I have to reformat the card. If you are not thrashing the card hard for some reasonable time (in my case I do a large self-hosted Catalina C compilation that takes 2 hours) then you might never realize it was not reliable. But in this test it fails 100% of the time - and it is the only card I have that has this particular behaviour.
@Wuerfel_21 said:
Oh, I never realized that's an extant limitation...
Though if it's about random access to a program file, my usual approach is to require the file to be in one contiguous fragment. That's almost guaranteed when copying a fresh file to a card.
Requiring programs files to be contiguous on the disk would be a bit restrictive - it would mean (for example) that you could not simply plug the SD card into your PC and read and write it.
It's really not. When copying a file, the PC always uses a contiguous area when a available. You only get problems when overwriting a file that already exists with a larger one or if there just isn't enough contiguous free space. Perfectly fine for the normal use case of the program file being read-only except for when it's updated on a PC. I know the FAT library that flexC's libc uses internally has a pre-allocate function for writing contiguous files, though I haven't used it yet.
My main use case ATM is using the uSD as a file system for self-hosted compilations, so I am often overwriting existing program files with new versions.
Will try it. Thanks. I wonder if this can also be used to get around the issue that the boot program on a P2 uSD has to be in the early sectors or the boot loader code can't use it. That one catches me out occasionally, and is a pain now that uSD cards have so much capacity. I don't remember all the details now - I think cluso99 mentioned it somewhere. I'll try and find it. EDIT: Found it - see next post.
The SD booter will only look in the first cluster size of directory entries on the SD card. In current sized cards this is 32KB meaning 64 sectors and there are a possible 32 file entries per sector. So the booter will only look at the first 2048 entries for the two boot files _BOOT_P2.BIX and _BOOT_P2.BIY. The filenames must be uppercase.
As far as I know, this is still true, and it is another good reason for using larger cluster sizes. Although I may be the only one who routinely writes 5000+ files to the SD card (most of these are Catalina's run-time library files).
So ... I finally got that 32GB Sandisk Ultra SD Card to work reliably! How? Simple - don't boot from it, is how.
To use older, smaller SD cards (anything 2GB or less) with the P2 you cannot use FAT32 but you can still use FAT16 by programming Catalyst into FLASH and booting from that instead of from the SD card. So it occurred to me (actually, I did it by accident, forgetting to flip the DIP switch on my P2 EDGE board) to try the Ultra using this method. Same SD card, same format (FAT32), same data on the card - but with the binary normally loaded from _BOOT_P2.BIX loaded from FLASH instead.
The result is 100% after 6 retries so far:
with the DIP switch set to boot from FLASH, the test works every time. My two hour compilation runs to completion.
with the DIP switch set to boot from the SD Card, the test fails every time. It boots up ok and runs the first few compilation steps, but locks up after about an hour.
The only difference is the setting of the FLASH DIP switch. I have looked at the schematics of the P2 EDGE, and I can see that the SD Card and FLASH share pins, but I cannot see why this should cause problems with only one SD Card, and not with any other cards.
A free copy of Catalina to anyone who can figure out the answer!
Comments
References for my comment above:
https://forums.parallax.com/discussion/comment/1551238#Comment_1551238
Also video showing the issue:
Is soft reset any more of an issue than hard reset?
On a Eval board, I would say yes. I believe the Eval board will powercycle the SD cards on hard reset. It doesn't on the Edge.
Correct, on the Eval boards, there is a separate regulator powering the EEPROM and SD slot. It is enabled via the external reset circuit from USB and push button.
Still testing this - it will take me more time (using different uSD cards) to get more conclusive results, but adding 8 clocks after every uSD access appears to have improved things. I can now reliably read and write to cards that periodically had issues because the card became unresponsive. Fingers crossed!
However, I would still like to see a list of uSD card types that are considered reliable, and those that are best avoided.
Ross.
to quote my earlier self:
(Required reading on such subjects: https://www.bunniestudios.com/blog/?page_id=1022 )
Depressing
I see our wholesaler has 512GB cards around the us$22 mark now, I'll get one next week for testing
Was just looking at the Lexus Dashcam option and see this:
Industrial-Grade MicroSD
A 16GB industrial-grade MicroSD is included for high-temperature and write-cycle durability.
I didn't know there were industrial grade uSD cards. That's interesting.
But Lexus only including 16 GB card? That does not sound high end...
It'll be SLC Nand Flash. 16 GB could be the very largest capacity available due to reduced layers compared to TLC/QLC products. Capacity takes a back seat to ruggedness.
PS: I'd have thought a video grade SD card would be the better suited though. It's not like the card has to hold critical system information for the car to operate.
An update:
Testing multiple software changes on multiple uSD cards takes a loooong time, especially when the errors are rare and unrepeatable. Here is what I have found so far:
I will also see if I can track down one of those "industrial grade" uSD cards!
Ross.
Flexspin's SD driver code doesn't use any commands for the ready check. It just keeps clocking more batches of 8 clocks, waiting for a 0xFF response (MISO pin returned high due to the pull-up resistor).
static int wait_ready (void) /* 1:OK, 0:Timeout */ { BYTE d; UINT tmr, tmout; tmr = _cnt(); tmout = _clockfreq() >> 1; // 500 ms timeout for(;;) { rcvr_mmc( &d, 1 ); if( d == 0xFF ) return 1; if( _cnt() - tmr >= tmout ) return 0; } }I do that too.
A mixed bag of updates!
I went to buy another Sandisk Ultra uSD card, but the store didn't have any. They did however have a 32GB Sandisk Extreme, so I bought one to replace my old one. I also bought a Verbatim 16GB card that was so cheap I just thought "why not?". That card turned out to be a real keeper!. Fast and reliable, and works with both the old software and the new! I will be going back to buy any they have left.
But the new Sandisk Extreme did not work at all - even the old software that worked very reliably on my previous Sandisk Extreme didn't work on this one. Clearly, as @Wuerfel_21 posted, you cannot rely on SD cards to remain the same even when they are sold as the same brand, class and size.
Worse, trying to make it work put the card into a state where even a Windows format would not make it work on the P2 - but luckily using the SD Association format program did. Whew! Repeated this a couple of times to make sure, and it happened every time. No idea why, but I have had previous issues using Windows format, and knew enough to try the SD Association one before concluding I had ruined the card (and how is that even possible???).
However, I did eventually manage to get this card working. Took out the CMD1 code and reinstated code similar to the "wait for 0xFF" code posted by @evanh - which I had already tried but which didn't seem quite as reliable as the CMD1 method. I think Sandisk must have updated their FLASH controller and the new one does not implement CMD1 correctly (which is still supposed to be mandatory, even though it is no longer used by modern cards).
More testing required, but I will issue an updated SD card plugin in the Catalina thread once I have re-tested all the cards I have. My key test is a 2-hour self-hosted Catalina C compilation that really thrashes the SD card, so this testing process can take a looong time!
What a joy SD working with uSD cards is!
Ross.
I just finished testing a new SD card plugin with all the uSD cards I have, and the results are that it works with every card EXCEPT my Sandisk Ultra 32Gb, which I now think may simply be faulty (can't confirm that until I can find another one).
Here are all the cards that DO work:
Here is the only one that still does NOT work:
Note 1: These are very old cards I use in various Propeller 1 boards, and any details other than the brand name are worn off.
Note 2: Windows won't format these cards as FAT32 with a 32k cluster size, and my test needs 32k clusters due to the size of the Catalina executables (e.g. the main compiler executable - rcc.bin - is 2.8MB). So instead I format them as FAT16. But the P2 cannot boot from a FAT16 uSD card so I test them by programming Catalyst into FLASH and booting from that instead of from the uSD. Once booted, Catalyst can use both FAT16 and FAT32 volumes, so it all works.
I will post the new SD card plugin on the Catalina thread (here).
>
I am very sure I posted the guiformat tool earlier in this very thread, but here it is again: http://ridgecrop.co.uk/index.htm?guiformat.htm Y'all should stop letting microsoft make your life worse...
(if it throws funny errors, use the stock windows format dialog once and then run guiformat)
These SD cards are too small for FAT32. Here is what I get from guiformat when I try to use 32k clusters ...
Oh, I never realized that's an extant limitation...
Though if it's about random access to a program file, my usual approach is to require the file to be in one contiguous fragment. That's almost guaranteed when copying a fresh file to a card.
Try formatting the 32 gb card as 16 gb
I don't seem to have a format program that has that as an option. What program do you use?
Don't remember, but maybe was diskpart:
https://superuser.com/questions/1303872/decrease-flashdrive-capacity
There was also a decent and fast fat32 formatter built into to the extended (non Rom) version of Taqoz
Ross I take it you are sure about the Sandisk Ultra 32GB? I found the Sandisk Ultra 16GB did work despite not releasing MISO, when I tested in the other thread,
https://forums.parallax.com/discussion/comment/1465472/#Comment_1465472
Perhaps I should repeat that testing with an Ultra 32GB rather than Ultra 16GB. Are these from officeworks?
I've also ordered a 512GB Team SDXC to pick up from the wholesaler perhaps today
Requiring programs files to be contiguous on the disk would be a bit restrictive - it would mean (for example) that you could not simply plug the SD card into your PC and read and write it. It would also increase the complexity of the Propeller file system when used for self-hosted development.
On the P1, the use of 32k clusters (the largest supported by FAT16) maximized the size of the program file that could be loaded because it minimized the size of the cluster list required to load the program, which was important given the very limited Hub RAM available on the P1. It would be less necessary on the P2 where there is much more Hub RAM available - but I took the easy option and adapted my P1 code for the P2 - and the load process is so complex that I can't face rewriting it yet again - every time I have to go back to work on it I start to sweat and my hands begin to shake!
Ross.
It's really not. When copying a file, the PC always uses a contiguous area when a available. You only get problems when overwriting a file that already exists with a larger one or if there just isn't enough contiguous free space. Perfectly fine for the normal use case of the program file being read-only except for when it's updated on a PC. I know the FAT library that flexC's libc uses internally has a pre-allocate function for writing contiguous files, though I haven't used it yet.
Thanks. I'll have another go at getting Taqoz working.
I am sure my Sandisk Ultra 32Gb has a problem, but I am not sure exactly what it is, or if it is just a one-off problem with this card. Yes, it was from Officeworks. But the nearest one is 100km away, and I have not been back again yet and I can't find one anywhere locally. If I don't get a chance to go back soon then I'll try ordering another one on-line to compare it with.
By adding the 8 extra clocks I was able to make this card work for a while ... but never long term. It sometimes works for hours but it always eventually locks up and becomes unresponsive. When it does so, it does so in such a way that I have to reformat the card. If you are not thrashing the card hard for some reasonable time (in my case I do a large self-hosted Catalina C compilation that takes 2 hours) then you might never realize it was not reliable. But in this test it fails 100% of the time - and it is the only card I have that has this particular behaviour.
Ross.
My main use case ATM is using the uSD as a file system for self-hosted compilations, so I am often overwriting existing program files with new versions.
Will try it. Thanks. I wonder if this can also be used to get around the issue that the boot program on a P2 uSD has to be in the early sectors or the boot loader code can't use it. That one catches me out occasionally, and is a pain now that uSD cards have so much capacity. I don't remember all the details now - I think cluso99 mentioned it somewhere. I'll try and find it. EDIT: Found it - see next post.
Ross.
For future reference, here is the SD card boot issue I was talking about ...
https://forums.parallax.com/discussion/171690/trouble-booting-full-taqoz-distro-from-sd
Here is the relevant comment from Cluso99 ...
As far as I know, this is still true, and it is another good reason for using larger cluster sizes. Although I may be the only one who routinely writes 5000+ files to the SD card (most of these are Catalina's run-time library files).
Ross.
"Riddle me this, Batman ..."
So ... I finally got that 32GB Sandisk Ultra SD Card to work reliably! How? Simple - don't boot from it, is how.
To use older, smaller SD cards (anything 2GB or less) with the P2 you cannot use FAT32 but you can still use FAT16 by programming Catalyst into FLASH and booting from that instead of from the SD card. So it occurred to me (actually, I did it by accident, forgetting to flip the DIP switch on my P2 EDGE board) to try the Ultra using this method. Same SD card, same format (FAT32), same data on the card - but with the binary normally loaded from
_BOOT_P2.BIXloaded from FLASH instead.The result is 100% after 6 retries so far:
The only difference is the setting of the FLASH DIP switch. I have looked at the schematics of the P2 EDGE, and I can see that the SD Card and FLASH share pins, but I cannot see why this should cause problems with only one SD Card, and not with any other cards.
A free copy of Catalina to anyone who can figure out the answer!
Ross.