SD Sector Editor?
VIRAND
Posts: 656
I have used disks by sector instead of file before.
I am wondering if such a program-specific use of SD cards is possible,
or does internal wear-levelling somehow require a filesystem otherwise likely
to corrupt sectors that are indexed directly as sectors?
I guess the analogy of the app is like a simple ordinary book with numbered pages that don't need to move.
Why use files if the sectors are always the right size for all of the pages? Sectors are just simpler I think.
Transfer as one big file would make sense when it needs to be done. But isn't that just a "disk image"?
Do I need any unexpected hints on how to use SD cards this way vs. doing the same
thing on a floppy? Is any structure necessary such as formatting or MBR if only sectors are needed?
Wear leveling should hopefully be not relevant since I don't think I'd ever rewrite
the same page of a book anywhere near the mean failure rates, just worried that
the lack of filesystem might cause automatic overwriting during wear leveling, if
SD's do that internally in PC-independent devices. Would an edited sector (practically) return
to the same physical sector or a different one? Would old pages appear where
new blank ones are expected to be? Would unexpected behavior mix everything up?
Another way of understanding my expectation is as if there is a shelf with lots of eeproms in cubbies
on it, and a robot grabs the one it is asked for, gives it to the propeller which reads and writes it,
then the robot puts it back on the shelf where it was, and will be the next time it is needed.
New eeproms that have never been used before should be found blank when they are used for the first time,
although this doesn't matter if the manufacturer wrote test patterns on them, since for the purpose
of the SD sector editor, it's blank if it doesn't start with something relevant like BOOKNAME PAGE #314
which would always be expected in sector 314 unless it had not yet been written. And if the sector failed,
the reader, as if someone spilled ink on or tore the page, would simply find it in that condition, and say,
page 314 is damaged, but will not be removed nor repaired nor replaced, but left "AS-IS".
Sorry, too many words to explain the specific and obvious nature of the "book" example.
Technically, readsector(n) and writesector(n) are sufficient for this purpose, without anomalies.
I am wondering if such a program-specific use of SD cards is possible,
or does internal wear-levelling somehow require a filesystem otherwise likely
to corrupt sectors that are indexed directly as sectors?
I guess the analogy of the app is like a simple ordinary book with numbered pages that don't need to move.
Why use files if the sectors are always the right size for all of the pages? Sectors are just simpler I think.
Transfer as one big file would make sense when it needs to be done. But isn't that just a "disk image"?
Do I need any unexpected hints on how to use SD cards this way vs. doing the same
thing on a floppy? Is any structure necessary such as formatting or MBR if only sectors are needed?
Wear leveling should hopefully be not relevant since I don't think I'd ever rewrite
the same page of a book anywhere near the mean failure rates, just worried that
the lack of filesystem might cause automatic overwriting during wear leveling, if
SD's do that internally in PC-independent devices. Would an edited sector (practically) return
to the same physical sector or a different one? Would old pages appear where
new blank ones are expected to be? Would unexpected behavior mix everything up?
Another way of understanding my expectation is as if there is a shelf with lots of eeproms in cubbies
on it, and a robot grabs the one it is asked for, gives it to the propeller which reads and writes it,
then the robot puts it back on the shelf where it was, and will be the next time it is needed.
New eeproms that have never been used before should be found blank when they are used for the first time,
although this doesn't matter if the manufacturer wrote test patterns on them, since for the purpose
of the SD sector editor, it's blank if it doesn't start with something relevant like BOOKNAME PAGE #314
which would always be expected in sector 314 unless it had not yet been written. And if the sector failed,
the reader, as if someone spilled ink on or tore the page, would simply find it in that condition, and say,
page 314 is damaged, but will not be removed nor repaired nor replaced, but left "AS-IS".
Sorry, too many words to explain the specific and obvious nature of the "book" example.
Technically, readsector(n) and writesector(n) are sufficient for this purpose, without anomalies.
Comments
However, you're just shooting yourself in the foot later when you may want to acess the stuff on a PC or in the future when you have forgoten the format of what you have done. Saving everything as txt files is much easier.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Nyamekye,
Wear leveling is completely internal. SD card takes care of reorganizing on sector level totally transparent to any kind of file system.
Most people can't understand why I see it that way.
Everything can be done faster without a PC.
The only things I do with PCs is use the internet and write Spin code and order more Propellers.
But if for some reason I did want to access the data with a PC for some reason, (like...hmmm...I give up, Why?)
since it is text, I'd generate a text file. (But why can't a PC read sectors just as easily... anymore?)
If I just wanted to print it, why not just connect the printer to the Propeller?
If I just wanted to process it, well, how do you program a PC to do that faster than a Propeller?
If I just wanted to put the data online, there's always those Netburner type things, or TELNET.
Most PCs are full of viruses and spyware and surrounded by people who think that's normal and OK.
Of course, that is more than enough reason to avoid all the trouble with them.
And strictly, I DISAGREE with EULA! Have you read it? It's so unconstitutional that EUROPE is suing Microsoft!
The ZiCog team decided not to worry about direct sector access to SD cards a long while back. We just do it. Basically when ZiCog runs CP/M then the entire CP/M disk image of 8Mbytes exists in a single file under FAT on the SD card. That file is written to a newly formatted card so as to be sure it exists in consecutive sectors. ZiCog finds the first SD block of that file on start up and from then on accesses blocks within it directly without using the FAT driver.
Originally ZiCog did not use FAT at all. It just assumed the CP/M disk image was at block zero on the card.
Problem is then how to get the CP/M disk images on and of a card from the PC with no FAT file system.
This is only a problem in the Windows world. In Linux you can write any data you like to any block position on the card using the "dd" command. dd will also read from anywhere.
For example:
dd if=somefile of=/dev/sg1
will read data from "somefile" and write it to the raw sectors of the SD card in a USB card reader/writer which has device name /dev/sg1 in this case.
dd has many options that allow reading/writing from specific blocks on the card. Why Windows cannot do such a simple thing is beyond me.
As a bonus using Linux gets you away from viruses, spyware, ELUAs and all that other stuff that makes PCs a millstone around your neck[noparse]:)[/noparse]
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
The way I understand it, it works something like this:
1. Flash is organized as blocks of some size, which may be larger than a 512 byte sector. There are typically more blocks than necessary. (So an 8GB card may have 9GB of blocks.)
2. Flash writes are done on a per-block basis. So a change to one sector in the block means the whole block is erased and re-written.
3. Instead of erasing and writing to the same block, instead one of the spare blocks is written, then the old block is erased and used for future writes. (Each block has some additional data containing the external address.)
4. Blocks may be pooled, so there may be only a small number of spare blocks for a set of active blocks.
"Wear Leveling" is a natural side effect of having spare blocks. I believe more sophisticated flash will also rotate in blocks which have not been overwritten, to ensure a more even distribution of use. The spare blocks also provide the excess capacity to replace any failed blocks.
It should also be obvious that it's much better (and faster) to write multiple sectors at once so the flash block doesn't have to be re-written for each sector. It's also better to try to spread writes over the entire "disk" instead of repeatedly updating the same sectors. (Because the spare blocks typically aren't shared across the entire disk. So if the OS is always writing to the first n sectors (e.g. the FAT) and those sectors are in a single pool, then the blocks in that pool will be used more heavily. It may be that some flash controllers reorder the address lines to automatically put consecutive sectors in different pools, but that doesn't help if the OS rewrites a single sector.)
Note: The flash assumes a block contains valid data until it is written to. So if only 1 block is used then it and the spare blocks in the same pool will be used while the other blocks will not be used. Again, some flash controllers try to resolve this inequity automatically.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Composite NTSC sprite driver: Forum
NTSC & PAL driver templates: ObEx Forum
OnePinTVText driver: ObEx Forum
As for Flash, it seems to me to be relatively unreliable and somewhat fishy compared to EEPROM.
Come to think of it, as I recall, there was an eeprom PIC whose newer flash equivalent wore out faster,
and an eeprom 8051 whose newer flash equivalent often failed to program more than once.
And then, there is a lot of weirdness going on even in some flash replacements for eeproms,
especially "security" features.
At this point I'm thinking flash is worse than a hard drive, while eeprom maybe better.
MMC maybe more reliable than SD.
What about SD DRM? (DRM is optional except for V/Moh/Win7, its not required by DMCA although hacking it is prohibited)
My best guess is that SD DRM is what some fools pay for when they license the special non-mmc SD protocol,
and I guess it normally uses the device serial number as an encryption key, which some STUPID digital cameras sometimes use.
I think that is all it is, so I don't know what other problem it might cause besides losing pictures when a camera is broken.
It is probably OK for me to use SD for the "book-like" app with no filesystem.
There will be few changes to pages and unlikely to wear them out.
There is unlikely to be a need for PC access since the app has nothing to do with PCs at all.
It is a microcontroller-specific app for self-sufficient Propellers and 8051s that a PC just can't do anymore.
If eeproms came in similar sizes for a similar price, they would be preferable.
A huge eeprom or NVRAM is preferable to something disk-like with blocks and sectors and filesystem is very irrelevant.
But SD is cheap and common so I can probably use it.
Your problem may be the OS you are using. I find Linux quite permissive in terms of what to do with devices and disks... like I have this file that is a filesystem I can mount it and write files to it. It may be possible in Window$ but I'm sure it is going to be messy and convoluted. Mac OSX is a bit more closed. But most things will work like in Linux but the people at Apple changed things just to make your life less fun.
The BSDs are that flexible too. May be you can get by using winblows just for the odd app that does not have a usable equivalent in Linux or Mac OSX (virtualized for instance) and the other for all the serious stuff like programming the prop. Do not forget that since BST happened life smiles at us
VIRAND:
All flash have some sort of sector. Atmel's Dataflash devices are quite nice and have a SPI interface. OK, they have no Gbyte storage.. but you probably do not need it either. The NAND flash devices can also be acquired, they have a parallel 8 or 16 bit interface. Nice little things. Sadly they come in fine-pitch TSSOP-48 packages...
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Visit some of my articles at Propeller Wiki:
MATH on the propeller propeller.wikispaces.com/MATH
pPropQL: propeller.wikispaces.com/pPropQL
pPropQL020: propeller.wikispaces.com/pPropQL020
OMU for the pPropQL/020 propeller.wikispaces.com/OMU
Post Edited (Ale) : 12/8/2009 1:34:46 PM GMT
DOS is good. Linux is good. Windows and hard drives are very bad. Now Mac is bad too?
Sure, I can agree with that. Even if you boot linux from CDROM you could still have a keylogger buggered flash BIOS.
Even a PROM BIOS branded American Megatrends Inc smells very phishy to me.