Working around Microsofts FAT patents

heater · 2009-07-02 14:59

For anyone working on FAT file systems here is an interesting trick the Linux guys have come up with to avoid any pesky patent problems.

arstechnica.com/open-source/news/2009/07/vfat-linux-patch-could-circumvent-microsofts-patent-claims.ars

(Of course if you are into file systems you probably heard this already)

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.

OwenS · 2009-07-02 15:03

Or if you read Groklaw [noparse]:)[/noparse] Groklaw has a nice analysis of it also.

Baggers · 2009-07-02 15:50

happy days had by all then [noparse]:)[/noparse]
like M$ need anymore money anyway lol

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
http://www.propgfx.co.uk/forum/·home of the PropGFX Lite

·

rokicki · 2009-07-02 16:46

I'd rather sidestep this whole question. This is one of the main reasons fsrw only supports 8.3 names
(and frankly, unless someone steps up to twist my arm, I have no plans to change that). Does anyone
really need longer names on the Propeller?

jazzed · 2009-07-02 16:50

Does fsrw have directory support? Don't remember exactly, but directory support would ease some of the naming issues.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
--Steve

Propalyzer: Propeller PC Logic Analyzer
http://forums.parallax.com/showthread.php?p=788230

heater · 2009-07-02 17:02

Well, I could be out of touch here, but I thought it was patent worries that put people of going to the latest and greatest FAT standard, with directories, bigger than 2G disks, long file names etc.

If it's only this short/long name mangling that's subject to patents this is a way around.

Personally I have no need for long file names and I'm yet to miss directories. I'm all for having the smallest possible code space taken up for a file system on the Prop seeing as space is always a pressing issue with Prop applications.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.

jazzed · 2009-07-02 17:05

I guess if you have volume C through Z, you don't need directories [noparse]:)[/noparse]

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
--Steve

Propalyzer: Propeller PC Logic Analyzer
http://forums.parallax.com/showthread.php?p=788230

LoopyByteloose · 2009-07-02 17:44

Microsoft has mainly moved on to NTFS because they need a more complex security system for networking computers on LANs. Fortunately FAT in all its forms remain on SDcards, USBmemorysticks and other devices.

Linux has created a 'super block' that can mount even NTFS systems and operate on files in the usual ways of read, write, and copy. But, for Parallax purposes FAT is just about all you need unless someone in Parallax wants to create a Propeller driven file server. I suspect we are a long way from that.

What Microsoft does do - it copies NTFS to SDcards in the more primative FAT32 format and the security features [noparse][[/noparse]like file encryption] are not transfered. One should read the NTFS article at Wikipedia to get some idea of what is really happening. Fortunately, I don't need 2gbytes storage with any Parallax device, but it is getting a bit harder to buy any storage under 1Gbyte. Save your old cards for hobby work and take in throw aways from other people.

Also, take a good look at FreeDOS for supporting FAT based hobby projects. You may be pleasantly surprised as they have been creating drivers for CD-ROMs and DVDs to support both FAT and DOS.

And by all means, try Ubuntu Linux as a means for learning what Microsoft is really trying to keep secret. Open Source code really rocks.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Ain't gadetry a wonderful thing?

aka G. Herzog [noparse][[/noparse] 黃鶴 ] in Taiwan

rokicki · 2009-07-02 17:50

Well, no dates can be promised, but we are actively working on a bunch of things. So far we have
significant progress on multiblock support, sdhc support, and a faster block layer in general, and
on FAT32 support. We are also working on subdirectory support, seek, and multiple files.

With bst and #ifdef (and a short Perl script to expand the #ifdef's for people not using bst), we can
give everyone the best of both worlds, let them cherry pick the features they need and only pay
the code space for those features.

It's been a real kick working with lonesock on this. If you want to monitor our progress, we are at

http://sf.net/projects/fsrw/

Note that we have not yet *released* anything but you can easily check out or browse all the files
to see where are and where we are headed, and if you are brave, even test out some of the
code.

ElectricAye · 2009-07-02 20:21

rokicki said...
...Does anyone
really need longer names on the Propeller?

If you're talking about names of files stored on an SD card, I'd be happier with longer names so that I can identify sensor, plus date and time just by looking at the file name. But I'm thrilled with your fsrw so don't call me ungrateful.

Oldbitcollector (Jeff) · 2009-07-02 21:51

rokicki said...
Does anyone really need longer names on the Propeller?

Pretty please... please.. please.. please........

OBC

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
New to the Propeller?

Visit the: The Propeller Pages @ Warranty Void.

ThePenguinMaster · 2009-07-02 21:58

Long file names! This would be useful! I would love to see this feature!

Rayman · 2009-07-03 00:12

I think long names and fat32 infringe on Microsoft (at least as I understand it) so would prefer to avoid both.

I'm perfectly happy with Rokicki (and Mike Green's) drivers but would like:

1. Faster read
2. Multiple file read
3. Directory support

PS:· I think SDHC compatiblility is practically only an issue if we're allowed to use FAT32.· If that's possible, it would be great!·

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
My Prop Info&Apps: ·http://www.rayslogic.com/propeller/propeller.htm

localroger · 2009-07-03 01:28

long file names may or may not infringe on MS$ legitimate or not legitimate patent, but they are a horrible kludge requiring buffers we don't need to be implementing in order to support.

When I get time to finish it I have a system implementing full directory support, attribute bits, real DOS wildcards, and as many files open as you might want at a time with far less than 512 bytes per channel cost. It does not however for example remember where you are in the directory tree so that you can put the path in your prompt; that would require an expensive arbitrary sized buffer I'd rather not allocate. (The file system itself lets you nav down from a directory through the .. entry in the subdirectory itself. No housekeeping required.) And long file names are even more expensive than that, taking three normal dir entries each plus a lot of case coding and oh yes conversion to unicode for the filename. It's a great big unnecessary mess for embedded work.

Cluso99 · 2009-07-03 02:16

My 2c:

Just stick to 8.3 and Fat16 or FAT32 for >2GB.
As a possible choice, perhaps we could have multiple FAT16's on >2GB but this would possibly mean we couldn't access the second FAT16 from PC's, but hopefully the first would be OK.
Directories would complicate matters, but remove the 8.3 limitation.
Using the date field would be an advantage. Since normally we don't have time and date, perhaps we can come up with some ideas here?
Just KISS where possible.

For the emulations, heater and I are just using single files for each disk drive and internally each file is CPM compatible. The N8VEM (Z80) group have adopted the same philosophy. But I am not sure about how this will work with Prop operating systems.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:

· Home of the MultiBladeProps: TriBladeProp, RamBlade, TwinBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: Micros eg Altair, and Terminals eg VT100 (Index) ZiCog (Z80), MoCog (6809)
· Search the Propeller forums (via Google)
My cruising website is: ·www.bluemagic.biz·· MultiBladeProp is: www.bluemagic.biz/cluso.htm

BradC · 2009-07-03 02:28

Cluso99 said...
My 2c:

Just stick to 8.3 and Fat16 or FAT32 for >2GB.
As a possible choice, perhaps we could have multiple FAT16's on >2GB but this would possibly mean we couldn't access the second FAT16 from PC's, but hopefully the first would be OK.

Seconded.
It would be trivially easy to use the four first primary partitions for 2GB filesystems on an 8GB card. Even windows is clever enough to figure that out.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Missed it by ->" "<- that much!

mynet43 · 2009-07-03 02:45

@ rokicki

Thanks for the heads up as to what you're working on.

I'm really interested in the FAT32 support. Don't care about the long file names.

I need to be able to update the firmware on a product in the field. The easiest way is to send them an SD card. The problem is, that <= 2GB SD cards are becoming harder to find.

I actually went to your web site and downloaded the routine called: mmcsdhc_spi.spin.

I tried to substitute this for the fsrw.spin routine, since it has the readblock( ) and writeblock( ) subroutines. I soon noticed that it didn't have the "mount" routine, so I substituted a call to the "start" routine.

I then ran into the problem that it doesn't contain the "popen" routine needed to open the file you're looking for. At this point I gave up.

You asked us to try your routines [noparse]:)[/noparse] I did. Can you let me know what I'm doing wrong, or at least the status of this routine, if it's a "work in progress"?

Thanks for your help. I can't wait until you get this working! Keep up the good work!

Jim

Praxis · 2009-07-03 08:26

Maybe this may of some help.

Cheers

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Caelum videre iussit, et erectos ad sidera tollere vultus

Certe, toto, sentio nos in kansate non iam adesse

lonesock · 2009-07-03 10:34

mynet43 said...
@ rokicki

Thanks for the heads up as to what you're working on.

I'm really interested in the FAT32 support. Don't care about the long file names.

I need to be able to update the firmware on a product in the field. The easiest way is to send them an SD card. The problem is, that <= 2GB SD cards are becoming harder to find.

I actually went to your web site and downloaded the routine called: mmcsdhc_spi.spin.

I tried to substitute this for the fsrw.spin routine, since it has the readblock( ) and writeblock( ) subroutines. I soon noticed that it didn't have the "mount" routine, so I substituted a call to the "start" routine.

I then ran into the problem that it doesn't contain the "popen" routine needed to open the file you're looking for. At this point I gave up.

You asked us to try your routines [noparse]:)[/noparse] I did. Can you let me know what I'm doing wrong, or at least the status of this routine, if it's a "work in progress"?

Thanks for your help. I can't wait until you get this working! Keep up the good work!

Jim

Hi!

The file mmcsdhc_spi.spin is the block layer manager that sits _under_ fsrw. You will want to do a quick edit to fsrw.spin:

obj
   sdspi: "sdspiqasm"

change "sdspiqasm" to "mmcsdhc_spi"

Then just use fsrw as normal. We really appreciate you testing this out for us! (just know that this is still pre-alpha code.)

thanks,
Jonathan

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
lonesock
Piranha are people too.

Kye · 2009-07-03 14:04

Mmm, okay I'll put this out here:

I've been working on an auto switching FAT16/FAT32 driver which supports all the features.

I have file and folder renaming.
I have the ability to change directories.
I have the ability to search directories.
I have the ability to list what's in the directory. (Creation Date/Time, Last Acess Date, Size, Attribute, Last Write Date/Time)

You can create sub directories. You can make new files.
You can delete directories and you can delete files.

You can also format the card as FAT16/32 in a way that is compatible with windows.

You can open one file at a time for reading and writing. Then you can read a byte or write a byte to it at will.
You can also use file seek to move up and down through the file.

Also support is the ability to can attribute bits.

8.3 file names are only supported since long file names require alot of extra code.

...

I'd love to help out on this effort and share my code for use. So far my whole driver set is smaller than the current FSRW and its block drivers. However, I have a spin only block driver... because of this my file system is not very fast.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Nyamekye,

Mike Green · 2009-07-03 14:21

How about switching to either Rokicki's or my (FemtoBasic) low level (block) SPI driver? If you use the FemtoBasic driver, you also get the ability to boot Spin programs and you get an I2C driver as well. For booting, you need a FAT16/32 driver interface routine that provides the absolute address of the start of a file and the length of the first file extent

mynet43 · 2009-07-03 16:24

@ lonesock

Thanks for letting me know how to link the mmcsdhc_spi.spin routine.

I made three changes to the fsrw routine and called it fsrw_jim:

obj
   sdspi: "mmcsdhc_spi" ' special sdpsi routine that reads all types of SD cards!

Added variable "card_type" to the mount routine:

pub mount(basepin) | start, sectorspercluster, reserved, rootentries, sectors, card_type

Changed the final line in the mount routine to return card_type, instead of "0":

if (totclusters > $fff0)
      abort(-25) ' too many clusters
return card_type

This allows me to output "card_type" to help with the debugging.

Unfortunately, when I ran it, it returns with an abort code of -20. (With either a 1GB or 8GB SD/SDHC)

if (brlong(@buf+$36) <> constant("F" + ("A" << 8) + ("T" << 16) + ("1" << 24)) or buf[noparse][[/noparse]$3a] <> "6")
      abort(-20) ' not a fat16 volume

Do I need to make some change to this line of code, or is something else wrong?

Here's the first part of the code I'm using. It works fine with the regular fsrw:

OBJ
  sd  : "fsrw_jim"              ' FAT/FAT32 File   System Read/Write
  i2c : "Basic_I2C_Driver"
  vga : "VGA_1024"

VAR
   long f0[noparse][[/noparse]128]

DAT
   filname   byte "WgtBin", 0        ' file name on SD card to copy to EEPROM

PUB CopyToEEPROM | i, j, sstart, start_time, card_type

'   start the VGA and Keyboard routines
    vga.start(BasePin, kbdflag, kbd_dpin, kbd_cpin, kbd_lock, kbd_auto)

    vga.cursloc(0,1)                 ' locate cursor x,y
    vga.str(string("VGA Started",13))
    
    i2c.Initialize(i2c#BootPin)
    vga.str(string("i2c Started",13,13))

    card_type := \sd.mount(0)    ' base pin 0
    
    if card_type < 0              
      vga.str(string("Can't mount SD card"))
    else
      vga.str(string("SD Card Mounted"))
      
    vga.str(string(" -- Card Type = "))
    vga.dec(card_type)
    vga.str(string(13))
      
    if \sd.popen(@filname,"r")
      vga.str(string("Can't open file",13))
    else
      vga.str(string("File Opened",13))

I've attached the two files I'm using to test this. Let me know what you recommend.

I'm happy to keep helping you test this.

Jim

LoopyByteloose · 2009-07-03 16:25

From what little I know, it seems that using FAT16/32 on an SDcard or USBstick allows you to utilize internal firmware that extends the useful life of the device by dispersing the information to physical memory evenly. But using a low level SPI interface, you have to create software to support that feature. Am I wrong??

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Ain't gadetry a wonderful thing?

aka G. Herzog [noparse][[/noparse] 黃鶴 ] in Taiwan

Mike Green · 2009-07-03 16:37

The wear levelling occurs "below" the low level SPI interface. To the SPI interface, the card appears as a series of 512 byte blocks addressable by byte address (regular SD cards) or block number (HD SD cards). Inside the card, the blocks are larger, grouped into sectors and chips, with some error checking added. Physically, the blocks can be anywhere, not necessarily in order. The SD card controller will move them around for wear levelling and remember where they were put.

rokicki · 2009-07-03 17:47

Hi guys.

It looks like there may be *two* other file system implementations under way.

Let's see the code, please.

-tom

Kye · 2009-07-03 21:43

Haha, well let me finish it first. I still have alot more todo. I'd say i'll be done in less than two weeks. FAT32 support has not really been added yet but I'm making almost all the parts of the code ignore the FAT type except for the low level parts.

If you would like to know,

I only need to get the writing and seek rountines working. However, I've been spending alot of time trying to figure out just how to write them so that they are highly optimized, currently I'm having a few problems there.

But, its coming along pretty good. I've managed to do away with aborts entirely and I have a whole return type sytem built in for error checking.

Basically, theres a mount rountine that mounts and the SD card and a flag that is set when the card is mounted. Whenever an error occurs the card is automatically unmounted causing any rountine being used currently to return false.

If the rountine fails in like finding a file or can't process a request it returns a string of zero length.

And if the rountine suceeds it returns a string with length (to the name of the file usually you operated on).

Since everything is setup like this almost all my functions are only a few lines of code each and all of them build off each other. The stack is larger but the code size is generally smaller.

Anyway, I'm gonna try to get file/folder deletion and creation working tonight. Gotta go.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Nyamekye,

rokicki · 2009-07-04 00:57

Ahh, no writing yet? Okay, well, we'll be waiting then. I hate to waste time writing code that you're
just going to write better.

rokicki · 2009-07-04 01:10

More constructively, at least take a look at the interface between fsrw and sdspi. If your block layer and file system use the same
basic API (I think start, stop, readblock, writeblock) then you can benefit from the awesome block layers that lonesock is writing
(or, write your own superior ones that *we* can benefit from).

This was suggested earlier; I just thought I'd emphasize it.

Kye · 2009-07-04 01:48

Oh, I really hope so. I tried writing my own asm driver block layer.

But, well, it didn't work and I couldn't figure out why.... and, well, $%^& it.

So, I'm giving up on that. I spent two days trying to get it working with no results. I have no clue why nothing was working. I only know the read and write byte rountines did not register action from the sd card.

...

Okay, so I have attached a good portion of the working code. Basically all the constructs that I want to fill in are lying around there.

Any code you see with commented parts are not finalized yet. However, uncommented code is finalized (mostly - not fully tested yet however).

The driver works like this...

First call the function fileEngine which starts the filesystem driver (the block driver), and then call mountcard which makes all the functions active and allows you to use the card.

After mount card is called you are thrown into the root directory.

I do not know if the driver will work yet on other systems because I have not finished it and fully tested it yet. Use the pin numbers in the constant section to change your sd card pins.

...And rokicki, I'm not trying to out do you or lonesock. I'm creating my driver·(and others)·for a student and hobbyist education platform I plan to release soon. However, to well, *own* the system since its mostly software I kinda need to write my own drivers but also understand each one throughly so that I can make upgrades and preform matience and etc.

· I felt like sharing this one however because there was alot of talk about it and well... the platform I'm working on will basically be released to the community for use also - but with a very different 4th generational programing language enviorment.

More on that however·in the future!

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Nyamekye,

rokicki · 2009-07-04 04:16

Now that's more like it! Thanks for the code. I understand your motivation; I just hoped
that somehow fsrw would suit your needs or could be extended to suit your needs.

Anyway, it's all good.

-tom

Kye · 2009-07-04 14:48

Okay, when I finish my driver I'll post it with the spin only block driver.

From there you can just replace the spin block driver with yours (making sure of course that it has all the functionality of the spin block driver) and all will be good.

As you can see the spin only block driver communicates through only one function which returns true on sucess and false on failure.

...

As for SDHC, how should that work? Because... well its really akward with the addressing and such since you have more that 2^32 bytes. I'll need to be very careful with FAT32 just to make sure that I don't have a spin only variable suddenly become negative... due to overflow.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Nyamekye,

Working around Microsofts FAT patents

Comments