Reading a text file from a SD card - Some suggestions please..

pacman · 2010-11-09 22:29

Hiya,

New project - I have a text file that lives on a a SD card (think edited in Windows) that _might have upto 256 lines in it (with a minimum of 1 line) and each line could have upto 35 characters in it.

Probably on a fat16 file syetm. Edit the file in Windows, and load it onto the SD card

So:

This could be a line
or this
even this
but a line this long would mean I needed to discard any text loner than the 35th position

But I see no easy way of reading in the text file without knowing the length of the file in advance. Where am I missing the obvious?

Is there an "SD_Card_Reader" object in the OBEX that allows me to read a single line at a time or byte wise till I get a CR? (I've had a look and I can't see the obvious "use me" solution). This "perfect object" would then give some kind of error code when I tried to read past the end of the file.... (it's good to dream...)

Even if you could point me to a _really_ (cause I need all the help I can get with string handling) good starting point that would be a huge bonus.

Thanks in advance

Dr_Acula · 2010-11-10 02:10

This question has been sitting here unanswered for a while, so sorry, I don't have a perfect answer, but I did confront this problem recently in sbasic and I ended up writing my own little routine that read the bytes in one at a time, but returned them a line at a time.

It can be a bit tricky though. Are the lines terminated with CR or CRLF? Is there an end of file character? Has a previous program padded out the file to 256 or 512 bytes?

When creating my own text files, eg INI type data, I tend to explicitly put "END" on the last line.

Sometimes there are ways to tell if you are at the end, eg good old Micro-Soft Basic had the EOF command, so you just kept reading till it went true.

Maybe take a look at some of Kye's SD code - he has put all sorts of goodies into the code. He also has some string routines.

It can be so tricky working with text files. Eg even when I knew I had a 3 byte text file, if I transferred it anywhere with xmodem, even to a board and then back again, xmodem padded it out with ^Z characters.

Do you have any control over what creates the text files?

pacman · 2010-11-10 03:32

Dr_Acula wrote: »

This question has been sitting here unanswered for a while, so sorry, I don't have a perfect answer, but I did confront this problem recently in sbasic and I ended up writing my own little routine that read the bytes in one at a time, but returned them a line at a time.

It can be a bit tricky though. Are the lines terminated with CR or CRLF? Is there an end of file character? Has a previous program padded out the file to 256 or 512 bytes?

Probably, could be either - {what is the standard for {say} notepad ?}. The 'plan' is to allow anyone to use a text editor to compose the messages - (in notepad ?) and then store the resultant file on the SD card.

The exact quantity of messages is open (but not more than 256), I envisaged that only the messages 'needed' would be in the text file.

As far as padding ... - don't _think_ so, but perhaps worth considering.

Dr_Acula wrote: »

When creating my own text files, eg INI type data, I tend to explicitly put "END" on the last line.

That is a good idea. Given the file would be written by the end user - the last line rule could be an excellent solution.

{always read 256 x 35 'bytes' of data - Look for {to be decided} end word. Discard everything in ReadBuffer after EndWord......Hmmm}

Dr_Acula wrote: »

Micro-Soft Basic had the EOF command, so you just kept reading till it went true.

Exactly what I was basing my 'idea' off. Seemed like a reasonable idea, perhaps one of the clever people on the list will think 'that's too easy' and rattle out a solution..

Dr_Acula wrote: »

Maybe take a look at some of Kye's SD code - he has put all sorts of goodies into the code. He also has some string routines.

Have been looking at Kye's excellent code (and not understanding 1/2 of it), but was thinking that as this had been discussed before that perhaps I was missing the obviously adaptable code from the {not immediately obvious name} object.

Must now go and try Kye's code reading a small file with a big buffer. See what sort of 'errors' it throws and if they are acceptable..

Thanks heaps for your input - Additional comments and thoughts welcome...

Dr_Acula · 2010-11-10 05:55

This is probably a strange bit of code to post here, but this is sbasic reading in a line of text

function LineInput(number=byte)=string
	var lineoftext=string
	var a=byte
	lineoftext=""
	a=0
		while a<>13 do
		begin	
			read #number;a
			lineoftext=lineoftext+chr$(a)
		end
	read #number;a
	lineoftext=left(lineoftext,len(lineoftext)-1) rem remove the return at the end
end = lineoftext

I think you could replicate something in spin. eg, read in a byte at a time, build up a string, check for an ascii 13, and when you find one, end the function and return the string minus the return.

This works for files that end in 13,10 ie CRLF so when you get a 13, you need to read in one more byte which will be the 10, and discard it.

I suspect Kye's code has the required building blocks.

If the line returned is "END" then it is the end of the file.

Dave Hein · 2010-11-10 08:51

pacman,

The following method would work with FSRW to read one line at a time.

PUB gets(str, num) | i, value
{{
  This routine reads a string from a file.  It will return a value of
  zero when the EOF is encountered.  Otherwise, it will read up to
  num-1 characters unless a linefeed or NULL is encountered.
}}
  result := str
  repeat i from 1 to num - 1
    value := \fsrw.pread(str, 1)
    IF (value < 1)
      IF (i == 0)
        RETURN 0
      QUIT
    IF (BYTE[str] == 0)
      QUIT
    if byte[str++] == 10
      quit
  BYTE[str] := 0

It returns a value of zero when it encounters the end of file. Unix/Linux text files terminate each line with a linefeed (decimal 10). Windows terminates each line with CR, LF (13, 10). In both cases the LF is the last character of the line.

If you know that the files will be no bigger than 35*256 you could use a single call to pread to read the entire file, and then replace all the linefeeds with nulls to make them null-terminated strings. pread will return a value indicating the number of bytes actually read.

Dave

MagIO2 · 2010-11-10 12:22

Easy, isn't it?

Simply have a buffer which has the size of your max. line-length (plus line end). You said 35 characters.
You read 35 minus number of unprocessed characters of the previous loop into the buffer start address plus the number of unprocessed characters.
For the first loop the number of unprocessed characters is 0, so you read 35 characters to the beginning of the buffer.

Then you search in the buffer for the lineend-character and replace it with $00, which is the string-end character in SPIN. You can now use the buffer-start-address in any string-function for example to print it on a TV.

When you're done with whatever you want to do you simply use the bytemove instruction to copy the rest of the buffer to the start of the buffer. And go back to the loop.

If the file is read completely the FSRW will tell you that you read less than 35 characters.

You can have a look at the ConfigReader I added a while ago to the Object exchange.

Of course reading character by character will work as well, but it will be much slower.

Reading a text file from a SD card - Some suggestions please..

Comments