writing and retrieving a long value to a file on Vinculum

PaulF

Hi,
I've been playing around with DataLoggerSPI and I'm trying to write a number (long) to a file, and then later read back this number. I just cannot seem to retrieve the correct value. Any suggestions?

Peter Verkaik

Have you tried the test program by selecting the correct clock settings
(either protoboard or spinstamp) and used pins?

regards peter

PaulF

Hi Peter, I have tried your test program and I'm able to write and read a string, but if i try writing a long variable to the file, the values i get back are different. perhaps I should save the number as a string but i thought this would introduce long delays. The final program needs to communicate with a factory floor printer at a baud rate of 115,200. This printer prints codes on products passing by on a conveyor at speeds of 10 per sec, so i cannot afford to have delays.

Peter Verkaik

The read/write functions operate on byte buffers,
so you need to pass the address of your long.
Here is a short setup that you must complete.

VAR
· value· long
··fw long
· fr· long

OBJ
· fs: "DataloggerSPI"

PUB try
· fs.DataloggerSPI(cs,sclk,sdi,sdo) 'connect to datalogger
· 'see test program for how to connect precisely

· value := $0123ABCD
· fw := fs.openFileForWrite(str("test.txt"),0) 'open file for write (fw must be <> 0)
· if fw
··· fs.writeToFile(@value,4,0)· 'write 4 bytes from specified address to file opened for write
··· fs.closeFile(str("test.txt"))
··· fr := fs.openFileForRead(str("test.txt"),0) 'open file for read (fr must be <> 0)
··· if fr
····· fs.readFromFile(@value,4,0) 'read 4 bytes from file opened for read
····· fs.closeFile(str("test.txt"))
····· if value <> $0123ABCD
······· 'display error message
····· else
······ 'display OK
··· else
····· 'display Cannot open file for read
· else
··· 'display Cannot open file for write

regards peter


Post Edited (Peter Verkaik) : 3/27/2008 11:34:36 AM GMT

PaulF

Thanks a million Peter, my problem was simply leaving out the @ symbol in the filewrite command

( fs.writeToFile(@value,4,0) )

Thanks again, and a lovely piece of work in developing the DataloggerSPI object.

Paul

PaulF

Sorry to bring this up again but although I've been able to write and read a long variable from a specific location in a file I run into problems when the file is longer than 32K. When I write the data to the location specified in the preceding seek command, the data is put in a completely different location. Is there anything I can do to ensure that the data is written to the correct location. I hope to use a file that is anywhere from 16MB plus.
Any help would be appreciated.

Peter Verkaik

1. Do you try to seek beyond the EOF position (equals filesize) ?
The effect is unspecified if you do.

2. Do you know that when writing to a file, the filesize gets truncated
to the current fileposition? Eg. you cannot open a file for write
and seek to some position to change some bytes. The moment
you open a file for write and it exists, it is first deleted
(the vinculum does that).

regards peter

PaulF

Hi Peter,
my program works as long as the file is less than 32K. I have a sequence of 8 digit codes in a file called 'Codes.txt'. The last four bytes are used to store the address within the file of the next code to read from the file. If I open the file for write, I am able to retrieve the address of the next code by using filesize - 4. I then use seek to go to that address, retrieve the code(readFromFile), print the code, increment the address used by 8 and then write this back to the original location (filesize - 4). The file remains the same length because I have not added any extra bytes to it.
The problem is that when the file is over 32Kb in length, the address is not written back to the correct location. A typical file length that I am using for test purposes is 80004 bytes long with the address of the next code held at location starting 80000.
Perhaps I should use an Sd card or some other means of storing the code list?
Regards
Paul

Peter Verkaik

What you describe is random access to a file (Open for write, seek to fs-4, read 4 bytes,
seek to that address, read code, add 8 to address, seek to fs-4, write address).
As far as I understand the vinculum manual, this is not possible (or at least not specified).
If you open a file for write, you can only write and seek, not read.
If you open a file for read, you can only read and seek,·not write.
If you can point me in the vinculum manual to where it says random access·is
supported, I can adapt my object to support that too.

Here is an extracted text from the manual:
6.2.8 Open File for Write (OPW)
Parameters:·
File Name – Up to 11 characters ASCII.·
File Date and Time – (Optional) 32 bits: file date and time.
The File specified in filename is opened for writing. The filename is normally specified as 8.3.
Data is normally appended to the end of an existing file or a new empty file is created if it does not exist. The Seek (SEK)
command can be used to move to an arbitrary point in the file and commence writing from there.
Note : The file will be truncated to the position of the file pointer when closed.

regards peter

PaulF

Peter,
on page 28 of the firmware manual it says:
"Only one file may be open at any one time. If a file is open for writing, then it may be written to or read from. If open for read then it may only be read from. A file pointer is maintained for the currently open file from where reads or writes will commence. The SEK command can be used to move the file pointer within a file. However, files open for write will be truncated at the file pointer when closed. The end of the file is moved to the position of the file pointer after a write operation."

I am both new to the propellor and to file access so I am struggling to explain my problem any better, but it does work for small files but not for larger files.

kind regards
paul

Peter Verkaik

I must have missed that, probably because it isn't mentioned
again·at the Open file for write command·section.
The last line indicates you cannot change some bytes in the middle
of a file because then the remaining of the file will be lost.
(EOF set to filepointer after write). You only change the last bytes
of a file so that should work.

Do you use the method filesize to retrieve the filesize before opening
the file for write? And·is this value valid, also for files => 32KByte?
If not, then the method filesize may be the cause.

regards peter

PaulF

Hi Peter,

·when I use filesize it returns the true length of the file. It seems to be just when I attempt to write to a location beyond 32K that my problems start. It seems to be able to read beyond 32K but not write.



Paul

stevenmess2004

Sounds like it could be a problem with pointer sizes. 32K would be a 16 bit pointer (or word). Could this be the problem? I'm not familiar with this device but its just an idea.

Peter Verkaik

Actually, 32K indicates a 15bit filesize.
The code for the DataloggerSPI object uses longs.

Paul, can you post your write code. It may well be a pointer problem,
if a word is extended to long·while bit15 is set, the long becomes negative.
I am not sure wether the vinculum accepts that. A negative long is an
unsigned value > 2^31, eg, larger than 2GigaByte which is the maximum
filesize supported I believe in FAT16/FAT32.

regards peter

PaulF

Will post the relevant code in about 3-4 hours as I'm away fom the development setup at the moment. Thanks again for the help, I will have a think about the poinetr issue. Could I simply set (or clear) bit 31 of the pointer before I use the seek command?

Peter Verkaik

If somehow a 16bit value gets sign-extended to 32bits
then just clearing bit31 is not sufficient.
For example,
$8000 becomes $FFFF_8000
You need to add $0001_0000 to get the right result $0000_8000
In spin:
· repeat while pointer < 0
··· pointer += $0001_0000

I assume you don't write blocks larger than 64KByte, in which
case the repeat while is only executed once, so you can
simplify it to

· if pointer < 0
··· pointer += $0001_0000

regards peter

PaulF

Hi Peter,
here's my code. It may not be the most optimised at the moment but I'm lucky to have any hair left after the last few days so...

PUB start | t, s, Index, a, Counter
vga.start(16, @array, @vgacolors, 0, 0, 0)
print($100)
Marker[noparse][[/noparse]0] :="W" 'MARKER IS USED TO MARK THE END OF FILE FOR DEBUG PURPOSES ONLY
Marker :="X"
Marker :="Y"
Marker :="Z"

serial.start(31,30,%0000,115200)

'start datalogger
usb.Start(SPI_CS,SPI_CLK,SPI_DATA_IN,SPI_DATA_OUT)
usb.receivePromptOrError(2000) ' get unsollicited message from datalogger
print_str(string("Test Program for Citronix CodeGen",EOL))
s := -1
waitcnt(clkfreq / 4 + cnt) ' Wait for 250ms initialization
CodeNo := 0
print_str(string("Filesize = "))
print_dec((usb.filesize(string("CODES.TXT"))) )
print(EOL)
print_str(string("There are "))
NoOfCodes := (usb.filesize(string("CODES.TXT")))-8 'LAST 8 DIGITS USED TO HOLD POINTER AND MARKER
print_dec(NoOfCodes/8) 'EACH CODE IS 8 CHARACTERS IN LENGTH
print_str(string(" Unique Codes in this Batch."))
print(EOL)
print_str(string("Starting from Code Number: "))
if usb.openFileForWrite(string("CODES.TXT"),0)
if usb.seek(NoOfCodes )
if usb.readFromFile(@buffer,4,0) 'GET ADDRESS OF FIRST CODE FOR PRINTING
CodeNo.byte := (buffer)
CodeNo.byte := (buffer)
CodeNo.byte := (buffer)
CodeNo.byte[noparse][[/noparse]0] := (buffer[noparse][[/noparse]0])
print_dec(CodeNo)
print(eol)



repeat

if usb.seek(CodeNo * 8)
if usb.readFromFile(@buffer,8,0) 'GET NEXT CODE
buffer[noparse][[/noparse]9] := 0
repeat Index from 0 to 7
buffer[noparse][[/noparse]Index] := buffer[noparse][[/noparse]Index] ^= xS[noparse][[/noparse]Index] 'SIMPLE XOR ENCRYPTION SCHEME
print_str(string("CodeNo = "))
print_dec(CodeNo)
else
print_str(string("Unable to read from file."))
else
print_str(string("Unable to go to seek location."))

repeat until serial.rx == $50 ' WAIT FOR OK FROM PRINTER
print_str(string("."))
serial.str(@buffer)
serial.str(string(13))
serial.rxflush
CodeNo ++ 'INCREMENT POINTER TO NEXT CODE
if usb.seek(NoOfCodes)
print_str(string("At seek location"))
print_dec(noofcodes)
if usb.writeToFile(@CodeNo,4,0) 'SAVE POINTER TO END OF FILE - 8 BYTES
if usb.writeToFile(@Marker,4,0) 'SAVE MARKER TO END OF FILE - 4 BYTES (DEBUG ONLY)
print(eol)
else
print_str(string("Cant find seek location"))


usb.closeFile(string("CODES.TXT"))


I hope you can help.

PaulF

sorry the formatting has gone on the post. What is the best way to send code samples?

Peter Verkaik

Attach the files.

regards peter

PaulF

Hi Peter,
here's the main program.

PaulF

Hi Peter,
I found something which surprises me. The file containing the codes was open going into the main repeat loop in the program.Then using seek I read the code for printing, wait for an Ok from the printer, send the code to the printer, then using seek I attempt to write to the end of the file. This is where the pointer gets written to the wrong location.
However if I close the file after reading the code then reopen before writing the incremented pointer back to the file, the program does write the data to the correct location. So apparently it is possible to 'only' read or write from/to a file that is open for write.
Do you think this is logical? I ask because if I have to constantly close and then open the file it introduces unacceptable delays as it only allows the app to send approx 260 codes per minute to the printer but i need to get approx 600 codes per minute.
Do you think I would be better using SD memory card or similar?

Kind regards
Paul

PaulF

Hi Peter,
still struggling with this. I was able to boost the speed up to approx 400 codes per minute by not closing the file after reading and simply opening the file for write before using seek to write the new incremented address back. Although this worked, I wasn't happy with doing this as it seemed strange. I therefore contacted FTDI who suggested that I upgrade to the latest version of VDAP (3.63) which overcame a bug they had with overwriting data at boundaries ( or something like that). However when I upgraded other issues have appeared where the file gets corrupted and cannot be opened with notepad for me to debug.
At this stage I am considering using battery backed RAM possibly the DS1307 as suggested by Mike or could anyone suggest a larger battery backed RAM module that is easily interfaced with the propellor. I will need approx 1-2 K of memory to be able to keep details of the different batches of codes.