SD card question
g3cwi
Posts: 262
Hi there
I need to read records from a card, manipulate and select a few and write them to a new file. Mike Green suggested that I need to have two copies of fsrw running to do this efficiently (at least I understood it that way). Do I therefore declare two objects i.e.
SDR: "fsrw"
SDW: "fsrw"
Do I need to mount the card etc for each? Can a card be opened for reading and writing by two instances of fsrw at the same time? If not, why do I need two running?
No doubt this is obvious to someone - but not to me!
Regards
Richard
I need to read records from a card, manipulate and select a few and write them to a new file. Mike Green suggested that I need to have two copies of fsrw running to do this efficiently (at least I understood it that way). Do I therefore declare two objects i.e.
SDR: "fsrw"
SDW: "fsrw"
Do I need to mount the card etc for each? Can a card be opened for reading and writing by two instances of fsrw at the same time? If not, why do I need two running?
No doubt this is obvious to someone - but not to me!
Regards
Richard
Comments
I am now trying to use the GPS_IO_Mini technique to read in the records for processing. The first thing I spotted was an error in GPS_IO_Mini whereby the array gps_buff is declared as a byte but cleared using longfill - in fact longfill seems to have the incorrect syntax anyway as using the correct syntax crashes the program. This has all sorts of potentially unwanted consequences. My code is:
CON _clkmode = xtal1 + pll16x _xinfreq = 5_000_000 obj term: "tv_text" sd: "fsrw" var byte tbuf [20] long altitude [10] long latitude [10] long longitude [10] long RECORD long comma_counter long r_stack[10] byte Recordb[68] long Recorda[20] long r_buff[100],Rx',cksum long cog,cptr,ptr,arg,j long Null[1] byte latlong_buf[10] pub go | x waitcnt(clkfreq*2 + cnt) x := \start term.str(string("Returned from start", 13)) term.dec(x) term.out(13) pub start | r, sta, bytes term.start(12) term.str(string("Mounting.", 13)) waitcnt(clkfreq/10 + cnt) sd.mount(0) term.str(string("Mounted.", 13)) waitcnt(clkfreq/10 + cnt) term.str(string("Dir: ", 13)) waitcnt(clkfreq/10 + cnt) sd.opendir 'Print list of files in directory repeat while 0 == sd.nextfile(@tbuf) term.str(@tbuf) term.out(13) term.str(string("That's the dir", 13)) term.out(13) r := sd.popen(string("APRS-SHT.CSV"), "r") term.str(string("Opening returned 1 ")) term.dec(r) term.out(13) read_record 'read the first record in the file term.out (summit_ref) sd.pclose term.out(13) term.str(string("That's, all, folks! 3", 13)) PUB read_record NULL[0] := 0 repeat longfill(@r_buff,0,100) 'clear the buffer ' cptr := 0 repeat while Rx <>= 13 'continue to collect data until the end of the record Rx := sd.pgetc 'get character from Rx Buffer if Rx == "," r_buff[cptr++] := 0 'If "," replace the character with 0 else r_buff[cptr++] := Rx 'else save the character term.out (r_buff) waitcnt(clkfreq + cnt) copy_buffer(@Recordb, @Recorda) pub copy_buffer ( buffer,args) bytemove(buffer,@r_buff,cptr) ' copy received data to buffer ptr := buffer arg := 0 repeat j from 0 to 3 ' build array of pointers if byte[ptr] == 0 ' to each if byte[ptr+1] == 0 ' record long[args][arg] := NULL ' in else ' the long[args][arg] := ptr+1 ' data buffer arg++ ptr++ ' now we just need to return the pointer to the desired record pub summit_ref return Recorda[0] pub summit_height return Recorda[1] pub Proc_Lat return Recorda[2] pub Proc_Long return Recorda[3]I was sort of hoping this would (as a test) read the dats from the CSV file:
G/SP-004,559,1779895,1432603
G/SP-013,385,1779537,1431506
G/SP-015,343,1778549,1431703
G/SP-001,636,1781278,1433843
G/SP-002,582,1781162,1435389
and print G/SP-004 to the TV. It does not work though, it just prints "G" lots of times.
Any ideas?
Cheers
Richard
How you save records is also how you read records. Did you save the data as a longs or as ASCII bytes? Post your database file.
pub copy_buffer ( buffer,args) bytemove(buffer,@r_buff,cptr) ' copy received data to buffer ptr := buffer arg := 0 repeat j from 0 to 3 ' build array of pointers if byte[ptr] == 0 ' to each if byte[ptr+1] == 0 ' record long[args][arg] := NULL ' in else ' the long[args][arg] := ptr+1 ' data buffer arg++ ptr++PUB read_record NULL[0] := 0 repeat longfill(@r_buff,0,100) 'clear the buffer ' cptr := 0 repeat while Rx <>= 13 'continue to collect data until the end of the record Rx := sd.pgetc 'get character from Rx Buffer if Rx == "," r_buff[cptr++] := 0 'If "," replace the character with 0 else r_buff[cptr++] := Rx 'else save the character term.out (r_buff) waitcnt(clkfreq + cnt) copy_buffer(@Recordb, @Recorda)Some good clues.
Processing the first record:
Recorda[0] = 559 and Recorda[1] = 1779895
I expected that Recorda[0] would be "G/SP-004" but for some reason it is not - but I feel close now!
Regards
Richard
Regards
Richard
Now the next challenge. My code processes 50,000 records (each with four fields). For each record it does a calculation (three calculations actually). Depending on the result of the calculation the record is either disguarded or needs to be kept in a new file on the SD card. The code that identifies the records is working fine.
There could be up to about 100 records that I need to keep. The SD card is read by fsrw and I cant see a pointer involved so I am guessing it is being read by clocking each byte out? If I detect a record that I want to save and try to save it, I am guessing that I will loose my place in the read file? So I need a method of saving records with one file open for reading and another open for writing.
Is this where I need two instances of fsrw or cant I have two files open on an SD card?
Expert advice welcome.
Thanks
Richard
PS my code takes 6 minutes to test all the records - seems mighty slow!
bytemove(sel_buff[selcnt],@r_buff,cptr)
...causes the program to crash
As does the end of file detection (EOF). Any thoughts on how best to do these?
CON _clkmode = xtal1 + pll16x _xinfreq = 5_000_000 CR = 13 LF = 10 Null = 0 COMMA = 44 SPACE = 32 TV_PIN = 12 Current_Lat = 1759224 Current_Long = 1430687 obj debug : "tv_text" sd : "fsrw" FS : "FloatString" Num : "Numbers" var byte tbuf [20] long r_stack[10] byte Recordb[68] long Recorda[20] long sel_buff[100] long SelCnt Byte Rx Byte r_buff[100] long cog,cptr,ptr,arg,j Byte EOF long diff, lat_dec, long_dec, lat_dec_diff, long_dec_diff pub go | x waitcnt(clkfreq*2 + cnt) x := \start debug.str(string("Returned from start", CR)) debug.dec(x) debug.out(CR) pub start | i,w,r, sta, bytes r:= 99 selcnt:=0 repeat i from 0 to 99 bytefill(@sel_buff[i], NULL, 100) debug.start(TV_PIN) sd.mount(0) debug.str(string("Mounted.", CR)) waitcnt(clkfreq/10 + cnt) debug.str(string("Dir: ", CR)) waitcnt(clkfreq/10 + cnt) sd.opendir 'Print list of files in directory repeat while 0 == sd.nextfile(@tbuf) debug.str(@tbuf) debug.out(CR) debug.str(string("That's the dir", CR)) debug.out(CR) r := sd.popen(string("APRS-SHT.CSV"), "r") waitcnt(clkfreq/10 + cnt) debug.str(string("Opening returned ")) debug.dec(r) debug.dec(w) EOF := 0 'test code repeat read_record 'read the records in the file If EOF == 1 quit lat_dec := Num.FromStr(Recorda[2], Num#DEC) lat_dec_diff := lat_dec - current_lat long_dec := Num.FromStr(Recorda[3], Num#DEC) long_dec_diff := long_dec - current_long diff := ||lat_dec_diff + ||long_dec_diff if diff < 2500 'range filter bytemove(sel_buff[selcnt],@r_buff,cptr) 'store record in array selcnt++ 'increment array record counter debug.out (CR) debug.str(recorda[0]) debug.out(SPACE) debug.dec(diff) sd.pclose debug.out(CR) debug.str(string("End", CR)) PUB read_record 'reads one record bytefill(@r_buff, NULL, 100) 'clear the buffer with nulls EOF := 0 'reset end-of-file flag ' cptr := 1 'start pointer after first null 'Fill the array r_buff with the first record repeat while Rx <>= CR 'continue to collect data until the end of the record Rx := sd.pgetc 'get character from SD Card 'detect end of file If Rx == -1 'if eof EOF := 1 'set eof flag Quit 'crash out of loop 'Replace special characters Case Rx LF : r_buff[cptr] := SPACE COMMA : r_buff[cptr++] := NULL 'null-delimit strings Other : r_buff[cptr++] := Rx 'else save the character If EOF == 0 copy_buffer(@Recordb, @Recorda) pub copy_buffer (buffer,args) bytemove(buffer,@r_buff,cptr) ' copy received data to buffer ptr := buffer arg := 0 'Build array of pointers to the 'data strings in the buffer '***not sure how this works!*** repeat j from 0 to cptr if byte[ptr] == NULL if byte[ptr+1] == NULL long[args][arg] := NULL else long[args][arg] := ptr+1 arg++ ptr++ ' now we just need to return the pointer to the desired record pub summit_ref return Recorda[0] pub summit_height return Recorda[1] pub Proc_Lat return Recorda[2] pub Proc_Long return Recorda[3]Regards
Richard
Regards
Richard
I agree with Dave. You should take a closer look at your code.
repeat i from 0 to 99 bytefill(@sel_buff[i], NULL, 100)sel_buff is a long array. This loop is zeroing out 100 bytes in sel_buff each loop, but your startng address increments by 4 bytes each loop. Also, your last loop will zero out the 96 bytes after the sel_buff array.It appears that you're treating sel_buff as if it were a 2 dimensional array of 100 elements with 100 bytes each. That would make it 10,000 bytes in size. Spin doesn't support 2 dimensional arrays, so you would need to do the 2D indexing yourself. You could declare sel_buff as "long sel_buff[2500]", and address each 100-byte chunk as @sel_buff[index*25], where index would have the range of 0 to 99. Is that what you intended?