When subdued, the program works great! 30 readings (60 writes) per second!
Also changed: text.start(12) to use base pin 0, since 12 was being used by the datalogger.
Now I need to get the time keeping program in there.
Also, I read that the ADC program should be capable of ~70,000 readings per second. Is there any way i can boost the speed of this program more? Or possibly take the average of all the numbers between writes?
Thanks
-Steven
[code]
con
_clkmode = xtal1 + pll16x
_xinfreq = 6_250_000
SD_DO = 8
SD_CLK = 9
SD_DI = 10
SD_CS = 11
obj
sd : "fsrw"
var
long ADCval
byte buffer[12]
byte buffer2[12]
repeat 10
if value => i
byte[address++] := value / i + "0"
value //= i
result~~
elseif result or i == 1
byte[address++] := "0"
i /= 10
byte[address]~ ' mark end of string
return address
DAT
shiftin
org 0
''Set the direction and state of the I/O pins and non-recurring events
''
or dira, OS ''Set OS to output
or dira, SCL ''Set SCL to output
or dira, CS ''Set CS to output
xor outa, SCL ''Set SCL pin high (1)
xor outa, CS ''Set CS pin high (1)
''
mov wcnt, #0 ''Zero wcnt
add wcnt, cnt ''Add system counter to wcnt
add wcnt, #13 ''Add 13 clock delay to wcnt
waitcnt wcnt, Tsucs ''Pause COG TSucs clocks
'rdlong ADCtemp,PAR
''==========================================================================================
''Start of conversion
:loop1 ''Loop1
writing small strings to the SD card is very inefficient, you need to write blocks at the size of the underlying hardware/software
binary to decimal conversion , it would be faster to dump the raw bytes to the SD. what tools are you going to use for data analysis?
These two things have me interested.
What is necessary to store the values in a block to write once per x? And would the values be saved while its writing that block, or would those values be lost?
Also, Using excel to manipulate the data after it is logged.
Steven,
I am using Etimer V 1.2 which writes to a global variable. I am also using Jon's fast adc object to read my adc. I am actually getting several ADC reads per ms of etimer count. In fact I am slowing my adc down by only reading when a motor drive pin is high. That is because I am measuring the current drawn by the motor when driven by a pwm pulse. If pins are not a problem, you could mod the etimer code to change the heartbeat to oncer per 10 ms instead of 1 per sec and read the adc once per heartbeat. (You can do this with waitpeq for the heartbeat to go high) Using a 128 long circular buffer you could shift your adc reading to the upper half of your reporting long variable add the current ms time reading and wrlong to your circular buffer. You then read the circular buffer once per second and store to the sd card. This would give you 100 readings per second each read one long. eXcel should be able to parse the data into time and data. 128 makes an easy number for circular buffers and should give you some overlap time to read data out while continuing to write new info in. When I use etimer, my debugging programing is reading results out of the global once every 12 milliseconds. I suspect that speed is determined by the spin program writing to the terminal at 115K.
What do you think?
RS_Jim
Steven,
With Etimer V1.2, you have a global variable that can be read at least once per millisecond. You only really need once per 10 ms or so. Go read the code from full duplex serial, it shows writing to a circular buffer in pasm and reading from that same circular buffer in spin. Basically what you do is maintain two pointers to a buffer that is a power of two 8,16, 32, etc. One is head one is tail. Put data in at tail, take it out at head. each time increment the pointer, and with $F $FF etc depending on size of the buffer. When the pointer rolls over at the end of the buffer the and resets it to the beginning. One head one tail pointer if head minus tail = 0 the buffer is empty.
RS_Jim
Well, so far I have made no developments from the last point on this.
I can't seem to find etimer V1.2 in the OBEX or on the site, so I'm still currently writing cnt as my time keeping.
The 10-bit ADC program is still writing 487 for 1.487 volts (as opposed to 1487). The range of the sensor is 0.3-2.6v so i think I may run into a problem here. I'm going to try to up the size of the variable and see if that helps.
Also, have not been able to learn anything about writing values in a circular buffer.
Still getting ~40 readings per second, which is fine. But 100 would be nice
Ok, after some reading, I get the idea that in order to write to eeprom I would need an external EEPROM chip? I thought the propeller had some onboard eeprom.
I am going to look into booting from the SD card at this point.
The Prop's bootloader can either load from an attached PC (by using the Propeller Tool or equivalent) or it can load from an attached EEPROM (I2C 32K bytes or larger). It can't load directly from an SD card. When people want to load from an SD card, they have an attached EEPROM which they preload with a program that boots from the SD card. All such boot loaders use the binary object code produced by the Propeller Tool. It's just saved from the Propeller Tool as a .binary file (with the extension shortened to .bin). It does have to be contained all in one file extent, but, if the SD card is formatted in 32K clusters, the largest possible .binary file will automatically fit in a single extent. The .binary file is just a copy of what would be downloaded to RAM or EEPROM.
The Propeller doesn't have any onboard EEPROM, but most Propeller boards include a 32K or 64K EEPROM.
Steven,
Sorry I havent gotten back to you sooner, been spending a lot of time at Mayo Clinic lately. First in answer to your question about circular buffers and fullduplex serial, look at the PUB rx_check, it gives you the detail of the spin side of removing bytes from a circular buffer. To change to words or longs, change the variable declaration. If you look at the cognew you will see where they set up the pointers for the circular buffers for the assembly program and down in the dat section there are seven lines of code stating with rdlong,par and ending with wrlong t2,PAR that shows how to write the data to the circular buffer in assembly. You will need to make your buffer longs not bytes and I reccommend you make it 128 longs in size as I previously mentioned. Next I appoligize on the reference to etimer v1.2, you are right it doesnot exist. Refer to this thread http://forums.parallax.com/showthread.php?128922-Passing-Memory-Address-I-am-stumped!&highlight=Variablesand check out V2. Just be sure that the clock is the same as the clock you are using. Changing the hb from once per second should be easy enough, mov tmp2,tmp1, and tmp2,#0F then use tmp2 instead of tmp1 in the remaining hb code. change #500 to #05 to turn the pin off again. You then use a waitpne for the heart beat in your adc program to start the conversion, do the conversion and shift the result left 16 and add the etimer ms data to the lower 16 and write the long to the circular buffer. When you remove the data from the circular buffer, remove it as words and insert the necessary formatting around it for excell before you write it the the sd card.
I hope this all makes sense to you.
RS_Jim
grimm,
How is your project going? I did some testing, Eight lines of code or so and I modified etimerV2 to output a a heartbeat pulse at 10 ms intervals. I used that to sync the adc then wrote the adc results along with the current time in ms to a single long and into a circular buffer. I haven't worked on the interface to the circular buffer as I don't have any way to write to a flash memory, but that should be fairly easy. I cold write a routine to send the contents to fullduplex serial, but I am not sure it can write out > than 100 readings per second particularly parsed so that excell could use it.
Jim
Getting about 40 readings per second right now on a PNY card, going to order a new card and see if its any better. I believe the speed it has right now should be fine though.
I would like to use etimer at this point, there are some issues from using the cnt frequency that result in an anomaly.
I am interested in how you modified the etimer program though. I will start to look into it when i get a chance.
Next I modified my ADC code to make it sync to the 10 Ms rate and write the time and adc data to a long.
I then tested it all using a spin program to write to fullduplex serial and was able to see every 10ms read of the adc thusly : 579,131
I used a circular buffer pickup in the spin method to read the buffered data put in by the ADC object. I don't have the hardware to read/write to a flash card so was unable to test the rest. However, I am confident that you can read 100 longs of data from the circular buffer,delimit it for xcell and write it to flash memory once per second. Using a 256 byte circular gives you plenty of time to format the data and write it to flash.
adcmain waitpeq hb,hb
' or outa,led
call #adcfn
rdlong tmp1,adc_head_pntr
' shl tmp1,#2
add tmp1,adc_buff_pntr
shl level,#16
rdlong tmp2,mspntr
add level,tmp2
wrlong level,tmp1
sub level,tmp2
shr level,#16
sub tmp1,adc_buff_pntr
add tmp1,#4
and tmp1,#$1FF
wrlong tmp1,adc_head_pntr
andn outa,led
' waitpne hb,hb ' wait for hb to end before starting next
jmp #adcmain
Steven,
At the begining of the Etimer file there are some constants that can be changed to switch etimer between 5MHz and 6.25 MHz Xtals. I was able to set up a spin program that would read out the time and leval info from the circular buffer and write it to a terminal using fullduplex serial. It would keep up with the 10MS rate so I was seeing every reading of the ADC time stamped. I don't have any hardware that will support the SD card function but, I am sure that you could do a write 1 per sec writing 200+ words that would give you all of the tiimer and ADC readings. I suspect that the eratic nature is comming from trying to write too many small values to the SD card instead of a large value once per second. You will note in my ADC code at the end of the loop I commented out the waitpne as I found the delay through ADC read function was causing the adc to read once every other 5MS interval. I remember an earlier post that suggested that you were trying to write to the SD too often.
My code seems to be rock solid and the scope on the 5MS pulse is very stable. I don't know how much more time your ADC takes to read than mine, that might be an issue. If you need it, I will see if I can find my code that read the info from the circular buffer and wrote it to fullduplex.
Jim
I think the primary timing issue will be caused by the abundance of calls to pflush. pflush forces FSRW to make sure all the FAT table entries are up to date, which totally derails the optimizations built into both FSRW and the SD card's circuitry to read and write consecutive blocks. If I was logging some data, I would probably make sure that pflush was called every couple of seconds or so. The idea is, if your project undergoes a sudden and gratuitous total power failure, your log data will be valid as of the last pflush.
Even with that, there are times when it just takes longer to write, notably when your log file fills up a cluster and has to go back to the FAT table to find the next cluster to begin writing. I've never seen it take 0.6 seconds though...what version of FSRW are you using? Actually, looking at your graph, how do you get *negative* 0.6 seconds between readings?
Jonathan
edit: 100 MHz is fine...it means the output data is clocked at 25 Mbps, which is exactly the SD spec's limit. I have one board running at 100 MHz as well.
Once you've done that, the speed will almost certainly be constrained only by
spin execution speed (the time to get the sensor data, convert the number
to decimal, and do the pwrite calls themselves).
Steven
In my second version of Test_ADC.spin (03-22-2011) I changed the method of writing to the SD card to double buffered 512 byte blocks similar to my "stupid video capture" program.
You seem to have completely ignored that one!
It seemed to work quite well to me with only one hesitation after a short time( probably when the SD driver allocated more memory for the file) but then ran for a complete hour with no noticeable glitches)
The "Stupid Video capture" writes out binary data at 30 fps, one video frame and roughly 266 audio samples, way above your requirements.
One way to speed up the process is to just write out the raw data and have the PC do the conversion to Excel.
I an not an Excel expert but a quick "google" of "excel binary import" showed it was possible get Excel to import binary data with a little bit of effort.
Interesting. I will have to look that program up in the obex and give it a try.
As far as uSD writing goes, I modified a file to write 5000 points with pflush between each write, followed by 5000 points with pflush once per cycle, and 5000 writes with pflush every 50 cycles.
The graph is attached above. There were still some erratic write times but they were more... similiar.
Comments
This was the problem that was causing everything to read random numbers:
ADC.start_pointed(Vo_p, Vn_p, Vclk_p, Vcs_p, 1, 1, 10, 1, @chanstate, @chanval, @chanmax, @chanmin)
When subdued, the program works great! 30 readings (60 writes) per second!
Also changed: text.start(12) to use base pin 0, since 12 was being used by the datalogger.
Now I need to get the time keeping program in there.
Also, I read that the ADC program should be capable of ~70,000 readings per second. Is there any way i can boost the speed of this program more? Or possibly take the average of all the numbers between writes?
Thanks
-Steven
[code]
con
_clkmode = xtal1 + pll16x
_xinfreq = 6_250_000
SD_DO = 8
SD_CLK = 9
SD_DI = 10
SD_CS = 11
obj
sd : "fsrw"
var
long ADCval
byte buffer[12]
byte buffer2[12]
PUB Start
waitcnt (80_000_000 + cnt)
cognew (@shiftin, @ADCval)
sd.mount_explicit(SD_DO, SD_CLK, SD_DI, SD_CS)
sd.popen(string("Online.txt"),"w")
repeat 1000
dec(cnt,@buffer)
write_to_SD(@buffer,strsize(@buffer))
write_to_SD(string(","),1)
dec(ADCval,@buffer2)
write_to_SD(@buffer2,strsize(@buffer2))
write_to_SD(string(13,10),2)
sd.pflush
sd.pflush
sd.pclose
sd.unmount
PUB write_to_SD(some_string,len) | di
di:=sd.pwrite(some_string,len)
sd.pflush
PUB dec(value, address) | i
if value < 0
-value
byte[address++] := "-"
i := 1_000_000_000
repeat 10
if value => i
byte[address++] := value / i + "0"
value //= i
result~~
elseif result or i == 1
byte[address++] := "0"
i /= 10
byte[address]~ ' mark end of string
return address
DAT
shiftin
org 0
''Set the direction and state of the I/O pins and non-recurring events
''
or dira, OS ''Set OS to output
or dira, SCL ''Set SCL to output
or dira, CS ''Set CS to output
xor outa, SCL ''Set SCL pin high (1)
xor outa, CS ''Set CS pin high (1)
''
mov wcnt, #0 ''Zero wcnt
add wcnt, cnt ''Add system counter to wcnt
add wcnt, #13 ''Add 13 clock delay to wcnt
waitcnt wcnt, Tsucs ''Pause COG TSucs clocks
'rdlong ADCtemp,PAR
''==========================================================================================
''Start of conversion
:loop1 ''Loop1
How inaccurate is using 'cnt' and clkfreq to get the time between values?
These two things have me interested.
What is necessary to store the values in a block to write once per x? And would the values be saved while its writing that block, or would those values be lost?
Also, Using excel to manipulate the data after it is logged.
Thanks
-Steven
I am using Etimer V 1.2 which writes to a global variable. I am also using Jon's fast adc object to read my adc. I am actually getting several ADC reads per ms of etimer count. In fact I am slowing my adc down by only reading when a motor drive pin is high. That is because I am measuring the current drawn by the motor when driven by a pwm pulse. If pins are not a problem, you could mod the etimer code to change the heartbeat to oncer per 10 ms instead of 1 per sec and read the adc once per heartbeat. (You can do this with waitpeq for the heartbeat to go high) Using a 128 long circular buffer you could shift your adc reading to the upper half of your reporting long variable add the current ms time reading and wrlong to your circular buffer. You then read the circular buffer once per second and store to the sd card. This would give you 100 readings per second each read one long. eXcel should be able to parse the data into time and data. 128 makes an easy number for circular buffers and should give you some overlap time to read data out while continuing to write new info in. When I use etimer, my debugging programing is reading results out of the global once every 12 milliseconds. I suspect that speed is determined by the spin program writing to the terminal at 115K.
What do you think?
RS_Jim
I will start by tying to modify the etimer program to display times more often.
Thanks
-Steven
it makes a csv file onto your pc directly. Max speed is 10 time per second but you could do something like this.
With Etimer V1.2, you have a global variable that can be read at least once per millisecond. You only really need once per 10 ms or so. Go read the code from full duplex serial, it shows writing to a circular buffer in pasm and reading from that same circular buffer in spin. Basically what you do is maintain two pointers to a buffer that is a power of two 8,16, 32, etc. One is head one is tail. Put data in at tail, take it out at head. each time increment the pointer, and with $F $FF etc depending on size of the buffer. When the pointer rolls over at the end of the buffer the and resets it to the beginning. One head one tail pointer if head minus tail = 0 the buffer is empty.
RS_Jim
I can't seem to find etimer V1.2 in the OBEX or on the site, so I'm still currently writing cnt as my time keeping.
The 10-bit ADC program is still writing 487 for 1.487 volts (as opposed to 1487). The range of the sensor is 0.3-2.6v so i think I may run into a problem here. I'm going to try to up the size of the variable and see if that helps.
Also, have not been able to learn anything about writing values in a circular buffer.
Still getting ~40 readings per second, which is fine. But 100 would be nice
PUB tx(txbyte)
PUB rx : rxbyte
in full duplex serial?
It will run the program fine (F10) but I get an error when trying to write it to eeprom (F11)
It gives me: EEPROM Programming Error on COM12
any hints?
thanks
I am going to look into booting from the SD card at this point.
What form does the file have to take to be able to successfully use: PUB bootPartition(filePathName) | bootSectors[64]
from kye's sd driver?
Thanks
The Propeller doesn't have any onboard EEPROM, but most Propeller boards include a 32K or 64K EEPROM.
Sorry I havent gotten back to you sooner, been spending a lot of time at Mayo Clinic lately. First in answer to your question about circular buffers and fullduplex serial, look at the PUB rx_check, it gives you the detail of the spin side of removing bytes from a circular buffer. To change to words or longs, change the variable declaration. If you look at the cognew you will see where they set up the pointers for the circular buffers for the assembly program and down in the dat section there are seven lines of code stating with rdlong,par and ending with wrlong t2,PAR that shows how to write the data to the circular buffer in assembly. You will need to make your buffer longs not bytes and I reccommend you make it 128 longs in size as I previously mentioned. Next I appoligize on the reference to etimer v1.2, you are right it doesnot exist. Refer to this thread http://forums.parallax.com/showthread.php?128922-Passing-Memory-Address-I-am-stumped!&highlight=Variablesand check out V2. Just be sure that the clock is the same as the clock you are using. Changing the hb from once per second should be easy enough, mov tmp2,tmp1, and tmp2,#0F then use tmp2 instead of tmp1 in the remaining hb code. change #500 to #05 to turn the pin off again. You then use a waitpne for the heart beat in your adc program to start the conversion, do the conversion and shift the result left 16 and add the etimer ms data to the lower 16 and write the long to the circular buffer. When you remove the data from the circular buffer, remove it as words and insert the necessary formatting around it for excell before you write it the the sd card.
I hope this all makes sense to you.
RS_Jim
How is your project going? I did some testing, Eight lines of code or so and I modified etimerV2 to output a a heartbeat pulse at 10 ms intervals. I used that to sync the adc then wrote the adc results along with the current time in ms to a single long and into a circular buffer. I haven't worked on the interface to the circular buffer as I don't have any way to write to a flash memory, but that should be fairly easy. I cold write a routine to send the contents to fullduplex serial, but I am not sure it can write out > than 100 readings per second particularly parsed so that excell could use it.
Jim
Getting about 40 readings per second right now on a PNY card, going to order a new card and see if its any better. I believe the speed it has right now should be fine though.
I would like to use etimer at this point, there are some issues from using the cnt frequency that result in an anomaly.
I am interested in how you modified the etimer program though. I will start to look into it when i get a chance.
-Steven
Ok so I hacked Jonny's V2.1_Par with the following to give me a 10 millisecond hb pulse that is 5 ms wide:
Next I modified my ADC code to make it sync to the 10 Ms rate and write the time and adc data to a long.
I then tested it all using a spin program to write to fullduplex serial and was able to see every 10ms read of the adc thusly : 579,131
I used a circular buffer pickup in the spin method to read the buffered data put in by the ADC object. I don't have the hardware to read/write to a flash card so was unable to test the rest. However, I am confident that you can read 100 longs of data from the circular buffer,delimit it for xcell and write it to flash memory once per second. Using a 256 byte circular gives you plenty of time to format the data and write it to flash.
This is the program i'm currently running:
The only problem i'm having with this is, as suggested, timing. It seems to be unstable.
Im currently trying to see if i can get the etimer to write to the uSD card alone, then move on to including the adc.
How would I be able to go about changing the update time to something quicker... 2 to 5 ms?
Thanks
I was able to record etimer values approximately every 0.024 seconds.
This timer has the same problem I was having with my propeller (no external clock)
The values are very erratic, most are around 0.024s between readings but there is the occasional -0.6s and scattered other values.
Is there any way to get rid of these? Or should I be looking into a timer chip? (which I cant find from parallax anymore)
Thanks
-Steven
Edit: The only thing I can think of is that the propeller is being run at 100 MHz instead of 80. Would that present a problem in any of the code?
Edit2: Looks like 80 MHz might actually be worse. I wonder what causes this to happen?
At the begining of the Etimer file there are some constants that can be changed to switch etimer between 5MHz and 6.25 MHz Xtals. I was able to set up a spin program that would read out the time and leval info from the circular buffer and write it to a terminal using fullduplex serial. It would keep up with the 10MS rate so I was seeing every reading of the ADC time stamped. I don't have any hardware that will support the SD card function but, I am sure that you could do a write 1 per sec writing 200+ words that would give you all of the tiimer and ADC readings. I suspect that the eratic nature is comming from trying to write too many small values to the SD card instead of a large value once per second. You will note in my ADC code at the end of the loop I commented out the waitpne as I found the delay through ADC read function was causing the adc to read once every other 5MS interval. I remember an earlier post that suggested that you were trying to write to the SD too often.
My code seems to be rock solid and the scope on the 5MS pulse is very stable. I don't know how much more time your ADC takes to read than mine, that might be an issue. If you need it, I will see if I can find my code that read the info from the circular buffer and wrote it to fullduplex.
Jim
I think the primary timing issue will be caused by the abundance of calls to pflush. pflush forces FSRW to make sure all the FAT table entries are up to date, which totally derails the optimizations built into both FSRW and the SD card's circuitry to read and write consecutive blocks. If I was logging some data, I would probably make sure that pflush was called every couple of seconds or so. The idea is, if your project undergoes a sudden and gratuitous total power failure, your log data will be valid as of the last pflush.
Even with that, there are times when it just takes longer to write, notably when your log file fills up a cluster and has to go back to the FAT table to find the next cluster to begin writing. I've never seen it take 0.6 seconds though...what version of FSRW are you using? Actually, looking at your graph, how do you get *negative* 0.6 seconds between readings?
Jonathan
edit: 100 MHz is fine...it means the output data is clocked at 25 Mbps, which is exactly the SD spec's limit. I have one board running at 100 MHz as well.
Your pflush is *very* expensive in several different ways.
-tom
Thanks
Once you've done that, the speed will almost certainly be constrained only by
spin execution speed (the time to get the sensor data, convert the number
to decimal, and do the pwrite calls themselves).
-tom
In my second version of Test_ADC.spin (03-22-2011) I changed the method of writing to the SD card to double buffered 512 byte blocks similar to my "stupid video capture" program.
You seem to have completely ignored that one!
It seemed to work quite well to me with only one hesitation after a short time( probably when the SD driver allocated more memory for the file) but then ran for a complete hour with no noticeable glitches)
The "Stupid Video capture" writes out binary data at 30 fps, one video frame and roughly 266 audio samples, way above your requirements.
One way to speed up the process is to just write out the raw data and have the PC do the conversion to Excel.
I an not an Excel expert but a quick "google" of "excel binary import" showed it was possible get Excel to import binary data with a little bit of effort.
Perry
As far as uSD writing goes, I modified a file to write 5000 points with pflush between each write, followed by 5000 points with pflush once per cycle, and 5000 writes with pflush every 50 cycles.
The graph is attached above. There were still some erratic write times but they were more... similiar.
A few changes taking suggestions from you guys, and it's now recording 700 readings per second!
Way too many, but I will take it.
Thanks for all the help