Deploying Bean's Frequency Counter Code Across Multiple Cogs

scocioba

Hi Folks,
Super noobie question here but how would you recommend going about turning the SPIN code for the "Freq Counter 3" into an object I can instantiate several of and pull the main output values to the serial terminal? I'm not super comfy with assembly yet so I'm tiring to figure where I can fit in a return command to be able to pass the port, cog, and resulting data from this program. Link here: forums.parallax.com/discussion/123170/propbasic-reciprocal-frequency-counter-0-5hz-to-40mhz-40mhz-now

This code works wonderfully with my TSL235R but I need 6 of them running in tandem and outputting to an SD card. I got all of the parsing and SD card parts working fine, just need to figure how to make this work like the jm_freqin demo code where I can index several of these objects and initialize them with appropriate pin values. Thanks so much in advance!

Cluso99

It would be best to post this in the Propeller 1 thread

Cluso99

To make this a standalone object, and without looking too deeply at the spin code...

It is outputting a string into hub called __DATASTART.

As a standalone object, to be used with a calling object...
These lines are not required or used as only the top object should define them (or in a hwdef spin file)
_ClkMode = XTAL1 + PLL16X
_XInFreq = 5000000 'XIN 5_000_000

Change the start method to receive a buffer parameter (address into PAR) and use COGNEW
PUB __Program(__DATASTART) 'PROGRAM Start ' Start program at label "Start:"
CogNew(@__Init, @__DATASTART)

Move the DAT section defining __DATASTART into the calling object, and make it like this (expand the ... section)
I will leave you to work out how to only define one InLineStr instead of six
DAT
__DATASTART
ascii1 BYTE 0[21]
InLineStr1 BYTE "Too low (< 0.5Hz)",0
...
ascii6 BYTE 0[21]
InLineStr6 BYTE "Too low (< 0.5Hz)",0


To start your 6 objects, call like this (expand ...)
object.__Program(ascii1)
...
object.__Program(ascii6)

Hope this helps

scocioba

Thanks for the clues, Cluso99! The backstory behind this is I am a biologist (and a mediocre coder at best) and we are working on a paper we wish to publish very soon on a new phenomena discovered using our open source hardware for bacterial growth studies. The devices are simple and I got away with using hardware interrupts on an Arduino Due to get two of these devices reading every second in tandem. I would like to take advantage of the propeller's 8 cores and run SIX simultaneous spectrometers at once. These devices use the TSL235S light-to-freq and give wonderfully resolute data if the counter works well. The device scales from 100kHz to 10Hz or so dark noise and has a wonderfully high dynamic range. They simply need to take a light reading once a second and dump the data to an SD card. I got all the dumping and the SD carding to work well with Johnny Mac's jm_freqin object but there are some really bad rounding errors or some kind of leveling to the data such that it gives me a series of values and no other numbers in between when the light changes. This quantized style of reading makes for huge oscillations in the data so the options are to either fix the jm_freqin object or to make this new one, that has a noise threshold of 0.1% (tested) and is working well, into a stand alone object to call 6 times.

Any significant help in this would mean I would cite you in the special thanks section of the paper once we publish. Might even make it to Nature! If that helps incentivize

Sorry for the desperation but having this working would literally be a lifesaver and the result could mean getting more students (middleschool and highschool) involved in actual peer-reviewed research. All will be open source and the paper will be too. Thanks for the help! I'll keep chugging along parsing what you wrote and learning more. Not giving up anytime soon!

Here is our little non-profit org if anyone is interested in what we do. Everything for the kiddos! https://binomicalabs.org/

Cluso99

WOW! Sounds quite interesting.
I am off to work shortly so I will comment further on the code later.

At those frequencies, a single cog should be able to do all 6 tests at once. I am no expert with the counters but they should have a better way. However since you have this object working it's a good way to progress by using it in 6 cogs.

JonnyMac

Can you do the math after the fact? That is, could you store raw counts to the spreadsheet and convert them to frequency after?

I was looking at Bean's code which has a 1s sample window -- this means you cannot update every second. Does this matter? What are the upper and lower frequency bounds that you'll see from the sensor?

It's lunch time for me so I'm going to knock together a little demo in Spin. It will use an embedded object which can be made stand-alone without a lot of fuss.

scocioba

Hi Cluso99 and JonnyMac,
Thank you both for the interest and support here! The reading taken from the turbidometers (600nm LED and TSL235R sensor) is a relative reading so raw counts are actually ideal in this case. I will be using the Beer-Lambert law -log(I/Io) where "Io" is the first reading (or an average of first few) and I is the current light intensity. This is a dimensionless number and it relies on the progression of light level with respect to the initial condition. As bacteria divide and grow they obscure the liquid media in which they live and we are recording the second by second changes in bacterial growth over the course of several days in a custom-made apparatus that can grow AND record the cells at the same time. Normally, a researcher has to pull a small sample of liquid media from the shaking flask in the incubator every hour. This gives a growth curve with the resolution of only a handful of data points. By being able to read the turbidity of the media THROUGH the culture vessel itself it allows for non-destructive readings at whatever interval desired by the researcher. 1 second is more than adequate for us. 86,400 data points per night, per spec. Fun times!

More info on the Beer-Lambert law here: https://chem.libretexts.org/Core/Physical_and_Theoretical_Chemistry/Spectroscopy/Electronic_Spectroscopy/Electronic_Spectroscopy_Basics/The_Beer-Lambert_Law

We are hot on the trail of a very subtle but significant "blip" in our data that has never been seen before due to poor resolution and lack of interest. We have witnessed several anomalous changes in the pattern of growth of E. coli that is easily reproducible and reliable so we will be focusing on this in our upcoming paper. It's amazing what one finds when they look close enough!

In short what is needed is a high precision and stable means of counting rising edges with minimal jitter that can be cast onto as many cogs as possible. If we could leverage the second counter on each cog to run an additional turbidometer, that would be amazing but I would rather sacrifice quantity for quality if having just one helps. These could most likely run for a month straight so any drift in the data will become significant over time. Keep that in mind if any accumulated lag errors may occur. We could also run a blank turbidometer to account for the expected drift and just subtract that from the data pool if necessary. Either way both of you are lifesavers and will be thoroughly credited in our paper without question! THANK YOU!!!!

jmg

JonnyMac wrote: »

...
I was looking at Bean's code which has a 1s sample window -- this means you cannot update every second. Does this matter? What are the upper and lower frequency bounds that you'll see from the sensor? .

FWIR of Bean’s code, there are 2 versions - a simple, gated counter that is easy to follow, and a more comprehensive Reciprocal Counter, that counts both Whole Cycles and dT, and this version has a more fluid gate time, and excellent dynamic range - gate time just needs to be longer than some minimum value, plus a whole period of Fin (which is why it can measure 0,5Hz with a nominal 1s gate time)
The 0.5Hz I suggested Bean add, to allow 1pps from GPS modules to calibrate the Xtal.

For 10Hz ~ 100kHz I would suggest Reciprocal Counting, as that auto-scales and avoids granularity issues.
To capture both dT and Whole Cycles with best precision, I think needs two counters, so that’s one reciprocal counter per COG.

P2 can manage many more Reciprocal Counters, via the smart pins.

Ariba

Here is a code that measures 6 TSL235 sensors with one PASM cog.
One periode of each enabled sensor is measured in a loop. If the lowest frequeny is 10 Hz this takes max. 0.6 s for 6 sensors.
You get the periode in sysclock ticks in 6 spin variables (tsl[0] to tsl[5]).

The test code generates 2 test frequencies with the counters and shows how to enable the sensor pins and how to calculate the frequency in Hz.
Don't enable pins whithout sensors connected, the pasm loop will hang if there are no edges at the pin.

Andy

Ariba

Code attached here

scocioba

Hi Ariba,
Thanks for the effort on the demo program! I connected all of the sensors and modified the code for the right serial readout. The results are as follows:

https://pasteboard.co/H8FDNMg.png

There is some reliable anomalous readings on the last sensor as well as a lack of change when i put my finger to block the light from sensors 3, 5, and 6. Also no reading on sensor 1 (mask 0) despite it working in another app. Seems to skip over sensor 1 on pin 0 entirely. Regardless of these bugs, the real issue is the low granularity of the data coming in. If you look at the data you will notice some recurring specific numbers which seem to be levels of values rather than continuous random readings. I had similar issues with Jon's demo so maybe it would be best to just read the raw counts? The actual calculation of frequency does not matter as long as there is a correlation between counts and light intensity. Whatever means of getting the frequency data as close to actual values as possible would be ideal. Calculation in post is fine, just need a number to apply Beer's law to. Thanks again!

Bean

scocioba,
You can measure two sources per cog on the propeller. As jmg stated, you should use a reciprocal counter for best resolution at low frequencies.

Use the hardware counters to count the cycles and WAITPNE or WAITPEQ to measure the time using CNT.

Basically for each cog to measure pin A and pin B do this:
Setup counterA to count rising edges on pin A
Setup counterB to count rising edges on pin B
Wait for pin A to go from low to high then record CNT in startCntA and clear PHSA
Wait for pin B to go from low to high then record CNT in startCntB and clear PHSB

Wait for the gateTime, no work to be done here.... Take it easy, put your feet up....

Wait for pin A to go from low to high, then record CNT in endCntA and record PHSA in cyclesA
Wait for pin B to go from low to high, then record CNT in endCntB and record PHSB in cylclesB

The frequency is: cycles / ((endCnt - startCnt) / 80,000,000)

I hope this helps, we sell this as a commercial product so I can't give away too much....
We make a 1U rack module that measures 32 channels of frequency simultaneously.

Bean

Bean

Here is a photo of a 128 channel system.
Each 32 channel module uses 4 propeller chips.

Bean

JonnyMac

@scocioba

I reformatted Andy's code (just so that I could be clear what he was doing) and packaged it in a format that may be easier for you to use. The problem that you're having with Andy's original demo is that you used pin numbers instead of pin masks.

The attached version lets you set you sensor inputs as constants and takes care of creating the masks for you. I also knocked together a simple method that converts the period ticks into a string representing Hz with 0.1Hz resolution. This string can be sent to the terminal (as in demo) or written to an SD card (as in your project).

Since I'm testing on a PAB I defined CH0 to pin 26 and CH1 to pin 27 as there are LEDs on the board. Also note that I set the constant N_SENSORS to 2 so that the code won't hang -- you'll need to change this for your project. The nice thing is this setup allows you to use 1 to 6 sensors. The current code uses N_SENSORS starting from CH0 and in a contiguous group. If you need to selectively enable/disable channels I can modify the code for you.

JonnyMac

Okay, I couldn't stop myself. Here's a version that allows you to selectively enable/disable non-contiguous channels. I'm a big believer in flexiblity; this give you the most. You need to set a constant called S_ENABLE with a bit mask of the channels to use. If you want all channels enabled, use this

  S_ENABLE = %111111

In the demo code I move one of the test channels to CH5 so I used %100001 for S_ENABLE which only monitors CH0 and CH5.

As a reminder, the channels in the S_ENABLE mask are MSB -> LSB, that is, 543210. If you wanted to enable CH0 and CH1 as I did in my first version, now you would set S_ENABLE to %000011.

JonnyMac

Whoops, here's the code and a screen-shot with changes highlighted.

varnon

Very interesting project. I am a biological psychologist (I suppose that is the best term) and I have done some similar work with the Propeller. I was using an I2C light sensor to detect light from bioluminescent algae. Once the data was obtained, I wrote it to an SD card. I used my Propeller Experiment Controller objects to do so. They are more for behavioral research, but they work for a lot of other occasions. My strategy was to write a measurement each second (I don't remember the exact rate) or so to a file on the SD card. It sounds like that approach will work well for you. I suppose you could also take a large number of measurements, and then write them all at once. I'm not sure the time frame for your experiment, but it sounds like you could easily set up your system to record for multiple days at a time. I had mine collect continuous data across a week.

It looks like you have the rest about figured out. I don't have this sensor, so I can't contribute more, but I think what you are aiming to do is very possible. Please keep us posted. I have the same thoughts as you about inexpensive open source technology in research, so if you get something published, I will likely cite it.

scocioba

See attached for results of Jon's recent version 2 monitors noise threshold versus the Bean's. I recorded the specs taking 100 readings with nothing inside them. The jitter is really significant on the jm_tslx6 one and the blip we are hunting for is very VERY subtle so the lower noise the better. I've included the code Bean wrote here with some minor minor tweaks for brevity's sake. Hope this helps and sorry for the fuss! Thank you, either way, all of you, for helping out here. This means the world to me! Obviously, final attributions will be given and credit will be given where it is due.

scocioba

Oddly enough all of the jm_tslx6 runs are around 5% deviation from the average except sensor 6 which is at 0.3% and looks a lot like Bean's results. Hmmm....

scocioba

If you look closely you can see the recurring pattern of numbers in Jon's output and a more natural-seeming sequence of reads from Bean's output. Why do you think there is such an odd stair-step effect? Due to the nature of noise, you should rarely see the same value twice so this seems like it is being chunked into larger steps. Thoughts?

JonnyMac

The code I posted today uses Andy's PASM driver -- I simply reformatted to adhere to my anal-retentive code formatting practices, and created a routine to convert the period ticks to a string.

Bean's code could be adapted to PASM but you would only get two channels per cog.

What I'm having a problem with is that Bean's code counts rising edges in a timed window; Andy's code uses two successive rising edges to determine the period. If you have noise -- that is, stray edges -- it seems like it would affect both drivers.

JonnyMac

This version uses edge counting as in Bean's code -- but both counters are used in a 0.5s sample window. Three cogs are used for six TSL channels. You can selectively disable any channel by setting its pin number to -1.

JonnyMac out.

Ariba

You said: "The device scales from 100kHz to 10Hz or so dark noise and has a wonderfully high dynamic range", but now you test with frequencies of 400..500 kHz.
The problem is that for such high frequencies it's better to count edges in a time window. But for low frequencies like 10 Hz you get a very bad resolution with that.
My solution counts the sysclock ticks for one periode. At 100 kHz you get still 800 ticks, which is a quite high resolution, at 500 kHz it's only 160 ticks.
Also at a resolution of 800 ticks you get steps in the frequency result, if you calculate the integer reciprocal. So it's better to store the periode ticks for further analysis.

Andy

scocioba

@ariba
My apologies for the mixup I completely forgot that the new TSL chips give a signal above 100khz. In my particular setup, I will tune the sensors to be no more than 500khz with no light obstructions and just checked the dark signal to be a little above 500Hz. The bacterial will never make the sensor fully dark because they are still a colloidal suspension in a liquid so plenty of room for stray light to sneak in.

@JonnyMac
The only remaining issue is this constant drift from the moment the spec's turn on. This is true if I use your code with all six sensors OR the addition of the SD card code. I notice the same tell-tale drift in Bean's original code as well. I am fine with a bit of sensor drift but the change in a mere 100 seconds is significant that across 24 hours (86,400 seconds) this will become a really catastrophic level of drift. I could hypothetically adjust for the drift in post but at the rate depicted in the graph, the value of the sensor will reach -1115544Hz in a day's time. I don't mind using one cog per sensor if it means a stable reading with similar precision. Curious what the underlying 20unit/reading is... Terribly sorry for the headache!

Bean

The drift could be the sensor or the propeller clock.
It seems like a lot of drift to be a crystal, so I would say it is from the sensor itself.
Could it be temperature related ?

Bean

scocioba

I've had the same sensors run in an edge counting sketch in arduino using interrupts and they have had stable readings for upwards of a few weeks. I switched to propeller so I can get more sensors in one experiment. They've also been sitting at room temp and have not moved much so not sure if that is the real issue here. If it's the crystal then is it doomed from the start? Maybe if I put an oscilloscope to the sensor outputs and check to see if the actual square wave drifts that quickly. The rate is fairly constant across all 6 of the sensors so if this was a sensor issue the rate of drift should vary enough to decern no clear pattern. The starting points of each of the sensors are off because I have not properly tuned the light levels yet but the downward slope is consistent across all the sensors. No clue how to proceed from here honestly. Everything else works perfectly and we are SO DAMN CLOSE to getting all this operational. Fingers crossed this isn't a terminal flaw in the system itself and just some lost edges due to a technical bug!

Either way, I am forever grateful for all the hard work and time you all put into this and my sincerest apologies for being a neanderthal in terms of PASM coding!

Bean

What are you using to clock the propeller ?
It is a parallax board ? or something you made yourself ?
You can get a cheap TCXO from digikey or somewhere that would be much more stable.
What is the time scale of the graph ? Seconds ? If so, then maybe the change will settle out after some time (1 Hour).

Bean

scocioba

I am using the Propeller ASC+ (arduino shield compatible) board at the moment. Good call on the external oscillator! Have not thought of that. I'll let the sensors run for an hour or so and see if the drift stabilizes. If it's just temp then joyous day everything works! Else I have no clue.

kwinn

scocioba wrote: »

I've had the same sensors run in an edge counting sketch in arduino using interrupts and they have had stable readings for upwards of a few weeks. I switched to propeller so I can get more sensors in one experiment. They've also been sitting at room temp and have not moved much so not sure if that is the real issue here. If it's the crystal then is it doomed from the start? Maybe if I put an oscilloscope to the sensor outputs and check to see if the actual square wave drifts that quickly. The rate is fairly constant across all 6 of the sensors so if this was a sensor issue the rate of drift should vary enough to decern no clear pattern. The starting points of each of the sensors are off because I have not properly tuned the light levels yet but the downward slope is consistent across all the sensors. No clue how to proceed from here honestly. Everything else works perfectly and we are SO DAMN CLOSE to getting all this operational. Fingers crossed this isn't a terminal flaw in the system itself and just some lost edges due to a technical bug!

Either way, I am forever grateful for all the hard work and time you all put into this and my sincerest apologies for being a neanderthal in terms of PASM coding!

There is one other possibility for the unstable readings, and that is chemiluminescence from exposure to external lighting, particularly fluorescent lights. That was a fairly common problem with liquid scintillation counters, and took some time in total darkness (10's of minutes to several hours) to decay. You may want to keep a sample in total darkness a couple of hours to see if that helps.

Cluso99

The drift is too large to be a crystal problem.

Scoping the prop pin doing the sampling might be a good way to see what might be wrong. Is there noise on the signal that might be causing false counts?

You need to lock down one sensor and get it working again. Then compare the different code that some have given you. Once that is working again we can help further.

Sorry I have been busy at work which has been overflowing to home.

Postedit. Just a thought. Can you post a pic of your setup, particularly the connection of the sensors to prop pins. It might be a problem here.

rjo__

?

Late to the party... I can't see any of the forum posted images. Windows 8.1 or Win 10