Help wanted - Oscilloscope!

For those of you that have played with the graphing feature in BlocklyProp - it's been my personal pet project since its inception, and I really want to make it something special.

Here's what's on my wish list - making a basic oscilloscope. I've tested the ability of the graphing engine in a web browser to redraw 1000 samples every 5 ms, which is more than fast enough on the computer/browser end (see example here).

There is also a simple library written for the ADC on the Propeller Activity Board (ADC124S021) that can collect ~100 ksps for 4 channels... Not super high frequency, but enough for classroom/learning/basic use.

Here's what I would need - some Propeller C code that can use this library (or a library of your own design) to send bursts of samples via the terminal. Ideally, you would set things like the trigger (none/rising/falling & threshold?), sample duration, and sample frequency (ie. sample for 2ms every 25 ms...)

The terminal in BlocklyProp can handle the full ASCII character set, so some sort of encoding scheme could be developed to transmit the samples to the computer very efficiently, and I can build all the the pieces for the browser, including blocks, settings, user interface, buttons, etc.

Anyone up for the challenge?
Matthew Matz
STEM/Robotics Educator
Parallax Inc.
«1

Comments

  • 53 Comments sorted by Date Added Votes
  • jmgjmg Posts: 12,621
    There is also a simple library written for the ADC on the Propeller Activity Board (ADC124S021) that can collect ~100 ksps for 4 channels... Not super high frequency, but enough for classroom/learning/basic use.
    For classroom learning, there are also Sound Card based scope programs.
    Not super fast, but most every PC has a Sound card.

  • Is this a paying gig?
  • David - I wish we could pay for it - but the idea is nicety, not a necessity, so no, it's more of an idea to throw out and see if there's any interest - especially from anyone who would like to see the functionality added to BlocklyProp.

    jmg - thanks! I'll see what I can glean from those repos. The UI part is actually the easy part for me as I'm more of a Javascript guy than an embedded C/C++ guy.

    I'm hoping that what I'm asking is simple - just getting lots a data from the ADC to the terminal as fast as 115200 bps will allow (since I'm not too keen on messing with the default terminal speed in BlocklyProp - that's a whole different can of worms). The UI/JavaScript end might even be able to determine things like the trigger/etc from raw data (although that's probably best handled on the Prop).
    Matthew Matz
    STEM/Robotics Educator
    Parallax Inc.
  • As for sound card scopes - yes, but the goal is for BlocklyProp to be a "one-stop-shop". Teachers have a hard enough job, so we want to make BlocklyProp a "swiss army knife" instead of them having to learn a different tool for each thing they want to teach.
    Matthew Matz
    STEM/Robotics Educator
    Parallax Inc.
  • jmgjmg Posts: 12,621
    ...
    I'm hoping that what I'm asking is simple - just getting lots a data from the ADC to the terminal as fast as 115200 bps will allow (since I'm not too keen on messing with the default terminal speed in BlocklyProp - that's a whole different can of worms). The UI/JavaScript end might even be able to determine things like the trigger/etc from raw data (although that's probably best handled on the Prop).

    hehe, "lots a data" and 115200 bps are something of a contradiction.
    To stream you would need binary, and 12b needs 2 bytes, so that's 5760 dots-per-second update. (that's about 1/8 the speed of a 'sound card scope' sample rate)
    If you pace conversions at or below that speed, Prop is just a bridge (SPI-UART), but that's modest speeds - however it is quite simple, and may be enough for your demos.
    You could do a music tuner with quite good precision, as the Prop crystal sets the timebase.


    The ADC chip can manage 200k updates, ( and there are 500k and 1M siblings, for modest price changes)
    Anything > 5760 needs single shot capture/send, and there Prop does need to manage trigger. It could grab ~100ms of one chan, or ~25ms each of 4 chans at 200k, but needs ~3.5 seconds to send those ~20k samples to the PC.

    Also useful could be a Logic-Analyser type capture. Prop could send 4 Logic channels streamed, at 17280 samples per second.
    Enough to show a servo working, but somewhat coarse.
    A time-stamp Logic Analyser would get to something under 1us edge precision, for a low average edge rate (eg useful for Servo, 9600 baud comms )
  • David - I wish we could pay for it - but the idea is nicety, not a necessity, so no, it's more of an idea to throw out and see if there's any interest - especially from anyone who would like to see the functionality added to BlocklyProp.
    No problem. Just thought I'd ask.

  • WhitWhit Posts: 4,004
    Matt - this is an exciting idea!
    Whit+

    "We keep moving forward, opening new doors, and doing new things, because we're curious and curiosity keeps leading us down new paths." - Walt Disney
  • jmg - thanks for doing the math :)

    My thought is that the scope would grab 2000 (or maybe only 1000) samples, then go into sending mode (to the PC/terminal), and when that's done, grab the next 1000-2000 samples (or at least pace things that way simultaneously across cogs). Our eyes can't perceive things much faster than that anyway.

    Whether the scale of the viewing window is 5ms or .05ms, you'd be looking at 1000-2000 samples. Do forgive me - since this is only at the very early stages of an idea, I haven't done the math on what's possible or not yet.

    Additionally, I'm okay limiting it to two inputs (analog), and maybe supporting 4 down the road in a logic analyzer mode.

    I think you're right in that the Prop would have to handle the triggers.

    The nice thing is that the graphing engine draws lines as SVG paths (bezier curves), so it "fills in the blanks", which might be advantageous, too.

    Matthew Matz
    STEM/Robotics Educator
    Parallax Inc.
  • And I'm not opposed to trying to up the baud rate...we do different rates for the scribbler and other prop hardware, so BlocklyProp has the capability. And, since we would be (at least initially) limiting this to the ActivityBoard, I don't think doubling it is out of the realm of possibility.
    Matthew Matz
    STEM/Robotics Educator
    Parallax Inc.
  • jmgjmg Posts: 12,621
    The nice thing is that the graphing engine draws lines as SVG paths (bezier curves), so it "fills in the blanks", which might be advantageous, too.

    I would be quite careful with that - can it also show the sampling dots ?
    A problem with polished smooth curves is, it can give the illusion of precision, when none exists....

    My thought is that the scope would grab 2000 (or maybe only 1000) samples, then go into sending mode (to the PC/terminal), and when that's done, grab the next 1000-2000 samples (or at least pace things that way simultaneously across cogs). Our eyes can't perceive things much faster than that anyway.
    Snapshot and send gives the highest sps (short term), but the continual send mode 'feels' more like an analog scope, in that you can vary volume / amplitude and get instant change.
    Continual send should be the easiest to get working initially.
    Prop could grab up to 20k samples, but that's then a quite long send-time, so a user choice of samples-per-packet-burst lets them trade off speed against X-axis reach. 1~2k may be a usage trade off.

    I'd suggest you also look at the .RAW file that LTSpice uses.
    That's a simple N-point, floating point number X=Time,Y=amplitude file, that you can load into LTSpice (et al) and measure/zoom.


  • Let's not get ahead of ourselves here.

    The topic was build a simple scope to demonstrate with blocky prop how a scope works.

    I would like to try and write that library.

    I have the original Activity Board and did not realize that there was not a library for that ADC.

    Have not had to use it so did not see a need to write a library for it.

    I think the Activity Board WX uses the same ADC.


    Mike
  • With the data speeds involved, your best bet is to create a "ticker tape" style graph where you only write data to the right most edge of the graph. An assembly routine (<-for speed I don't see any other way) would then take the entire scope defined graphics region and shift it left by one pixel thus grabbing the data you are writing to the right most edge. The rate at which you shift the region and sample your data would be your TIME/DIV. I have already done this once but it was lost somehow in the OBEX and forum upgrade, but what I have described is the way you need to think for this project with limited data throughput. You simply don't have enough bandwidth to update the entire graphics frame in any human readable format that wouldn't be painful to watch.


    Beau Schwabe -- Submicron Forensic Engineer
    www.Kit-Start.com - bschwabe@Kit-Start.com ෴෴ www.BScircuitDesigns.com - icbeau@bscircuitdesigns.com ෴෴

  • iseries wrote: »
    Let's not get ahead of ourselves here.

    The topic was build a simple scope to demonstrate with blocky prop how a scope works.

    I would like to try and write that library.

    I have the original Activity Board and did not realize that there was not a library for that ADC.

    Have not had to use it so did not see a need to write a library for it.

    I think the Activity Board WX uses the same ADC.


    Mike

    Exactly! Simple is the goal. There is a library (I linked it in the original post), but it's only just the SPI driver for the onboard ADC - we'd need something that collects, does some minimal processing, and transmits them via serial. It does need to be fast, but I don't think we need we need high refresh rates.

    Keep me posted on what you are able to accomplish - If you're up for it, let's make this happen!
    Thanks!
    Matt

    Matthew Matz
    STEM/Robotics Educator
    Parallax Inc.
  • Where's blockly prop come into play? Will the C code that you're asking for be turned into a blackbox block? Does it need to be both fast enough to meet your requirements AND simple enough to be understood by new programmers, or are advanced techniques allowed? Along the same lines, C only or is C++ allowed? The talk about read from adc in bursts and then send over serial is weird when we're talking about a multicore chip... Are we restricted to a single cog due to blockly? I'm thinking it'd be easier if one cog read from the adc and dropped into hub RAM while another read from that queue and shot it over uart to the PC.
    David
    PropWare: C++ HAL (Hardware Abstraction Layer) for PropGCC; Robust build system using CMake; Integrated Simple Library, libpropeller, and libPropelleruino (Arduino port); Instructions for Eclipse and JetBrain's CLion; Example projects; Doxygen documentation
    CI Server: http://david.zemon.name:8111/?guest=1
  • MattMatzMattMatz Posts: 76
    edited July 4 Vote Up0Vote Down
    Im open to any idea really. c (not c++) is a hard restriction (although I've heard that a c++ library with a c wrapper is a thing?), and 115200 is a soft restriction. It should be a library, and would be added to SimpleLibraries (at least the blockly branch). Blockly currently only compiles with -O0 (none)/cmm. Black-boxed is okay, as it won't serve much purpose beyond this application. Performance is preferred over comprehensibility - for example: I bought my nail gun to make furniture, not to learn how nail guns work.

    By all means use the cogs :).
    Matthew Matz
    STEM/Robotics Educator
    Parallax Inc.
  • jmgjmg Posts: 12,621
    DavidZemon wrote: »
    The talk about read from adc in bursts and then send over serial is weird when we're talking about a multicore chip... Are we restricted to a single cog due to blockly? I'm thinking it'd be easier if one cog read from the adc and dropped into hub RAM while another read from that queue and shot it over uart to the PC.

    The need to capture, then read, is more related to the quite slow serial port rule of 115200.

    Looking at the ADC, it has a simple fixed 16 clock frame, with 2 index bits shifted in, and 12 ADC bits shifted out.
    Interleaving an ADC read with UART TX in a single COG, looks doable at a glance, for a 20 bit-slot total 2 byte frame. ie Very simple, fixed rate, bridge streaming.

    A full burst read at ADC peak speeds of 16 Clks at 200k (+?) would need 2 COGS as you say.
    Playback at that speed would need ~ 4MBd, which starts to thin out the candidate UARTS. eg CP2102N can be set to 4MBd, but may need HW handshake enabled to hit that reliably
    Possibly a round 100k sps could support over a 2MBd continual stream link ?
  • This thing won't be anywhere close to a professional scope, and I think that's okay. Here's what 180 per channel and two channels sent every 65 ms (1/15 of a second refresh rate) looks like:

    jsfiddle.net/8tq7xvmg/73/

    180 x 2 channels samples every 65 ms happens to be what can be squeezed into 115200 bps.

    The axes can be scaled however we want. Since the ADC chip on the ActivityBoard can do 200 ksps, that means that the X axis could zoom down to ~1 ms. Even at 100 ksps, a zoom down to 2ms isn't bad - enough to show servo pulses or medium-frequency audio.

    Would definately want an edge trigger capability, as well as letting the prop prep all of the data, since there wouldn't be any wiggle room in the sending of that data.
    Matthew Matz
    STEM/Robotics Educator
    Parallax Inc.
  • jmgjmg Posts: 12,621
    This thing won't be anywhere close to a professional scope, and I think that's okay. Here's what 180 per channel and two channels sent every 65 ms (1/15 of a second refresh rate) looks like:

    jsfiddle.net/8tq7xvmg/73/

    180 x 2 channels samples every 65 ms happens to be what can be squeezed into 115200 bps.

    That's quite nifty. I like the 'curves + dots' :)

    The axes can be scaled however we want. Since the ADC chip on the ActivityBoard can do 200 ksps, that means that the X axis could zoom down to ~1 ms. Even at 100 ksps, a zoom down to 2ms isn't bad - enough to show servo pulses or medium-frequency audio.

    Would definately want an edge trigger capability, as well as letting the prop prep all of the data, since there wouldn't be any wiggle room in the sending of that data.

    I'm not sure the Prop needs to prep all the data, and the more I look at an interleaved bridge code, the faster that can get.
    ie You might pack this into one COG, with no RAM cost, which means it becomes rather more a clip-on useful item for debug, as well as education.

    For this, you run the UART and SPI at the same time, and one UART bit is sent, for every SPI bit R/W.

    A little bit of 'housekeeping footwork is needed in the middle', when you have SPI bits arriving and no TX slot yet, but that's a primed FIFO to sort.
    Most bits are 4~5 lines, one bit is 6~7 lines, unless I swap the lower nibble bit order, in which case that drops to 4~5 lines.
    Bit reversal you would do via a table anyway, so nibble order is manged when that table loads.
    Hmm, thinking some more, I think the ADC will tolerate a paused clock during read, in which case this gets simpler, as no mini-FIFO is needed.

    Do you have any FT2232H / FT232H modules there, to do some real Serial crunch tests ?

    ie What baud rate can your javascript keep up with ?
    Does your 'redraw 1000 samples every 5 ms' include a serial buffer read ? - that's ~200ksps, which is quite decent.
  • To be honest, I don't think we've tested the baud limits on the browser's serial connection. I've seen websockets pass an incredible amount of data, so I don't think it will be the limiting factor. The example I linked is to demonstrate the lowest speed and resolution that still looks fluid and reasonably precise. You can crank that up quite a bit in the browser, but it starts really sucking CPU resources. It's also just inside the limit of a 115200 baud rate. I can conceivably double it (230400), but I'm not sure we should go any higher. That would be more than sufficient for this application, but I'd be totally okay with something far more capable that's actually slowed down for this use.
    Matthew Matz
    STEM/Robotics Educator
    Parallax Inc.
  • jmgjmg Posts: 12,621
    ... You can crank that up quite a bit in the browser, but it starts really sucking CPU resources.

    So was that 'redraw 1000 samples every 5 ms' a flat-out rate ?

    3 of the streaming 20 bit-slots have spare time, so they can poll a RXD for Start/Stop type commands for simple pc-side channel/mode/go on the streaming. (you would send a double 0x00,0x00 to force a stream break )
    .. I can conceivably double it (230400), but I'm not sure we should go any higher. ..
    Why not ? Useful to know just where the ceiling is, on Baud + javascript.
  • DavidZemonDavidZemon Posts: 2,559
    edited July 4 Vote Up0Vote Down
    I'm definitely intrigued. This sounds like a fun project and I have the time and (I think) the ability.
    Blockly currently only compiles with -O0 (none)/cmm.

    Really? -O0? Why not -Os? Cmm I get, but it's not like anyone's looking at the generated assembly for Blockly apps, right?
    c (not c++) is a hard restriction (although I've heard that a c++ library with a c wrapper is a thing?)

    C-wrapper around C++ is quite easy. For instance:
    /**
     * @file MyFoo.h
     */
    class MyFoo {
      public:
        MyFoo(int x) : m_x(x) { }
        void print() { std::cout << this->m_x << std::endl; }
      private:
        int m_x;
    }
    
    /**
     * @file MyFoo_c.h
     */
    void init_MyFoo(void *foo, int x);
    void print(void *foo);
    
    /**
     * @file MyFoo_c.cpp
     */
    #include <MyFoo_c.h>
    #include <MyFoo.h
    void init_MyFoo(void *foo, int x) {
      foo = (void *) new MyFoo(x);
    }
    void print(void *foo) {
      ((MyFoo *)foo)->print();
    }
    

    With that in mind, what would you think if I wrote it using PropWare classes? That'd be easier for me of course because I know those classes inside and out, but it would entail dropping a bunch of new code into Simple (such as my own UART object, and probably the Printer, and others). I'm guessing that's not ideal. But... if that doesn't scare you off, I'm probably up for it! However... I do worry about that -O0 flag. No idea what kind of code space the PropWare classes will consume with that flag.
    David
    PropWare: C++ HAL (Hardware Abstraction Layer) for PropGCC; Robust build system using CMake; Integrated Simple Library, libpropeller, and libPropelleruino (Arduino port); Instructions for Eclipse and JetBrain's CLion; Example projects; Doxygen documentation
    CI Server: http://david.zemon.name:8111/?guest=1
  • Yeah - my computer could handle 1000 samples every 5ms, but I don't think that's representative after giving it more thought - My machine has an i5 with a lot or RAM, so that's not a good benchmark for the average user. I'll have to try it on a Chromebook and see what it looks like.

    The -O0 flag is because for some time, we couldn't figure out how to get BlocklyProp to generate "volatile" for variables shared across cogs. We have figured it out since, but have not done the testing required to implement the change (which would then allow other optimization settings to be used). It's on the list, but it's priority is medium...

    Since I don't know anything about PropWare classes, ????

    I'll try to do some experimenting with the baud rate, and 1) see if I can change it on the fly within BlocklyProp, and 2) see how fast it can go.
    Matthew Matz
    STEM/Robotics Educator
    Parallax Inc.
  • Since I don't know anything about PropWare classes, ????

    The PropWare classes are a collection that I wrote with the intention of providing a complete HAL, much like Simple and libpropeller. However, unlike Simple, I used a few more complex language features than Simple (as well as C++ in general) to achieve better performance and higher code reusability, including inline assembly + fcache, interfaces/polymorphism, and templates. Usage of PropWare definitely requires a more mature grasp of C++ - it was never my intention to entice grade school children - but what you get is pretty cool. One of my favorite demos in PropWare is the one for my "Runnable" class: https://david.zemon.name/PropWare/api-develop/Runnable_Demo_8cpp-example.xhtml showing off the Runnable class for executing code in separate cogs, the Printer interface, the synchronous printer interface which lets you print from multiple cogs without requiring a dedicated cog for the UART driver as well as blinking an LED.

    Doxygen docs: https://david.zemon.name/PropWare/api-develop/namespacePropWare.xhtml
    Source code: https://github.com/parallaxinc/PropWare/tree/develop/PropWare

    So, it's a vastly different approach. On the one hand, the PropWare-ness would be completely hidden from the user in this "black box" that is Simple. I'd write a bunch of C++ code, get the job done, and then when all is said and done and tested, I'd wrap it with some C, copy/paste all the necessary files in Simple, test it one last time from SimpleIDE instead of PropWare's build system, and call it done. The user of said library would never know how different it is.

    But does Parallax want a bunch of code in their repository (which Parallax will ultimately be responsible for maintaining/supporting) that follows such a vastly different philosophy and coding style?
    David
    PropWare: C++ HAL (Hardware Abstraction Layer) for PropGCC; Robust build system using CMake; Integrated Simple Library, libpropeller, and libPropelleruino (Arduino port); Instructions for Eclipse and JetBrain's CLion; Example projects; Doxygen documentation
    CI Server: http://david.zemon.name:8111/?guest=1
  • jmgjmg Posts: 12,621

    Here are some test results, COM ports at likely baud rates. Uses FT2232H (HS-USB) as the 'emulate P1' TX test generator, because that can stream without gaps.
    Config with 2 Terminals, sending large files (up to 500k Bytes) , and look for issues..
    ~~~~~~~~~~~~~~~~ real UARTS Streaming Test Test results ~~~~~~~~~~~
    
    FT2232H transmit c8 ->   EXAR  XR21B1420   OK at 1MBd, 2MBd, 4MBd 100,000 chars continual, 8,N,1,  4.8MBd  8,M,2 OK, 6MBd  8,M,2 OK, 6MBd  8,N,1 OK, 8MBd  8,N,1 OK,(wow)  So conclude good margin on the XR21B1420
    
    FT2232H transmit c8 ->   CP2102N           OK at 1MBd, 2Mbd, 2.181818MBd, 2.4MBd, 2.666666MBd  3MBd, 3.428571MBd OK,  
    Drops Chars at 4MBd 8,N,1, Drops at 4M,8,N,2, but is looking OK at 4M 8,M,2 - so might be ok to pace CP2102N ? 
    
    FT2232H transmit c8 ->   FT232R            OK at 1MBd, 2MBd, 3MBd, No Support for 4MBd
    
    FT2232H transmit c8 ->   FT231X            OK at 1MBd, 2MBd, 3MBd, No Support for 4MBd
    
    CP2101N notes: ~~~~~~~~~~~~~~~~~~~~~~~~~
    4MBd with extra bits : > 20M/((20*5)+5+5+5+5) (2 stop + mark) = 166666.6666  20M/((20*5)+5+5+5+5+5) = 160000 - one more bit slot.
    or choose 
    3.428571 MBd = 24M/7/20 = 171428.57 sps, but are 20Mhz quanta, so ..
    20M/(24M/7) = 5.833333'   (6+6+6+6+6+5)/6 = 5.83333' every 6 samples, so can match 18, then 2 more 6's, for actual sample rate of 
    20M/((6+6+6+6+6+5)*3+2*6) = 170940.1709 sps ACTUAL rate at 3.428571 MBd  Actually (20.00MHz/117)
    or, if prefer round numbers, can pack 2 stop BITS sightly for  20M/((6+6+6+6+6+5)*3+2*10) = 160000 sps  (20.00MHz / 125)
    
    FT231X,FT232R notes:~~~~~~~~~~~~~~~~~~~~~
    3.000 MBd can be 6.7.7.6.7.7.6.7.7 during bit times, looks to be +/- 16ns edge variance on 333.3333ns ideal, likely fine. and can make 18 averags 6.2/3 with 2 more 7's at STOP bits, so firm rate is  
     N=0;;N=N+1;round(N*(6+2/3))/20M % 333.3333n
    ans = 1.66667e-8    ie  +16ns
    ans = 3.166667e-7  ie  -16ns
    ans = 1.0e-13
    20M/(18*(6+2/3)+7+7) = 149253.731 ACTUAL samples per second(max) at nom 3MBd  ( actually 20.00MHz / 134)
    
    EXAR  XR21B1420 Notes ~~~~~~~~~~~~~~~~
    The XR21B1420 manages even 8MBd data, so is the best performing FS-USB UART, and it should comfortably do 200k sps (20.00MHz/100)
            
    

    The Prop+ADC124S021 can manage 200ksps, (assumes 80MHz), but the common UARTs are the weak links...

    The best performing FS-USB uart is the EXAR XR21B1420, which can cope with an impressive 8MBd continual, but they are less common and a slight price premium.

    The CP2102N is next best, with more baud choices than FTDI, (2Mbd, 2.181818MBd, 2.4MBd, 2.666666MBd 3MBd, 3.428571MBd, 4MBd) but it just coughs at 4MBd continual 8.N.1.
    Works ok at 4MBd with 1 added stop bit plus mark parity, or is testing ok at 3.428571 MBd
    for an exact sample speed of (20.00MHz/117) = 170940.1709 sps

    FTDI parts seem to manage 3MBd, tho hard to gauge margins as they have no other choices close.

    FT231X for example, could do exact sample speed of 20.00MHz/134 = 149253.731 sps
    Yeah - my computer could handle 1000 samples every 5ms, but I don't think that's representative after giving it more thought - My machine has an i5 with a lot or RAM, so that's not a good benchmark for the average user. I'll have to try it on a Chromebook and see what it looks like.
    ...
    I'll try to do some experimenting with the baud rate, and 1) see if I can change it on the fly within BlocklyProp, and 2) see how fast it can go.

    I tried to gauge PC loading, and with large files and two terminals active (doing 4MBd one each way), it's hard to nudge over 10% CPU here, on a good desktop PC.

    Faster baud should be easy enough to set, as the drivers do all the work. Be interesting to see what javascript can handle.
    A useful feature of the hard-bridge coding, is you know the exact sample rate as that is set by the Prop Crystal, so should be within a few ppm.

    It would be nice to stream at max rates for 2~5 seconds, so close to 1MByte buffer. 'Scope like' Display zoom can be by skip of 2.5.10.20.50.100 samples,
    If you can support a > 1 second max, you can use GPS 1pps systems to calibrate the Prop Xtal, and turn this into a quite useful frequency counter.

    A good example of a good Frequency counter, using low sample rates, is http://www.daqarta.com/

  • MattMatzMattMatz Posts: 76
    edited July 5 Vote Up0Vote Down
    Okay - so let's take this a bit further.

    1) I took a peek at the code in BlocklyProp - and the baud rate doesn't look like it would be too hard to change, so let's say we can set that to anything we want within reason. I'll try to do some testing to see how fast we can go before we start losing data, because I don't think calculating a checksum is worth it, and then report back what I learn when I'm able to do this...

    1b) FYI - serial communication between the USB port and the browser is base64 encoded. Doesn't affect the prop, but there is a speed penalty of 75%. It's this way because upper ASCII (and some command ASCII bytes) would cause things to really flip out...

    2) Let's stick to the hardware on the Propeller Activity Board. That means that we have the ADC124S021, FTDI FT231X, and 5 Mhz crystal (16xPLL/80Mhz clock) to work with.

    3) Let's define what the data being sent to the browser looks like (open to suggestion, but here's some initial thoughts.
    a) 5 byte packets:
    byte 1: LSB [0-5] channel 1 low [6-7] 0b00
    byte 2: LSB [0-5] channel 1 high [6-7] 0b00
    byte 3: LSB [0-5] channel 2 low [6-7] 0b01
    byte 4: LSB [0-5] channel 2 high [6-7] 0b01
    byte 5: LSB [0-5] time, in microseconds (or 0.5 us or 2 us or 5 us increments?), since last measurement [6-7] 0b10, except FIRST measurement in a burst (tells the computer to refresh start collecting new) is 0b11.

    4) Let's define a command set that can be sent to the Prop at any time - REALLY open to suggestions here...
    a) command packet size?
    i) trigger (none, ch 1 rising, ch 1 falling, ch 2 rising, ch 2 falling, others?)
    ii) trigger voltage?
    iii) sampling frequency?
    iv) one shot or continuous?

    5) Not too worried about precision...any "measuring" (min/max/avg/frequency/etc.) could probably be done on the computer/browser side, but again, open to ideas.
    Thoughts?
    Matthew Matz
    STEM/Robotics Educator
    Parallax Inc.
  • jmgjmg Posts: 12,621
    Okay - so let's take this a bit further.

    1) I took a peek at the code in BlocklyProp - and the baud rate doesn't look like it would be too hard to change, so let's say we can set that to anything we want within reason. I'll try to do some testing to see how fast we can go before we start losing data, because I don't think calculating a checksum is worth it, and then report back what I learn when I'm able to do this...
    Sounds good. Those speeds above are with a compiled Terminal code, but even at the full 4MBps tests, it's hard to move the CPU load above 10%.
    Not sure how that maps onto javascript, but USB drivers and readers usually specify some buffer size, so I'd expect full speeds are possible, with the right buffer eizes.

    2) Let's stick to the hardware on the Propeller Activity Board. That means that we have the ADC124S021, FTDI FT231X, and 5 Mhz crystal (16xPLL/80Mhz clock) to work with.

    Yes, the 3MBd rate above with a dual-byte (16b SPI to 20b UART) encoding maps on FT231X : Gives exact sample speed of 20.00MHz/134 = 149253.731 sps

    1b) FYI - serial communication between the USB port and the browser is base64 encoded. Doesn't affect the prop, but there is a speed penalty of 75%. It's this way because upper ASCII (and some command ASCII bytes) would cause things to really flip out...
    Hmm. that's more of an issue - both from a raw speed cost, and a peak reading rate loss.
    Exact Base64 would not stream at 3~4MBd, but the encoding used below might just be good enough, if used with care ?

    My current mapping looks like this - it has expansion room for two types of Logic Analyzer / Digital streaming too, but that could be command based.

    ADC, stream, Binary packed First bit tagged, a/b = 00 for AnalogCH0, 01 For AnalogNextCH, 10 for packed digital(3x4b nibbles), 11 for timestamp digital
    [0.a.b.ADC11:7] - always sent paired a.b are tag bits, and a. can be 1, which is nearly base64, (if you ignore the bit reversal issue)
    [1.ADC6:0]

    It's encoded like this bit-order because the ADC has a few clocks preamble and then delivers data MSB first.
    Usually this would feed through a fast re-map table lookup, to swap the bit order & shift/add, which can (hopefully) be done before any escape testing ?
    The PC knows how many samples in the stream eg 1000~1000000 ?
    4) Let's define a command set that can be sent to the Prop at any time - REALLY open to suggestions here...
    a) command packet size?
    i) trigger (none, ch 1 rising, ch 1 falling, ch 2 rising, ch 2 falling, others?)
    ii) trigger voltage?
    iii) sampling frequency?
    iv) one shot or continuous?

    To keep the prop side simplest, right now the command variables are just
    * A SampleCount (in 20.00MHz/134 units), (may as well be 32b )
    * Channels to scan (ADC can do 1,2,3,4)
    * A possible live Break from the PC (in case the sample count is too large, or perhaps a stream forever to disk may be possible ? Numeric limit of 32b at 3MBd is ~ 8 hours.)

    because it can stream at a good enough speed, the PC can manage Trigger/Trigger voltage/ sampling frequency (by skip)
    The sample count means the PC can know exactly how many bytes to store as samples, and bypass the base64 handler.
    (of course, assumes no lost bytes, but there is a unique end-of-frame possible, that could serve as a double check )
    ie Request N Readings. repeat read until EOF, and check N were stored.
    5) Not too worried about precision...any "measuring" (min/max/avg/frequency/etc.) could probably be done on the computer/browser side, but again, open to ideas.
    Precision comes for free, with packed transmit, as the timestamp is built in :)
    The bridge code has an exact, deterministic time base, that is Crystal Accurate, and the Crystal can be GPS calibrated, so single digit ppm (or better) looks achievable.

    Might as well use/ show off the full abilities of the Prop :)



  • DavidZemonDavidZemon Posts: 2,559
    edited July 5 Vote Up0Vote Down
    I'm still digesting some of you said, but one part jumped out at me
    3) Let's define what the data being sent to the browser looks like (open to suggestion, but here's some initial thoughts.
    a) 5 byte packets:
    byte 1: LSB [0-5] channel 1 low [6-7] 0b00
    byte 2: LSB [0-5] channel 1 high [6-7] 0b00
    byte 3: LSB [0-5] channel 2 low [6-7] 0b01
    byte 4: LSB [0-5] channel 2 high [6-7] 0b01
    byte 5: LSB [0-5] time, in microseconds (or 0.5 us or 2 us or 5 us increments?), since last measurement [6-7] 0b10, except FIRST measurement in a burst (tells the computer to refresh start collecting new) is 0b11.

    This is storing quite a lot of state in the messaging system. Are you worried about the PC-side of the world loosing track of things, or bytes coming in out-of-order? I'm not sure why you need the high bits in every single byte, especially the final one for time, where you've only left yourself with the ability to count up to 63 microseconds, which is almost 16kHz. As for the last byte, the time between each sample - the PC is the one setting the sample rate, why can't the PC keep track of that? No need to repeat the same number over and over again.

    What about this simpler idea:

    byte 1: Low byte of channel 1
    byte 2: [0-3] high bits for channel 1, [4-6] Channel for this and the previous byte, [7] Unused?
    Repeat these two bytes for each channel currently enabled

    David
    PropWare: C++ HAL (Hardware Abstraction Layer) for PropGCC; Robust build system using CMake; Integrated Simple Library, libpropeller, and libPropelleruino (Arduino port); Instructions for Eclipse and JetBrain's CLion; Example projects; Doxygen documentation
    CI Server: http://david.zemon.name:8111/?guest=1
  • David -
    This is storing quite a lot of state in the messaging system. Are you worried about the PC-side of the world loosing track of things, or bytes coming in out-of-order? I'm not sure why you need the high bits in every single byte, especially the final one for time, where you've only left yourself with the ability to count up to 63 microseconds, which is almost 16kHz. As for the last byte, the time between each sample - the PC is the one setting the sample rate, why can't the PC keep track of that? No need to repeat the same number over and over again.

    What about this simpler idea:

    byte 1: Low byte of channel 1
    byte 2: [0-3] high bits for channel 1, [4-6] Channel for this and the previous byte, [7] Unused?
    Repeat these two bytes for each channel currently enabled

    To tweak your suggestion slightly:

    bit [7] could mark the first byte in a burst.


    jmg -

    the base64 thing happens between the BlocklyProp-client and the browser. More exactly, USB data comes into the BlocklyProp-client RAW, then gets base64 encoded, and sent to the browser via a Websocket. When the browser gets it, it is then base64 unencoded. This is because certain ASCII chars would break the Websocket. When we move to an offline version, we won't need the Websocket because the client and browser will be within the same app, so at that point the base64 encoding goes away. Unfortunately, there isn't really a way to conditionally use encoding or not - it's sort of an all or nothing thing. That being said - some speed tests will prove out what's possible.

    Both - I was mistaken in thinking the packets should be timestamped - if the prop sends at regular intervals, that should be sufficient. So I like David's simple packet suggestion.

    I do suspect that we will want to Prop to manage the trigger...but again, open to what's possible, and letting testing prove out what's best.


    I'll start cooking up some browser HTML/Javascript code to do some testing with this.
    Thanks!
    Matthew Matz
    STEM/Robotics Educator
    Parallax Inc.
  • Matt,

    If it would help at all, you're welcome to use and modify my SpinScope HTML/Javascript:

    https://forums.parallax.com/discussion/157982/spinscope-a-virtual-oscilloscope-for-the-propeller/p1

    My original code doesn't display analog, but Heater modified it so that it could (same thread).

    -Phil
    “Perfection is achieved not when there is nothing more to add, but when there is nothing left to take away. -Antoine de Saint-Exupery
Sign In or Register to comment.