Forum Update - Announcement about May 10th, 2018 update and your password.

SpinScope: A Virtual Oscilloscope for the Propeller

2456

Comments

  • turbosupraturbosupra Posts: 1,088
    edited November 2014 Vote Up0Vote Down
    This is exactly the unit I was looking at when I worded my previous comment. I really like the Logic8 and will probably be buying one very soon, for what it does it is very well priced IMO.

    Publison wrote: »
    Parallax just ran out of an inexpensive Logic Analyzer from Saleae, that does all that for $149.00 US. Unfortunately, Saleae discontinued that product and came out with a new one that has some analog capabilities at $199.00US. Still not a bad price
  • TorTor Posts: 1,934
    edited November 2014 Vote Up0Vote Down
    http://www.bitscope.com/product/BS05/
    Someone posted this Saleae alternative on another forum. They have various models. Looks interesting.I think I'll order that BS05 model and a 10:1 analog probe when I'm back in Japan (less VAT)
  • mklrobomklrobo Posts: 420
    edited November 2014 Vote Up0Vote Down
    Awesome job! I will use it for a serial sniffer, for other
    projects! Thanks!
  • tritoniumtritonium Posts: 225
    edited November 2014 Vote Up0Vote Down
    Could somebody point their browser at http://the.linuxd.org/lab/

    Very nice in Chrome (once its powered up (the 'scope that is))
    In IE11 it works but I get a stationary image unlike Chrome where the image jitters, which I suspect is what is meant to happen.

    Does it NEED a power button?

    Dave
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 21,922
    edited November 2014 Vote Up0Vote Down
    trionium wrote:
    Does it NEED a power button?
    Yes. The "power" button is what connects and disconnects the server from the Prop so the Prop can be reprogrammed. The same serial port cannot be open on two apps at once.
    __________

    Heater, your signal looks good here. I'm not sure that streaming the data is going to help with the update rate, though. The real bottleneck is between the server and the Propeller, not between the browser and the server.

    -Phil
    “Perfection is achieved not when there is nothing more to add, but when there is nothing left to take away. -Antoine de Saint-Exupery
  • Heater.Heater. Posts: 20,779
    edited November 2014 Vote Up0Vote Down
    tritonium,
    Does it NEED a power button?
    Yes.

    Well no. It's only part of the whole retro scope idea.

    It's like that thing where you get an image of a floppy disk on the save button of applications. Why for God sake? Icons are supposed to indicate function but young people today have never seen a floppy disk and probably get puzzled as to what it means.

    And why the green trace? Who has ever seen an old analogue scope with a green phosphor cathode ray tube? And why the need for an image of a scope screen bezel at all?

    Me, I love it all. I like the way it lights up when you hit the power button :)

    Now, the image should be jittery and wiggly, it has 10% or so of random noise added to it.

    I'n curious that you might see a single static image in IE11. That implies that at least one AJAX request is working but then not the following ones. Strange.
  • JDatJDat Posts: 103
    edited November 2014 Vote Up0Vote Down
    Hmm. Looks great on Sony Xperia Z with Crome browser over WiFi...
  • Heater.Heater. Posts: 20,779
    edited November 2014 Vote Up0Vote Down
    Phil,
    The "power" button is what connects and disconnects the server from the Prop so the Prop can be reprogrammed....
    Ah yes, of course, interesting.


    Of course what I did here was not what you had in mind. I put the thing on the net where there could be potentially many people viewing. As opposed to the GUI being the single source of control.


    But then again in a classroom setting you may even want the possibility of multiple viewers of the same machine for demo purposes.
    I'm not sure that streaming the data is going to help with the update rate, though
    Quite so. I subsequently thought about it and realized that we are only good for about 11 updates per second anyway at 115200 baud from the Propeller.


    I think my next angle of attack then has to be connecting the web page directly to the serial port. Only for Chrome extentions though.


    Now to decipher your serial port protocol...
  • PublisonPublison Posts: 10,124
    edited November 2014 Vote Up0Vote Down
    turbosupra wrote: »
    This is exactly the unit I was looking at when I worded my previous comment. .

    You should visit the Parallax Store more often. :)
    Infernal Machine
  • Heater.Heater. Posts: 20,779
    edited November 2014 Vote Up0Vote Down
    Publison,

    What, you mean these Parallax forums guys also sell stuff ?

    :)
  • PublisonPublison Posts: 10,124
    edited November 2014 Vote Up0Vote Down
    Heater. wrote: »
    Publison,

    What, you mean these Parallax forums guys also sell stuff ?

    :)

    Sometimes..

    I hear there is a P2 coming.m :)
    Infernal Machine
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 21,922
    edited November 2014 Vote Up0Vote Down
    JonnyMac wrote:
    You declare cogno as a byte, yet check for less than 0 -- this should never work, correct? If cognew returns -1, cogno will be truncated to 255. If cognew for xcrv fails, should you not add a cogstop for cogno before returning false?
    Right on both counts, Jon! I was in a hurry to have it ready for class and got a little careless.

    -Phil
    “Perfection is achieved not when there is nothing more to add, but when there is nothing left to take away. -Antoine de Saint-Exupery
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 21,922
    edited November 2014 Vote Up0Vote Down
    Heater,

    The serial protocol is simple: DLE Cmd Data ... Data DLE EOT. If a data byte happens to be a DLE, a second DLE is inserted into the data stream and removed at the receiving end. Perl's regexes make it easy to parse out whole messages from the serial stream and do the DLE pruning.

    -Phil
    “Perfection is achieved not when there is nothing more to add, but when there is nothing left to take away. -Antoine de Saint-Exupery
  • turbosupraturbosupra Posts: 1,088
    edited November 2014 Vote Up0Vote Down
    I have in the past, but this has caused fights with my wife ;)
    Publison wrote: »
    You should visit the Parallax Store more often. :)
  • Heater.Heater. Posts: 20,779
    edited November 2014 Vote Up0Vote Down
    Phil,

    Regular expressions, blech. They are impossible to read and always buggy. Real men write their own stream parsers :)

    More seriously, JS can of course do all that regexp stuff, except I have no idea how. I guess we need to use some kind of \x01 syntax to catch those control characters.
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 21,922
    edited November 2014 Vote Up0Vote Down
    Heater,

    Regexes are both terse and very fast. You can't process text in a scripting language a character at a time and expect to match regex performance. Using regexes ranks near the top of the programming skills I've learned over the years. Here's how I use them to parse the serial data stream from the Prop:
    $Stream .= $Port->input;
    while ($Stream =~ m/\x10([A-Z](\x10\x10|[^\x10])*)\x10\x04/osg) {
      my $cmd = $1;
      $cmd =~ s/\x10\x10/\x10/osg;
      do_command($cmd);
      $nxt = pos($Stream);
    }
    $Stream = substr($Stream, $nxt) if defined $nxt;
    


    The m/.../ looks for matches. \x10 is the leading DLE that has to be followed by a capital letter [A-Z]. That gets followed either by a pair of DLEs, \x10\x10 or | a non-DLE [^\x10], repeated 0 or more times *. That gets followed by a DLE EOT, \x10\x04 to end the message. The group in the outer parentheses is broken out into variable $1 if an entire match occurs.

    The s/\x10\x10/\x10/osg just replaces all pairs of DLEs with singular ones.

    The global matching mechanism remembers where the last match ended, associating pos with the input string, so that the matched stuff can be chopped off the beginning of the string with substr.

    Simple! :)

    -Phil
    “Perfection is achieved not when there is nothing more to add, but when there is nothing left to take away. -Antoine de Saint-Exupery
  • Heater.Heater. Posts: 20,779
    edited November 2014 Vote Up0Vote Down
    Phil,
    You can't process text in a scripting language a character at a time and expect to match regex performance.
    That sounds like a challenge!

    Your code snippet demonstrates my point. It looks like the baud rate was set wrong on whatever link received it. And that's just the Perl parts never mind the regexps! :)

    You might have to explain what is going on there.

    The way I see these things is that we are receiving a stream of input tokens and we want to extract the data framed by something like "startToken, data, data, data, data......data, data, endToken"

    Problem is that when we get some bytes from the stream we may only have "startToken, data, data, data, data".

    If we parse that, with a regexp or manually, we do not have a data packet. We have to wait for more bytes to arrive.

    What to do?

    1) Get the bytes. Parse them, and if not complete wait for some more bytes. Append them to what we have got. Parse the whole buffer again. And so on until a match, or more, is found.

    2) Build a state machine that throws away bytes until it sees a startToken, appends bytes to a packet until it sees an endToken. Acts on the packet.

    I have to meditate on your code some more but it looks like you are doing option 1). Repeatedly parsing whatever byes you have until something interesting is accumulated there.

    It's not clear to me how this is more machine efficient than a handmade state machine.

    Time for another experiment...I have so many today.
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 21,922
    edited November 2014 Vote Up0Vote Down
    Heater,

    I never wait for more bytes if there's an incomplete message. I just return and come back later. The $Stream variable is global, persistent, and acts as a secondary queue that can be read as deeply as necessary to parse an entire message before it's removed. Stuff gets added to the end from the serial-port queue and taken off the beginning in whole-message chunks. No bytes are ever thrown away.

    -Phil
    “Perfection is achieved not when there is nothing more to add, but when there is nothing left to take away. -Antoine de Saint-Exupery
  • Heater.Heater. Posts: 20,779
    edited November 2014 Vote Up0Vote Down
    Phil,

    I'm a bit too tired to follow this. But surely you run your regexp over partial packets again and again until a match is found when you eventually have enough bytes to make a whole packet.

    For certain the "thrown away" bytes I as alluding to are the tail end of some packet that were received when your program starts up in the middle of an incoming packet. They cannot be used as you don't know where you are at that point.
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 21,922
    edited November 2014 Vote Up0Vote Down
    Heater,

    My program never statts a match in the middle of a packet. The first byte of $Stream is always the first byte of a packet. You may have missed the period in this statement:
    $Stream .= $Port->input;

    -Phil
    “Perfection is achieved not when there is nothing more to add, but when there is nothing left to take away. -Antoine de Saint-Exupery
  • Heater.Heater. Posts: 20,779
    edited November 2014 Vote Up0Vote Down
    Phil,

    Hmm..now I'm even more tired than I was before so this gets harder...but:

    I don't see how you can say your program never starts in the middle of a packet. Given that the Prop and the Perl code can be stopped and an restarted independently at any time synchronization is not assured.

    I did not miss the ".". I puzzled about it a bit and still have no idea what it means. :(

    In the event of errors on the line we cannot be sure where we are in a packet. OK, perhaps we can assume there are no errors and this never happens.

    Perhaps I'm wrong but surely if your program gets 10 bytes of a 100 byte packet it runs the regexp to see if anything interesting arrived.

    Then if another 10 bytes arrives the regexp is run on the 20 accumulated bytes. Again nothing there.

    This continues until all 100 bytes are received.

    Which does not sound very efficient.

    Or have I totally misunderstood that code snippet?
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 21,922
    edited November 2014 Vote Up0Vote Down
    heater wrote:
    ... now I'm even more tired than I was before ...
    Do you never sleep? :)
    Given that the Prop and the Perl code can be stopped and an restarted independently at any time synchronization is not assured.
    Okay, but once the first fragment has passed, and barring further serial-port interruptions, my statement holds.
    I did not miss the ".". I puzzled about it a bit and still have no idea what it means.
    It's Perl's string-append assignment operator.
    Perhaps I'm wrong but surely if your program gets 10 bytes of a 100 byte packet it runs the regexp to see if anything interesting arrived. Then if another 10 bytes arrives the regexp is run on the 20 accumulated bytes. Again nothing there. This continues until all 100 bytes are received. Which does not sound very efficient.
    In theory, that may be true. In practice, having tight assembly code do the regex match is still quicker than having Perl interpreted code running a byte-by-byte state machine.

    -Phil
    “Perfection is achieved not when there is nothing more to add, but when there is nothing left to take away. -Antoine de Saint-Exupery
  • Cluso99Cluso99 Posts: 13,760
    edited November 2014 Vote Up0Vote Down
    WOW, this is just beautiful Phil!

    Love the concept of using the browser. Makes it much more machine independent.

    It would be possible to use a different serial port and a cheap USB-Serial (about $3-4) on eBay for the scope section, keeping the propplug for download.
    Should also be possible to send via WiFi and the ESP8266 ($3-5).

    BTW I see some previous "sync" life in those DLE-xxx uses ;)
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 21,922
    edited November 2014 Vote Up0Vote Down
    Thanks, Ray!
    It would be possible to use a different serial port and a cheap USB-Serial (about $3-4) on eBay for the scope section, keeping the propplug for download.

    Good point. There's no reason to restrict SpinScope I/O to P30/31. I could add a start_explicit routine that lets the user select the I/O pins.

    This kind of feedback is what I love about the forum! :)

    -Phil
    “Perfection is achieved not when there is nothing more to add, but when there is nothing left to take away. -Antoine de Saint-Exupery
  • Heater.Heater. Posts: 20,779
    edited November 2014 Vote Up0Vote Down
    Phil,
    Do you never sleep?
    ....zzz...zzz...zzz..... Seven hours later....

    Yep. Just got my 7 hours in. Often I find it very hard to get to sleep at the right time. It's happening more and more in recent years. Very annoying.

    Having all those dots and dollars and other line noise which is the Perl language nagging at my mind does not help with a sleep problem :)

    I think you have convinced me that parsing with regexps is going to be faster than a manually written parser. In some odd cases that might not be true (large packets received in small chunks) but that's not what we have here.

    Now how to detect your protocol in JS...
  • TumblerTumbler Posts: 323
    edited November 2014 Vote Up0Vote Down
    Nice work Phil

    I like the http server for showing output into a brouwser.
    I'm sure you like these components too (maybe for future use)
    Tadaaaah!
  • WurlitzerWurlitzer Posts: 237
    edited November 2014 Vote Up0Vote Down
    Wow! Thanks a ton!
    This will be very helpful on my Organ Props for when the program somehow goes off into the weeds to at least be able to see if all the various Shift register clock, data, strobe signals are active without having to pull the console from the wall, and schlep the scope downstairs or to the 3rd floor organ chamber to connect it to the Prop.
  • Heater.Heater. Posts: 20,779
    edited November 2014 Vote Up0Vote Down
    Phil,

    You got me going with that regexp stuff. I always have a big struggle getting regexps to do what I want. Never can be sure when or if they are correct. Anyway the JavaScript version of your parser turns out to look like this:
    function parse (input) {
        var reArray,
            endPos,
            cmd,
            re = /\x10([A-Z](\x10\x10|[^\x10])*)\x10\x04/g;
    
        stream += input;
        while (reArray = re.exec(stream)) {
            cmd = reArray[1];
            cmd = cmd.replace(/\x10\x10/g, '\x10');
            doCommand(cmd);
            endPos = reArray.index + reArray[0].length;
        }
        stream = stream.substring(endPos, stream.length);
    }
    
    The parse loop is pretty much the same once the foibles of JS regexps are figured out.

    Now I'm all set to get SpinScope working as a Chrome App.

    One thing bothers me though. The payload data is binary. So who is to say those DLEs and EOTs don't show up in the data and cause the parser to get confused?
  • Heater.Heater. Posts: 20,779
    edited November 2014 Vote Up0Vote Down
    Damn, it seems a \x10 is leaking through into doCommand for single character-no-data commands. It's not clear to my why your code does not do that.
  • TorTor Posts: 1,934
    edited November 2014 Vote Up0Vote Down
    The Perl regex system is very powerful as well as efficient. And more complete than just about any other regex system. Not too difficult to learn either. I maintain a quite large commercial Perl setup for a customer, there are some bottlenecks in the code but the regex (and there are a lot of them) are never part of the bottlenecks. I like Perl, although unlike Phil I like C too :) - The customer likes it too because they get to see the source code.. and can try out changes before they suggest updates to us. Oh, and for quickly getting a piece of Perl code with or without regex working - get the Perl Cookbook, it's one of the best support books out there.

    I'm looking forward to where SpinScope goes.. this can become very interesting indeed. Good stuff Phil!
Sign In or Register to comment.