Shop OBEX P1 Docs P2 Docs Learn Events
Fix Parallax Serial Terminal so $0D can be user choice of newline or carriage return? — Parallax Forums

Fix Parallax Serial Terminal so $0D can be user choice of newline or carriage return?

Chris B.Chris B. Posts: 18
edited 2013-10-14 18:48 in General Discussion
In my experience $0D strictly means carriage return. Thus my serial code sends $0D0A for a newline for compatibility with the defaults in Putty, Hyperterm, etc.

Parallax Serial Terminal is the first I've ever encountered which treats $0D like a Unix newline and no apparent means to modify. Am I just missing a setting that's right in front of my eyes?

It seems like an easy fix, such as in Putty, have two extra check boxes in the Prefs "Implicit LF with each CR" and "Implicit CR with each LF". With the former being checked by default so behavior is unchanged from what Parallax Serial Terminal users are accustomed to.

Any interest from the company? Is this the right place to propose this, or should I contact someplace other than the forums?

Thanks for your consideration!

Best regards,
Chris
«1

Comments

  • msrobotsmsrobots Posts: 3,709
    edited 2013-10-09 18:02
    +1

    I dislike this also - the whole CRLF thing is a mess.

    as of ansi definition CR should return to the beginning of the same line and LF should do a new line.

    as any teletype printer did.

    and dos/windows does.

    Then came C/C++ and Linux. and redesigned that concept. Declared CRLF is to much to type so LF should be enough.

    Then came MAC and decided CRLF is to much to type so CR should be enough.

    And that is where we are now. MAC uses CR, Linux uses LF and windows does what is supposed to be there anyways and uses CRLF

    to get rid of the extra lines in PST you can disable LF (or CR).

    but this is still wrong IMHO

    Enjoy!

    Mike
  • Don MDon M Posts: 1,652
    edited 2013-10-09 18:10
    +2........
  • Mike GMike G Posts: 2,702
    edited 2013-10-09 19:24
    You can disable/enable ASCII control characters in PST preferences.
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2013-10-09 19:56
    But if you disable them, don't they show up as weird text?

    -Phil
  • Mike GreenMike Green Posts: 23,101
    edited 2013-10-09 20:07
    The terminal emulators I use (like CoolTerm and MacWise) give you a choice of line endings: CR, LF, or CR+LF. There's all sorts of other configuration possible. The Parallax Serial Terminal ought to be able to do the same.
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2013-10-09 20:12
    Without any changes to PST itself, the Parallax Serial Terminal.spin object would be easy enough to modify to provide the sought-after flexibility.

    -Phil
  • Heater.Heater. Posts: 21,230
    edited 2013-10-09 22:01
    msrobots,

    Do not rewrite history. "Then came C/C++ and Linux. and redesigned that concept." is so incorrect.

    In the early 1960s the ISO standard allowed use of either CR+LF or LF alone as a newline. There was no ANSI at that time.

    In 1964 the MULTICS operating system used LF for newline. Unix followed along in the early 1970s. Other systems followed that choice later.

    The C standard uses "\n" but says when writing/reading text mode files it should be translated to/from what ever sequence the hardware uses.

    Now MSDOS did not turn up till 1980 or so, nearly 29 years after the begining of this story. They proceeded to get it wrong whilst using the wrong path separator character as well.

    Macs are also much later and just wierd. Mac after Mac OS 9 do not use CR. They realized thier mistake.

    Clearly having a single character to get you to the begining of the next line is the correct thing to do. Even ancient typewriters only had a single operator input to do both. Clearly LF is the way to go because that what everyone standardized on from the get go,

    The program dos2unix fixes these Microsoft quirks:)
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2013-10-09 22:13
    Heater,

    But you ignore the Model 33 KSR/ASR teleprinter and its predecessors. CR returned the printhead to the beginning of the same line (useful for obfuscating typed passwords), LF scrolled the paper up one line without moving the printhead. It took both to do a proper newline. Microsoft simply copied the functionality of equipment that had already been in use for years.

    -Phil
  • Heater.Heater. Posts: 21,230
    edited 2013-10-09 22:38
    I don't forget the teleprinters. They are hardware. I see no reason why the details of hardware should be encoded into my text files. And that is what the C standard says. Although such abstractions were "a new idea" at the time. Should all my files be in postscript because that is what my old laser printer uses?

    When it come to terminal emulations they should all be VT100 compatible anyway:)
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2013-10-09 22:45
    heater wrote:
    I see no reason why the details of hardware should be encoded into my text files.
    And yet we still use the abbreviation CR for carriage return. How odd. :)

    -Phil
  • Heater.Heater. Posts: 21,230
    edited 2013-10-09 23:04
    Hmm...so what did they call that action you had to do on old mechanical typewriters, swiping that big lever from right to left, that did both a return and a new line at the same time?
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2013-10-09 23:34
    You got a CR by pushing on the carriage itself; a LF, by turning the knob. That lever thingy was just a macro that could be "programmed" for a 1-, 1.5-, or 2-line automatic advance.

    -Phil
  • Heater.Heater. Posts: 21,230
    edited 2013-10-09 23:45
    A 'macro'. I like it.

    Same kind of macro as in "\n" in the C standard then:)

    No body ever figured out what "\t" should do consistently either.
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2013-10-10 00:16
    Funny thing about "\n": it performs the expected operation across multiple OS's when sent to STDOUT. But ord("\n") (in Perl at least) always returns 10 (LF).

    -Phil
  • Heater.Heater. Posts: 21,230
    edited 2013-10-10 02:48
    In C/C++ "\n" and "\r" are escape sequences that produce single characters representing new line and carriage return. They are not required to be the same as the ASCII LF and CR although they pretty much always are.

    Somewhere in the standard it tells that when working with text files "\n" is to be translated by the system to whatever sequence is required to get a "new line + carriage return". Similarly such newline + carriage return sequences are translated back to "\n" when reading.

    To files are opened as text mode by the fopen function unless you add the "+b" to the mode line in which case you get binary mode with no translation.

    When working with serial port devices you can set up "raw" or "cooked" mode.

    Perl is not lying to you.
  • Mike GMike G Posts: 2,702
    edited 2013-10-10 04:17
    But if you disable them, don't they show up as weird text?
    No, not using pst.str(). I just checked it to make sure I was not crazy.
  • Martin_HMartin_H Posts: 4,051
    edited 2013-10-10 06:43
    Do you want a real CR versus CRLF horror movie mess? Work on a software project with a shared source code repository on Linux, but the GUI designers use Macs, the front end developers use Windows, and the back end developers user Linux. Now have a few team random members improperly configure their source code control client to use the wrong line endings for their platform. At first you don't notice because the revision control systems use deltas, so unchanged lines are unaltered, while only changed lines are corrupted.

    So at first code like this:
        if (test)
        {
           \\ a comment
           ... code...
        }
    

    becomes this:
        if (test)     {        \\ a comment
           ... code...
        }
    

    Which is completely harmless, and even looks fine. But like the blob from the 50's horror film, the problem grows out of site until it becomes an uncontrollable monster. Pretty soon some files end up with changes that aren't harmless, but still compile, however tests start breaking and no one knows why. Finally the $#!% hits the fan and the build breaks as some file gets all slammed together. At that point you have to hunt for the person who's messed up their machine, and do an audit of your source code looking for modules they changed. But some of those people are in India, and have no idea what you are talking about. Oh and you can't change revision controls systems because it's a corporate standard, and the people who decided on the standard also have no idea what you are talking about.

    You'll wake up screaming.
  • Heater.Heater. Posts: 21,230
    edited 2013-10-10 08:00
    Martin_H,

    Oh God yes. Some twat on Windows ends up converting a whole file from unix style to dos style and then manages to check it in.
    Great, now "git diff" tells us that every single line of code has been changed.
    That's even before we get onto the guys who like to just reformat everything to their own personal indenting standards before they even look at a file.
  • jazzedjazzed Posts: 11,803
    edited 2013-10-10 12:16
    Oivey!

    Qt works fine regardless of the platform :)
  • Heater.Heater. Posts: 21,230
    edited 2013-10-10 12:26
    Oh yeah. And now you can compile Qt Apps to JavaScript and run them in a browser. Very soon SimpleIDE will be a web app!
    http://vps2.etotheipiplusone.com:30176/redmine/projects/emscripten-qt/wiki/Demos
  • KyeKye Posts: 2,200
    edited 2013-10-10 14:57
    I don't know why this is a problem, you can easily handle all three cases for display purposes on the terminal without any major issues:
    // Please assume variable initialization else where... this code is just for example...
    
    ...
    
    bool firstCharSeen = false;
    
    char prev = curr;
    char curr = getChar(); 
    
    switch(curr)
    {
      ...
    
      case '\n': 
      {
        if((!firstCharSeen) || (prev != '\r')) 
        {
          doNewLine();
        }
        
        break;
      }
    
      case '\r'
      {
        doNewLine();
    
        break;
      }
    
      ...
    }
    
    firstCharSeen = true;
    
    

    If the user sticks to all '\n' it works, if they do just '\r', it works, and if they do '\r\n' it works. They can even mix them too.

    ---

    And yes, of course if the data stream is corrupted there will be issues... but, the data stream is corrupted anyway so who cares... hit the clear button.
  • KyeKye Posts: 2,200
    edited 2013-10-10 15:00
    @Heater - Not sure how the backend would run :)
  • msrobotsmsrobots Posts: 3,709
    edited 2013-10-10 17:21
    here from Wikipedia ...http://en.wikipedia.org/wiki/Newline#History

    ASCII was developed simultaneously by the ISO and the ASA, the predecessor organization to ANSI. During the period of 1963–1968, the ISO draft standards supported the use of either CR+LF or LF alone as a newline, while the ASA drafts supported only CR+LF.
    The sequence CR+LF was in common use on many early computer systems that had adopted Teletype machines, typically a Teletype Model 33 ASR, as a console device, because this sequence was required to position those printers at the start of a new line. On these systems, text was often routinely composed to be compatible with these printers, since the concept of device drivers hiding such hardware details from the application was not yet well developed; applications had to talk directly to the Teletype machine and follow its conventions. In particular, this explains the names of the control characters, which signify the actions of an ordinary mechanical typewriter of the times: "Line Feed" is the action to switch to a new line, carried out by the paper feed mechanism of the typewriter carriage, and "Carriage Return" is the action to position the typing device at the beginning of the line, carried out by the sliding movement of the carriage.
    Most minicomputer systems from DEC used this convention. CP/M used it as well, to print on the same terminals that minicomputers used. From there MS-DOS (1981) adopted CP/M's CR+LF in order to be compatible, and this convention was inherited by Microsoft's later Windows operating system.
    The separation of the two functions concealed the fact that the print head could not return from the far right to the beginning of the next line in one-character time. That is why the sequence was always sent with the CR first. In fact, it was often necessary to send extra characters (extraneous CRs or NULs, which are ignored) to give the print head time to move to the left margin. Even many early video displays required multiple character times to scroll the display.
    The Multics operating system began development in 1964 and used LF alone as its newline. Multics used a device driver to translate this character to whatever sequence a printer needed (including extra padding characters), and the single byte was much more convenient for programming. The seemingly more obvious choice of CR was not used, as a plain CR provided the useful function of overprinting one line with another to create Boldface and Strikethrough effects, and thus it was useful to not translate it. Unix followed the Multics practice, and later systems followed Unix.

    Enjoy!

    Mike
  • Heater.Heater. Posts: 21,230
    edited 2013-10-10 21:53
    msrobots,

    Yes I do enjoy that because it supports my claim that you were rewritting history by saying that C/C++ and Linux had messed up the line ending thing after Mac ond DOS had got it right. The Article clearly states that it was the other way around.

    Thank you :)
  • Chris B.Chris B. Posts: 18
    edited 2013-10-11 07:01
    Heater. wrote: »
    I don't forget the teleprinters. They are hardware. I see no reason why the details of hardware should be encoded into my text files.

    We're not talking about files. We're talking about RS-232 control characters. CR means CARRIAGE Return, it moves the print head, it does not move the paper feed. There is a whole set of RS-232 control characters with their functions defined by standard. We're not going to adopt a certain shell of a certain operating system's proprietary behavior for the RS-232 standard. It's not DOS, Windows or Unix, it's RS-232.

    All I'm suggesting is that the Parallel Serial Terminal .exe tool add check boxes for "Implicit CR with LF" and "Implicit LF with CR" so it can behave properly for a proper serial stream (so that it doesn't linefeed twice when CRLF is received). Gobs and gobs of serial devices out in the wild terminate a serial line with CRLF and gobs and gobs of software expect CRLF as a 16-bit line end.

    This has got nothing to do with Unix or or a Propeller program. Also, vulgarities and namecalling don't help the discussion.

    Regards,
    Chris
  • Chris B.Chris B. Posts: 18
    edited 2013-10-11 07:14
    Martin_H wrote: »
    Do you want a real CR versus CRLF horror movie mess?

    This thread is not about your C++ code example. It's not about Linux or Windows or Mac or any other "platform". It's about a text terminal program that always double-spaces the CRLF sequence, despite the fact that very many RS-232 serial appliances send CRLF to terminate a line intending and expecting to cause only a single line space.

    I think CRLF is the de facto line-end standard for serial peripherals, and the proper behavior for sending a CR is to restart on the same line and not move down to the next line. Uncommonly one finds a device that only sends CR, necessitating going into terminal preferences to tick the box Add Line Feed for Carriage Return.

    Right now I'm receiving weather station data using a propeller and the weather station is sending CRLF. This is not relevant to Parallax Serial Terminal.exe, it is just makes the point that most serial devices behave this way, no matter the opinion of me or anybody else. When I write a serial device, I use CRLF because it's more common than not, and practically every terminal program defaults to not adding LF after CR.

    Web forums are littered with complaints about O/S specific issues surrounding new line, such as "I can't talk to my GPS with minicom thanks to this eternal religious war over newline." The problem with programming on a platform is that ever-clever O/S developers will mangle streams trying to 'help' the programmer not care about things she should care about and thus defeating the programmer's intent. Or that application developers will hardcode their software to their opinion of the "correct" way to do it, instead of being flexible and forgiving of it being done other ways. My take is that 'special' characters in printf() should be escaped all the way to the shell/environment and handled according to rules, and that if a developer wants to send a CRLF specifically, then \015\012 should not be escaped by the compiler but should always make it all the way though the shell and O/S and device driver gantlet onto the wire as two literal bytes as if they were typeable characters, without being mangled. But nobody listens to me.

    Back to the point of this thread I started: I'm simply proposing that Parallax add check boxes to the Windows application Parallax Serial Terminal.exe tool preferences for "Implicit CR with LF" and "Implicit LF with CR", so that a serial device sending CRLF as a line terminator can be correctly displayed. Parallax should take no stand on the newline war, just simply support everything, the way Putty does. It's not a big deal, I can just use Putty instead of the Parallax serial tool or deal with the double line. And I certainly didn't intend to start a flame war.

    Regards,
    Chris
  • Dave HeinDave Hein Posts: 6,347
    edited 2013-10-11 07:28
    Chris B. wrote: »
    I'm simply proposing that Parallax add check boxes to the Parallax Serial Terminal .exe tool preferences for "Implicit CR with LF" and "Implicit LF with CR", or maybe merely the latter, so that a serial device sending CRLF as a line terminator can be correctly displayed.
    I have made a similar proposal for the SimpleIDE terminal. Check boxes for LF and CR make a lot of sense to me. Currently, when I use the PST with C programs I disable the LF character. This is a workable solution, but a checkbox would be better than completely disabling the LF character. I think we would only need the checkbox for the CR character, which would enable/disable a linefeed along with the carriage return.
  • Heater.Heater. Posts: 21,230
    edited 2013-10-11 08:31
    Chris B.

    I do agree that the Parallax Serial Terminal should have options to deal with the interpretation of CR and LF.

    You are right, we are not talking about files. The issue of line endings in text in files or over a communication medium or driving terminal screens is very closely related though.

    We are NOT talking about RS232 standards either. RS232 the physical characteristics of a link, voltages, currents, timing, connectors pin outs etc. The RS232 standard does not define any character encodings.

    It is true that thus has nothing to do with Unix or any OS I was commenting on the history of how this mess came to be.

    I am not aware of any "vulgarities and name calling" I may have perpetrated. I certainly have not intended any. Please point them out and I will make appropriate apologies and delete the post if need be.
  • Jeff MartinJeff Martin Posts: 758
    edited 2013-10-11 11:28
    Chris B,

    "Parallax should take no stand on the newline war, just simply support everything..."

    We certainly don't want to take part in the war and would rather no war erupts in the first place. There are reasons that PST supports what it does that I won't get into here.

    Still, we understand that there are difficulties experienced with the way PST is being used and that there may be a solution like the one you proposed. I never was a big fan of making check boxes to disable (effectively ignore) the defined control characters via the preferences window because it meant that, while the user could solve their immediate problem by unchecking certain things, it would lead to confusing behavior when interfacing with other code that was supposed to use those control codes as was intended by PST.

    For that reason, if we can avoid it, I'd like us to solve this another way if possible.

    I don't know of any system in the last 20 years that outputs LF by itself to mean "move to next line, keeping the cursor in the same horizontal position." Also, I don't know of any that outputs CR by itself to mean "move to the first position of the current line." And if a user wants to build a custom system that has such capability, PST already supports moving the cursor in such ways using other control characters.

    In order to support everything, what if we:
    • Change PST's LF interpretation to be the same as its current CR interpretation,
    • Make it automatically ignore the first LF (if any) immediately following a CR,
    • Make it automatically ignore the first CR (if any) immediately following an LF

    Multiple CR,CR, or LF,LF, or CR,LF,CR,LF patterns will be treated as two or more "new lines," but CR,LF or LF,CR pairs would be treated as only a single new line.
  • Chris B.Chris B. Posts: 18
    edited 2013-10-11 14:00
    Hi Jeff,
    I don't know of any system in the last 20 years that outputs LF by itself to mean "move to next line, keeping the cursor in the same horizontal position."
    Well that's the common behavior for terminal software. And, incidentally, that's what Parallax Serial Terminal does, too. Jumper TX and RX, and hit CTRL-J. :-)
    Also, I don't know of any that outputs CR by itself to mean "move to the first position of the current line." And if a user wants to build a custom system that has such capability, PST already supports moving the cursor in such ways using other control characters.
    Somebody mentioned either in this thread or one of the previous it has come up that it is sometimes the case that a CR is used as a 'go back to the start of the same line' to overwrite. It's the default behavior on most terminal software.
    In order to support everything, what if we:
    • Change PST's LF interpretation to be the same as its current CR interpretation,
    • Make it automatically ignore the first LF (if any) immediately following a CR,
    • Make it automatically ignore the first CR (if any) immediately following an LF
    Multiple CR,CR, or LF,LF, or CR,LF,CR,LF patterns will be treated as two or more "new lines," but CR,LF or LF,CR pairs would be treated as only a single new line.

    I prefer positive control of an app by setting preferences vs magic behavior, but that's just my two cents.

    Radio button: (*) Enter button sends $0D$0A, ( ) Enter button sends $0D, ( ) Enter button sends $0A.
    Check box: [ ] Incoming $0D also moves cursor to next line (implicit LF)
    Check box: [ ] Incoming $0A also moves cursor to left margin (implicit CR)

    The defaults for these settings in preferences would mimic the current behavior, naturally.

    Best regards,
    Chris
Sign In or Register to comment.