Network connection to P2 Serial Stream - RealTerm

Tubular · 2020-12-29 01:32

During recent zoom meetings there was discussion about installing a TCP/IP socket to redirect the serial stream, so other display utilities can 'listen in' and perhaps display graphs or log data

RealTerm already has this option, under the 'echo port' display tab. I was able to successfully telnet in from a remote computer and interact with MicroPython REPL

There is also a raw (non telnet) mode that applications might prefer

hinv · 2020-12-29 18:43

Very nice!

octetta · 2020-12-30 03:12

Nice demo Tubular (also "live" during the P2 call!)

Let's try to zero-in on what's useful.

From my point-of-view two things have been discussed:

1. How can PropTool "play nice" with external tools.
2. How can we make the P2 "network aware".

RealTerm + "echo port" is a nice and simple answer to #2, although long-term, I hope we can put something like PPP or SLIP in the codebase, I like this b/c it doesn't prescribe any particular network hardware.

For #1, I'd suggested an additional DEBUG message to send a bunch of bytes to a UDP port, but after talking myself in a circle I'm open to ideas on how to best present this to Chip, particularly since I have NO IDEA what Delphi code looks like for network programming. I did find a Stack Overflow article that hints at how to do this (look at the answer): https://stackoverflow.com/questions/3931484/handling-tudpsocket

For those not familiar with network programming, maybe a simple Python program for server and client would be useful, since that might be more universal than Delphi for explaining things.

# server... run in one console window
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # make a UDP socket
addr = ("0.0.0.0", 56001) # 0.0.0.0 means listen on all network interfaces, 56001 was chosen randomly
s.bind(addr) # attach this socket to the aforementioned addr
while True:
  what, who = s.recvfrom(1024) # receive upto 1024 bytes of the next waiting UDP message
  print(what, who) # "what" contains the received message, "who" is the IP address and port of the sender
  s.sendto("ok dude", who) # send a contrived response, the below client ignores this message

# client... run in another console window
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # make a UDP socket
s.sendto("blah-blah", ("127.0.0.1", 56001)

Because of the way UDP works, the programs can be started in any order... if there's no server running, the client's UDP messages just silently disappear.

My idea is that a PropTool DEBUG UDP message does the equivalent of the client code above, so whoever happens to be listening can do custom work with the data. UDP sockets handle binary data just fine, but maybe we could just use ASCII between the ` and end-of-line like Chip's graphics do.

Michael or Tubular suggested that the DEBUG tool being able to receive responses would be useful, but since that introduces "protocols" (about what happens after a message) and timeouts, which from my perspective, makes thing more complicated than needed for the moment.

octetta · 2020-12-30 18:31

Big apologies to ESR (and DaveH?), since I totally forgot about his(their?) 9P filesystem implementation inside loadp2.

From the very nice docs he wrote (https://github.com/totalspectrum/loadp2#readme), he wraps that protocol like this:

File server messages from the device start with the two byte magic escape sequence 0xff, 0x01. After that the standard 9P protocol data follows, as described in the Plan 9 manual pages (see http://man.cat-v.org/plan_9/5/). All protocol messages start with a 4 byte message length, followed by the message payload.

Maybe a similar method could be adopted by other P2 tools?

While 9P is overtly a file-system protocol, it's designed to encapsulate structured communications (in the vein of open/read/write/close/+++). Maybe more layers than we need here, but good to think about nonetheless.

Hoping Eric and Dave might chime in here with their wisdom.

On the Python side, I believe there are abstract libraries for 9P access, but I've only used this for C and Lua code.

octetta · 2020-12-30 18:44

octetta wrote: »

...two byte magic escape sequence 0xff, 0x01... ...All protocol messages start with a 4 byte message length, followed by the message payload.

Without knowing 100%, it seems Chip's protocol starts with a tilde and ends with a LF? If so, I don't know if he wants to go from 2 bytes to 6 bytes of overhead to every message or not... hoping he has time to skim this conversation too.

Tubular · 2020-12-30 20:55

Like you said, I think the real "value" would be just concentrating on the one way flow of data out of the P2. UDP makes sense. Whether its binary or ascii data can be set by the P2 program.

You're right in that there's some overlap with Eric's Plan9. I would think Plan9 would be good for logging data to local (host PC) disk, whereas the UDP approach useful for storing data remotely across a network.

I've seen a couple of projects using Node-Red / InfluxDB / Grafana recently, hence the interest in the networked approach

octetta · 2020-12-31 20:07

Thanks for the feedback Tubular.

How about we propose to Chip that he add a DEBUG statement(?) that takes a UDP port number as its argument that PNut does a "sendto" localhost/127.0.0.1 with the data in the parameter following it. Something like:

DEBUG("`UDP 56001", ...)

where the "..." is one of the many output functions listed in Chip's documentation, and in this example would send to UDP port 56001. Then another program running on the same PC as PNut that had bound to ("0.0.0.0", 56001) and that was waiting on a "recvfrom" would receive the data sent.

(*someone on the call suggested that PNut might have a configurable output address to send the data to somewhere other than localhost...)

The best I can tell (and I alluded to this earlier), Delphi has some basic winsock capabilities built-in. I have absolutely no way to test this, but found an example via google-ing from (https://lists.freepascal.org/pipermail/fpc-pascal/2001-July/001699.html). Text of that message below:

Subject: [fpc-pascal]Send String via UDP


> I want to send a simple String via the UDP-Protocol to an unknown number
> of computers on our intranet.
>
> I've tried some things, but had no success in working with the
> Socket-Unit.
> Is there anyone who can give me an example on how do do it?

Here's a small example program. Since you didn't tell for what OS you want
your program, it is a Win32 one. For Unix / OS/2 / * you've to search the
RTL / packages / whatever for the units to be used for the different calls
and remove all WSA* stuff. Note that I didn't use the winsock unit supplied
with the RTL but the winsock 2 header translation from Alexander S. from the
contribs page since the RTL ones aren't fully compatible to the Delphi ones.
Though it might work with it too.

{$mode delphi}
uses
  winsock;

type
 Int32 = Longint;
 UInt32 = DWord;

const
  MYPORT = 9034;

var
  sock : TSocket;
  send_addr : TSockAddrIn;
  hostentry : PHostEnt;
  numbytes : Int32;
  hostname : String;
  sendstring : String;
  wsadata : TWSAData;

begin
  if (paramcount < 2) then begin
    Writeln('Usage : ' + paramstr(0) + ' hostname message');
    exit;
  end;
  // initialize winsock
  if (WSAStartup($0202, wsadata) <> 0) then begin
    Writeln('Couldn''t initialize Winsock 2');
    exit;
  end;
  // resolve host
  hostname := paramstr(1);
  hostentry := getHostByName(PChar(hostname));
  if (hostentry = nil) then begin
    Writeln('Couldn''t resolve host');
    exit;
  end;
  // get socket
  sock := socket(AF_INET, SOCK_DGRAM, 0);
  if (sock = INVALID_SOCKET) then begin
    Writeln('Couldn''t retrieve socket');
    exit;
  end;
  // fill in target address
  with send_addr do begin
    sin_family := AF_INET;
    sin_port := htons(MYPORT);
    // neat typecast from PChar to UInt32 :-)
    sin_addr.s_addr := UInt32((@hostentry^.h_addr^[0])^);
  end;
  // send data
  sendstring := paramstr(2);
  numbytes := sendTo(sock, sendstring, length(sendstring), 0, send_addr,
sizeof(send_addr));
  if (numbytes = SOCKET_ERROR) then begin
    Writeln('Error sending string');
  end else begin
    Writeln('Sent ', numbytes, ' bytes to ', inet_ntoa(send_addr.sin_addr),
' successfully');
  end;
  // clean up
  closesocket(sock);
  // shut down winsock
  WSACleanup;
end.

>
> How can I reach all computers (of course with a running
> Client-Application) on the net?
> It is said I have to use 255.255.255.255 as the receiver-adress?!

With this address you reach all computers on the 'net regardless of them
having installed your client application or not afair. Guess you won't make
much friends when doing that =I

> Any hints?

Hope this helps.

Regards,
  Thomas

A small warning is this example seems to be a UDP multi-cast which isn't what I had in mind and I guess if the "paramstr(0)" is set to "127.0.0.1" that it would do what I intend.

Notice this is on the FreePascal mailing list, so I don't know what other problems this might have with real Delphi.

Happy New Year!

-joe

octetta · 2020-12-31 21:37

I'm learning about PPP after many years of not having a dial-up modem line.

The reasons are that I think it'd be really cool if a P2 could use a simple UART connection to send/receive data to a connected device.

This will be the first of several times I share notes as I get back into this with the hopes someone else can benefit or participate.

My setup:

MacBook pro with a FTDI USB-to-UART adapter (A).

Raspberry Pi 3 with a FTDI USB-to-UART adapter (B).

Ground of A and B connected, Rx/Tx connected "crossed over" between the two.

1. Open a console port on the RPi
2. Install ppp via "sudo apt-get install ppp"
3. Learn the device name of the FTDI (usually /dev/ttyUSB0 unless you have more than one connected.
4. Start ppp via

sudo pppd nodetach debug maxfail 0 dump xonxoff nocrtscts local lcp-echo-failure 1 lcp-echo-interval 5 192.168.1.3: /dev/ttyUSB0 115200

5. Open Terminal.app on MacBook
6. Learn the device name of the FTDI via "ls /dev/tty* | grep usb". Mine showed up as "/dev/tty.usbserial-A904ML0O"
7. Start ppp via

sudo pppd -detach debug maxfail 0 dump xonxoff nocrtscts local lcp-echo-failure 1 lcp-echo-interval 5 192.168.1.2: /dev/tty.usbserial-A904ML0O 115200

If this all works, you'll have a ppp interface on both machines, visible via ifconfig on both machines.

RPi

$ ifconfig | grep ppp
ppp0: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST>  mtu 1500
        ppp  txqueuelen 3  (Point-to-Point Protocol)

MacBook

$ ifconfig | grep ppp
ppp0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1500

The RPi is assigned IP address 192.168.1.3 and the MacBook is assigned 192.168.1.2 (set via the respective invocations of pppd).

At this point you can ping either side from the other.

Next experiments will be on how to create a minimal ppp implementation in either spin2 or flexc.

Tubular · 2020-12-31 21:55

NIce experiment. That could be quite potent if you can get it onto P2.

Regarding getting Parallax interested in adding UDP to PST or PropTool, they're probably pretty stretched right now. But a proof of concept as well as demonstrated application never hurts

Network connection to P2 Serial Stream - RealTerm

Comments