What do you want in a Serial Driver?
Cluso99
Posts: 18,069
There seems little interest in the serial driver I posted a couple of days ago
forums.parallax.com/discussion/171906/clusos-serial-driver-pasm-standalone-cog-incorporates-most-of-rom-monitor-debugger#latest
My serial driver is derived from my P2 ROM Serial/Monitor/Debugger code.
forums.parallax.com/discussion/170638/p2-rom-monitor-debugger-rev-1-rev-2-silicon
I will be providing simple spin2 calls to my serial driver (pasm in its' own cog) shortly, provided there is interest.
So, do you want a standard serial driver?
If yes, what features (ie calls/routines) do you want?
What languages do you want to call it from?
What interface (eg hub mailbox) do you want?
Do you want the serial driver to reside in its' own pasm cog?
If not, then what features, cog/lut or hubexec?
Here are the calls for the P1 FullDuplexSerial (not it used its' own cog for the pasm, and could only be called from spin)...
Here are the calls supported (or planned) in my serial driver...
Can be called from pasm, spin, or any other language, via a simple hub mailbox (any language that provides hub read/write access).
I find the biggest and easiest features of my serial are the TxList and Monitor features when debugging.
A simple call to TxList can display/dump line(s) of memory in both hex and ascii
So, what do you want, if anything???
forums.parallax.com/discussion/171906/clusos-serial-driver-pasm-standalone-cog-incorporates-most-of-rom-monitor-debugger#latest
My serial driver is derived from my P2 ROM Serial/Monitor/Debugger code.
forums.parallax.com/discussion/170638/p2-rom-monitor-debugger-rev-1-rev-2-silicon
I will be providing simple spin2 calls to my serial driver (pasm in its' own cog) shortly, provided there is interest.
So, do you want a standard serial driver?
If yes, what features (ie calls/routines) do you want?
What languages do you want to call it from?
What interface (eg hub mailbox) do you want?
Do you want the serial driver to reside in its' own pasm cog?
If not, then what features, cog/lut or hubexec?
Here are the calls for the P1 FullDuplexSerial (not it used its' own cog for the pasm, and could only be called from spin)...
PUB Start(rxPin, txPin, mode, baudrate) : okay PUB Stop PUB RxFlush PUB RxCheck : rxByte PUB RxTime(ms) : rxByte | t PUB Rx : rxByte PUB Tx(txByte) PUB Str(stringPtr) PUB Dec(value) | i, x PUB Hex(value, digits) PUB Bin(value, digits)
Here are the calls supported (or planned) in my serial driver...
Init initialise: (rxpin, txpin, bitper, rxbuf_address) TxChar send 1-4 characters, options to reverse byte order TxStr send $0 terminated string TxHex send long in hex, options to reverse byte order, add space in pairs, 1-8 digits TxList send/dump line(s) of hex & ascii from hub (cog/lut not relevant), option to display longs, give address range TxBin send binary, 1-32 digits TxDec send decimal, signed/unsigned RxChar (not tested) receive a char, options to send prompt 1-4 chars first, echo, strip <lf>, wait or return $1FF in not-avail RxStr (not tested) receive string, options to send prompt 1-4 chars first, echo, strip <lf>, supply new hub buffer address. Can edit with <bs> Monitor call Monitor/Debugger (as in P2 ROM). Returns with Q<cr>Note: RxChar serves both Rx and RxCheck functions of FullDuplexSerial. All other functions are available.
Can be called from pasm, spin, or any other language, via a simple hub mailbox (any language that provides hub read/write access).
I find the biggest and easiest features of my serial are the TxList and Monitor features when debugging.
A simple call to TxList can display/dump line(s) of memory in both hex and ascii
01AE0: 70 05 00 00 EC 7C BB B9 0A E0 A4 2D 8A 3A FA A1 'p....|.....-.:..' 01AF0: 07 95 34 04 09 3E 02 0E 53 CA 27 FC F4 D2 72 FE '..4..>..S.'...r.'And a simple call to the Monitor enables interactive display and modification of memory, and when done can return to your program.
So, what do you want, if anything???
Comments
The other thing is with serial transmit is that I prefer to simply send it or wait to send it, but use a high baud rate so that it shouldn't have to wait either. The trouble with buffering output is that you can end up filling the buffer and waiting anyway, plus there is a lot more overhead in buffering and transmitting under interrupts.
Anyway, it seems that obex drivers could do with a standard API so that one comprehensive and standard set of formatting routines are totally reusable, as are any new formatting routines.
P.S. Features wise you could include break detection with optional automatic user specified action, be it reset or abort etc, but from the driver itself. I also have character sequence detection where up to 4 characters are matched to a 32-bit constant, so you could make this another driver level feature that works whether or not the user code is working properly.
Normally I would expect that a different driver would be loaded (with identical routines) to support VGA, HDMI or whatever, plus some form of input such as a keyboard. Since a standard hub mailbox is used, the driver can be switched on the fly, without the user program even knowing.
Agreed, and that is precisely what I do. Receive is a different matter tho'. Here I am up for discussion. I think Rx and variants is different to RxStr (rx string) where a buffer is used and it can be edited in place using <bs>, just waiting for a <cr> before being returned (with $0 appended) to the calling program.
That was a big annoyance to me!
Not only that, but even the basic routine to send/display a character had different names (eg tx, out, send, etc)
However, the problem is what do you write it in if you do not include it in the actual driver?
My routines were originally written in pasm for lmm hub mode, then hubexec for the P2 ROM, and now they also work in cog (pasm).
IMHO having it embedded in the driver makes the most sense as long as the calls are the same between drivers.
Of course the original intent was to have these routines resident in P2 HUB as loaded by the ROM. But now we cannot use that method anymore.
In TAQOZ the VGA word is used to redirect output to a VGA output routine which also processes control controls and sequences etc, just as if it were a device on the remote end. So, no special routines for the application, just talk to the "device" much like we do in Linux.
A "page" ($0C) command clears the screen on the VGA the same way it does on the serial terminal. CR moves to the left column but does not advance to the next line since that's what LF does etc.
* P1 required spin whereas P2 does not. Therefore the spin interpreter may not exist and so the code needs to be in the language of either the user (in the users code space) or in the driver (in the drivers language and code space)
* If it resides outside the driver, it will need to be ported to all languages
* If it resides within the driver, then while it needs to be duplicated for each driver (the code can be a common include if it is the same language), residing within the driver has the advantage that it can be executed within the drivers' time-frame and typically within the cog/lut and therefore using less hub space.
I've seen first-hand the issue of making these routines work from hubexec (for users space). You need to minimise the resources used - PA/PB/PTRA/PTRB, internal call stack, cog register space, all at the expense of code (hub) space. BTW these pasm hubexec routines can be called from any language, provided you can give up 16 cog registers.
When used in a driver, the space and registers can be shared to be efficient for the driver concerned, and they don't impact on the users space or language.
Jon, your thoughts on this please? (please don't take this the wrong way because I value your thoughts)
We really need to try and get this right before we have too many fragmented drivers/routines.
Oh, and I missed the python way. Unfortunately I see lots of HL languages make all sorts of ways to format strings with data. I guess the reason we haven't seen a standard way is because none of them seem to be easy to use.
TAQOZ can do all sorts of things in minimal code.
But don't we need the 64KB code base to get to it?
And we can't call it from any program?
So, how can we leverage TAQOZ without needing to understand it first?
BTW, the TAQOZ footprint is actually way smaller and even fully expanded doesn't use up 64k.
I prefer them to be in the driver because this doesn't use resources within the calling program, and it can be run in parallel (ie without waiting) if so desired. BUT I absolutely want the call to be the same, no matter what the input/output form (eg serial, vga etc).
I would expect most standard drivers will use pasm, so it would make most sense for these routines to be in pasm so that they can be "lifted" into other drivers. So I guess what is required is that these general routines (Jon has them in jm_nstr.spin) should be available as a pasm object that can be "included" in different drivers.
It would be great to have a handy type of IO driver/object API that can be used from PASM/SPIN2, or other languages that can input/output formatted data and which is re-directable to different devices such as serial, video, network etc if low level character I/O is available from those device drivers.
It defintely seems wasteful for every driver to have to provide the same formatting code if something common could do it and the drivers just output characters.
You'd want this to be multi-COG capable: maybe one COG is printing debug info to serial while another is writing characters to a video console. Of course sharing the same output may become problematic unless it is line based, or commands are allowed to block while the resource is busy etc.
You could probably still use hub exec for a character formatting IO driver, and callable directly even from PASM2 if you can preserve or not need any special regs like PTRA, PTRB, etc and run with some pre-allocated hub stack area etc.
It would be nice to somehow tap into the existing SPIN2 SEND capability too as this has variable length argument capabilities which is convenient for debug printing and is also re-directable in its own right. Someone clever mentioned using parameters values > 255 for pointers to strings or other data, and <256 for individual characters which is a good idea. This may allow mixing format strings and other data if some format string is always expected as the first argument for example, like a printf().
For some generic input type functionality it would either need to be a able to read a single character, word, long etc, or do string reading until end of line or a given number of bytes if a buffer is provided. Probably needs to have blocking and/or timeout options.
For something like reading from a USB keyboard, supporting input may need an additional helper COG to continually poll the source of data or have an interface where its data can be collected/buffered in the IO driver. Serial receivers also may need to have something polling them regularly or be interrupt driven in a COG to avoid overflow, though some of that polling overhead could be pushed to the client side.
IMHO it's not wasteful if it resides within the driver in cog/lut tho. And this way it can be buffered allowing the user program to continue. Just load up whatever driver you need, which also can be done on the fly too since the newly loaded routine would use the same mailbox and commands.
This is done in my ROM serial/debugger/monitor. It is hubexec and only requires 16 longs for registers of which 4-5 need to be preserved while the others are temporary scratch registers. The internal stack is used. No special registers are used. A hub buffer is required for rxstring only.
I have a version that can be relocated to any hub address and any group of 16 cog registers.
RxString in the ROM serial does this (character and string). String includes editing with <bs> and terminates with <cr> and appends $0 for $0 terminated string. No blocking/timeout option but could be easily implemented in the soft version.
I see this as all being in its' own cog.
In P1 we had a smart UART with FullDuplexSerial. But it was only callable from spin, and had a spin interface where the formatting was done in spin (slow) and in the calling cogs' code space and timeframe.
In P2 we can have a super-smart UART which can have format processing within the same driver cog.
One interesting point of my serial/debugger is that it has its' own monitor. You can call this from your user program and this then gives you interactive memory display and modification. It can also actually locate and read a FAT32 file from SD and even run it. I have this in the current serial driver too, although for now I disabled it to save needing lut code space. The monitor also has downloading capability too.
So, the serial driver could have a key sequence (ie from the keyboard/PC) which could drop into the monitor without the user program even knowing!
Always pleased to get your input. Even you're doing your own thing is valuable.
So may I ask why?
What will your serial do?
Will it encompass what I am doing?
Maybe I don't need to continue and do something else?
https://github.com/parallaxinc/propeller/tree/master/libraries/community/p2/All/jm_serial
My 2 cent
If you want a serial driver that can set a standard and be used by anyone and everywhere using its dedicate cog than, in my opinion, it should have:
- at least 2 (if not 4) serial ports (otherwise for one serial a cog is a waste)
For each serial port:
- it must be configurable in terms of used pins
- every serial port should have its own baudrate (if not too complicated perhaps it could be differentiated also between rx and tx)
- it must support 7,8,9 data bits (if possible also 5) and parity control
- when in 9 bit mode it should support internally at least 2 addresses comparison (one for itself, one for broadcasts): if data is not addressed to one of this two it can be discarded.
- it must have optional auto rts/cts hw flow control configurable using additional io pins of course
- it must support half-duplex mode and have optionally TxEn io pin to drive also RS485/422 buses
- it can recognize a few user-configurable bytes eg (STX, ETX)
- it should have two 64 byte FIFO for RX and TX
- as a bonus, when using 9bit data in RS232, and P2 pins are not a concern it could have double rx/tx pins eg (RX, RX', TX, TX'): when using rs232 daisy chain communication you need 2 rs232 connectors (in and out) on the same serial controller. You can have external AND/OR ports to wire them, but having the internal (P2) AND/OR function can save on BOM (and whatever comes with it: design, soldering, ...).
So 4 configurable pins for every serial channel:
- standard RX
- standard TX
- optional CTS/RX'
- optional RTS/TX'/EN
And I agree with others that formatting functions should be left outside. Perhaps a hub-exec block of pasm on which Spin2/TAQOZ/Other wrappers can be made available to other cogs and other languages.
The mailbox communication is also ok for me. .... and, at the end, also formatting routines can be hosted internally in the driver (if it is cog/lut it not hurts) but not at expense of the above requirements.
I also agree that formatting routines should be independent. The core serial object should implement tx, rx, and maybe the equivalents of Unix read/write (for efficiency). Everything else can be done in a formatting object.
Which brings up the question of how the formatting object should work, namely how one specifies the tx routine to use. Just passing it as a method pointer seems reasonable. SEND would be another option, but that's harder to get working in other languages (like MicroPython).
Eric has #include but unfortunately it's not in pnut or PropTool. There is the file command but that only imports a binary (ie not source code)
I have thought about breaking out the formatting (pasm) code into a module that could be included in various drives. It would be easy to do except for the missing #include in pnut. IMHO it really needs to be in the driver as that saves the code from needing to be available in various languages to be included into the users code.
Some languages have their own formatting methods, and that's fine for them too. They can still use the xxx driver without utilising the drivers' formatting code.
No one else seems interested anyway
Basically this is smarter then #include since it does not include the routines into the source of every caller including it, it calls one object from multiple calling objects using it.
You need to put all variables of the subobject you relay on into the VAR section, so every caller has its own set.
So in some form Spin1 and Spin2 have a include function.
Enjoy!
Mike
I guess what I am thinking about for SPIN2 is probably a type of common wrapper layer which I called "iodrv" below that knows how to format and can be used to redirect. Method names used below are just for example only and not a proposal. Maybe for inputting interactive data as a string it could include some optional capability that also allows simple backspace/del editing and/or can echo chars. This sort of common input/output stuff has to be repeated in everyone's code otherwise and gets tedious.
VAR long buf[100] OBJ io : "iodrv" ' common io redirection object vid : "p2videodrv" ps2 : "ps2drv" ' e.g. ps/2 kbd driver uart : "smartserialdrv" PUB demo() | x, y io.start(@vid.tx, @ps2.key) ' initializes input/output default path to use video console and PS/2 keyboard io.hex($1234, 4) ' hex output 4 digits io.dec(123) ' output unsigned dec io.bin(x, 32) ' binary output 32 bits io.tx("A") ' output single char io.print(string("Hello world")) ' output a null terminated string io.printf2(string("%d %8x"), x, y) ' output two formatted arguments (would need a bunch of these APIs) io.write(@buf, 100) ' outputs 100 bytes from a buffer io.setOutput(@uart.tx) ' now redirects output used by for this COG to a serial port io.print(string("Sending to UART")) io.rx() ' read a single char/key/value io.input(@databuf, 32, 13) ' reads a string to a buffer, 32 = size limit to read, 13=terminating char is CR io.read(@databuf, 100) ' raw read 100 bytes io.poll() ' check if data is ready? io.ctl(x, y) ' set some driver specific parameter x to value y (like an IOCTL)PUB start(addr) : okay ' start the serial driver in its' own cog stop() mailbox := addr okay := cog := coginit(16, @ser_command, ptr_mailbox) + 1 PUB stop() if cog cogstop(cog-1) PUB init(rxpin, txpin, @ser_buffer, _bitper) ' initialise the serial driver call_serial((equ._CMD0+equ._INIT), _bitper, ser_buffer, (rx_pin<<8 + tx_pin)) PUB txchar(char) ' send a character(s) call_serial((equ._XMIT), char, 0, 0) PUB txstr(@str) ' send a $0 terminated string call_serial((equ._XMIT+equ._STR), str, 0, 0) PUB txcr() ' send <cr><lf> call_serial((equ._XMIT+equ._CRLF), 0, 0, 0) PUB txdec(val) ' decimal unsigned call_serial((equ._DEC), val, 0, 0) PUB txdecs(val) ' decimal signed call_serial((equ._DEC+equ._SIGN), val, 0, 0) PUB txhex(val, digits) ' hex if digits > 7 call_serial((equ._HEX+0), val, 0, 0) else call_serial((equ._HEX+digits), val, 0, 0) PUB txbin(val, digits) ' binary if digits > 31 call_serial((equ._BIN+0), val, 0, 0) else call_serial((equ._BIN+digits), val, 0, 0) PUB txlist(addr1, addr2) ' list/dump memory (address range) if addr2 == 0 call_serial((equ._LIST), 0, addr1, 0) else call_serial((equ._LISTequ._ADDR2), 0, addr1, addr2) PUB rxchar() ' receive char (wait) call_serial((equ._RECV), 0, 0, 0) repeat while LONG[mailbox[0]] <> 0 ' wait for serial mailbox empty char := LONG[mailbox[1]] ' get recv char PUB rxcheck() : char ' receive char (return -1 if none avail) call_serial((equ._RECV+equ._CHECK), 0, 0, 0) repeat while LONG[mailbox[0]] <> 0 ' wait for serial mailbox empty char := LONG[mailbox[1]] ' get recv char if char == $1FF char := -1 'PUB rxflush() ' flush any chars in the buffer PUB rxstr(@buf) ' receive string, <cr> terminates with <cr>+$0, editing with <bs> call_serial((equ._RECV+equ._STR+equ._ADDR), 0, buf, 0) PUB monitor() ' call the monitor call_serial((equ._CMD0+equ._MONITOR), 0, 0, 0) ' some extended methods require an additional parameters - these calls can use the following call directly. PUB call_serial(cmd, val, addr1, addr2) ' generalised serial call repeat while LONG[mailbox[0]] <> 0 ' wait for serial mailbox empty LONG[mailbox[1]] := val LONG[mailbox[2]] := addr1 LONG[mailbox[3]] := addr2 LONG[mailbox[0]] := cmd ' do this lastIn my above post, I used "ser" as the object name, but it could be anything. It is actually up to the driver cog to take the data from the mailbox, and deal with it - ie I/O to serial, VGA/Keyboard, etc.
To change the I/O would be a simple matter of starting a "the driver" cog with a new driver using the very same mailbox. This can be done on the fly. With a small VGA/composite driver, it could be in the same cog as the serial driver. Thus, it would be an extended mailbox command that would switch between different inputs and outputs.
The main point of this is that the compiled user program does not need to be recompiled for an alternate input and output driver. It's the basis of an OS which is almost what I had with P1.
To change the I/O from Serial to TV/Kbd (or something else) I just change a single OBJ line and re-compile one program (the OS base program - it’s like a bios program). There is no call change to switch the I/o driver. I never got around to permitting a driver change on the fly even tho it’s a simple matter. All I/O goes via a fixed hub mailbox. This is why the OS commands do not require re-compilation.
So, it’s a tried and proven solution.
Fair.
Maybe Spin2 should get #include on top of 'file' in the new Version. There seems already a seed in Chips mind that #ifdef and Co could be implemented with some preprocessor.
Fastspin already uses a standard include for formatting, but that might not work in Spin2 since it uses variable count of parameters for some printf routine.
I really like eric's extension for variable count of parameter with defined standard values very much. That is missing in Spin2, but basically the send thing is doing it in Spin2, so pestetering @cgracey to extend his Spin2 to allow this for all methods too would be very useful.
My FullduplexSerial2 driver adapted them to provide 2 ports with buffers in lut. just needs 12 longs mailbox in HUB
Mike
1) dead code elimination from image, saves binary image memory by not including the methods that are not referenced - this would be ideal for building up a good set of library routines, e.g. graphics, that could bloat the code otherwise.
2) #include for source (maybe #ifdef), though perhaps this is something a cpp front end could possibly do too.
3) variable number of arguments, done in Fastspin with the default values, and is quite handy for those printf type functions. We do seem to have some form of it with SEND already so perhaps it might be possible.