P2 Multiport x16 Serial Driver working (based on JonnyMac's FullDuplexSerial)
Cluso99
Posts: 18,069
Rather than take over @DiverBob's thread I thought I should start a new one.
forums.parallax.com/discussion/172485/creating-a-multi-port-serial-object#latest
Thanks to Ken, there is now a Quick Bytes here
https://www.parallax.com/multiple-serial-port-16-object/
Update 29-Jan-2021: V0.40 code posted with working demos for 1, 2 & 8 fullduplexports (2, 4 and 16 ports/pins). Tested with flexspin and pnut.
https://forums.parallax.com/discussion/comment/1515662/#Comment_1515662
Update 27-Jan-2021: V0.31 code posted with working demos for 1, 2 & 8 fullduplexports (2, 4 and 16 ports/pins). Tested with flexspin and pnut.
http://forums.parallax.com/discussion/comment/1515415/#Comment_1515415
Update 26-Jan-2021: V0.30 code posted showing two full duplex ports working.
http://forums.parallax.com/discussion/comment/1515306/#Comment_1515306
Update 22Jan2012: posted working code below. This will support up to 16 total uni-directional ports (ie up to 16 pins that may be any mix of transmit or receive).
Just for fun, here is a 64-Port version. For serious use I would need to tweek the driver.
https://forums.parallax.com/discussion/172821/p2-multiport-x64-serial-driver-working-just-for-fun/p1
Please note the following implementation detail has changed in the final versions.
JonnyMac made an amazing object (need a link) that includes formatting strings with binary/decimal/hex/etc. I've added a few personal bits like dumping hub memory with the usual 16 bytes per line in hex plus ASCII. I've starting breaking down the PASM driver into a separate object so that it can be loaded separately from the spin code. The reason for this is to be able to make the PASM driver able to stay resident when loading other objects, as is required when making a P2 OS (operating system). There are other benefits too, such as being able to use the spin code to format strings, etc, while changing the PASM driver underneath to say VGA and Keyboard, and still retain the hub buffer interface (ie hardware virtualisation).
The first part is to define the hub interface. This works well in Jon's code but I would like to take the time to expand on a couple of things before adding the multiple buffer sets required for each port. Here is the current interface for a single port...
The real part of the interface is this section...
Since we have a reasonable 512KB, providing a few extra longs to convey some additional info would be nice. What might these be?
* tx and rx pin
* baud
I would also like to add in
* rxsize
* txsize
This would make the buffer sizes flexible tho we have to account for this.
The padded string(s) would be defined in the spin section so it would not be part of the buffer interface (or mailbox). And the driver cog number is probably not required here.
Question(s)...
* Would a word (16 bits) be good enough for the head, tail and size? This is 64KB after all !!!
Why? Well it depends on whether we want to pack the heads/tails/sizes for all ports into one contiguous group and the buffers separate to these. There are some advantages to using PTR[offset] for getting these values.
So here are two alternatives for packing the buffers and pointers (I've just used longs for now)...
Repeat for each port
or
If you want to be able to start and stop ports at will, then the interface will likely need an extra interface for port number, txpin, rxpin, baud, txbuf and rxbuf addresses and txsize and rxsize for sizes, and a flag indicating if it is running. txpin, rxpin and flag can be bytes within a long, baud is a long, and the txbuf, rxbuf need to be a hub address so 20 bits minimum so a long each.
Ideas???
forums.parallax.com/discussion/172485/creating-a-multi-port-serial-object#latest
Thanks to Ken, there is now a Quick Bytes here
https://www.parallax.com/multiple-serial-port-16-object/
Update 29-Jan-2021: V0.40 code posted with working demos for 1, 2 & 8 fullduplexports (2, 4 and 16 ports/pins). Tested with flexspin and pnut.
https://forums.parallax.com/discussion/comment/1515662/#Comment_1515662
Update 27-Jan-2021: V0.31 code posted with working demos for 1, 2 & 8 fullduplexports (2, 4 and 16 ports/pins). Tested with flexspin and pnut.
http://forums.parallax.com/discussion/comment/1515415/#Comment_1515415
Update 26-Jan-2021: V0.30 code posted showing two full duplex ports working.
http://forums.parallax.com/discussion/comment/1515306/#Comment_1515306
Update 22Jan2012: posted working code below. This will support up to 16 total uni-directional ports (ie up to 16 pins that may be any mix of transmit or receive).
Just for fun, here is a 64-Port version. For serious use I would need to tweek the driver.
https://forums.parallax.com/discussion/172821/p2-multiport-x64-serial-driver-working-just-for-fun/p1
Please note the following implementation detail has changed in the final versions.
JonnyMac made an amazing object (need a link) that includes formatting strings with binary/decimal/hex/etc. I've added a few personal bits like dumping hub memory with the usual 16 bytes per line in hex plus ASCII. I've starting breaking down the PASM driver into a separate object so that it can be loaded separately from the spin code. The reason for this is to be able to make the PASM driver able to stay resident when loading other objects, as is required when making a P2 OS (operating system). There are other benefits too, such as being able to use the spin code to format strings, etc, while changing the PASM driver underneath to say VGA and Keyboard, and still retain the hub buffer interface (ie hardware virtualisation).
The first part is to define the hub interface. This works well in Jon's code but I would like to take the time to expand on a couple of things before adding the multiple buffer sets required for each port. Here is the current interface for a single port...
var long cog ' cog flag/id long rxp ' rx smart pin long txp ' tx smart pin long rxhub ' hub address of rxbuf long txhub ' hub address of txbuf long rxhead ' rx head index long rxtail ' rx tail index long txhead ' tx head index long txtail ' tx tail index long txdelay ' ticks to transmit one byte byte rxbuf[BUF_SIZE] ' buffers byte txbuf[BUF_SIZE] byte pbuf[80] ' padded strings
The real part of the interface is this section...
long rxhead ' rx head index long rxtail ' rx tail index long txhead ' tx head index long txtail ' tx tail index
Since we have a reasonable 512KB, providing a few extra longs to convey some additional info would be nice. What might these be?
* tx and rx pin
* baud
I would also like to add in
* rxsize
* txsize
This would make the buffer sizes flexible tho we have to account for this.
The padded string(s) would be defined in the spin section so it would not be part of the buffer interface (or mailbox). And the driver cog number is probably not required here.
Question(s)...
* Would a word (16 bits) be good enough for the head, tail and size? This is 64KB after all !!!
Why? Well it depends on whether we want to pack the heads/tails/sizes for all ports into one contiguous group and the buffers separate to these. There are some advantages to using PTR[offset] for getting these values.
So here are two alternatives for packing the buffers and pointers (I've just used longs for now)...
Repeat for each port
long rxhead ' rx head index long rxtail ' rx tail index long txhead ' tx head index long txtail ' tx tail index long rxsize ' rx buffer size long txsize ' tx buffer size byte rxbuf[BUF_SIZE] ' buffers byte txbuf[BUF_SIZE]
or
long rxhead1 ' rx head index long rxtail1 ' rx tail index long txhead1 ' tx head index long txtail1 ' tx tail index long rxsize1 ' rx buffer size long txsize1 ' tx buffer size long rxhead2 ' rx head index long rxtail2 ' rx tail index long txhead2 ' tx head index long txtail2 ' tx tail index long rxsize2 ' rx buffer size long txsize2 ' tx buffer size long rxhead3 ' rx head index long rxtail3 ' rx tail index long txhead3 ' tx head index long txtail3 ' tx tail index long rxsize3 ' rx buffer size long txsize3 ' tx buffer size long rxhead4 ' rx head index long rxtail4 ' rx tail index long txhead4 ' tx head index long txtail4 ' tx tail index long rxsize4 ' rx buffer size long txsize4 ' tx buffer size byte rxbuf1[BUF_SIZE1] ' buffers byte txbuf1[BUF_SIZE1] byte rxbuf2[BUF_SIZE2] ' buffers byte txbuf2[BUF_SIZE2] byte rxbuf3[BUF_SIZE3] ' buffers byte txbuf3[BUF_SIZE3] byte rxbuf4[BUF_SIZE4] ' buffers byte txbuf4[BUF_SIZE4]
If you want to be able to start and stop ports at will, then the interface will likely need an extra interface for port number, txpin, rxpin, baud, txbuf and rxbuf addresses and txsize and rxsize for sizes, and a flag indicating if it is running. txpin, rxpin and flag can be bytes within a long, baud is a long, and the txbuf, rxbuf need to be a hub address so 20 bits minimum so a long each.
Ideas???
Comments
The UART manager cog could check a mailbox that would have a long broken into a word and two bytes
word[1] is the hub address of the port structure
byte[1] is add or remove
byte[0] is the port (0 to 7) to add or remove
If there are not port changes a value of 0 in the mailbox would indicate there is nothing to do but check on the buffers.
If a buffer is added, a bit flag is added to available ports and the cog reads the configuration data it needs (mostly pointers) from the hub. If a port is removed the bit is cleared so that those pins are no longer checked on scans.
The foreground Spin can configure the smart pins for serial to take that burden off of the cog -- this keeps things simple (I think).
Thanks for commenting so quick Jon.
I think we would need a table of bits for the currently in-use ports so a byte for 8 ports. I do this for mapping used stay-resident cogs in my P1 OS and it woks nicely.
I think the hub address of the port structure should be at least 20 bits but we could combine the port number and add/remove bit within the long, so your mailbox would work.
BUT, the problem comes with multiple cogs trying to get a port concurrently. We would need to use a lock or we have to have separate longs for each port and continuously read them all by the driver cog.
Now if we used 2 longs (ie 8 bytes) as just a flag for each port, the driver could read these 2 longs using a setq #2-1 & rdlong and compare with zero, OR the driver could read each long using the wz flag. Neither of these would take too long to interfere with the rx/tx scanning. A further set of 8 longs would follow with one dedicated to each port. This would be quicker than using locks.
Smartpins may as well be set/cleared by the driver cog as there is plenty of cog space and it's only a couple of instructions. Might just need to be interleaved within the port scanning. Thinking more, yes, could be better for the calling cog to set the smartpins as the bitper needs to be calculated from baud and clkfreq .
alignl byte port0 ' 0 = nothing to do, %01 = enable portn , %11 = disable portn (p_portn is hub pointer) byte port1 byte port2 byte port3 byte port4 byte port5 byte port6 byte port7 long p_port0 ' hubaddr-of-port-structure (?? 1<<31 if port active) long p_port1 long p_port2 long p_port3 long p_port4 long p_port5 long p_port6 long p_port7Do you see that the data buffers for each port should be in a contiguous area in hub? Or do you think separate areas for each port would be better?
IMHO it would be nice to have separate hub areas but it's more complex and might be even more confusing to newcomers.
' -------------------------------------------------------------------------------------------------- ' Main hub interface (allows up to 8 ports) ports_command byte 0[8] '\ %00 = nothing to do, %01 = enable port[n], %11 = disable port[n] long 0[8] '/ hub pointer to port[n]'s config & buffers ' Port[n] configuration and buffers (up to 8 ports, one set for each port) ports_config word 0 '\ rxhead word 0 '| rxtail word 0 '| rxsize word 0 '| rxpin word 0 '| txhead word 0 '| txtail word 0 '| txsize word 0 '| txpin byte 0[rxbufsiz] '| rx buffer byte 0[txbufsiz] '/ tx buffer ' --------------------------------------------------------------------------------------------------And if they are contiguous then there is no need for the hub pointers for each port.
And INCMOD allows sizes to not be a binary multiple.
Yes, I stole it from you. It just needs to be a variable
If rx and tx buffers are all contiguous...
' -------------------------------------------------------------------------------------------------- ' Main hub interface (allows up to 8 ports) port_control byte 0[8] '\ control (8 ports) %00 = nothing to do, %01 = enable port, %11 = disable port word 0[8] '| rxpin << 8 | txpin (8 ports) $FF = no pin/port long 0[8] '| rxhead << 16 | rxtail (8 ports) long 0[8] '| txhead << 16 | txtail (8 ports) long 0[8] '| rxsize << 16 | txsize (8 ports) byte 0[rxbufsiz0] '| rx buffer port[0] byte 0[txbufsiz0] '| tx buffer port[0] '| ..... byte 0[rxbufsiz7] '| rx buffer port[7] byte 0[txbufsiz7] '/ tx buffer port[7] ' --------------------------------------------------------------------------------------------------Or if we allowed tx/rx buffer pairs to be anywhere in hub, then this...
' -------------------------------------------------------------------------------------------------- ' Main hub interface (allows up to 8 ports) port_control byte 0[8] '\ control (8 ports) %00 = nothing to do, %01 = enable port, %11 = disable port word 0[8] '| rxpin << 8 | txpin (8 ports) $FF = no pin/port long 0[8] '| rxhead << 16 | rxtail (8 ports) long 0[8] '| txhead << 16 | txtail (8 ports) long 0[8] '| rxsize << 16 | txsize (8 ports) long 0[8] '/ p_rxbuf (8 ports) hub address where rxbuf starts and txbuf follows ' --------------------------------------------------------------------------------------------------I've not looked at this code/design in detail but one (probably helpful) reason to separate port control into a byte per port is that it allows the independent port control settings to be setup/changed independently from different COG contexts without extra code needed to preserve any other port state during the change. i.e. it potentially avoids the read-modify-write cycle needed to change the value for the port control byte when the byte containing it in hub memory spans two actual port control nibbles, and you can then just do a single byte write atomically instead and simply clobber its prior value as you change it. There's no additional nibble shifting needed either. In this case sure it might burn an extra long of hub memory, but with 512kB I'd say it would be well worth it.
In my embedded/middleware work over the years I've found it's useful to do little things like that for assisting multi-threaded code, particularly when state can change dynamically after initialization. If you try to pack things a bit too tight you can come unstuck in some areas so it's a tradeoff as usual.
Maybe make two versions, one supporting eight ports with a lower top speed, and the other with four ports for higher speeds.
This has led me to the following thoughts and I'd like to know if this makes sense...
* Sixteen uni-directional ports each with its' own control byte in an array (solves read-modify-write or lock problems)
* Each port is for 1 pin only and can be either a receive or a transmit pin (ie you can have 16 receivers if you want)
* Each port will have a p_head and p_tail which are pointers to hub addresses (ie not offsets)
* At initialisation the p_head and p_tail will point to the first and last+1 hub address of the buffer
* The calling code will setup and disable the smart pin (ie the driver will not program the smart pin associated with the port)
' Main hub interface (allows up to 16 uni-directional ports)
port_control 'byte 0[16] '\ control (16 ports) %atpppppp where...
'| a: 1 = port active, t: 1 = tx, 0 = rx, pppppp: = port pin
byte 1<<7 | 0<<6 | 63 '| port[0]: active, rx, P63
byte 1<<7 | 1<<6 | 62 '| port[1]: active, tx, P62
byte 0[14] '| port[2]-[15]: = inactive/idle
port_params 'long 0[2][16] '| max (16 ports) of 2 longs... (note: only need to reserve as many ports as max required)
long @xbuf_0 '| port[0]: p_head (initially set to start of buffer)
long @xbuf_0_end '| p_tail (initially set to end of buffer)
long @xbuf_1 '| port[1]: p_head (initially set to start of buffer)
long @xbuf_1_end '| p_tail (initially set to end of buffer)
' long 0[2][14] '/ port[2]-[15]: (no need to fill/reserve unused ports)
' buffers (no need to follow port_params)
xbuf_0 long 0[128] ' port[0]: buffer
xbuf_0_end
xbuf_1 long 0[128] ' port[1]: buffer
xbuf_1_end
This minimises the hub when only a small number of uni-directional ports will be used. Just leave space for the maximum number of supported pins (ie uni-directional ports).
This simplifies the number of instructions needed to be executed in the pasm driver, and therefore maximises the number of ports and speed that can be supported.
This has led me to the following thoughts and I'd like to know if this makes sense...
* Sixteen uni-directional ports each with its' own control byte in an array (solves read-modify-write or lock problems)
* Each port is for 1 pin only and can be either a receive or a transmit pin (ie you can have 16 receivers if you want)
* Each port will have 4 long parameters...
- p_head pointer to current head hub address of the buffer (ie not an offset)
- p_tail pointer to current tail hub address of the buffer (ie not an offset)
- p_first pointer to first hub address of the buffer (ie not an offset)
- p_end pointer to last+1 hub address of the buffer (ie not an offset)
* The calling code will setup and disable the smart pin (ie the driver will not program the smart pin associated with the port)
' Main hub interface (allows up to 16 uni-directional ports) port_control 'byte 0[16] '\ control (16 ports) %atpppppp where... '| a: 1 = port active, t: 1 = tx, 0 = rx, pppppp: = port pin byte 1<<7 | 0<<6 | 63 '| port[0]: active, rx, P63 byte 1<<7 | 1<<6 | 62 '| port[1]: active, tx, P62 byte 0[14] '| port[2]-[15]: = inactive/idle port_params 'long 0[4][16] '| max (16 ports) of 4 longs... (note: only need to reserve as many ports as max required) long @xbuf_0 '| port[0]: p_head long @xbuf_0 '| p_tail long @xbuf_0 '| p_first long @xbuf_0_end '| p_end (last+1=p_first+bufsize) long @xbuf_1 '| port[1]: p_head long @xbuf_1 '| p_tail long @xbuf_1 '| p_first long @xbuf_1_end '| p_end (last+1=p_first+bufsize) ' long 0[4][14] '/ port[2]-[15]: (no need to fill/reserve unused ports) ' buffers (no need to follow port_params) xbuf_0 long 0[RX_BUF_SIZE] ' port[0]: buffer xbuf_0_end xbuf_1 long 0[TX_BUF_SIZE] ' port[1]: buffer xbuf_1_endThis minimises the hub when only a small number of uni-directional ports will be used. Just leave space for the maximum number of supported pins (ie uni-directional ports).
This simplifies the number of instructions needed to be executed in the pasm driver, and therefore maximises the number of ports and speed that can be supported.
By having p_first and p_last available it reduces the number of instructions necessary to calculate the start and end and can be accessed easily by both spin and pasm objects.
It’s probably advisable for the setup code to walk the table to look for pin conflicts before adding new entries. That wouldn’t be particularly difficult to achieve.
Same applies to configure the smart pins. The caller setting up the smartpins also lets the caller configure baud and number of bits.
I’m almost ready to test it out but i will not have time tomorrow due to day job. Mondays are always hectic.
While it’s unusual, having half-duplex (ie single pin uni-directional) ports seems like it will work nicely. You can have any mix of receive and transmit pins (ports) to a maximum of 16 pins. So you can have 1 transmit and 15 receive, or any other mix to a total of 16.
Using SETQ and rdlong minimise the number of clocks when getting parameter blocks from hub. P2 has some really neat instructions.
Yes, it does.
Yes, it's easy enough for the caller to walk the table and check the pin's not in use. Of course it cannot know if the pin is being used by a different driver eg I2C.
'' RR20210117 006 untested PUB start(p_config) : cog cog := coginit(COGEXEC_NEW, @uart_start, p_config) + 1 ' start uart cog (PTRA=p_config=@port_control) dat { smart pin uart/buffer driver } org 0 uart_start mov p_params, ptra ' PTRA = ptr to port_control (16 bytes) add p_params, #16 ' p_params = ptr to port_params (up to 16*4 longs) main_loop setq #4-1 ' get port_control from hub (all 16 bytes) rdlong control, ptra ' ... .port0 mov portnum, #0 ' set port[0] mov control, control wz ' all ports 0-3 inactive? if_nz call #proc_active ' n: go process .port4 mov portnum, #4 ' set port[4] mov control, control+1 wz ' all ports 4-7 inactive? if_nz call #proc_active ' n: go process .port8 mov portnum, #8 ' set port[8] mov control, control+2 wz ' all ports 8-11 inactive? if_nz call #proc_active ' n: go process .port12 mov portnum, #12 ' set port[12] mov control, control+3 wz ' all ports 12-15 inactive? if_nz call #proc_active ' n: go process jmp #main_loop ' all done ' process active ports (try all 4 port subsets) proc_active .p0 testb control, #7 wc ' port +0 active? if_c call #proc_this_port ' add portnum, #1 ' port++ shr control, #8 ' .p1 testb control, #7 wc ' port +1 active? if_c call #proc_this_port ' add portnum, #1 ' port++ shr control, #8 ' .p2 testb control, #7 wc ' port +2 active? if_c call #proc_this_port ' add portnum, #1 ' port++ shr control, #8 ' .p3 testb control, #7 wc ' port +3 active? if_c call #proc_this_port ' ret ' done +0..+3 ' process active port[n] proc_this_port mov xpin, control ' set pin# testb xpin, #6 wc ' t: 1=tx and xpin, #$3F ' if_c jmp #tx_serial ' j if tx ' recv serial... rx_serial testp xpin wc ' anything waiting? if_nc ret ' n: abort if nothing ' get port[n] params... mov ptrb, portnum ' current port[n] (0-15) shl ptrb, #2 ' *4 = offset to current port[n] params add ptrb, p_params ' add base setq #4-1 ' read... rdlong p_head, ptrb ' ... current port[n] params (4 longs: head/tail/start/end) ' read serial... rdpin chr, xpin ' ch := read from uart shr chr, #24 ' align lsb wrbyte chr, p_head ' rxbuf[head] := ch add p_head, #1 ' p_head++ cmp p_head, p_end wz ' end of buf? if_e mov p_head, p_start ' wrap _ret_ wrlong p_head, ptrb[0] ' write head index back to hub ' xmit serial... tx_serial rdpin chr, xpin wc ' check busy flag? if_c ret ' y: abort if busy ' get port[n] params... mov ptrb, portnum ' current port[n] (0-15) shl ptrb, #2 ' *4 = offset to current port_params[n] add ptrb, p_params ' add base setq #4-1 ' read... rdlong p_head, ptrb ' ... current port_params[n] (4 longs: head/tail/start/end) ' anything to xmit... cmp p_head, p_tail wz ' byte(s) to tx? if_e ret ' n: abort if nothing to xmit ' write serial... rdbyte chr, p_tail ' ch := txbuf[tail] wypin chr, xpin ' write to uart add p_tail, #1 ' p_tail++ cmp p_tail, p_end wz ' end of buf? if_e mov p_tail, p_start ' wrap _ret_ wrlong p_tail, ptrb[1] ' write tail index back to hub ' -------------------------------------------------------------------------------------------------- control res 4 '\ port_control (always 16 ports) %atpppppp where... '/ a: 1 = port active, t: 1 = tx, 0 = rx, pppppp: = port pin p_head res 1 '\ current port_params[n]: p_head p_tail res 1 '| p_tail p_start res 1 '| p_start p_end res 1 '/ p_end (=last+1=p_start+bufsize) p_params res 1 ' ptr to start of port_params portnum res 1 ' current port no. 0-15 xpin res 1 ' current port byte %atpppppp chr res 1 ' chr ' -------------------------------------------------------------------------------------------------- fit 472True.
Code looks ready for testing to me; Seems to be striking a good balance between clarity and speed.
Yes. Just ran out of time to test it with a new front-end. I will build at least two front-ends. One will just be a full duplex (ie 2 port pins) object that will be pre-configured for the masses. The other will be an extendable version and it will probably undergo a few revisions to get it easy to use.
Only tested with FlexProp v5.0.7
pnut and PropTool testing to come soon. pnut working.
mpx_multiportserialdriver.spin2
This is the pasm driver that is loaded into it's own cog. The code is working and AFAIK with no bugs. It will support up to 16 half duplex ports - any mix of receive ports (pins) and transmit ports (pins) up to a total of 16. Smartpins are setup in the calling program, as are the port control and parameters.
mpx.demo.spin2
This is a demo program and just calls mpx_fullduplexserial.spin2. This is a simple demo program that uses a few methods to display and receive some serial data. Nothing major here.
mpx_fullduplexserial.spin2
This program is based on @JonnyMac 's jm_fullduplexserial.spin2 code. This is great code which includes methods to display strings with embedded hex/decimal/binary etc. A really great object.
I have added some additional methods to display such things as hub dumps which includes both hex and ascii representation.
Also, this is now using my new mpx_multiportserialdriver.spin2 driver.
Warning: I know there are some problems in the receive section because I haven't completed the process to use the new style receive buffers. I have just released it so you can get a preview. AFAIK the transmit methods should work correctly.
If you find any bugs please let me know here, aside from the receive methods (rxcheck, rxavailable, etc) which haven't been converted yet.
See next post for latest code
txflush, rxcheck, available, etc should now be working
Please report bugs here.
Glad to hear you have this running. I do hope that diverbob is following this thread as I know he needs at least 6 serial ports for his robot.
Jim