New Generation of Low Level Driver for P2
Kaio
Posts: 253
In the past each low level driver did include a more or less comprehensive interface implementation of methods which were necessary to have it to formatting data to send or receive data via the low level driver.
Thanks to the method pointer feature now available in Spin2 it is possible to make a code boundary between the low level driver itself and necessary higher level formatting functions purposeful. This means that a low level driver must not provide a specific formatting implementation instead of it only need to provide an interface at the lowest level necessary for a specific driver. If one need some formatting functions used together with a low level driver one can use any generic formatting object to have the necessary functions available as needed.
This can be your own preferred interface implementation derived over many years or another interface implementation from any person. There's only one thing that this implementation must be rely on and that is the use of the SEND and RECV method pointers as low level interface.
As example I have stripped the multi-port serial driver from Ray (@Cluso99) which now also includes some improvements on the provided interface as I suggested (https://forums.parallax.com/discussion/comment/1515837/#Comment_1515837). The pasm serial driver is unchanged.
Then I have extracted the interface of formatting methods derived from the original full-duplex serial from Jon (@JonnyMac) and modified it to act as a generic formatting object.
Now Jon and others are able to use the new generation multi-port serial driver as replacement of the full-duplex serial. Only the initialization for a serial communication of both is different, see code comments to try.
This example shows you how powerful the method pointer concept is. In compare to the original multi-port serial driver there was no need to change some formatting methods to have a better naming. One can still use one's preferred interface with any low level driver (serial, VGA, HDMI or whatever).
The benefit of this concept is that one don't need to know the differences between the interfaces of several drivers as one can use the same interface for all. And for developers it is easier as they need only to implement the lowest level interface. This speeds up the developing and testing of new drivers. Finally it saves memory wasted before for unnecessary functions as everybody can implement its own interface as needed without any change on the driver itself. This was not possible before with the P1.
As Dave mentioned below method pointers were possible on the P1 with his object.
Comments
It was possible to do method pointers with the P1, but nobody ever used it. I posted a method pointer object to the OBEX ten years ago.
Kaio's code also looks to be a good reference for spin2 memory pointers in general.
@""Dave Hein"
I did check your MethodPointer object for P1 and you did a great job. But it wouldn't be fair to compare your solution with the method pointer concept introduced with spin2 and one shouldn't conclude that it will not be used on P2 too.
I think some people need time to evaluate the method pointer concept on the P2 to see the benefit which will be possible if they would use it. I see no reason why one should not use this feature now as it is so easy to use it.
@Kaio : I like the direction your object is going in, but I think it's still more complicated that it needs to be. For example, instead of:
ser.fstr0(string("Demo of New Generation Low Level Driver\r\r"))you can just write:
send("Demo of New Generation Low Level Driver", 13, 13)SEND is more powerful than just a method pointer; the compiler also treats calls using SEND specially, and will translate calls with multiple arguments (like strings) into multiple calls to the pointer.
Have you looked at my ers_fmt object in the github OBEX? I've attached a zip of it here.
Yes, I know my MethodPointer object is a kludge, but it worked.
I used it to support C function pointers in CSPIN. I then was able to port my chess program that was written in C to P1 spin. I like the method pointers that were added to Spin2. Along with all the other syntax changes Spin2 has almost become a subset of C++, except with a funky indention syntax. At this point, all Spin2 needs is some braces and it's basically a simpler version of C++.
I like your low level driver idea. It will be interesting to see if it catches on.
@ersmith Thanks Eric for your clarification and I'm aware of that. I did already check your code provided with FlexProp.
For some people it might be difficult to change their mind of how to code some stuff if using driver objects. Using directly the SEND method pointer would be a big change for these people.
Hence, my intention was to show that one can use the same interface implementation for several drivers if one would use the method pointer concept provided by spin2. This is also possible if one want not to use for some reason the provided formatting methods of the driver as with the method pointer concept one is independent from the implementation of such methods of the driver. In this case the developer of the driver would have waste his/her time if the users would avoid using the provided methods and the users would waste the memory of the propeller with code of unused methods.
To avoid this scenario I would suggest that developers need only to implement the lowest level methods of a driver and the user can use one’s preferred interface to formatting data by using the method pointer concept. If the SEND method pointer is used directly in this case or via a generic formatting interface object is the choice of the user. In both cases the user only have to assign the low level method of the driver to a method pointer in his/her code.
@Kaio
Took a quick look yesterday. Hopefully I’ll get some time to look deeper today. Currently I’m missing something with send as I don’t see how it’s getting around, or simplifying anything to my drivers. Your other changes helped.
Currently I’m working on modifying my LCD driver to use the new buffers. I needed to add text scrolling first, which I did yesterday (currently it’s a quick kludge as I haven’t worked out if I can use the LCD to do it, or else read back the contents so I can scroll without maintaining an independent screen buffer).
@"Dave Hein" Thanks Dave, you are welcome. Maybe you or other P2 users have some more ideas for which the method pointer concept would be helpful.
@Cluso99
Usual a driver did include some formatting methods in the past. And you did reuse the interface of formatting methods derived from the full-duplex serial driver from Jon to rework for a new multi-port serial driver.
I did also reuse the interface of formatting methods derived from the same full-duplex serial driver, but I extracted it to a separate file to be used as a generic formatting interface. I did also remove all formatting methods from your multi-port driver in my example code. So, there are only the lowest level methods remaining on the driver.
Now one can "connect" both together, the multi-port driver and the separate formatting interface, by using the method pointer concept. And one can use the same formatting interface with any other driver via the method pointers.
The benefit for the user is that one can develop one's own formatting interface or use a preferred one from another developer. The benefit for the developer of a driver is that one needs less time for development and testing of a driver as only the lowest level methods need to be implemented.
The benefit for all P2 users would be that one would have the choice of several formatting objects and to use always the same interface or different interfaces depending on the purpose of the project. And one would not waste the memory of the propeller with unused methods of a driver.
I'm still trying to understand the send concept and how it's an advantage. I stayed around on the zoom meeting this morning but missed the opportunity.
I've been trying to get my LCD into shape to be used as a pure driver. Currently it has a mix of spin and the pasm driver is in a separate cog.
@Kaio
I like what you've done with the calls, and port allocation, even tho I prefer to know the port number I guess it's not essential.
I don't agree with auto-configuring the rx pin as it's definitely not a standard for the rx to be tx+1.
Do remember that I'm trying to replace ALL I/O excluding disk (SD and FLash) using the same mechanism. So I do have ports > 16 that have a different port_control interface because of the need to pass more pins. This means that I might not be limited to the 16 set of buffer controls - I haven't got that far yet.
I now see that what you're redirecting with send is the bottom tx(chr) and recv is rx(chr).
The first problem I see is that rx(chr) isn't always called because sometimes you do not want to pull a character out of the buffer, just know if there's one there. Also, you do not want to wait if there isn't one in there. So I guess there's a problem for these.
Next question I have. How does send work when you have two fdx ports? Can you have two send and recv sets? For example, I have some code that outputs to both tx pins using ser.tx...(..) and ser2.tx...(...) so how does this work? I cannot see why send and recv are better. I only have to swap a pointer to port_params (or two for the tx and rx pair) to swap where the output is going. Very easy to do in software.
And another question. I'm not sure if there is an easier way, and not sure whether it's relevant here, but I'll describe it anyway.
I have a need to xmit (in particular) messages (similar to Chip's debugger) from different levels of objects to the one serial port. So, I have a top object that outputs various messages as it gets to certain points. It also has an object (same cog), and this object also has the need to output messages to the same serial port. Perhaps there's an easier way, but for now, that object has a copy of the same serial object as used in the parent object. In order to utilise this same object I just pass it the same port_param reference and all is fine. because all this code runs in the same cog, there's no contention or mixed messages happening. Ideas?
Ray, here is the SEND explanation from Spin2 doc (the current forum software is not showing the indents, so I'll use periods):
SEND
SEND is a special method pointer which is inherited from the calling method and, in turn, conveyed to all called methods. It's purpose is to provide an efficient output mechanism for data.
SEND can be assigned like a method pointer, but it must point to a method which takes one parameter and has no return values:
SEND := @OutMethod
When used as a method, SEND will pass all parameters, including any return values from called methods, to the method SEND points to:
SEND("Hello! ", GetDigit()+"0", 13)
Any methods called within the SEND parameters will inherit the SEND pointer, so that they can do SEND methods, too:
PUB Go()
..SEND := @SetLED
..REPEAT
....SEND(Flash(),$01,$02,$04,$08,$10,$20,$40,$80)
PRI Flash() : x
..REPEAT 2
....SEND($00,$FF,$00)
..RETURN $AA
PRI SetLED(x)
..PINWRITE(56 ADDPINS 7, !x)
..WAITMS(125)
In the above example, the following values are output in repeating sequence: $00, $FF, $00, $00, $FF, $00, $AA, $01, $02, $04, $08, $10, $20, $40, $80 (but inverted for LEDs)
Though a called method inherits the current SEND pointer, it may change it for its own purposes. Upon return from that method, the SEND pointer will be back to what it was before the method was called. So, the SEND pointer value is propagated in method calls, but not in method returns.
Chip,
In your example above, I'm confused...
I understand setting send := @setled so that when send is used it converts/redirects the call to the setled call.
If the parameters in the send such as send(flash()... it will call the method within the setled called flash.
-- How does it know flash is part of the setled object?
-- I see flash() will output $00,$FF,$00 twice ie it is just running flash()
-- I see flash will return $FF but where does this go? Or is this if retnval := send(flash()... then retnval will get $FF ?
I don't see how the $01,$02...$80 will go to the setled method, or is that because there are no other method names in the send(flash(),.... so the default will be the original @setled ?
So in the old case like fullduplexserial, we could
send := @tx(char)
So send(txstring(@somestring)) would output to the method in the tx(chr) object the contents of the string @somestring ?
And we can chain them like send(txstring(@somestring),13,10,txstring(@secondstring)) etc ?
Presuming this is correct, now suppose I have two fullduplexserial objects which use two different sets of pins. I would define them like
OBJ
ser1 : "fullduplexserial"
ser2 : "fullduplexserial"
Currently I would output to the different serial ports by..
ser1.txstring(@somestring))
ser2.txstring(secondstring))
How does send work with two objects ?
Ray, to use SEND for two different objects' methods, you would have to assign each method to SEND, as needed, one at a time.
You would only use SEND if you wanted the advantage of inheritance by called methods and the multiple parameters feature.
I'm not understanding the first part of your question.
Chip,
I'm not seeing how your example above calls various methods within an object.
However, as I suspected, the problem when you have multiple instances needing the send function breaks down. This has been where I'm coming. Perhaps I didn't explain it very well.
For the multiport serial, I just load up multiple "formatting" objects, and the one driver (in its' own cog) can handle up to 16 serial pins of any mix of transmit pins and receive pins. In fact I have an extended version which handles up to 64 pins! I've tested it to 58 pins (29 full duplex serial uarts) daisy chained (P0-55 + P62-63) on the P2-EVAL. If we didn't fit a flash or sd we could use the whole 64 pins for 32 fdx uarts - not bad for a P2 hey!
For each instance of a port (can be a bidirectional port ie supports tx methods and rx methods, or a unidirectional port ie only supports either tx methods or rx methods), you create multiple references to the same object - and object that handles the formatting (such as string, Jon's formatting strings, etc). Each instance gets a prefix such as ser1. and ser2. in my example above. This object is the same for outputs such as serial, vga, video, hdmi, lcd, or any other device you can reasonably think of. Your code just outputs to the standard formatting object. This duplicated object does not know what is on the other end, be it serial, vga, etc. In fact, that underlying object can indeed be changed on the fly by simply changing the co-operating objec (a serial driver, a vga driver, an lcd driver, etc). The common part is simply a buffer (or two buffers if it handles both tx and rx) and one (or two) port-parameters of 4 longs which consist of 4 hub address pointers - p_head, p_tail, p_start, p_end.
The drivers (serial, vga, lcd, keyboard - usb, ps2, serial, etc, mouse, gamepad, etc) all use this port_parameters 4 long interface (or two) to take characters from the buffers and send them out the real pins, and/or take characters from the real pins and put them in the buffers.
The formatting objects and the drivers are totally decoupled, allowing a simple substitution of the drivers at will.
The buffer and pointer mechanism is extremely simple, identical, and tiny. Really it's just a slightly more modern version of what you used in fullduplexserial. However, they are in separate objects that can be totally decoupled and replaced.
If you go back to the CPM days, there was a main port and an auxilary port (both bi-directional) and IIRC a printer port and a punch port, a reader port and something else. Again, IIRC there were 4 bi-directional ports.
This might give you an idea as to why I want to specifically deploy the buffers and pointers, and the code in specific locations within hub ram. I want to be able to load binaries to replace current binaries, all while keeping other code running in other cogs. This is the heart of an OS - the services remain operational while you load user binaries. Even DIR, TYPE, COPY etc are actually binaries which rely on parts of the OS remaining resident and operational. And it's so much better/easier with multiple cogs as you don't need to worry about interrupts either. Imagine my OS loading multiple copies of space invaders! I've been loading Z80/CPM from my OS in P1 for more than 10 years now! Believe it or not, it was way easier to do on the P1. I still haven't achieved it properly on P2 as I'm having issues with the OS. P2 Z80/CPM has been running for many months now.
@Cluso99 That's a matter of opinion. The standard serial pins on P2 are in that order. So, it will be enough if one provide only one pin. Maybe an additional method with parameters for rx and tx pin could be added so that the user has the choice of use.
I have used your 16x multi-port serial driver as base. If you need more ports managed by my driver you have only to change MAX_PORTS in the CON section of multiportserial.spin2. It's the same as in your original driver. But don't forget to update the pasm driver too as it depends on the size of the control structure to work correctly. As I did say already in your thread on discussing of my suggestions, there's no need to have the control structure outside of the driver.
You're right, I have used the blocking rx method with the method pointer RECV. But you can also assign the non-blocking rx method to RECV.
If you would need both rx variants in your program you can simply create another method pointer in the VAR section (e.g. recvNB for non-blocking) and assign the non-blocking rx method to this one. In this case you have to specify the number of returned values if you use this method pointer for a call, e.g. recvNB():1. On the built-in RECV method pointer this is not necessary as it is designed to return always one value.
Already answered by Chip.
Ray, you don't need to initialize the same port again in another object. In the top object you would have located the driver objects, you open the port and assign the tx method to the SEND method pointer. You can simple use the SEND method pointer (send(...)) in any other object to send data to the same port as this method pointer is a global variable in the interpreter. (I could not check this behavior yet as I have not my board in place, but from interpreter code it looks like.). This works as long all objects use the same cog. In a different (spin) cog you could used the SEND method pointer for other things.
@Cluso99
Ray, I'm also a great fan boy of CPM with a deep knowledge of this OS. But I will not repeat myself again. If you need this behavior with buffers and pointers for your propeller OS you can do this in your project for yourself. But I see no reason why other P2 users should do the same if it is not necessary as the control structure should be encapsulated in the multi-port driver itself. That this is possible and working I have presented in my code example available in the first post.
In the way you will use the multi-port driver it is out of scope for "a truly standard spin driver for P2". Sorry, that I must have to say this so clearly.
@Kaio
Rather than require I’ll refer to paragraph numbers.
1. The order of tx and rx is only by the choice of p63 an p62. But that doesn’t imply or mandate any other pins need to be pairs, although likely, and especially the order. Consider two p2s connected together - it would be likely their p63 would go to p62 and visa versa. This has been done on p1s. We always specified both pins on P1 and I really think not supplying both is a huge mistake here.
Just for reference IIRC the ps2 connections which have a clock and a data pin would likely be similar, but if you look at parallax’s reference design IIRC they were swapped for different ports. The big benefit of p1and P2 is your not constrained to using any particular pins.
I think you’ll see later when I get other objects working together with these, then having the port_parameters outside the drivers will make more sense. It’s just not clear at the moment.
Yes, there may be a way around this.
Chip answered. From my point of view, this takes away any benefits of send and recover ;(
I need to check this out as this is a huge advantage for using send because it’s during debugging that it’s particularly relevant. I have used objects in multiple levels in P1 but they were not I/O objects if I recall correctly, so it was just as easy to re-declare them.
General...
As you can see, I’m not sure precisely how the send call mechanisms work to call the other methods. That can wait though.
Many thanks for your comments and discussions. Well end up with a better set of objects, drivers and mechanisms because of this.
Ray.
I agree.
It looks like there's some missing at the end of your post.
Sorry, go back and read the rest. I accidentally hit post on my iPad. It’s 3am here!
I'm really enthused to see some conversation around the DRY features of Spin2. When it comes to generalizing driver interfaces, I think this is a very good idea that is easier to do now. There is also a need in some cases to have direct control of buffer spaces when you are doing glue work between an application level and multiple drivers. They have to fit in memory and play nice together, and last I checked, we don't have a standard malloc interface, nor any equivalent for pins. (It might be a good idea for someone to build a memory and pin allocation library, that can be passed to drivers on init)
It would be nice if the SEND pointer delegation could be made into a generalized feature that you could use with any method name. However nice it is, it feels a little strange having a special case of behavior for a couple method names. It should be possible to define other pointers or values that are managed by the compiler, not just for methods. However, I do see need for caution as well, since the only real indicator that you are using an implicitly defined pointer is that you are using SEND or RECV, and those stand out as special cases once you know about them.
Maybe a generalized mechanism for this could be called out with a syntax which is not going to be confused with anything else. I'm not a huge fan of symbol creep, but explicitly tying syntax to behavior is generally also a very good thing. Where there can be ambiguity between "I defined this symbol at the method level" and "I defined this two layers ago in the stack", there is plenty of room for the kind of bugs that drive new coders crazy.
We'll see. You know my point of view.
That's fine.
You're right with the P1. But on the P2 you could do it by another way.
It's as easy. The meaning of "assigned method" in the description below is that you have assigned a method to the SEND method pointer before use of send, e.g. send := @obj.method. Then you can call send(p1, p2, func1(p3),...).
If you provide a single value (byte, word or long) as parameter then it is hand over to the assigned method.
If you provide a string as parameter the assigned method is called with each byte of the string multiple times until the string is processed. It's like the str method work in your driver. But you don't need to use the string function to create a string, only quotes are necessary.
If you provide a method as parameter it is called as usual and its return value is hand over to the assigned method. You can also provide parameters to this method.
The benefit of the SEND method is that you can mix all these variants of parameter types in the parameter list as needed. There is no limit for the count of parameters I know of. It's all handled by the spin interpreter.
Ray, you're welcome.
Thomas
@Twyyx
This might be necessary for few cases, e.g. for propeller OS, but usually you will not need this if using spin including pasm routines.
The malloc thing is not easy to do with spin.
For pins its already available, the multi-port driver can manage all pins. But currently there's only an implementation to use it as a serial driver. Support for other types of drivers needs to be added in a general way.
I'm not sure what do you mean? You can assign any method to the SEND method pointer which expects one parameter.
And of course you can create your own method pointer to use it for a method with more parameters. But there is the following limitation as of the Spin documentation.
"There is no compile-time awareness of how many parameters the method pointed to actually has. You need to code your method pointer usage such that you supply the proper number of parameters and specify the proper number of return values after a ":", so that there is agreement with the method pointed to."
Did you read this already?
What makes you confusing regarding the method pointer concept?
Ok, I see that part now. I was confused with Chips example and thought it was calling multiple methods rather than just bulking up the string and passing it to the tx method.
This is exactly what I meant. There could/should be a general way to delegate the hub memory and/or pin allocations to a single module which could be passed to drivers which support the interface. My comments were more on the language level and general driver portability, not specifically to this one.
If we wanted to get 4KB aligned at 1KB boundary, it wouldn't be difficult to ask a heap-based utility to do this for us. Same for pins: "Give me 16 pins, aligned mod 8". The only thing that really has to change for this to be useful is that a driver needs to allow the user to provide an appropriate delegate function, maybe as an alternate initializer path to the default.
What I meant was that maybe I want to use another method like BUFFER, and have its parameters handled in a similar way to SEND or RECV. Being able to designate a method as a consumer of varargs style arguments has general utility value.
I'm not saying I'm confused about the concept. I'm saying that allowing the calling semantics of methods to be implicit for the same syntax may lead to surprising bugs. The other part of what I was saying is that for SEND and RECV it's not such a big deal as these are clearly marked out as special cases. However, if you could do something like BUFFER:=@MY_BUFFER_FUNC, then perhaps it would be better to make it more explicit, maybe requiring calls to @BUFFER to look different, as in @>BUFFER("My Data Here") <-- I know this is different, but that's the point. BUFFER would behave differently, so it's appropriate to call this out to avoid ambiguity.
There are no "standard" serial pins on P2. That's a standard that you classified as such, and once you start to design lots of hardware, you may well find that even you yourself won't adhere to it.
There are pins that are used by default by the bootloader, but the use of those pins for anything else is entirely optional, and the serial function of the bootloader can well be disabled - then these pins won't act as serial pins by default unless the first flash page becomes corrupt. In many of the designs I do for work, the built-in bootloader is not available, because there's a stage 2 bootloader that does what I need done. I have a serial connection usually, but it uses whatever pins make sense for the layout. I assign the pins based on the routing needs of the board. That's part of the reason why I'm using P2 after all: so that the board routing is much easier. In a design that uses a single VCCIO (i.e. no precision/low noise analog functions), all the I/O pins are exactly equivalent (with exception of the HDMI serdes, when I use one), and I route for shortest traces, not to appease anyone's idea of what is "standard".
So, if you care about your users, you'll provide means to actually leverage P2's unique feature - that all pins are equal. Otherwise you can't expect everyone to just take your "standard" and force themselves to use it.
@kuba You're right, that changed my mind. Thank you.
@Cluso99 I was right with what I said. I've checked this with a little test. (Please comment out the constant _xtlfreq I forgot.)
The SEND and RECV method pointer are global per spin interpreter instance (cog). But self defined method pointers are local to the object where it is located in the VAR section.
@Kaio
Thanks Thomas.
Just looking now.
This line explained what I was trying to ask Chip above. I didn't understand send's working but his line shows me what I was missing.
send("Test from top object via send",ser.CR,ser.CR)
It's certainly a more complex call but it could be broken into separate lines which makes it easier to read if the method has complex parameters - ser.CR is simple so it's not a good example.
send("Test from top object via send")
send(ser.CR)
send(ser.CR)
So now I think I can do this
send(ser.fstr1(string("Port-params %8x\r\n"), addr2))
But it doesn't allow this
send(ser.fstr1(string("Port-params %8x\r\n"), addr2))
send(ser1.fstr1(string("Port-params %8x\r\n"), addr2))
because you would need to add an intervening line to redirect send, and then another line after to return to the original. Correct?
Demo looks nice.
BTW It doesn't compile with FlexSpin. Perhaps Eric is not supporting RECV()?
C:/P2/_RetroBlade2/mpx_multiportserial/Kaio2/mp2_test.spin2:50: error: Object called is not a function
C:/P2/_RetroBlade2/mpx_multiportserial/Kaio2/mp2_test.spin2:50: error: __recvptr is not a function
This is the offending line (3rd line c:=recv()
PUB getChar() : c
'does work as the RECV method pointer is global variable in the spin interpreter
c := recv()
You can actually pass method pointers anywhere within an application and use them. They hold (and point to) complete contextual data. They needn't be run at the same level they were made at. You can get a pointer to any method in your current object and any PUB method in any child object. Once you have that method pointer established, you can pass it and use it anywhere in your application's scope. It is, after all, just a long.