Language Agnostic API for P2 HAL Implementations
DavidZemon
Posts: 2,973
Wouldn't it be cool if every (major/large) library for the Propeller 2 implemented a similar API? Of course it would - then jumping from one language to another would only be as hard as learning the new syntax rather than learning both a syntax and a new library API. It's going to be hard. Without a dictator, there will be religious wars over what is the "right" or "best" signature for a certain method. If there was a dictator, there would be a mutiny. But if we can make it happen, if we can agree on a single language-agnostic API that HAL (hardware abstraction library) authors can implement (or at least, draw inspiration from), wouldn't that be COOL!?
I think now is the time to start talking about this. We're early enough that very little work has been done toward HAL implementations yet far enough into P2 development that we have a variety of tools at our disposal that will let us start implementing the APIs across different languages, and trying them out to see what works and what doesn't.
(I think Peter's ROM code probably has the most work put into it, and hopefully we can learn something from the Forth API, but I don't know how well that will translate to stack-based languages)
I would propose using a slightly modified Spin syntax to describe this API. This should be sufficiently legible by anyone with the most basic of programming experience, while also providing the necessary information to translate to other languages
Note that parameters do not have types associated with them in this psuedo code, and the return type should be as broad as possible. I can imagine valid types being limited to "boolean", "string", "number" or the name of a type described elsewhere in the API (for instance, maybe a function returns an instance of a UsbBus, or something like that).
An example might look like this:
You could then translate this to C, C++ and Python as (these are just ideas - don't get caught up on my implementation of the above API):
In the spirit of open source software, perhaps this API can be documented in a Markdown file (or a collection of files) and stored in GitHub, on github.com/parallaxinc. This allows us to have fights discussions over new proposals via the Pull Request system and avoids the problem of one person leaving the community and preventing access to edit the top forum post anymore.
What do you guys think? Is this feasible? Is it a pipe dream that can never happen? Most importantly, if such a document existed and contained an API with functions that performed a task you wanted to implement, WOULD YOU FOLLOW IT even if you disagreed with it? Would you accept that conformity is better than perfection? Or would you simply see the API and think "well that's dumb, they should have done it this way," and proceeded to write your own HAL with a completely different API?
Let me also lay down some of extra "guidelines" I think should be followed:
Language standards come first. If a language has a standard (think Python's PEP) or overwhelming convention, always follow that over any convention in the language agnostic API. For instance, I used camelCase for parameter names but PEP dictates underscores. The Python implementations should therefore use underscore for all parameter names.
On a similar note, C and Spin don't have exceptions and therefore must report errors through a return code (please don't crucify me if that's incorrect about Spin... I haven't written Spin in a long time). However, the Python implementation should use the Pythonic way of handling errors: raise an exception and DO not return an "error code."
If this gains traction, I will update this first post with any pertinent information.
I think now is the time to start talking about this. We're early enough that very little work has been done toward HAL implementations yet far enough into P2 development that we have a variety of tools at our disposal that will let us start implementing the APIs across different languages, and trying them out to see what works and what doesn't.
(I think Peter's ROM code probably has the most work put into it, and hopefully we can learn something from the Forth API, but I don't know how well that will translate to stack-based languages)
I would propose using a slightly modified Spin syntax to describe this API. This should be sufficiently legible by anyone with the most basic of programming experience, while also providing the necessary information to translate to other languages
function_name(parameterName1, parameterName2) : returnType # Any extra description necessary, such as what the return value is and whether or not exceptions might be raised
Note that parameters do not have types associated with them in this psuedo code, and the return type should be as broad as possible. I can imagine valid types being limited to "boolean", "string", "number" or the name of a type described elsewhere in the API (for instance, maybe a function returns an instance of a UsbBus, or something like that).
An example might look like this:
start_pwm(pin, frequency, duty) : number # Returns an error code if any parameters are invalid
You could then translate this to C, C++ and Python as (these are just ideas - don't get caught up on my implementation of the above API):
error_t start_pwm(const uint32_t pin, const uint32_t frequency, const uint8_t duty);
class Pin {
public:
Pin(const uint8_t pinNumber);
void start_pwm(const uint32_t frequency, const uint8_t duty) const; // Throws an exception if any parameters are invalid
};
class Pin: def __init__(pinNumber) def start_pwm(frequency, duty) # Raises an exception if any parameters are invalid
In the spirit of open source software, perhaps this API can be documented in a Markdown file (or a collection of files) and stored in GitHub, on github.com/parallaxinc. This allows us to have fights discussions over new proposals via the Pull Request system and avoids the problem of one person leaving the community and preventing access to edit the top forum post anymore.
What do you guys think? Is this feasible? Is it a pipe dream that can never happen? Most importantly, if such a document existed and contained an API with functions that performed a task you wanted to implement, WOULD YOU FOLLOW IT even if you disagreed with it? Would you accept that conformity is better than perfection? Or would you simply see the API and think "well that's dumb, they should have done it this way," and proceeded to write your own HAL with a completely different API?
Let me also lay down some of extra "guidelines" I think should be followed:
Language standards come first. If a language has a standard (think Python's PEP) or overwhelming convention, always follow that over any convention in the language agnostic API. For instance, I used camelCase for parameter names but PEP dictates underscores. The Python implementations should therefore use underscore for all parameter names.
On a similar note, C and Spin don't have exceptions and therefore must report errors through a return code (please don't crucify me if that's incorrect about Spin... I haven't written Spin in a long time). However, the Python implementation should use the Pythonic way of handling errors: raise an exception and DO not return an "error code."
If this gains traction, I will update this first post with any pertinent information.
Comments
Its clear that things are starting to move with other language options, which makes it a good time to at least discuss. I'm sure where will be a wide variety of opinions, and consensus might be hard to reach in all areas, but lets have a go, particularly around smartpins
as Mr. Spock would say.
Please send me some links so I can understand more.
Thanks
Martin
The broad idea has merit, but I guess it depends on how close you want to be to the silicon, and how much code you want inside the function calls.
My preference would be to start from the Smart Pin Docs, and code APIs from there up, so you get full silicon coverage.
Other APIs could call those base ones, if that was required.
Let's look at the smart pin DOCs
"%01001 = PWM sawtooth
This mode overrides OUT to control the pin output state.
X[15:0] establishes a base period in clock cycles which forms the empirical high-time and low-time units.
X[31:16] establishes a PWM frame period in terms of base periods.
Y[15:0] establishes the PWM output value which gets captured at each frame start and used for its duration. It should range from zero to the frame period.
A counter, updating at each base period, counts from one up to the frame period. Then, Y[15:0] is captured, IN is raised, and the process repeats.
At each base period, the captured output value is compared to the counter. If it is equal or greater, a high is output. If it is less, a low is output. Therefore, a zero will always output a low and the frame period value will always output a high.
During reset (DIR=0), IN is low, the output is low, and Y[15:0] is captured.
So the first problem is frequency is not a natural parameter here, nor is duty. Also uint8_t duty is going to be too coarse for many uses, and does not use all the silicon capability.
I also find the Smart Pin docs a little obtuse, so a formula based description in the API would help explain to users what they need to fill out, for what outcome.
To me, the natural parameters here, which are pretty much chosen by the HW of the Smart Pin itself, are
I think PWM_Per = 0xffff is legal and gives PWM_Pin = HI, no matter what PWM_Per is set to. ie that's PWM_Comp >= PWM_Per : Hi
And expect some dumb questions in the beginning.
https://github.com/DavidZemon/HelloP2GCC/blob/master/common.h
And I agree about 8-bits not being precise enough for duty cycle in a general purpose library. I was thinking 0-100% in 1% increments, but finer granularity will almost certainly be appreciated. This is, of course, why it's important that fine details like bit widths be left out of the language agnostic version.
@pilot0315 questions are welcome
As an example, a recent discussion for Servo pulse control via smart pin, had me reaching for the natural PWM - but that has 16b fields so limits the granularity.
Someone else spotted a sub mode in another smart pin mode, that allowed one-shot pulses to 32b precisions.
If those bit-widths are too hidden, users get no guides of the underlying silicon limits and capabilities.
Just keep me in a beginners loop so I can get up to speed. And understand the basics.
Thanks
Here are some of my thoughts on the subject, in no particular order:
- We may actually want two layers: a HAL (hardware abstraction layer) that hides/abstracts hardware details, and a lower level (I'll call it a hardware interface layer, or HIL) that directly exposes the P2 hardware. As an example, in the HAL we might have a function like "set_system_frequency(freq)" that sets the system clock to a requested frequency. This would calculate an appopriate P2 clock mode and call the lower level HIL function "clkset(mode, freq)" which would issue hubset instructions to actually do the change.
What I'm envisioning is that HIL calls would map to 1 or at most a few P2 instructions, but HAL ones might be more complicated. The idea is that code written to the HAL could run on P1, P2, or future chips (within reason, of course; if a chip doesn't support ADC then HAL analog to digital conversion functions will report an error). The HIL would be pretty much processor specific, or close to it.
- I think the HAL and HIL function proposals probably should include types for parameters and return objects. Some languages (like Spin and Python) may not need them, but other languages (like BASIC and C) do, and they could be useful for documentation purposes as well. Incidentally, fastspin accepts but does not require type annotations like "=long" or "=@byte" on parameters and return values.
- I'm not a Tachyon expert at all, but I think @"Peter Jakacki" has probably covered some of this ground already, and any FORTH words he already has defined would probably be a good starting point.
- In general I think if we can get Tachyon, p2gcc, and fastspin to support some API then it's probably going to become a de-facto standard. Obviously it would be good to hear from @cgracey about his Spin2 plans as well. Hopefully he'll contribute his ideas and/or support what we come up with.
What about object names? Is it worth making the API object oriented? If spin2 supports or will support multiple instances of the same object, I'd say yes. Otherwise, I'd say make the psuedo code procedural and let OO languages convert to objects.
I'm happy to use types, though definitely surprised. What do you think of switching to C syntax? That lets us be precise with const and pointers and references where necessary.
Watching with interest
That said, a procedural API would be more generally applicable than an object oriented one; FORTH, for example, doesn't have any idea of objects. So probably we should start there.
I think C syntax would work well. The only restriction C has is that it only allows single return values, whereas fastspin/spin2 allow multiple return values. But only a few languages support multiple returns, and if we really have to we can get something similar in C by returning structs (or having output-only pointer parameters, but I think that's ugly and hurts optimization).
Multiple return values is a really good point. Let's have the API allow for that. Python supports it too, and I fully expect the C implementation to use "output parameters" when it needs to both return an error code and return a value. So now we're looking at something like this for the language-agnostic part:
Now... I'm starting to have problems with specifying types. I'm tempted to write the following API function
and then be 100% okay with specifying no return value, because pin_t would enumerate each and every one of the 64 pins, therefore making it impossible to pass in any invalid values. But what does a dynamically typed language like Spin or Python do? I'm starting to think, again, that the language-agnostic part of this API should really just be focused on:
1. What functions are exposed
2. What are the parameters for those functions
3. What values and errors might the functions return
4. How will modules be organized (procedural and OO languages must have some kind of organization, whether that's just folders, classes, packages, file names, etc doesn't matter)
What about a different idea. What if return values are specified separately from errors? What if a "!" in the signature says "everything after me is an error scenario that might (or might not) need to be handled"
This could expand to the following C code:
Note that C doesn't need an error codes because the pin can be fully enumerated and therefore catching any invalid arguments at compile time. However, the error does need to be handled in Python:
class Pin(object): def __init__(self, pinNumber: int): """ :param pinNumber Pin number (1-indexed) to be used :raises Exception: If `pinNumber` greater than maximum number of pins (64) """ pass def start_pwm(frequency: int, dutyCycle: float) -> None: """ :param frequency Frequency (in Hz) for the signal :param dutyCycle Duty cycle as a percentage from 0.0 to 100.0 :raises Exception: If duty cycle is less than 0 or greater than 100 """ passNo, let's not rely on the type checking to prevent errors -- even in C the programmer could cast an illegal value to a pin_t and pass it to the function. For that matter, I would strongly suggest that we should stick to a small set of types that will be available for all languages (things like "int32_t", "uint8_t", "string", or pointers to those). Some languages (e.g. assembler) don't really have types, and we also don't want to burden the reader of the documents with having to learn a bunch of P2 API specific types like "pin_t".
I really wouldn't sweat the details of how the pseudo-code declarations look; we can always change those later. Just pick something simple that should be widely supported (like C declarations) and write the API that way. We can always change it later if we get stuck. Remember Knuth's adage: "Premature optimization is the root of all evil".
I think that HIL should be coded first, to give full access to smart pin modes, and it also guides HAL work, as some things will be more natural than others.
Taking the PWM example, since it is appearing here many times, the HIL actually needs 4 parameters, and does not 'think' in Hz units.
Instead, it accepts u16 dividers and compares, and avoids needing to know any SysCLK.
An HAL that tries to define just Hz,Duty has to know SysCLK, and also needs to take a guess on the split of 2 u16 values that are chained to deliver an approximation to the user requested Hz.
The HIL u16 Compare value knows it is relative to the u16 Period value, whilst an HAL that passes DUTY needs to remember what split was chosen for Hz, to scale the duty, and the decimal point location is unclear here too.
The biggest problem is Hz,Duty gives the illusion of portability and precision, whilst what you actually get will vary with language library implementation decisions.
TAQOZ already has a HIL that is close the the silicon, here is a PWM example : Reverse polish, so defines 47 then PIN to map which pin, and passes 3 parameters to Smart pin Sawtooth PWM mode.
This is 4/8 = 50% duty, SysCLK/1/8
I'd follow a standard if we were to implement one.
Cheers,
Jesse
Just want you to know that I did read your response yesterday and my gut reaction is to disagree. I'll hold off a lengthy response until my brain can come to terms with my gut. Maybe I'm just being stubborn and need time to come around.
ALSO! Anyone have a name suggestion for this thing?
Not sure which part you disagree with, but if it's the "let's not use user defined types in the API" I wasn't saying that we should avoid them in any particular implementation. If you want to define types for everything in PropWare go ahead! I just meant that in the language agnostic specification we shouldn't assume that the language has the ability to define and/or enforce types, and should use just a minimal set of types that are common to all languages.
That's a little bit of it, but I was already on the same page with "i can just write propware my own way" because that's a small enough deviation that I'm not concerned about it.
What I was more concerned with was whether or not to even specify types in the language-agnostic API (LAAPI? I'm getting really tired of typing that out). It comes down to: where do we want to draw the line. How much type information do we want to provide in the LAAPI? You seem to want to draw the line as basic primitive types (primitive for any language other than C, that is
I'm having a hard time getting behind anything other than no types or JSON types in the LAAPI ("lap-ee"? "L-A-A-P-I"? "lae-pee"? life is hard...). I think anything more specific than that should be a decision based on the language and no attempt should be made to standardize that across different languages.
Btw, I just want to re-iterate one thing: this idea of mine does include language-specific APIs as part of the standard. The LAAPI by itself is not the entire goal. I want a repository with the LAAPI and language-specific APIs for a few different languages. What my idea explicitly excludes is any executable/compilable code. Just APIs/prototypes/signatures/whatever you want to call them.
...................................
...................................
Also, I had a separate idea this morning. Once this thing reaches a point where people start writing their own HALs that implement the standard API, it's almost certain that someone will create something that matches some aspect of the standard but not all. It would be GREAT if we gave users a fast way to gauge how closely a specific HAL matches the standard. Here's my proposition:
Gold: The HAL uses the exact same prototypes as the language-specific API in the standard. This implies that code written for one gold-standard HAL can be copied/pasted and used with another gold-standard HAL of the same language without any errors (or if there are errors, those would be considered bugs worth reporting to the author... no one is perfect).
Silver: The HAL uses the same function names and parameter names as the language-specific API in the standard. Return values, if applicable, also match the API standard. All parameters appear in the same order as the language-specific API in the standard. Types for parameters may differ.
Bronze: The HAL drew inspiration from the standard but no guarantees are made about compatibility. Differences might include but are not limited to: package structure, object/function names, parameter types/names, parameter orders, etc.
These "badges" can and should be self-administered. I don't think anyone wants to be in charge of reading through someone else's HAL and giving them the "gold stamp of approval." If you're a HAL author, figure out for yourself whether you have gold/silver/bronze compatibility and document your library as such - no need to ask permission.
We're also going to want to version this project (semantic versioning but without the "patch" digit since this is only an API and can't have "bugs"). This will let people say "PropWare is a C++ HAL for the Parallax Propeller 2 with a Gold-standard rating on API v2.1."
Once the repo is up, people can start submitting pull requests. I'll make sure the README has clear instructions and/or examples for how submissions should be formatted.
I'm going to lock down the master branch so that all commits have to go through a PR, but I will hopefully be able to give permission to collaborators to approve PRs. What I want is that no one person should be changing the standard without agreement from some portion of the community. Maybe that's one other person. Maybe that's 10 other people. Maybe that's two specific people and one other person. No idea yet. I won't implement any limits at all right off the bat other than no committing directly to master.
I have high hopes for this!
PAPI for Propeller API...
Mike
Love it
-Phil
Interesting trivia. I don't know what to do with this information though lol. Is it meant simply as random trivia, or are you trying to push for or against the name?
-Phil
Noted. Thanks
Mike
It is synonymous to « grandpa » in French