The reason I commented on code documentation and the need for some sort of minimal standard is that some of the virtual or soft peripherals routines truly suck in that department. And is a lot of code self-documenting? No. When I first got started on the Prop and downloaded some SP's, it's was a exercise in head scratching, it was abundantly clear some of the coders wouldn't know documentation from a poison arrow frog.
IMO some sort of core Parallax approved VP or SP's are almost mandatory if they want to sell the PropI and Prop II to industry. Look it one thing to target hobbyists and curiosity seekers who will put up with routines with almost zero documentation which forces them to either re-write the routine in question or hopefully badger the original coder to cough up some answers. I don't think that potential corporate customers are going to be so forgiving when time is money for them.
I don't disagree on the need, just raising the bar to contribute. The code there is MIT license, which means "take it, and do as you will, and take it as is". That's because it's difficult to expect people to produce production quality code and contribute it for gratis. On the other hand, it's really easy to say there is a shared repository, contribute some, take some, no worries. Maintaining that is very important. That dynamic is the core of why there is a nice body of code in there in the first place.
Not that they won't do it to the nines. Some will, but some won't either. Lots of reasons too, so it's not a personal thing. Just depends on how people use their time and what value they place on various things, that's all.
Bottom line is, I want all the contributions, good, bad, well documented, code only, and everything in between. Worst case, somebody is faced with writing some code, or parsing through some that is written to learn how it works, and having done that, could very easily add that work back to the repository, same as the original contributor did. The license permits that, so no worries. If it's needed, those that need it can organize to do it, and do the work to make it easier for others to do it too. No worries on that either.
Forcing it changes things, which is why I advocate we promote and republish instead of raise the bar for contributions.
I wanted to drop a quick "thank you" for starting this thread. Its really cool that Parallax has people that care so much about the Propeller.
I was in a meeting on Tuesday with Ken and other members of the engineering group and made a short presentation about how to structure our new OBEX Gold Standard. After reading your posts in this thread, I made a PowerPoint slideshow to help bring your voices out. Check out the presentation, if you like - I have attached it.
For now, I think we will be sticking with the name "OBEX" and "object". Technically, code in the objects is instantiated at compile time. Not to mention that it would be difficult to change all references from object to software peripheral in all of the Propeller documentation and books that Parallax has published.
So far, I think that we need to define a standard for an OBEX Gold Object. This will include how to properly comment and document the object, a standard interface for similar objects (for drop in interoperability), and restrictions on new cog and lock declarations (to ensure interoperability among all gold objects).
I would also REALLY like to see a SourceForge-like presentation of all user submitted objects. This front end web interface would let people upload new revisions to objects - the idea being that someone could upload their code and others could submit changes, comment updates, and bug fixes. Users would also be able to rate and leave comments for uploaded versions. In this way, new objects can be fast tracked to gold certification.
Anybody would be able to submit an object for gold certification, but it must meet the standards defined by Parallax. Side note, I hope you guys will chime in with suggestions on what the standard will be. Also, I would like to see a Suggestions/Ideas page created where people can suggest an idea for an object they would like to see. Then, others who might already have code written or have the know how to write the object could do that.
I also emphasized support of gold objects. Prof_Braino brings up an excellent point that Parallax needs to take responsibility for objects that it is going to certify as bug free and completely working in all cases.
Let me know what you think about all of this. Questions, comments, suggestions - all are welcome. Im a big fan of advancing the state of things and everyone benefiting.
I would not state bug free and completely working in all cases. I would say, "supported". "bug free" is difficult given the wide range of use cases a gold object would be subjected to. Without a use case scope statement, "bug free" is falling into the "never say never" trap. Bad idea, IMHO.
"in all cases" only works again, "where all cases" is a defined set of cases; otherwise, the object should be "supported".
Normally, I wouldn't write this, but over time spent interacting with users in this space, I've seen considerable variation in what is a solid use case, and what might be fringe, or excessive, or just risky. Given that isn't very well nailed down, it is very important to leave Parallax, and potential object authors and supporters the "out" they need to insure that support is a value add, and not some blanket expectation that could prove burdensome and difficult to meet.
Edit: On a side note, I just have to chuckle when I see "quote from potatohead" on corporate powerpoint! It is a strange and wonderful community we have here. Thanks for that.
Great news Daniel.
I think the Gold Standard Objects should be just another part of the obex, so everyone realises it's there. If you make is so others can contribute (~sourceforge), make this the same in the current obex too.
I think perhaps there should be a few sections of the gold obex, just like the standard obex. Perhaps we could have a tree structure (in the standard obex too please). So, we could have...
Gold Obex
Generic Software Peripherals (just a basic driver) (or something along these lines to get the word out there that the prop has lots of flexible peripherals!)
UART
I2C
SPI
VGA
TV (Composite Video)
Audio
Specific Software Peripherals (includes intelligent/specific handling of the driver)
Serial (e.g. PST, FullDuplexSerial)
SD Card - with FAT and File Access
VGA - Text, Graphics, Tiles - various resolutions
TV (Composite Video) - Text, Graphics, Tiles - various resolutions
RTC
Counter Applications
PWM
ADC
etc
OS (Operating System) (various types of mini-operating systems)
Development Tools (Obviously PropTool & Propellent, but also possibly with bst, homespun, catalina, propbasic with the caveat these are not MIT. In other words, IMHO Parallax could endorse other objects and compilers/tools here that are not MIT.)
Code Snippets (IMHO, we should have some gold code snippets such as Bin -> String, etc; and we should also have them in the basic obex too. The best ones come here.)
Also, IMHO, not all objects may run under all languages. Some objects may require specific resources, that could normally be considered as not making it general enough for the gold standard. I do not believe this matters, as long as it is clearly stated what the restrictions are.
While I touched on the Props advantages, a gold demo with 8 UARTs, SD, SPI Flash, I2C RTC, TV (rather than VGA so we save some pins to spare), stereo out, a couple of switches and leds, just to show off the flexible peripherals.
The only problem with using source control (ALA sourceforge, etc...) is such tools are not well equipped to handle that silly Parallax UTF16 font. If you want to use SVN or other source control, it will mean eliminating the font. Think different
I'm actually pretty happy with OBEX as is.
But, I think some of the concerns here could be addressed if OBEX could be sorted by #downloads.
I think that's the real indication of usefulness...
Maybe the main OBEX page could list the top 25 downloads.
Actually, if the main OBEX page could add the 10 newest objects + the top 25 downloads, that would be great.
Ok, also I'd like to add a hot 5 list, that list the 5 most downloaded in the last month or so...
PS: While they're at it... If the Parallax webmasters could change the search tool so that hitting the "enter" key has the same effect as the "go" button, that would be great.
Hmmm. It's hard to get details on objects in the current OBEX without really drilling in.
- I think that object compatibility between languages is important.
- In addition to Cluso99's list, a FAT32 SD util w/ wav (DAC) would be good to have as a gold standard obj.
Yes, pleas put a high priority on language neutrality for all gold standard objects that are mainly a PASM core and a Spin interface wrapper.
It should be possible to use such PASM "cores" with any language / OS etc without needing that Spin wrapper. That is to say there is no "linkage" between Spin and the PASM core. Only PAR is used to set parameters in at start up and a HUB memory "mail" box used to exchange commands and data at run time.
Such cores can already be extracted with the BST compiler and HomeSpun I believe.
Whilst this may appear to be a put down to Spin, I think it will be even more important for the Prop II where other languages the BASICs and Cs become even more practical with it's larger HUB memory space.
I don't think language neutrality should be a prerequisite. A contributor may wish to indicate whether his contribution is language-neutral or not, but there are too many otherwise good objects that might be excluded by a neutrality requirement.
I would be totally against any review of objects other than the current system on the grounds that if objects are reviewed it would serve as an impediment to posting. Objects are posted by folks with all kinds of experience (and time to do it) the best review we have now is the download history and the stars. I do not wish to see that change.
I personally do not care if we get down to the file level source control. IMHO, that's complex for people who have not used it before. Not that they shouldn't learn --they should. It's a great thing. I personally would use it for more complex objects, and would prefer not to for simple ones, simply because the overhead wouldn't warrant the return.
Here's a thought. Sorry. Lots of those today. Scroll me, if you need to, because I know I'm chatty at the moment. One case for doing this would be to introduce users to revision control. Right now, with the simple tools we've got, people need to do this manually, and I don't think how to do that in a way that actually preserves code history is obvious.
It really should be, because the simple undo doesn't cut it, particularly if people are saving work, and need to revert. Once somebody "wants to go back to that block of code they had last thursday", realizing it's gone, they will be very interested in source control. I seriously wonder whether or not it makes sense to just offer that as a web service. Sign up to the forum, and get a little repository? Could be really great, but a lot of work on the back end. Still, I wonder whether or not it's practical to set that up in such a way that a user can just point it to their current project directory and have this largely done for them, or at least made structured and easy. It might seriously rock, compared to the usual, work from the bottom up, save as, edit OBJ block, repeat, save as dance done right now to accomplish this.
The archive function is a nice short cut, but not very pretty too. Maybe this is out of scope, but I'm gonna put it out there anyway, just because it's a very high value thing worth thinking about. It could contribute to the development of "gold" quality stuff too. Who knows?
Put that aside, and just having a interface where packages can be stored, with meta-data, such as revision, change log, contributing authors, and links to documentation and potentially discussion, on those packages would be a excellent minimum too.
The trade off for the font issue is rather good internal file documentation. Many people put schematics, tables and other stuff directly in the file, and I like this a lot! Plus, I like the font and how it reads. It's got issues, but not that many issues.
That said, some of the larger scale projects that are in progress, or that will begin to happen would be hobbled by this, particularly at the larger scale of Prop II. Strong case for both here for sure, IMHO.
This topic would benefit from some greater discussion of the trade-offs before some concrete path is taken.
Perhaps it makes sense to split the middle? A gold object presumably would come with documentation sufficient to render the advantages of the font moot, meaning ordinary ASCII could be used for source code, and that opens the door for revision control, Source Forge style. As long as simple, discrete packages are available, no worries. Just want to avoid having a user do a check out to access the code, as that's a barrier to entry for use of the code.
Same philosophy applies for use as for contributions. Somebody should be able to do the minimum and get something out of it. For a contribution, that means just sticking it in there. Done! Thanks! For a use, pulling out a package, unzipping it, and hitting compile is probably the useful minimum there.
As they learn, the other tools would make sense. Or, if they don't, and just are happy, then those tools are not a significant distraction.
One thing that really interested me in this stuff again was the low overhead. Back when I decided to jump back in (and I'm really glad I did, but my retro computers are sad as they haven't seen the play they once did), that low overhead really was a significant contributor. It was very easy to hook up the Demoboard I bought, load something, hit F10, and see it go. The graphics_demo.spin was the first thing I ran back then, and I ended up tinkering with it a lot to grok what Spin was all about, and did it again with PASM.
Addressing Pro class users, or just seasoned people, is a must. Make no mistake. The need is clear now. But, I really think paying attention to that overall barrier to entry is important.
IMHO, it's worth noting that a whole lot of this discussion really could not have happened earlier, because of the boot-strapping that has happened over the last few years. It quite literally took a while to sort out how to really use Propellers, and that is still happening right now. Look at the Catalina project, Zog, etc... LMM, XMM, and it's clear to me, we've evolved lots of core bits needed for this to really take off. Just want to put that out there. Not sure why. Maybe just to address the "why not sooner?" meme that is out there. It can happen now though, which is a darn good thing.
Re: Objects vs Virtual Peripherals.
Agreed in that a overall documentation revision is a PITA. Don't see a lot of value return on doing that either. Seems to me, some wrapper text, presented to the user of the OBEX, as well as a general expansion of the term "Virtual Peripheral" is worth more than passing thought.
I see a split where the current education targeted material remains more or less unchanged, with references added over time to get them across the bridge to "pro" land makes sense. In "pro" land, the use of "object" is just as likely to be a exception as it is a benefit. Consider using "virtual peripherial" in that context instead.
Because Parallax will build out a division, entity, whatever to address the "pro" class users, this makes great sense, and shouldn't really cause a lot of trouble, if the right expectations are set. Somebody walking in the "pro" door, finding that it's tough, can just walk over to the "educational" door, get up and rocking hard, then return, no worries, right?
On the other hand, the interested lay person, student, or part time semi pro can walk in the education door, and know it's education, and have the expectation set that the goal is to grow beyond that, step by step into pro land, where the little issues that simplify are stepping stones, not barriers or exceptions to those that are in a state of personal development to see them as such.
So the trick there is to just be perfectly open and honest about the split, why it exists and what the value of having it is. "Sell" that, and the rest is just cake, IMHO. Again, most of the people, more happy and productive more of the time.
Seems to me, that's the kind of feel that would unify a lot of this discussion making more of us happy more of the time, and honestly, from where we are now, that's as good as it gets.
Edit: (big, fat edit)
This also seems to speak to "Gold" objects having both a pro package, and educational package, each documented, or demonstrated in a way that makes the best sense overall. It's a bit more work, but for what I would call "foundation" objects, that work is worth it as a lot of higher level stuff will be built off of them, just as we have seen happen with the current set of "as shipped" Reference objects and schematics.
I also want to put the term "Reference" out there. I've used it off and on in my own writings and adventures here, and it's got some merit. A "Reference Implementation" of a virtual peripheral, or object, is a solid use case all around, balanced, appropriate for use, or building, depending on what the target use case happens to be.
What I like about this is the expectations are set correctly without a lot of explanatory text being required. Nobody uses the reference case for bleeding edge work. On the other hand, somebody looking for overall direction, would totally go for the reference case, so it all just kind of works. Would like to see that concept embodied in this somehow, because it adds a lot of value without a lot of work.
Finally, and it seems I'm writing a small book, some discussion on the method of communication between objects needs to be done.
I think it's pretty safe to say the majority of current active users are either rapidly seeing the advantage of a binary object, where possible and practical, or are already there trying to build new ones, or retool existing ones. The binary object communication has seen a few different variations:
1. The mailbox method, where a few longs are used to communicate.
2. The par block method, where a set of HUB memory locations are passed and use to communicate.
There are other methods too, involving pins, locks, etc... I highlighted those two, because they are potent, in use now, and not at all difficult. Others may be warranted. I just don't have that visibility in my own adventures here so far, other than the one used in graphics.spin, which appears like a combination of the two to me, with a mailbox taking commands, and a par block communicating other details needed for it all to work.
My only reason for commenting on this is that each method has it's advantages, and COG code requirements. For things like video, the par block is very nice, as often those parameters are "live" where the driver will act on them right then and there, changing a color definition, or screen buffer location, etc... Operates a lot like a memory mapped graphics chip.
I've seen the mailbox play well too, for things like compute objects that do math, etc...
Unifying these may not always be possible. Some discussion of each is probably warranted, just as a overall bit of documentation that would serve to significantly reduce the amount of required documentation for each individual "Gold" object. Common terminology, and a reference implementation, in the form of code snippets and examples could be reviewed by object authors, or used by object promoters in their work to create new, or improved "gold" resources. Maybe just those two, and incorporate what graphics.spin does in tandem with TV and VGA.spin, as a general case example.
(yes, I said resources, because I am just letting some terminology slide, until such time as it's better realized by everyone as a whole)
Along with the "Gold" requirements then, should come one reference "Gold" object, demonstrating both techniques, or a few could demonstrate the techniques, and how to interoperate at the user code level. IMHO, I've not seen where these are mutually exclusive either, meaning it's probably a solid expectation to see one or more communication techniques in use. Limiting this to one method might be overly constraining too, and I'm very curious to hear what everybody has to say about that aspect of things.
Agreed with Phil, BTW. Some contributors may be constrained on time and skill, unable or unwilling to commit to a full language neutral contribution. That can be added by others over time, and probably will be, given we structure this effort correctly. That's the inherent beauty of the MIT license. Anyone and everyone gets to do that. So far, we've not seen it done too much. Probably that's just because everybody works together well and is active, but that will eventually change as people come and go. Set the expectation that additions to a objects, or a change, review, etc... isn't a bad thing, and the language ports will happen organically, IMHO.
Yes, pleas put a high priority on language neutrality for all gold standard objects that are mainly a PASM core and a Spin interface wrapper.
This requires clear thinking and some coding. coffee ++ and alcohol --
Ok, first bit of code, untested, is a demo cog. I want to put an array in hub memory, put a number in the first element of the array, then call this cog code and have it return the second item in the array as equal to the first item plus one. The array is an array of longs. Does this code look ok?
CON
_clkfreq = 80_000_000
_clkmode = xtal1 + pll16x
PUB Main
coginit(1,@entry,0) ' cog 1, entry, dummy value
DAT
org 0
entry ' start of the cog code
mov arraypointer,par ' arraypointer is the address of the array
rdlong array0,arraypointer ' array0 = array[0]
mov array1,array0 ' array[1]=array[0]
add array1,#1 ' array[1] += 1 change the value so see cog works
add arraypointer,#1 ' pointer += 1
wrlong array1,arraypointer ' write array[1] back to hub
jmp #entry
arraypointer long $00000000 ' pointer to array[0]
array0 long $00000000 ' first array location
array1 long $00000000 ' second array location
fit 496
Next is a way to read this. I have written some C code that takes this, turns it into a hex array and pastes it back into the C program (not with this exact pasm code, for those that can read hex!)
void mycogject(int cognumber, unsigned long parameters) // this name copied from the .spin name in the pasm section - names must match eg void mycogject matches mycogject.spin. Also first code after this must be the .h array file. Put your code after the };
{
/**
* @file mycogject_array.h
* Created with spin.binary PASM to C Array Converter.
* Copyright (c) 2011, John Doe
*/
unsigned long mycogject_array[] = // dummy data or compiled above code or #include file.h
{
0x003c0602, 0x5c7c0000, 0x00001388, 0x00000041
};
unsigned long par_cogject[] = { 1, 2, 3 }; // data to pass to cog - ignore if not used
_coginit((int)par_cogject>>2, (int)mycogject_array>>2, cognumber); // array name built from spin file name
}
But reading through this threat, I am no longer sure this is the best approach. The code is now tied to C.
Maybe one could think about converting to hex formats for Spin, C, Basic ($, 0x &H)
But I have also been looking at an approach that creates binary files that are on an sd card. Compile the above code to a binary and rename it as .cog, and put it on a card. I've got Kyedos v3 with xmodem, and I have an IDE that can do the compilation and download with one keypress. So getting .cog files onto an sd card is now quick and simple.
Next challenge is getting that code into a program. The application here is mainly for external memory programs. If you are running an external memory program, it will generally be a big program so it will need to be loaded off an sd card and into external memory. And so if you are already doing that, then there is no real extra cost in loading in .cog files separately rather than as part of the big program.
So the next bit of code is to be able to read binary files. Kyedos has some examples of doing this in Spin. In C, this is a little program that creates "Hello World" as a text file, and then reads it back as a binary file.
#include <stdio.h>
// *************************************************
// Typedef
// *************************************************
typedef unsigned char UCHAR;
typedef unsigned long DWORD;
typedef unsigned long UINT;
// *************************************************
// System Variables
// *************************************************
// *************************************************
// User Global Variables
// *************************************************
static FILE *FP1;
// *************************************************
// Standard Prototypes
// *************************************************
int EoF (FILE*);
char* BCX_TmpStr(size_t);
// *************************************************
// User Prototypes
// *************************************************
void ReadFile (char *);
void CreateFile (char *);
// *************************************************
// User Global Initialized Arrays
// *************************************************
// *************************************************
// Main Program
// *************************************************
int main(int argc, char *argv[])
{
CreateFile("new.txt");
ReadFile("new.txt");
return 0; // End of main program
}
// *************************************************
// Runtime Functions
// *************************************************
char *BCX_TmpStr (size_t Bites)
{
static int StrCnt;
static char *StrFunc[2048];
StrCnt=(StrCnt + 1) & 2047;
if(StrFunc[StrCnt]) free (StrFunc[StrCnt]);
return StrFunc[StrCnt]=(char*)calloc(Bites+128,sizeof(char));
}
int EoF (FILE* stream)
{
register int c, status = ((c = fgetc(stream)) == EOF);
ungetc(c,stream);
return status;
}
// ************************************
// User Subs and Functions
// ************************************
void ReadFile (char *Filename)
{
char a;
memset(&a,0,sizeof(a));
printf("%s\n","now read file back as a binary file");
if((FP1=fopen(Filename,"rb"))==0)
{
fprintf(stderr,"Can't open file %s\n",Filename);exit(1);
}
fseek(FP1,0,0);
while(!EoF(FP1))
{
a=getc(FP1);
printf("% d\n",(int)a);
}
if(FP1)
{
fclose(FP1);
FP1=NULL;
}
}
void CreateFile (char *Filename)
{
printf("%s\n","Open file and create file");
static FILE* MyFileHandle;
memset(&MyFileHandle,0,sizeof(MyFileHandle));
if((FP1=fopen(Filename,"w"))==0)
{
fprintf(stderr,"Can't open file %s\n",Filename);exit(1);
}
MyFileHandle=FP1;
fprintf(MyFileHandle,"%s\n","Hello World");
if(FP1)
{
fclose(FP1);
FP1=NULL;
}
printf("%s\n","Closing file");
}
and this is the same program in Basic
' ************ main **********
CreateFile("new.txt")
ReadFile("new.txt")
' ************ end main **********
Sub ReadFile(Filename as String)
dim a as char
print "now read file back as a binary file"
OPEN Filename FOR BINARY INPUT AS FP1
Seek FP1,0 ' start at beginning
While Not EOF(FP1)
a=getc(FP1)
print a
Wend
Close FP1
End Sub
Sub CreateFile(Filename as String)
print "Open file and create file"
DIM MyFileHandle@ ' BCX Reserves @ for "C" FILE* data types
OPEN Filename FOR OUTPUT AS FP1
MyFileHandle@ = FP1
FPRINT MyFileHandle@, "Hello World"
Close FP1
print "Closing file"
End Sub
The rules for a .cog program are very simple. You get to pass one long which the cog code then reads with "par" and if required, reads in an array of data (which might contain pointers to other variables or arrays).
On the Spin/Big Spin/C/Basic side, variables are passed in an array. Many objects do this already, but sometimes don't explicitly say this is what they are doing. Sometimes you get a clue, as in a comment saying "don't change the order of these variables".
Maybe we could think about the "gold standard" containing clearer comments about why the order should not be changed? And how many variables are in the list?
So on the Spin side of the code, maybe it is just a matter of better commenting rather than necessarily changing code? Or - perhaps you put the values into an array?
Ok, first bit of code, untested, is a demo cog. I want to put an array in hub memory, put a number in the first element of the array, then call this cog code and have it return the second item in the array as equal to the first item plus one. The array is an array of longs. Does this code look ok?
CON
_clkfreq = 80_000_000
_clkmode = xtal1 + pll16x
PUB Main
coginit(1,@entry,0) ' cog 1, entry, dummy value
DAT
org 0
entry ' start of the cog code
mov arraypointer,par ' arraypointer is the address of the array
rdlong array0,arraypointer ' array0 = array[0]
mov array1,array0 ' array[1]=array[0]
add array1,#1 ' array[1] += 1 change the value so see cog works
[COLOR="red"]add arraypointer,#4 ' pointer += 4[/COLOR]
wrlong array1,arraypointer ' write array[1] back to hub
jmp #entry
arraypointer long $00000000 ' pointer to array[0]
array0 long $00000000 ' first array location
array1 long $00000000 ' second array location
fit 496
One minor thing, hub RAM is byte addressed. So in order to get to the next long you want to add #4.
I agree with Phil & potatohead. I would hate to see any impediment to posting objects and also for objects to be considered Gold. If there are restrictions, then they should be clearly stated, but that is it. Why, because it could be some great Gold Object just does not work for say, Basic. What about a Gold "Forth" Object? Clearly it is not platform compliant. Preference yes, but requirement no.
There are a lot more objects in these forums than are in the Obex.
ptoatohead: your analysis is great. There must be levels of objects. Take your older TV Driver as an example. Originally it was well documented and easy to read. I came along and re-ordered the code to get it faster so you could get more characters per line. Now it is unreadable for many. Which one do we choose??? Well, obviously the latter for functionality, but for education, the former is better. So a classification needs to be done for beginner, experienced, expert. Just because you cannot read an object easily doesn't mean it is no good. As we push the envelope, it becomes quite difficult to understand what is happening. It does not really matter, as long as it works. For example, do you know how your ABS braking works - no, just the basics because you use it every day, and to use it you do not need to understand the how-it-works.
Another example is with FPGA's. You may buy an IP Block. This does not mean you will get the source code, just a running code. Same here.
The other thing coming clearly through is the need to mention "Virtual Peripherals" or similar. So, this needs to be a separate section in the Obex so we can try and get this across. A virtual peripheral will most likely be a pasm driver and hopefully operable amongst all languages. For example, it would really help to have ONE Gold low-level fast pasm driver for the SD card, instead of the many we currently have. The next layer can take care of other complexities. Put as much as we can into this bottom layer.
Parallax has control over the libraries and examples that are packaged with the Propeller tool. Putting onerous burdens on OBEX contributors will stymie creativity of people who wish to share there. The gold standard is what Parallax chooses it to be.
I have no problems with the way the OBEX exchage currentlly setup and how Parallax controls it. It wouldn't be a bad idea to have a peer review board for checking code. But, we need to remember that the code posted by someone else is used as is, as it stated in the MIT licence. That's why I like what Potatohead stated this
@Cluso: Yes. Exactly. I actually was thinking of that case too. Funny, on the next revision, it's a significantly more readable readable driver(IMHO), and it's still fast, and smaller. Another aspect of the MIT license. Might not have been me that did the next one, could have been somebody else. I tried doing a "kernel" this time, isolating the core problem in a generic context, and Eric was able to provide a superior solution fairly quickly. it was less trouble to do that, than it was to build something up, only to tear into it near the end, like we did the first time. Dev time and readability appear to both be positive experiences this time.
@bsnut: Another yes, exactly. Not everybody who contributes may be able to, or willing to support those contributions over extended periods of time. They may well just toss it in there, and let others do what they will. I think that's a great expectation to set. People are free to be active, but should not be required to be active. That's a barrier, IMHO. That dynamic is the core of why I advocate we adopt a promote and republish model as the core of this OBEX change / expansion.
Comments
The reason I commented on code documentation and the need for some sort of minimal standard is that some of the virtual or soft peripherals routines truly suck in that department. And is a lot of code self-documenting? No. When I first got started on the Prop and downloaded some SP's, it's was a exercise in head scratching, it was abundantly clear some of the coders wouldn't know documentation from a poison arrow frog.
IMO some sort of core Parallax approved VP or SP's are almost mandatory if they want to sell the PropI and Prop II to industry. Look it one thing to target hobbyists and curiosity seekers who will put up with routines with almost zero documentation which forces them to either re-write the routine in question or hopefully badger the original coder to cough up some answers. I don't think that potential corporate customers are going to be so forgiving when time is money for them.
Not that they won't do it to the nines. Some will, but some won't either. Lots of reasons too, so it's not a personal thing. Just depends on how people use their time and what value they place on various things, that's all.
Bottom line is, I want all the contributions, good, bad, well documented, code only, and everything in between. Worst case, somebody is faced with writing some code, or parsing through some that is written to learn how it works, and having done that, could very easily add that work back to the repository, same as the original contributor did. The license permits that, so no worries. If it's needed, those that need it can organize to do it, and do the work to make it easier for others to do it too. No worries on that either.
Forcing it changes things, which is why I advocate we promote and republish instead of raise the bar for contributions.
I wanted to drop a quick "thank you" for starting this thread. Its really cool that Parallax has people that care so much about the Propeller.
I was in a meeting on Tuesday with Ken and other members of the engineering group and made a short presentation about how to structure our new OBEX Gold Standard. After reading your posts in this thread, I made a PowerPoint slideshow to help bring your voices out. Check out the presentation, if you like - I have attached it.
For now, I think we will be sticking with the name "OBEX" and "object". Technically, code in the objects is instantiated at compile time. Not to mention that it would be difficult to change all references from object to software peripheral in all of the Propeller documentation and books that Parallax has published.
So far, I think that we need to define a standard for an OBEX Gold Object. This will include how to properly comment and document the object, a standard interface for similar objects (for drop in interoperability), and restrictions on new cog and lock declarations (to ensure interoperability among all gold objects).
I would also REALLY like to see a SourceForge-like presentation of all user submitted objects. This front end web interface would let people upload new revisions to objects - the idea being that someone could upload their code and others could submit changes, comment updates, and bug fixes. Users would also be able to rate and leave comments for uploaded versions. In this way, new objects can be fast tracked to gold certification.
Anybody would be able to submit an object for gold certification, but it must meet the standards defined by Parallax. Side note, I hope you guys will chime in with suggestions on what the standard will be. Also, I would like to see a Suggestions/Ideas page created where people can suggest an idea for an object they would like to see. Then, others who might already have code written or have the know how to write the object could do that.
I also emphasized support of gold objects. Prof_Braino brings up an excellent point that Parallax needs to take responsibility for objects that it is going to certify as bug free and completely working in all cases.
Let me know what you think about all of this. Questions, comments, suggestions - all are welcome. Im a big fan of advancing the state of things and everyone benefiting.
I would not state bug free and completely working in all cases. I would say, "supported". "bug free" is difficult given the wide range of use cases a gold object would be subjected to. Without a use case scope statement, "bug free" is falling into the "never say never" trap. Bad idea, IMHO.
"in all cases" only works again, "where all cases" is a defined set of cases; otherwise, the object should be "supported".
Normally, I wouldn't write this, but over time spent interacting with users in this space, I've seen considerable variation in what is a solid use case, and what might be fringe, or excessive, or just risky. Given that isn't very well nailed down, it is very important to leave Parallax, and potential object authors and supporters the "out" they need to insure that support is a value add, and not some blanket expectation that could prove burdensome and difficult to meet.
Edit: On a side note, I just have to chuckle when I see "quote from potatohead" on corporate powerpoint! It is a strange and wonderful community we have here. Thanks for that.
I think the Gold Standard Objects should be just another part of the obex, so everyone realises it's there. If you make is so others can contribute (~sourceforge), make this the same in the current obex too.
I think perhaps there should be a few sections of the gold obex, just like the standard obex. Perhaps we could have a tree structure (in the standard obex too please). So, we could have...
- Gold Obex
- Generic Software Peripherals (just a basic driver) (or something along these lines to get the word out there that the prop has lots of flexible peripherals!)
- UART
- I2C
- SPI
- VGA
- TV (Composite Video)
- Audio
- Specific Software Peripherals (includes intelligent/specific handling of the driver)
- Serial (e.g. PST, FullDuplexSerial)
- SD Card - with FAT and File Access
- VGA - Text, Graphics, Tiles - various resolutions
- TV (Composite Video) - Text, Graphics, Tiles - various resolutions
- RTC
- Counter Applications
- PWM
- ADC
- etc
- OS (Operating System) (various types of mini-operating systems)
- Development Tools (Obviously PropTool & Propellent, but also possibly with bst, homespun, catalina, propbasic with the caveat these are not MIT. In other words, IMHO Parallax could endorse other objects and compilers/tools here that are not MIT.)
- Code Snippets (IMHO, we should have some gold code snippets such as Bin -> String, etc; and we should also have them in the basic obex too. The best ones come here.)
Also, IMHO, not all objects may run under all languages. Some objects may require specific resources, that could normally be considered as not making it general enough for the gold standard. I do not believe this matters, as long as it is clearly stated what the restrictions are.While I touched on the Props advantages, a gold demo with 8 UARTs, SD, SPI Flash, I2C RTC, TV (rather than VGA so we save some pins to spare), stereo out, a couple of switches and leds, just to show off the flexible peripherals.
But, I think some of the concerns here could be addressed if OBEX could be sorted by #downloads.
I think that's the real indication of usefulness...
Maybe the main OBEX page could list the top 25 downloads.
Ok, also I'd like to add a hot 5 list, that list the 5 most downloaded in the last month or so...
PS: While they're at it... If the Parallax webmasters could change the search tool so that hitting the "enter" key has the same effect as the "go" button, that would be great.
- I think that object compatibility between languages is important.
- In addition to Cluso99's list, a FAT32 SD util w/ wav (DAC) would be good to have as a gold standard obj.
It should be possible to use such PASM "cores" with any language / OS etc without needing that Spin wrapper. That is to say there is no "linkage" between Spin and the PASM core. Only PAR is used to set parameters in at start up and a HUB memory "mail" box used to exchange commands and data at run time.
Such cores can already be extracted with the BST compiler and HomeSpun I believe.
Whilst this may appear to be a put down to Spin, I think it will be even more important for the Prop II where other languages the BASICs and Cs become even more practical with it's larger HUB memory space.
-Phil
I personally do not care if we get down to the file level source control. IMHO, that's complex for people who have not used it before. Not that they shouldn't learn --they should. It's a great thing. I personally would use it for more complex objects, and would prefer not to for simple ones, simply because the overhead wouldn't warrant the return.
Here's a thought. Sorry. Lots of those today. Scroll me, if you need to, because I know I'm chatty at the moment. One case for doing this would be to introduce users to revision control. Right now, with the simple tools we've got, people need to do this manually, and I don't think how to do that in a way that actually preserves code history is obvious.
It really should be, because the simple undo doesn't cut it, particularly if people are saving work, and need to revert. Once somebody "wants to go back to that block of code they had last thursday", realizing it's gone, they will be very interested in source control. I seriously wonder whether or not it makes sense to just offer that as a web service. Sign up to the forum, and get a little repository? Could be really great, but a lot of work on the back end. Still, I wonder whether or not it's practical to set that up in such a way that a user can just point it to their current project directory and have this largely done for them, or at least made structured and easy. It might seriously rock, compared to the usual, work from the bottom up, save as, edit OBJ block, repeat, save as dance done right now to accomplish this.
The archive function is a nice short cut, but not very pretty too. Maybe this is out of scope, but I'm gonna put it out there anyway, just because it's a very high value thing worth thinking about. It could contribute to the development of "gold" quality stuff too. Who knows?
Put that aside, and just having a interface where packages can be stored, with meta-data, such as revision, change log, contributing authors, and links to documentation and potentially discussion, on those packages would be a excellent minimum too.
The trade off for the font issue is rather good internal file documentation. Many people put schematics, tables and other stuff directly in the file, and I like this a lot! Plus, I like the font and how it reads. It's got issues, but not that many issues.
That said, some of the larger scale projects that are in progress, or that will begin to happen would be hobbled by this, particularly at the larger scale of Prop II. Strong case for both here for sure, IMHO.
This topic would benefit from some greater discussion of the trade-offs before some concrete path is taken.
Perhaps it makes sense to split the middle? A gold object presumably would come with documentation sufficient to render the advantages of the font moot, meaning ordinary ASCII could be used for source code, and that opens the door for revision control, Source Forge style. As long as simple, discrete packages are available, no worries. Just want to avoid having a user do a check out to access the code, as that's a barrier to entry for use of the code.
Same philosophy applies for use as for contributions. Somebody should be able to do the minimum and get something out of it. For a contribution, that means just sticking it in there. Done! Thanks! For a use, pulling out a package, unzipping it, and hitting compile is probably the useful minimum there.
As they learn, the other tools would make sense. Or, if they don't, and just are happy, then those tools are not a significant distraction.
One thing that really interested me in this stuff again was the low overhead. Back when I decided to jump back in (and I'm really glad I did, but my retro computers are sad as they haven't seen the play they once did), that low overhead really was a significant contributor. It was very easy to hook up the Demoboard I bought, load something, hit F10, and see it go. The graphics_demo.spin was the first thing I ran back then, and I ended up tinkering with it a lot to grok what Spin was all about, and did it again with PASM.
Addressing Pro class users, or just seasoned people, is a must. Make no mistake. The need is clear now. But, I really think paying attention to that overall barrier to entry is important.
IMHO, it's worth noting that a whole lot of this discussion really could not have happened earlier, because of the boot-strapping that has happened over the last few years. It quite literally took a while to sort out how to really use Propellers, and that is still happening right now. Look at the Catalina project, Zog, etc... LMM, XMM, and it's clear to me, we've evolved lots of core bits needed for this to really take off. Just want to put that out there. Not sure why. Maybe just to address the "why not sooner?" meme that is out there. It can happen now though, which is a darn good thing.
Re: Objects vs Virtual Peripherals.
Agreed in that a overall documentation revision is a PITA. Don't see a lot of value return on doing that either. Seems to me, some wrapper text, presented to the user of the OBEX, as well as a general expansion of the term "Virtual Peripheral" is worth more than passing thought.
I see a split where the current education targeted material remains more or less unchanged, with references added over time to get them across the bridge to "pro" land makes sense. In "pro" land, the use of "object" is just as likely to be a exception as it is a benefit. Consider using "virtual peripherial" in that context instead.
Because Parallax will build out a division, entity, whatever to address the "pro" class users, this makes great sense, and shouldn't really cause a lot of trouble, if the right expectations are set. Somebody walking in the "pro" door, finding that it's tough, can just walk over to the "educational" door, get up and rocking hard, then return, no worries, right?
On the other hand, the interested lay person, student, or part time semi pro can walk in the education door, and know it's education, and have the expectation set that the goal is to grow beyond that, step by step into pro land, where the little issues that simplify are stepping stones, not barriers or exceptions to those that are in a state of personal development to see them as such.
So the trick there is to just be perfectly open and honest about the split, why it exists and what the value of having it is. "Sell" that, and the rest is just cake, IMHO. Again, most of the people, more happy and productive more of the time.
Seems to me, that's the kind of feel that would unify a lot of this discussion making more of us happy more of the time, and honestly, from where we are now, that's as good as it gets.
Edit: (big, fat edit)
This also seems to speak to "Gold" objects having both a pro package, and educational package, each documented, or demonstrated in a way that makes the best sense overall. It's a bit more work, but for what I would call "foundation" objects, that work is worth it as a lot of higher level stuff will be built off of them, just as we have seen happen with the current set of "as shipped" Reference objects and schematics.
I also want to put the term "Reference" out there. I've used it off and on in my own writings and adventures here, and it's got some merit. A "Reference Implementation" of a virtual peripheral, or object, is a solid use case all around, balanced, appropriate for use, or building, depending on what the target use case happens to be.
What I like about this is the expectations are set correctly without a lot of explanatory text being required. Nobody uses the reference case for bleeding edge work. On the other hand, somebody looking for overall direction, would totally go for the reference case, so it all just kind of works. Would like to see that concept embodied in this somehow, because it adds a lot of value without a lot of work.
Finally, and it seems I'm writing a small book, some discussion on the method of communication between objects needs to be done.
I think it's pretty safe to say the majority of current active users are either rapidly seeing the advantage of a binary object, where possible and practical, or are already there trying to build new ones, or retool existing ones. The binary object communication has seen a few different variations:
1. The mailbox method, where a few longs are used to communicate.
2. The par block method, where a set of HUB memory locations are passed and use to communicate.
There are other methods too, involving pins, locks, etc... I highlighted those two, because they are potent, in use now, and not at all difficult. Others may be warranted. I just don't have that visibility in my own adventures here so far, other than the one used in graphics.spin, which appears like a combination of the two to me, with a mailbox taking commands, and a par block communicating other details needed for it all to work.
My only reason for commenting on this is that each method has it's advantages, and COG code requirements. For things like video, the par block is very nice, as often those parameters are "live" where the driver will act on them right then and there, changing a color definition, or screen buffer location, etc... Operates a lot like a memory mapped graphics chip.
I've seen the mailbox play well too, for things like compute objects that do math, etc...
Unifying these may not always be possible. Some discussion of each is probably warranted, just as a overall bit of documentation that would serve to significantly reduce the amount of required documentation for each individual "Gold" object. Common terminology, and a reference implementation, in the form of code snippets and examples could be reviewed by object authors, or used by object promoters in their work to create new, or improved "gold" resources. Maybe just those two, and incorporate what graphics.spin does in tandem with TV and VGA.spin, as a general case example.
(yes, I said resources, because I am just letting some terminology slide, until such time as it's better realized by everyone as a whole)
Along with the "Gold" requirements then, should come one reference "Gold" object, demonstrating both techniques, or a few could demonstrate the techniques, and how to interoperate at the user code level. IMHO, I've not seen where these are mutually exclusive either, meaning it's probably a solid expectation to see one or more communication techniques in use. Limiting this to one method might be overly constraining too, and I'm very curious to hear what everybody has to say about that aspect of things.
This requires clear thinking and some coding. coffee ++ and alcohol --
Ok, first bit of code, untested, is a demo cog. I want to put an array in hub memory, put a number in the first element of the array, then call this cog code and have it return the second item in the array as equal to the first item plus one. The array is an array of longs. Does this code look ok?
CON _clkfreq = 80_000_000 _clkmode = xtal1 + pll16x PUB Main coginit(1,@entry,0) ' cog 1, entry, dummy value DAT org 0 entry ' start of the cog code mov arraypointer,par ' arraypointer is the address of the array rdlong array0,arraypointer ' array0 = array[0] mov array1,array0 ' array[1]=array[0] add array1,#1 ' array[1] += 1 change the value so see cog works add arraypointer,#1 ' pointer += 1 wrlong array1,arraypointer ' write array[1] back to hub jmp #entry arraypointer long $00000000 ' pointer to array[0] array0 long $00000000 ' first array location array1 long $00000000 ' second array location fit 496Next is a way to read this. I have written some C code that takes this, turns it into a hex array and pastes it back into the C program (not with this exact pasm code, for those that can read hex!)
void mycogject(int cognumber, unsigned long parameters) // this name copied from the .spin name in the pasm section - names must match eg void mycogject matches mycogject.spin. Also first code after this must be the .h array file. Put your code after the }; { /** * @file mycogject_array.h * Created with spin.binary PASM to C Array Converter. * Copyright (c) 2011, John Doe */ unsigned long mycogject_array[] = // dummy data or compiled above code or #include file.h { 0x003c0602, 0x5c7c0000, 0x00001388, 0x00000041 }; unsigned long par_cogject[] = { 1, 2, 3 }; // data to pass to cog - ignore if not used _coginit((int)par_cogject>>2, (int)mycogject_array>>2, cognumber); // array name built from spin file name }But reading through this threat, I am no longer sure this is the best approach. The code is now tied to C.
Maybe one could think about converting to hex formats for Spin, C, Basic ($, 0x &H)
But I have also been looking at an approach that creates binary files that are on an sd card. Compile the above code to a binary and rename it as .cog, and put it on a card. I've got Kyedos v3 with xmodem, and I have an IDE that can do the compilation and download with one keypress. So getting .cog files onto an sd card is now quick and simple.
Next challenge is getting that code into a program. The application here is mainly for external memory programs. If you are running an external memory program, it will generally be a big program so it will need to be loaded off an sd card and into external memory. And so if you are already doing that, then there is no real extra cost in loading in .cog files separately rather than as part of the big program.
So the next bit of code is to be able to read binary files. Kyedos has some examples of doing this in Spin. In C, this is a little program that creates "Hello World" as a text file, and then reads it back as a binary file.
#include <stdio.h> // ************************************************* // Typedef // ************************************************* typedef unsigned char UCHAR; typedef unsigned long DWORD; typedef unsigned long UINT; // ************************************************* // System Variables // ************************************************* // ************************************************* // User Global Variables // ************************************************* static FILE *FP1; // ************************************************* // Standard Prototypes // ************************************************* int EoF (FILE*); char* BCX_TmpStr(size_t); // ************************************************* // User Prototypes // ************************************************* void ReadFile (char *); void CreateFile (char *); // ************************************************* // User Global Initialized Arrays // ************************************************* // ************************************************* // Main Program // ************************************************* int main(int argc, char *argv[]) { CreateFile("new.txt"); ReadFile("new.txt"); return 0; // End of main program } // ************************************************* // Runtime Functions // ************************************************* char *BCX_TmpStr (size_t Bites) { static int StrCnt; static char *StrFunc[2048]; StrCnt=(StrCnt + 1) & 2047; if(StrFunc[StrCnt]) free (StrFunc[StrCnt]); return StrFunc[StrCnt]=(char*)calloc(Bites+128,sizeof(char)); } int EoF (FILE* stream) { register int c, status = ((c = fgetc(stream)) == EOF); ungetc(c,stream); return status; } // ************************************ // User Subs and Functions // ************************************ void ReadFile (char *Filename) { char a; memset(&a,0,sizeof(a)); printf("%s\n","now read file back as a binary file"); if((FP1=fopen(Filename,"rb"))==0) { fprintf(stderr,"Can't open file %s\n",Filename);exit(1); } fseek(FP1,0,0); while(!EoF(FP1)) { a=getc(FP1); printf("% d\n",(int)a); } if(FP1) { fclose(FP1); FP1=NULL; } } void CreateFile (char *Filename) { printf("%s\n","Open file and create file"); static FILE* MyFileHandle; memset(&MyFileHandle,0,sizeof(MyFileHandle)); if((FP1=fopen(Filename,"w"))==0) { fprintf(stderr,"Can't open file %s\n",Filename);exit(1); } MyFileHandle=FP1; fprintf(MyFileHandle,"%s\n","Hello World"); if(FP1) { fclose(FP1); FP1=NULL; } printf("%s\n","Closing file"); }and this is the same program in Basic
' ************ main ********** CreateFile("new.txt") ReadFile("new.txt") ' ************ end main ********** Sub ReadFile(Filename as String) dim a as char print "now read file back as a binary file" OPEN Filename FOR BINARY INPUT AS FP1 Seek FP1,0 ' start at beginning While Not EOF(FP1) a=getc(FP1) print a Wend Close FP1 End Sub Sub CreateFile(Filename as String) print "Open file and create file" DIM MyFileHandle@ ' BCX Reserves @ for "C" FILE* data types OPEN Filename FOR OUTPUT AS FP1 MyFileHandle@ = FP1 FPRINT MyFileHandle@, "Hello World" Close FP1 print "Closing file" End SubThe rules for a .cog program are very simple. You get to pass one long which the cog code then reads with "par" and if required, reads in an array of data (which might contain pointers to other variables or arrays).
On the Spin/Big Spin/C/Basic side, variables are passed in an array. Many objects do this already, but sometimes don't explicitly say this is what they are doing. Sometimes you get a clue, as in a comment saying "don't change the order of these variables".
Maybe we could think about the "gold standard" containing clearer comments about why the order should not be changed? And how many variables are in the list?
So on the Spin side of the code, maybe it is just a matter of better commenting rather than necessarily changing code? Or - perhaps you put the values into an array?
There are a lot more objects in these forums than are in the Obex.
ptoatohead: your analysis is great. There must be levels of objects. Take your older TV Driver as an example. Originally it was well documented and easy to read. I came along and re-ordered the code to get it faster so you could get more characters per line. Now it is unreadable for many. Which one do we choose??? Well, obviously the latter for functionality, but for education, the former is better. So a classification needs to be done for beginner, experienced, expert. Just because you cannot read an object easily doesn't mean it is no good. As we push the envelope, it becomes quite difficult to understand what is happening. It does not really matter, as long as it works. For example, do you know how your ABS braking works - no, just the basics because you use it every day, and to use it you do not need to understand the how-it-works.
Another example is with FPGA's. You may buy an IP Block. This does not mean you will get the source code, just a running code. Same here.
The other thing coming clearly through is the need to mention "Virtual Peripherals" or similar. So, this needs to be a separate section in the Obex so we can try and get this across. A virtual peripheral will most likely be a pasm driver and hopefully operable amongst all languages. For example, it would really help to have ONE Gold low-level fast pasm driver for the SD card, instead of the many we currently have. The next layer can take care of other complexities. Put as much as we can into this bottom layer.
@bsnut: Another yes, exactly. Not everybody who contributes may be able to, or willing to support those contributions over extended periods of time. They may well just toss it in there, and let others do what they will. I think that's a great expectation to set. People are free to be active, but should not be required to be active. That's a barrier, IMHO. That dynamic is the core of why I advocate we adopt a promote and republish model as the core of this OBEX change / expansion.
I agree with you 100% on your point.