My attempt at a "Gold Standard" process
Circuitsoft
Posts: 1,166
So, I've set up Gerrit code review on my home server. You can access it at http://vdubshouse.zapto.org/parallax/. As a first change, I submitted a very basic README file and, like any other commits, would like comments/suggestions before fully committing it.
So, I'll do my best to watch this thread, but Gerrit has conversation threads associated with each change, so code discussions should happen there.
Thanks, everyone, and if you have any questions, please post them here.
Oh, and, should this gain traction, there should be little difficulty moving the entire thing into Parallax' hands.
So, I'll do my best to watch this thread, but Gerrit has conversation threads associated with each change, so code discussions should happen there.
Thanks, everyone, and if you have any questions, please post them here.
Oh, and, should this gain traction, there should be little difficulty moving the entire thing into Parallax' hands.
Comments
Gerrit uses the Git version control system to hold the repository. Every change gets submitted to Gerrit and is held in limbo until a Process Administrator approves it. In order for a Process Administrator to approve a change, it must be given a +2 vote, which only a Senior Developer has permission to do.
For now, that's up to the admins, but I would offer the idea of leaving each change up for at least a week, watching what comments it garners, and accepting/rejecting changes based on where commentary seems to go.
If you think you can help the process in either capacity, please PM me and let me know. I will try to look into how helpful you've been on the forums, and may or may not require more discussion before giving you the permissions in question. While this probably sounds horribly arbitrary, I really don't intend it to be, and hope to choose the same people that the community-at-large would if it were voted on.
Click Register in the top/right corner, and log in via an OpenID provider. If the provider you use is not your primary email, please click "Settings" in the top-right corner, click "Contact Information", and "Register New Email" to add your primary email account. It is imperative that one of your accounts match the email address used by Git in your commit messages, as Author.
Use Git to clone "ssh://YourUsername@vdubshouse.zapto.org:29418/Parallax Library" and follow the directions in the DEVEL.txt.
Start by looking at each file in the change. Double-click on a line to start a new "draft" and add comments for that particular line. When you're done, click on "Up to change" and click the Review button. From there, you can enter overall comments, vote on the viability of the change, and submit your comments in total.
For example, what traits for the item have to display to be considered "gold", and how would this be measured?
While I, personally, currently have admin access to the site and can submit changes, I don't intend to do so, because I don't think I'm knowledgeable enough to approve Propeller code. I'm only keeping the role in a technical administrator context.
And on that note, everyone, please register and sign in. There's no real reason not to, and it cuts down barriers down the line, when you feel need to comment. If you want to see Gerrit used on another project, check out OpenOCD (experience).
If you want to elicit change and implement code standards then simply do it! Be a leader... Stop with the they should and I can't because...
How do you go about doing that? Well, pick a goldification candidate (ask Parallax if needed), assemble a team, write the specs, build the tests, write the code, refactor, and document along the way. The repository is secondary.
I'm not writing standards, because I don't think I'm qualified to do so. I've always been better at IT/IS type things, and like sitting on the engineering side of that fence.
For now, I'd like people to play with the commenting mechanism, just on my DEVEL.txt document, and get that up to a complete/usable spec before submitting it.
I am looking forward to the discussions on a object/library/entry bases, in the past discussions were always lost in an avalanche of other forum topics.
Thanks, good work !
Eldonb46
#1000001
I am looking forward to it too. It has been needed (and wanted) a long time.
Seeker (John)
#1000002
I only get notified of new posts... not edits.
Edit to add: But I know to look.
How is a item measured to determine if its "gold" or not? Is this to be subjective (some group of individuals "likes" it) or objective (it meet these specific criterion)?
Or is the "gold" qualification simple review status, "reviewed" versus "not reviewed"?
The answer to these questions will give us a fairly accurate estimate of the likelihood of success for this effort.
I've decided to create two groups with different permissions.
Process Admins
I imagine making you (prof_braino), and others who have a solid handle on process, Process Admins. It is up to the Process Admins' discretion whether any object/change/update is accepted into the repository. Mostly, your job is to look at the discussion surrounding and object and determine whether consensus has been achieved or not. However, you can't submit something until it has been tagged as both "Good Code" and has been Verified.
Senior Developers
This group will hold the people who have proven to be the most technically competent. I'll start with Kuroneko, Chip, Bill Henning, and a few others. It is up to their discretion to give something either a "+2, Good to submit" or "-2, bad idea, never submit" tag. All users can vote +1/-1, but they're not actually cumulative, so a bunch of +1s does not make a +2.
As for requirements, I would refer to requirements made by others. There were a few posts to that end in the [THREAD=144536]Whatever happened to the "Gold Standard"?[/THREAD] thread. I'm mostly trying to provide the mechanism to enforce and encourage a reasonable process, rather than implement my own personal coding standard.
One of the main points about Gerrit, is that it's a great place to hold things that are on their way to Gold Standard, but not quite ready. I'm in the process of trying to convert Lonesock's ffdsv1 driver to C, and I expect there to be a lot of discussion and revisions surrounding it before it's submitted. I'm hoping I can use it as an example to show off the process.
As for assigning people to groups, I can't do that until they've made their own accounts. That's why I asked for PMs in the FAQ entry on this thread.
I suppose it would be most accurate to say it's subjective, but that it's depending on the whims of two distinct "senior" groups.
Since Gerrit offers a lot more options than just Reviewed or Not Reviewed, I hope I answered this above.
Well, really, any process is going to need community buy-in. In the most recent Gold Standard thread, twc said This is why I started this effort. I'm hoping several people will sign up and work with it for a while. If it goes somewhere, great. If not, it's a failed experiment, but at least I tried it. But no matter what anyone does, anything will fail without at least some community buy-in for the attempt.
Folks need to know what they are being asked to buy into. Right now, we don't know. Leaving the determination of "gold or not" to a select group of individuals relying on their interpretation of "gold" in not something we can NOT buy into. It is undefined.
However, the data suggests that there is an easy way to do this right. I reference the Carnegie Mellon Software Institute CMMI body of knowledge. There's a LOT of material there, here is the cliff notes version. (just as a starting point)
We need to build a discrete set of qualifications for be "gold". These must be Yes/No or True/False testable. For example:
After those are answered, we can talk about:
So, the REPOSITORY is the last part of the process. The CRITERION is the first part of the process, otherwise we can never get to code and review. And these two lists can be a starting point point, if participants can agree. The lists are to be refined as we gain knowledge by experience.
Anywho, I know something about this, and its not so hard once we know the trick. And the trick is saying what we want. We can get what we want when we can be specific when we ask for it.
Is there any interest in this?
May I add that this is an excellent start, and the only thing lacking is clearly defined criterion for what qualifies as gold.
So, please, do not butt-out. Given comments you've made in the past, I would very much like you on the "Process Admin" team. I expect things to be submitted for review that have a good idea, but aren't really ready yet. This provides a forum, attached to the code/documentation, with integrated automated testing, so that sub-standard submissions can be developed to the point of achieving the standard.
Thanks for the invite. I've signed up.
Incidentally, this is the point at which the Parallax effort stopped last summer/fall; we never agreed what if any criterion would be used to measure a submission to the Gold Standard repository, or how such criterion would be evaluated. Understandably so, since the advice was that bulk of the process should be "organically driven" by the community, and Parallax should have a minimal role (perhaps "final approver") rather than the more resource intensive "soup nazi" role found in heavily regulated industries. The final roles will likely be somewhere in between.
In any case, this will be a good experiment. I'm glad you are leading the exploration of this option.
Quick howto: Click on "All" in the top-left corner, then "Open" to see a list of all changes (currently there is one, "Committing setup instructions"). Click on it to open its changeset pane. From there, you can see the files in the commit (again, only one, "DEVEL.txt"). Click on DEVEL.txt to open the diff (against nothing since it's a new file). From there, double-click on any line in the file to insert a comment. This is called a "Draft" and can be edited later. When you have all your comments in the file, click "Up to Change" then the "Review" button. You can give it a +1 or -1 vote (or not if you don't want to), and add final comments to the whole change. Once they're submitted here, I can see them and respond to them, either with more comments, or a new version of this change.
Ever seen this? http://www.parallaxsemiconductor.com/support/goldstandard/specifications
I don't agree with all of it, but looks like a great starting point if you really want criteria.
Happy New Year,
--Steve
Sure did, that's why we needed the discussion. The material on that page only represents the LAST 5-10 percent of the work.
The ninety percent not addressed is regarding what is the item intended to achieve, how it expects to go about doing it, how this can be tested, whether the tests pass.
Also needed are whether anyone reviewed the item, is the material complete and understandable.
So yes it is a great starting point. There is substantial work before the goals of the Gold Standard effort are achieved.
That last 5 to 10% as you call it is actually the hardest part because you need a consensus that works. The standard is the end product that submitters and reviewers must follow faithfully to have a solution.
The best process you will get (more or less) from contributors is:
2) black box description of each public interface - testers get a white box because source is available.
3) some proof that it works - a functional demo program (not a full test harness).
4) standard compliance - and peer review acceptance of readable code.
The mechanics of review is like a clock: it is something that should work well and be not over-bearing. Looks like Circuitsoft has a good first cut at that.
The problem with too much thinking is having no time left for action. It shouldn't take a year to get this done.
I presume this is only for SPIN/PASM to start with. Other languages will have various proportional challenges depending on the basic elements.
--Steve
So, I've not seen any comments from Braino or Jazzed (or anyone else with a more process-oriented view) on my and twc's previous idea of just hashing something out until everybody likes it, then examining what about the final form is necessary/desirable in a standard.
Thoughts?
I don't have the git installed yet, but its still on my list.
Re: jazzed - knowing what the code is intended to do, and how to verify that it does something in particular is the only thing of interest. If we don't establish these, it does not matter whether code conforms to a standard or not. Conforming to a coding standard is trivial if all the participants agree to it, and not possible when the participants do not agree.
The items you list (description of the function, description of the interfaces, some type of tests, and peer review that says the previous three are sufficient) is a good start. The process by which we review these and determine if they are sufficient is the challenge at hand.
I suppose that means you and I agree on those points.
BTW, most of that is in the Parallax Gold Standard document, although it is not explicitly stated.
That's all I have to contribute. I wanted to add more, but I have plenty to worry about as it is. I just wish Parallax was more involved in this. That they were involved is some consolation.
Personally, I would never review a piece of code in the depth that it required unless I needed to use it. In which case, I dive down into the internals and modify to suit. I've always imagined that the "Gold Obex" would have to work the same way: the only people who will do more than a superficial review are those who need to use it in a project. This naturally leads to the conclusion that you will rarely (never?) have a group of volunteers actively working on a single object to bring it to Gold Status.
And on that note, my superficial comments about the FFDS1.c/h files are:
1) I think the dat section should be volatile. Also, with spin2cpp you can use --dat with --gas to make a separate .s file (apparently: I've never tried it, although I will tomorrow).
2) Should line 25 of FFDS1.c really be static? That would imply to me that you can only have one of the objects, which seems like a problem in some situations (I'm not at all sure of this, though).
3) Code fluf, but it's inconsistent with the placement of "{": sometimes it's on the end of a line, sometimes by itself.
4) If you don't use the specific memory types (int32_t, etc) then you don't need to #include <stdint.h>.