PropellerGCC on GitHub?
SRLM
Posts: 5,045
I'd like to suggest that PropGCC and friends move to GitHub. Google code is falling into the dust, and not keeping up in features or capabilities. In fact, they're dropping them. Remember when search disappeared? I do. Remember when downloads for new projects disappeared? I do. The user interface is old and difficult to navigate. Why isn't the issues list used? Because it's a hassle and the interface doesn't make it any easier. Why don't we get more diverse contributors? Maybe it's because there's a strong fear of breaking the build, and no easy to use mechanism to suggest changes.
GitHub, on the other hand, is much better.
I've tried the three major tools (GitHub, BitBucket, Google Code) and many smaller tools as well. GitHub is by far the best of them. It allows for easy usage, easy contributions, and discussions that don't feel like they're lost to time. It makes it easy to collaborate. Look at any project that has more than a few users: there are dozens of contributors. The layout is top notch, and ... I could go on. But GitHub has a nice features page here, and there are numerous discussions on Google Code vs GitHub. GitHub almost always wins.
What would PropGCC lose by going to GitHub? Not much. In particular:
1. External URLs that link to the source.
2. ???
I understand that this may take some effort. So, I'll commit to helping. If PropGCC is moved to GitHub I'm willing to port over all the Issues and documentation (wiki, HTML, etc.). The mvn to git process is automated. Finally, that just leaves sorting out the accounts for the developers, which isn't all that hard.
ps: If PropellerGCC is moved to GitHub, can we get rid of the propgcc sites.google pages and move them to GitHub? I'd be willing to do that as well.
GitHub, on the other hand, is much better.
I've tried the three major tools (GitHub, BitBucket, Google Code) and many smaller tools as well. GitHub is by far the best of them. It allows for easy usage, easy contributions, and discussions that don't feel like they're lost to time. It makes it easy to collaborate. Look at any project that has more than a few users: there are dozens of contributors. The layout is top notch, and ... I could go on. But GitHub has a nice features page here, and there are numerous discussions on Google Code vs GitHub. GitHub almost always wins.
What would PropGCC lose by going to GitHub? Not much. In particular:
1. External URLs that link to the source.
2. ???
I understand that this may take some effort. So, I'll commit to helping. If PropGCC is moved to GitHub I'm willing to port over all the Issues and documentation (wiki, HTML, etc.). The mvn to git process is automated. Finally, that just leaves sorting out the accounts for the developers, which isn't all that hard.
ps: If PropellerGCC is moved to GitHub, can we get rid of the propgcc sites.google pages and move them to GitHub? I'd be willing to do that as well.
Comments
Cheers!
Daniel
Github can be used to host the website as well as the code, and I don't think we would lose direct URLs (maybe I misunderstand what you meant there).
I've used it for a few months and I thought it was very easy to learn and get used to. Collaboration is very easy, and it's very easy even for non-members to contribute by forking repositories to their own account, making the necessary changes and posting pull requests to get the changes integrated. Github makes excellent tools available for offline use (though I have to admit I only worked with the Windows version). It may not be as good as Perforce (which is what I use at work) but it's definitely not as bad as some other services that are completely jumping the shark (*cough*sourceforge*cough*).
===Jac
Just a quick note to let Cody and Jac know that I'm taking a little more detailed look into this proposal with the intent of supporting your request.
Give me a day please. We're honored to have community contribution of this kind.
Ken Gracey
I have pretty much hated every version control system I ever had to use. And there have been many, Clear Case, Visual Source Safe, cvs, svn, git and a bunch of others that are now thankfully lost to history. They have all been slow, awkwark, cumbersome, unreliable, complicated, annoying etc.
The emphasis here is on "had to use". One would never inflict that pain on ones self.
Then there is git. God damn that thing is mind bending. A billion commands and options and concepts to deal with. Reading the documentation is scarey and many of the "howtos" are disturbing.
But, here I am using git for nearly everything in our company and personal work for some years now. How did that happen?
Thing is, once you have you basic work flow sorted out and a few commands and steps committed to memory git is easy to get on with. It's fast. It works when you are off line. It makes every clone repo a equal peer of the original (working back ups everywhere). Branching and merging are a doddle.
Git becomes something you can use, hour by hour, day by day. It helps you without being that chore you have to do to get your code into a repo after a stint of development.
Github helps here a lot. It's a breeze.
As for moving propgcc. I don't know. Shouldn't propgcc be in whatever system the upstream GCC is in?
P.S. I have "cloned" the loader from propgcc into github here https://github.com/ZiCog/pi-propeller-load
If it was Git, then yes
You really shouldn't be storing binaries in a repository... I don't want to have to clone 1GB of material just to work with the code. But if you want to you can take the path that I did. Create two repositories: one for the code, and one for the binaries. Since people won't really be cloning the repository with the binaries you can feel free to delete it whenever you want and start over. In my case, I'm doing this for the Doxygen generated documentation since it's all new every time.
Edit: it looks like we're already downloading a bunch. The current PropGCC repository size is 655.4 MB.
Not sure if that helps much with the 655MB download problem.
It is incredibly slow to pull down the repo, the first time at least.
No idea why.
Regards,
Eric
Github gives us:
1. Effective searching of code
Google code doesn't allow searching. Why Google can't do search is beyond me, but that's the way it is. Github has a nice search interface that makes it easier to find things, especially in a project like PropGCC where the entire GCC sink is in there.
2. Easier collaboration tools. Right now, it's really hard for people like me (little experience) to contribute.
Github is well known for it's fork and pull features. This gives confidence to people like me who would be worried about breaking the build. Two recent cases where this could be helpful:
a - Heater's work with ARM. He could have created a fork that does whatever he needs done.
b - My recent challenge with getting it to build on Ubuntu 13.10. While searching that process I thought I would have had to make changes to several files. It's easier to do that in a fork than a forum post.
3. A unified documentation host. We can have good looking wiki's and hosted HTML pages (via rawgithub)
Where's the documentation for ...? I'm always asking myself that, and I've been doing PropGCC for a year now. Parallax used to be known for their excellent documentation. I don't think they have that for the Propeller, and I'd propose that part of that is too many places. There's sites.google.com, Google Code Wiki, some HTML, some Learn.parallax.com, SimpleLibrary API, and these forums.
Where do you go to learn about memory models? Where do you go to get the current status of SimpleIDE? Where do you go to get an introduction to PropGCC? What about install instructions? These topics are all over the place.
I'd compress everything down to two places: GitHub and learn.parallax.com. Learn would host the learn tutorials, and GitHub hosts everything else.
4. Better bug tracking. Why doesn't anybody use the Google Code bugs? I don't know, but I would guess because it doesn't make it easy.
In two years there have been 62 issues reported. This year, the last one was a month ago. Has there really only been less than 3 issues per month? It seems that most people (myself included) report issues on the forums. This is both inefficient and bad task tracking. Issues are one of the easiest ways to hook new contributors. They can pick an issue out of the list and fix it. Issues that are reported on the forums (or Google Code!) are lost to time, never to be seen again.
5. A better culture. Yes, this is a benefit. Github is "home of the open source project".
Github is nearly synonymous with modern open source. That's where everybody is now days. PropGCC is a software project, not hardware project. It should be run like a software project and use the tools of the open source world.
One of the problems with the current setup is there is no central "home page". There's no place where the real status updates are, at a granular level. One thing that I would like to see is a roadmap.
A second issue is contributors. Right now, it's pretty much just David, ersmith, and Jazzed doing everything from new features to most of the documentation and support. You've done alot, and I for one am very grateful. I can't say thanks enough. But do you want to be maintaining this for life? I think that anything that makes the project "owner agnostic" is a good thing, and Github is the tool to make that happen.
Just FYI for everyone, here are the number of commits by individual:
1148 Eric Smith 773 David Betz 174 Steve Denson 87 Ken Rose 29 Dave Hein 20 Ted Stefanik 2 srlm 2 Daniel Harris 1 Bill HenningYes, 655Mb is rather close to a hard 1GB ceiling, it may pay to clarify the road map of the ceiling before moving.
Mercurial has the same ability to fork as Git, except that it calls forks "clones". You won't break the build if you pull down a copy (clone) and do whatever you want locally -- edit files, commit them, etc. To get changes back to the server you have to do an explicit "push". You can also clone a copy of your local repository for experimental work, in which case "push" from the experimental clone will go to your local repository, and it has to be pushed from there back to Google Code.
If we were to start the project over again knowing what we do now, I think we would start it at GitHub rather than Google Code. My major concern with moving to GitHub is that we don't want it to be a distraction to development. Maybe we could do a "staged" move with some someprojects (like documentation, which seems to be where GitHub is best) moving first.
Eric
With Google Code I'd have to make a copy, edit it, then somehow get the changes back to where somebody smart (like you, Jazzed, or David) can review it. The "Get it back to you" is the part that is lacking right now. There is no easy mechanism that allows you to review the changes without having them committed to the repository.
A move to GitHub probably wouldn't benefit the core developers of PropGCC very much. After all, you seem comfortable with Google Code and rightly so. The move is to benefit everybody like me who want's to dip their toes in without diving in.
While 655MB is a large amount, I think the large majority of the files don't change at all, ever. Many files are only in the PropGCC repo because they're in the upstream repo. So I don't think it's "cutting it close" to 1GB, even if you regard history as part of the size (and it's not clear from the page linked above whether they do or not).
I don't think the future changes to the Propeller GCC repository will fill up the 369MB (or 345MB if we're talking MeB here) of headroom any time soon, but even if it does, there are several possible scenarios:
- First of all, it's probably fairly easy to determine which files aren't necessary for the Propeller build of GCC. There are tools that let you log which files are accessed by a program, I know I've done this myself with a Windows project using Process Monitor by Sysinternals (now Microsoft). Under Linux it should be even easier though I'm not sure how to do it. Unused files can be obliterated (i.e. history deleted) after packing them up in a zip file that's not part of the repo, so that they can be restored for a pushback into upstream GCC.
- It's possible to split GCC, binutils and documentations into separate repos. I don't recommend doing this from the get-go, though: I have several repos myself for a couple of Arduino libraries, and I regard that decision as a mistake, because, in theory, if someone would already have an old version of library A, and then gets the latest version of program B that depends on a newer commit of A, their local version of A doesn't get automatically updated with the cloning of program B. I strongly recommend starting out with everything in one repository.
- In the future, I expect Propeller Binutils and GCC to be merged into upstream. At that point, a new repository can be created that starts out with a clean slate (the state of Binutils and GCC from upstream, immediately after the integration). With no initial history for this new repo, the size should be small enough again to remain under any Github limit for a while (unless Binutils and GCC grow too big themselves).
Also, if Parallax would have a paid account, Github might be more inclined to "look the other way" if the repo does get bigger than what they see as comfortable (I couldn't quickly find any page on github.com that would indicate this, so this is speculation).
===Jac
If you go above it they may eject you from the service. I even offered to pay and they say they still can't go above 1 GB. I believe the limit is because of how GIT works. GITHub is just not compromising since everyone would want a bigger repo if there was no hard limit.
Three reasons:
1. It requires commit changes to the repository, so that prevents new people from contributing without getting approved.
2. It's really scary to modify the core repository for a newcommer like me. That's why for my two commits I started a thread about the changes and got approval via the thread, then very carefully made the change via the web editor so that I didn't mess it up.
3. It doesn't scale. Google code keeps the list of all branches in the dropdown, even the closed ones. What about down the road when there are dozens of contributors? How do you keep track of the state of each branch? By forking, it puts maintenance on the forker, not the original project.
We will be removing the huge newlib folder from the repository entirely. It is not used.
BTW, ~650MB of SimpleIDE+PropellerGCC binary (Linux) doesn't mean the source is that size. On Mac, the full install is ~270MB.
Does hg commit --closebranch not prevent display of those old branches?
Also, if googlecode is truly "turning to dust" perhaps simply shifting the existing hg repo to bitbucket would forstall further dustiness?
Lastly, regarding a workflow for submitting patches from occasional contributors, one way to do that is for the contributor in question to get their own bitbucket account and push their work there. Then the project maintainers can pull in various fixes as they see fit. That is how the hg community does things at least. You can also email patches and bundles, although I don't have that much experience with those mechanisms.
All the best,
Tom
Would a move lose our version history?
http://stackoverflow.com/questions/35837/what-is-the-difference-between-mercurial-and-git
I'm starting to look at differences now. Lots to learn.
Long way to go yet, but one primary difference is Mercurial prevenst people from making non-recoverable mistakes.