Bringing Perl5 to GitHub

Even though Git sometimes gives me heartburn, I'm a huge fan of GitHub. The fork & pull workflow makes it sooo easy (and fun) to contribute to other projects. And as a project owner, it is very easy for me to manage incoming contributions. My absolute favorite feature is the discussions that are inline with the code. To me, that is much more natural than talking about code in a separate mailing list.

So I started thinking...what if Perl5 was developed on GitHub[1]?

Moving a large and mature project like Perl5 to a new set of tools and workflows will certainly not be easy and I expect a lot of resistance. I don't expect perfect feature parity either -- some capabilities will be gained and others will be lost. But GitHub has worked really, really well for a lot of projects (including Perl6). So I think it is definitely worth investigating.

In my heart, I hope that GitHub eventually becomes the primary platform for Perl5 development. But I don't want to alienate everyone who is comfortable with the current development tools & processes either. So I'm going to try and get the best of both worlds by creating a two-way bridge with GitHub. For example, submitting a ticket to RT could automatically create an issue on GitHub (and vice-versa). Or making a pull request on GitHub could automatically generate a patch and send it to the p5 mailing list for review.

Of course, this is all speculative. But I've already taken the first steps: I have established a Perl organization in GitHub with a (mostly empty) Perl5 repository. Next, I'll look into marshalling commits back and forth between GitHub and perl5.git.perl.org. And I'm especially eager to try migrating all the tickets from RT into GitHub's issue tracker.

I don't know how this will all work out, but I am optimistic. I'm even hopeful that it might bring new energy and new blood into Perl. So give a shout if you like the idea. And please contact me if you'd like to pitch in.

[1] Yes there is a mirror of the Perl5 repository on GitHub, but it is read-only. I'm talking about doing real development of the P5 core directly on the GitHub platform.

11 Comments

I like the idea of doing this. Perl 5 has gotten easier to contribute to over the years, but this would make it even easier.

That said, the core is still going to be an impenetrable mass of C and macros, which is probably the biggest barrier to entry for many types of improvements.

Hey, cool! I like this idea. I'm not sure if I'll ever contribute a line of code to Perl's core, but if I do, this'll make my life easier.

The problem I see is that new code should come in via a smoke-me branch, and I'm not sure if there is an easy way for a project administrator to take a pull request on one branch and pull it in to a new branch prefixed with "smoke-me". I would love to see this sort of development style incorporated into PDL, so if you find a set of steps to make this possible, please post them somewhere so we can use them, too! :-)

An amazing alternative would be to find or develop a mechanism for adding github pull requests to the smoke queue. Then again, somebody could put arbitrary code into a pull request, so maybe automating that sort of thing would introduce a security vulnerability for the smokers... OK, maybe it would be good for the p5p to add other users' repo/branch to the smokers after they've had a chance to look over the change set. What do you think?

This would make it much easier for the casual, drive by fix, especially to the non-code things.

I'm sure there's some special magic that p5p gets out of hosting their own git server. If someone lists what special hooks and so on it uses, another person can probably be able to figure out how to do it in GitHub so p5p knows they don't lose anything.

And, maybe someone will adapt perlbug to use Github issues as a proof that it can be done. I know `/anick has done similar things, but using non-core modules too. With the recent discussion about perlbug, maybe there's the political will to bring it into the present now. :)

I feel the real let-down of github issues is its very weak on metadata vs RT. Metadata tools are limited to tagging and milestones, and thats basically it, and for seriously large projects this gets seriously out of hand.

ie: https://github.com/CPAN-API/metacpan-web/issues?state=open

That said, I'm moving all my bugs to it, because its just more convenient.

Howdy, I've actually done a good deal of work getting a separate git repo posting to github via a post-receive hook (@ SocialFlow.com), and was planning on doing some work with regards to syncing pull requests and maybe some other stuff?

Anyhow, the important bits are in this Makefile: https://gist.github.com/ediblenergy/4950549

the idea being that github_mirror@yourserver has keys to the repo, and git@private-git-host has keys to github_mirror@yourserver.

We've been using this to keep a sort of 'read-only' repo up to date.

I know there was a project that imported their entire bug-report database into GitHub. ( may have actually been Rakudo )

For casual drive by fixes to non-code, we already have the perl-doc-cats github organisation that took over maintenance of perlfaq when brian quit. I'd love to see that publicised more.

For code fixes, these are going to need to be discussed on perl5-porters anyway - and it's already an open access mailing list, so I don't see that as our primary accessibility problem.

Discussions inline with code happen via "putting a patch in an email" - it's a pretty usable approach, and works fine for a lot of people.

Notably, it works fine for the people already contributing, who're the people who can provide enough help to get this stuff ready.

I'd be fascinated to see a two way gateway between github pull requests and perl5-porters - that way the github generation could interact in the way comfortable for them, while still allowing those of us who work better via email to continue to do so.

To be entirely honest, I think the key obstacle to -code- contributions is that it's quite hard to get an overview of the project - there's no "map" as such that lets people figure out where to look for what. I'd love to see such a thing - for the sake of argument call it 'perlsourcemap' - and that would, I think, lower the barriers to contributing substantially more than "now you can click on a web UI rather than posting to a mailing list".

So, in summary -

(1) perl-doc-cats already exists on github for non-code contributions, so please support that rather than duplicating the effort

(2) working on a perlsourcemap would, in my opinion, help lower the barrier to entry more than any tooling change would

(3) allowing the existing mirror to accept pull requests would be interesting -provided- they can be two way gatewayed to the mailing list so that existing contributors aren't forced to use github

(4) syncing to issues is probably less important; our problem isn't people being able to comment on bugs (perl5-porters already has plenty of people who comment but never actually contribute), it's getting more people to contribute, and that means making writing patches more accessible more than it means making submitting them more accessible

(5) please, please rename the perl organisation you created to something else (thaljefs-fascinating-and-cool-experiments would be fine by me ;) because it's making this come across as a backseat driving exercise and while I know that's not your intent, the rest of the world doesn't

(6) perl-doc-cats. perlsourcemap. pull requests. in that order. I tell you three times! :)

(and, thaljef, much love for the fact that you're not saying "other people should do this" but instead "if I can figure out what should be done, I'll try and do it" - no matter what the rest of the outcome, we need more people with your attitude!)

-- mst, out

Hmm, I agree with these comments. It's possible to duplicate efforts ... even with forking and pulling :-) Proliferating organizational repos can confuse people (or at least people like me) who are used to the well-herded namespaces of CPAN!

I think I am encouraged that Matt Trout thinks something like perlsourcemap is a good idea. Well, I am either encouraged or just less afraid :-)

Leave a comment

About Jeffrey Ryan Thalhammer

user-pic Hacker, speaker, author, dad.