QA Hackathon 2015 (Berlin)
I was at the QA Hackathon in Berlin this year.
Everyone else writing about it has an introductory section about what it is, so I needn’t concern myself with one… as well we’re on blogs.perl.org here after all.
In a stupid move, I forced myself to constantly look for some other hacker to mooch power from – I left my own power supply at home. Fortunately as a semi-reluctant initiate to the Apple cult, I found plenty who were of the same hardware persuasion, whom I could bum some laptop juice of life from. I hope to not repeat this part.
But the main reason that I chose to be present at all was for the consensus discussions scheduled this year.
If I am frank, I cannot say whether I mattered to the discussions as much as they mattered to me. But their outcome mattered greatly to me, and so I am glad to have been there, and glad of the course they eventually took. I tried to do the best I could to help steer them in that direction, but in the end it was not me who made the difference.
I am deeply grateful that Neil Bowers was there. Without him, my true feelings about that part of the hackathon might have had to be very different.
Thank you, Neil.
(I am left to contemplate what might have resulted had we had a fifth day, and had we scheduled a from-scratch repeat of the conversation from day 1 for that ever-to-be-hypothetical day 5.)
If you weren’t there, you will be reading about this subject on his weblog in the time to come.
As for hacking, going in, I registered my interest to work mainly on the Plackification of PAUSE and other PAUSE issues. Well, I didn’t end up working on any of that. And with Kenichi Ishigaki on the ground, it truly was unnecessary – he set the tarmac on fire, with an initial working port in hours.
Instead, on the first day, Ricardo Signes made an off-hand remark about a branch that adds more tie magic to Module::CoreList (tie is like violence, apparently…). “Thoughts?” Thus it is that I went on a still-ongoing quest to raze Module::CoreList as we know it to the ground, to replace it with something that is actually designed.
The plan for that is as follows:
Complete the design and implementation of a module for providing the data currently stored in Module::CoreList. This comprises two concerns: a data model and storage format for the data on one hand, and an interface to query the data on the other hand.
Cut the guts out of Module::CoreList and shim them with reimplementations in terms of the new module (using – of course – some more
tie… as I said…).Repeat the same for all the CoreList-related tooling for the Perl release engineering process.
There is some complication in that these steps are not independent; working on #2 and #3 is necessary to inform the design in step #1, by providing specific requirements to cover, and changes in #1 then spiral back to #2 and #3. A holistic learning process, as all software development is.
The pace of this project has slowed since I got back home, inevitably, but I have continued the pursuit. The design for #1 seems to have stabilised now. I hope to get to the point of finishing up #2 fairly soon. When I get there, I’ll post patches for review on p5p. I will also be writing up which design constraints I chose and why, and how and why I chose to fulfil them.
There has been a curious thread through my hackathons – Bitter Lemon. The Act hackathon in Lyon last year taught me that apparently you cannot order it in France. At the QA hackathon in Berlin this year, I taught Ricardo Signes and David Golden (and others maybe) that you can order it in Germany. It seemed quite the surprise to Rik. And I noticed that while David had been demurring with “just water” when the others would order their Weizens and their ales and whatnot, after that, he’d have Bitter Lemon at nearly every opportunity.
Yes, it was good to hang with that crowd. Often I enjoy only fragments of the experience, and it falls on those to make the whole worthwhile. But not here so much.
Besides Neil Bowers, my sincerest thanks go to Tina Müller and aides for doing all the legwork it took to make this event happen. (Metaphorical and physical. Four tall flights up!) And of course I would be remiss to close the post without a mention of the sponsors who made it possible: thinkproject!, amazon Development Center, STRATO AG, Booking.com, AffinityLive, Travis CI, Bluehost, GFU Cyrus AG, Evozon, infinity interactive, Neo4j, Frankfurt Perl Mongers, Perl 6 Community, Les Mongueurs de Perl, YAPC Europe Foundation, Perl Weekly, elasticsearch, LiquidWeb, DreamHost, qp procura, mongoDB, Campus Explorer, as well as individual contributions from Ron Savage, Christopher Tijerina, Andrew Solomon, Jens Gassmann, Marc Allen, and Michael LaGrasta.
It was worth it.
Waxing philosophical
Regarding a rework of Module::CoreList... I'm interested in writing another module with the same kind of role: Git::Features.
At first, I thought about stealing from Module::CoreList, or convincing BinGOs to upload Module::CoreList::TieHashDelta as a separate module I could use.
Now that you've exposed your plan, I'd be interested to participate, at least in the position of a potential user that is not Module::CoreList itself.
Please don’t use Module::CoreList::TieHashDelta… for anything. It is a terrible design with terrible performance.
You will not get much code reuse out of Perl::ReleaseInfo (that’s what I named the new module). That is on purpose, to avoid exposing the internal storage format, so that in the future we will be able to avoid having to wrap variables with ties in order to extend the module. With no internal interfaces, there is nothing you can reuse directly.
But you are absolutely welcome to copy-paste and tweak the design to your needs, of course. You will probably want to do that anyway, since you are dealing with only one type of thing (I think), whereas I have three (versions of perl, modules, and utilities (the latter of which I have not addressed yet, but it’s on my list)), so the API as is probably does not make sense for your case.
As long as I am not ready to post patches, you will find the code in my private branch. (For the time being, it is totally undocumented, but I hope it is not exactly cryptic.) (Note that my workflow is very rebase-heavy.)
The major difference internally is the way the data is stored. The internal format of Perl::ReleaseInfo is designed to avoid repeating the module names over and over, which is the biggest bloat in Module::CoreList. That makes it < ¼ the file size. And because the API does not expose any data structures at all, it does not need ties to compute and cache redundant data. (So it also ends up being much faster than Module::CoreList. MCL’s own test suite runs 2.5× faster when its innards are replaced with wrappers around PRI.)
(All of which and more I will document in due time.)
But there isn’t really any magic going on code-wise. As well there shouldn’t be! Something is very wrong if a job this simple requires deep magic. So, you should be able to make a module with the same design quite trivially.