Kenichi Ishigaki

At the previous QA hackathons, I spent most of my time on improving various aspects of CPANTS. However, I usually couldn't see what I implemented there online, because it takes about a day to analyze everything. All I could do was to start the analyzer before I fly back and confirm the result at home.

This year, things went differently for me. I spent three days on porting PAUSE Web UI using Plack toolkit, and was able to actually see the result there.

Why ported?

Accoring to the history, the current form of PAUSE was (re-)written in 1999. As were serious web applications at that time, PAUSE was built on mod_perl version 1 and thus, Apache 1.3. Though PAUSE has still been running pretty steadily since then, both of its requirements have been retired for years (mod_perl 1 was last released in 2009, and Apache 1.3.42 in 2010), and it's getting harder for new contributors to install them, especially with newer compilers and newer perls. This was the main reason why I proposed porting.

Web UI of PAUSE is made with about 9000 lines of code, including HTML and sparse comments (excluding blanks). It may not be trivial, but it's not so big either. If it were my personal project, I wouldn't probably have hesitated to rewrite it from the scratch with a modern web framework.

However, I got to consider it better to keep my changes as little as possible instead (at least for this time). Partly because I wanted to lower migration cost for those who have actually been maintaining PAUSE, and partly because I wanted to avoid conflicts and/or cherry-pickings as Ricardo Signes had also started his hacks on PAUSE. So I set my goal just to drop mod_perl dependency and replace necessary parts with Plack equivalents.

With a kind consent for porting from Andreas König, I started looking into the actual code in the early morning on Day 2 (actually, I got up at three, that was ten in JST, not uncommon for me).

The first step to port a mod_perl application with Plack is to find a main (Perl content) handler (or handlers) to be converted into a Plack application. Shortly I found it in config.pm, in which a main (manager) object was created to do the rest (which means, this manager object could be easily replaced with a modern router/dispatcher).

Now I found an entry point, I created a local branch for prototyping, then a minimal app.psgi to pass a Plack::Request object to the main handler, and started dropping everything under the Apache namespace. I kept fixing issues I saw in the console until Plack server stopped complaining. In a few hours, all the Apache/Apache::Request objects including the ones in Apache::HeavyCGI (a mod_perl framework used in PAUSE) were roughly replaced with equivalent Plack::Request/Response objects, and almost everything became visible from a browser. Plack really rocks.

With a few more tweaks in PrivatePAUSE.pm (which you need to create to configure local paths and dsns etc), I also succeeded to run paused and mldistwatch (the PAUSE indexer) to fetch and index a locally uploaded tarball. An important trick was to set $PAUSE::Config->{INCOMING} to a file:// URI, which saved me from setting up a local FTP server.

After a short test, I created another (public) branch and repeated what I did in the morning but more cleanly. I wondered if I should release the ported version of Apache::HeavyCGI to CPAN or just put it into the PAUSE repository for now, and asked Ricardo for his advice. I renamed and pushed it into my branch as a starter. I called it a day when I finished redoing.

Day 3 was rather short because I couldn't help but fall back asleep after I woke up at three, and we stopped hacking a bit earlier to visit the Computerspielemuseum.

I started the day with porting authentication and session management. I wrote a wrapper for Auth::Basic middleware and let it use PAUSE's auth handler. I first thought I also needed to port a custom session manager PAUSE uses because of its name (Apache::Session::Counted), but with a closer look, it turned out unnecessary. An unfortunate thing was I had to replace a logger middleware because of an obscure error. I also attended (actually, listened to) the discussion of the day because one of the proposed topics was PAUSE version 2 optimized for maintainability.

On the last day, I set up an nginx proxy to see if PAUSE on Plack still works behind a proxy correctly. I also reviewed all the usage of "param" method because I vaguely remembered it had small incompatibility. I reread Apache configuration to add a few more Plack middlewares, and ported a fixup handler for index.html.

I asked Andreas to deploy my port on a staging environment so that we could fine-tune it under more realistic environment. We needed to fix a few more issues on Perlbal configuration and URL handling with the help of Peter Rabbitson, Slaven Rezić and Tatsuhiko Miyagawa. Finally we could make it live. We actually uploaded modules with it.

If you'd like to try, PAUSE on Plack still lives at https://pause2.develooper.com/pause/query as of this writing. You'll encounter an SSL warning because it uses the same certification as pause.perl.org. So, CAVEAT EMPTOR.

The following is a rough procedure to set up your local PAUSE on Plack:

Clone "plackify" branch
Plackify branch has not been merged into Andreas's master yet. So clone from my fork.

git clone https://github.com/charsbar/pause -b plackify

Install dependencies
Run cpanm --installdeps . or something equivalent. You might also need to install unzip, MySQL, SQLite3, gpg, rsync etc. with your system package manager.

Create databases
Schemata are found in doc/*.schema.txt (for MySQL), or doc/schemas/*.sqlite (for SQLite). As of this writing, you still need to use MySQL for various reasons. Add an admin user to authen_pause (and mod). Read doc/README for details.

Create PrivatePAUSE.pm
You need to create a module named PrivatePAUSE.pm to modify PAUSE configuration. It's safer to create it out of the cloned directory not to push it accidentally. Make sure perl can find it.

See $PAUSE::Config definition in lib/PAUSE.pm to find what to modify. You usually need to set (AUTHEN|MOD)_DATA_SOURCE_(NAME|USER|PW), and it's better to override email addresses not to send test mails accidentally. Setting ML_MIN_INDEX_LINES/ML_MIN_FILES to 0 is also important if you'd like to test with a small number of distributions (instead of whole CPAN).

Create necessary paths
Unless you have overriden everything above, you probably need to create a few paths, and change owner/permission for some.

Set up a proxy
PAUSE on Plack works without a proxy (in most cases), but it's usually better to set up one if you'd like to access via a browser.

Run some tests
This is not mandatory, but it's nice to run tests to see if your PrivatePAUSE is working correctly.

Start a server
Run plackup. Perldoc plackup for useful options. Run also paused/mldistwatch when you upload a tarball.

There are still things we can (or need to) do for PAUSE on Plack, such as writing tests for further migration, adding more middlewares for security, better static file serving, to name a few. I hope I can work on some at home by the next time.

Aside from porting PAUSE, I asked Neil Bowers for a few improvements on PAUSE::Permissions, replied to a ticket for Parse::PMFile posted by Olaf Anders (on Day 5 while I was in town), fixed an issue on Module::New suggested by Olivier Mengué, and fixed issues on DBD::SQLite.

I'd like to thank Tina Müller (and others who helped her) for organizing this awesome event, Wendy van Dijk for food and liquors and more, and our generous sponsers: thinkproject!, amazon Development Center, STRATO AG, Booking.com, AffinityLive, Travis CI, Bluehost, GFU Cyrus AG, Evozon, infinity interactive, Neo4j, Frankfurt Perl Mongers, Perl 6 Community, Les Mongueurs de Perl, YAPC Europe Foundation, Perl Weekly, elasticsearch, LiquidWeb, DreamHost, qp procura, MongoDB, Campus Explorer, Ron Savage, Christopher Tijerina, Andrew Solomon, Jens Gassmann, Marc Allen, Michael LaGrasta.