December 2009 Archives

Investing in contributor docs

Like many projects, Plumage wants as many active contributors as it can get. In order to bring in new contributors, any project needs to do a few things:

  1. Get users interested in contributing (more on this another time)
  2. Let them know how to contribute
  3. Make it as easy as possible for them to do so

One big way to address that list is to invest in contributor (AKA "hacking") documentation. The word "invest" is intentional -- you will need to put quite a bit of time into writing the documentation, and you hope that it will bring enough additional contributions (that you wouldn't have otherwise gotten) to more than make up for the time you invested.

Hacking docs are over and above the low-level API documentation, covering object attributes, method signatures, exception types, and so forth. Those API docs will certainly make development easier for your contributors, but they don't capture the workflows, mental checklists, and rules of thumb that guide you when implementing new features or tracking down bugs.

For example, imagine you're working on a calendar application that handles culture-specific calendars and holidays. What are the tasks necessary to add a new culture to your application? Obviously you'll need to update the database of rules that determines start of year, groupings (such as weeks and months), day names, and so forth. And you'll need to add entries to the holiday tables for the top-level culture and all of its subcultures and local variants ... which means that you need to update the subcultures table too. All of these new values will need translations for the user interface as well. Did you remember to check if the minimum supported Unicode version needs to be bumped? And tests for both functionality and interface need to be written, and the QA lead notified of the test changes. Oh and then there's the cross-culture date translator ....

Faced with this kind of complexity, new contributors are likely to follow one of the following courses of action:

  1. Give up and run away screaming.
  2. Write a snippy blog post about how impenetrable your code is.
  3. Spend an hour trying to figure out all the things they need to do, and then proceed to #1.
  4. Produce a patch for the 50% of necessary changes they can actually figure out how to do, leaving you to try to track down what they missed.

This situation is, as they say, suboptimal. In a complex enough project, even the core committers are likely to forget tasks on a disturbingly regular basis if they try to keep the checklists in their head. And of course, every time a new feature results in a new task on the checklist, the change won't be communicated to everyone (and heavens forfend one of you is sick or on vacation for an extended period).

Thus, for your own sanity and that of your teammates present and future, an investment in proper contributor docs would be a wise decision. The fact that it will vastly improve your chances of recruiting and keeping new contributors is pure profit.

Of course, there's the small matter of actually investing all that time. If you try to sit down and write all of it at once, you'll probably choose action #1 and run away screaming ... which is why it pays to do it a little at a time, as you go. You'll be surprised how much useful documentation you can produce when you write it in little pieces.

Over the last couple weeks, I've made it a personal policy that any time I work on a Plumage task I've done more than once, I document the procedure in hopefully sufficient detail that anyone else can repeat it. A few times I found myself about to document something particularly bletcherous, and this forced me to fix the code or improve the process in a few places just so I could face the mirror in the morning with a clear conscience.

While I wrote these task-specific documents, I spent some time cross-linking the docs where one workflow related to another. Then I added a general Internal Design document to make it easier for readers unfamiliar with the code to find their way around. Finally, a general Contributing document lets new users know how they can contribute, and lists our (pretty minimal) rules.

I can tell there are quite a few more of these just waiting to be written, but already anyone should be able to add a new build tool, add a project action, or add a new plumage command within an hour of cloning the Plumage Git repository. That's a pretty good feeling. And the documentation work has already paid for itself by shining a light on some dark corners of the code.

But I'm still hoping to snag some new contributors too. :-)

Interested in our future progress? You can follow Plumage here or on twitter or; or for the real time experience, come by #parrot and chat:

A month of Plumage work

It has been another month since the last time I posted a Plumage work log, back on the old u.p.o blog. Mostly this was because I was getting actual work done on Plumage, so I hope you'll forgive me.

Just like last time so much time had passed, I'm just going to post the (mildly edited) #parrotsketch reports here; the format is a bit terse, but it should give you an idea of the pace of development:

* Support fperrad++'s setup.pir system in Plumage
* Fix several portability issues, leading to ...
* ... Plumage's first successful clean test run and project install on Win32!
* Much more use of new NQP-rx features
* Switch over to parrot-nqp, no longer following NQP-rx git master directly
* More Glue.pir -> Util.nqp conversions
* Refactor action functions to reduce pointless duplication
* Add simple version of 'status' command
* Commit metadata additions/edits from fperrad++
* Reorganize more of the tests to make it easier to identify testing gaps
* fperrad++   # setup.pir, Win32 testing, new & improved metadata ...
* pmichaud++  # NQP-rx features, w00t!

* Keep up with pmichaud's NQP-rx improvements (and push for more :-)
* Keep up (mostly) with fperrad's bug reports and metadata updates
* Various Glue.pir improvements
* A few more tests
* Big refactoring of main "brains" to OO classes
* Document refactored code
* About 50% done with planned refactors before next feature push
* fperrad++   # Copious metadata updates, suggestions, and bug reports
* pmichaud++  # NQP-rx feature-o-rama
* dukeleto++  # Initial pass at qx() tests

* Lots more metadata updates, fperrad++, dukeleto++, vadrer++
* Now at least 23 projects have Plumage metadata
* Convert Glue.pir code to native NQP-rx features as they become available
* Util.nqp reduce() implementation that doesn't entirely suck to use
* Improve output of projects command
* Continued major refactoring.  About 80% done now.
* Details of remaining refactoring unclear, need more code to clarify path forward.
* fperrad++  # Metadata updates, Win32 testing, assisting conversions to setup / plumage
* dukeleto++ # Metadata updates
* tene++     # try/CATCH
* cotto++    # Parrot build GUID/UUID suggestion

* Hacking docs!  Reviews requested: , start with the contributing.pod doc.
* More metadata updates from fperrad++
* Add accessor methods to Plumage::Project
* Add project-dir command to plumage
* Move responsibility for determining build sequence to Plumage::Project
* Move responsibility for mkpath($build_root) to Plumage::Project's fetch action
* Excise %STAGES and supporting code from plumage.nqp
* Improve API of perform_actions{,_on_projects}
* Add stub of update stage to Plumage::Project, reusing (and falling back to) fetch
* Make harness exit(1) after failing test runs
* Fix broken test
* Tell plumage command_ subs what command invoked them
* Collapse all plumage project action commands into one sub

* Documentation
  * Add/improve comments
  * Update/improve usage info
  * Update existing hacking docs
  * Add more hacking docs
* Internals
  * Save away a copy of the project metadata at install, remove at uninstall
  * Improve metadata validation, resilience, and error reporting
  * Several other improvements to error reporting
  * Add hash() coercion to Util.nqp
* New features
  * Add update type 'parrot_setup'
  * Make update fall back to fetch if source dir is missing
  * Make update the default way to start a build cycle
  * Add support for Git submodules
  * Add support for Mercurial repos
  * Add support for Rake build tool
  * Add new project actions: smoke, clean, realclean, uninstall
    * uninstall only works with setup.pir-based projects for now
* Misc
  * Lots of metadata updates, fperrad++
  * Fix several bugs reported by fperrad++
  * Fix several bugs discovered by me(--,++)
* Convert remaining Glue.pir items to NQP and move to Util.nqp
* fperrad++   # Testing, suggestions, and metadata updates
* #ps++, pmichaud++, moritz++, Tene++, (anyone I forgot)++   # Great discussions, and suggestions for improving Plumage messaging outside the Parrot core team
* Fat arrow (=>) syntax in nqp-rx

Interested in our future progress? You can follow Plumage here or on twitter or; or for the real time experience, come by #parrot and chat us up:

Welcome, and say Hello to my friend Plumage

Stability seems to be coming to these shores, so I guess it's about time I created an account here and started posting. For a while at least, most of my posts will be about Plumage, for which I am the current lead developer.

Plumage is a cross-language module ecosystem, designed to work with CPAN, LuaRocks, RubyGems, Rake, distutils, and so on. Plumage itself is implemented in NQP (Not Quite Perl 6), which is part of the PCT toolset used to implement languages and modules for Parrot.

The original design document for Plumage is available on the Parrot wiki, and the source repository is hosted on Gitorious. You can check it out by doing the following:

# Fetch, build, and 'make install' parrot first.  Then:
$ git clone git://
$ cd parrot-plumage
$ parrot-nqp Configure.nqp
$ make
$ make test

Now you can install a language:

$ ./plumage install rakudo

And maybe some modules:

$ ./plumage install mt19937 parrot-linear-algebra

If you want to see what dependencies are needed by a particular project, Plumage can help:

$ ./plumage showdeps lua-batteries

There's already a lot more that Plumage can do; this will give you a quick list:

$ ./plumage usage

So that's the first taste of things to come -- please give it a try and don't be shy with comments or suggestions.

You can follow Plumage's progress here or on twitter or; or for the real time experience, come by #parrot and chat us up:

About Geoffrey Broadwell

user-pic My JAPH side: Perl 5, Perl 6, Parrot, Plumage, oPenGL (cough). See also and