Perceived Extinction - Some context

Perl is tired, old, crusty, and ready for the farm where its spirit can run free with the likes of COBOL and Fortran.

Or at least that's what some people outside the community say.

We know this couldn't be farther from the truth: Perl 5 is alive, "Modern Perl" is buzzing on everyone's lips, Moose has brought Perl OO into the 21st century, frameworks like Catalyst and Mojolicious tame monstrous web projects, and ORM libraries like DBIx::Class make database interaction fun again!

A widely held view of Perl in the realm of large-scale software development, however, is that the language and community are dying. Despite the obvious absurdity of this to those of us in the Perl trenches every day, the perception of Perl can and will become self-fulfilling. Perception is -- or at least begets -- reality.

The Future of Perl Documentation:

1. Perceived Extinction - Some context
2. The Power of Shine, and Harnessing the Web (coming soon) - Arguments for rethinking documentation practices
3. An Evolution in Perl and CPAN Documentation (coming soon) - Proposal and RFC to reshape how we write documentation

This is the first of three posts where I will be discussing how a focus on revamping our documentation can help to revitalize and grow the Perl community.

The extinction of Perl? Aren't you taking this a bit far?

If your experiences have been to the contrary, that's great news! You should skip this post and move along to part 2. This first part merely serves as a backdrop on the stage - a bit of context to explain the story that follows. The changes I would like to propose in the following posts can be worth discussing no matter how strong the future of Perl might be.

And while we are framing the argument, allow me to make something perfectly clear. My experiences and frame of reference are primarily with Perl as a tool for large-scale software development. Aristotle Pagaltzis made a comment on the p5p list recently reminding us that there are multiple strata in which Perl is used. It might be safe to say that people still, and increasingly turn to Perl in other strata, such as writing glue code, data munging utilities, one-off sysadmin scripts, and the like. Especially those tasks in which limited experience with large frameworks and best, modern practices are (arguably) lesser concerns.

That being said, let's turn our focus back to large-scale software development.

Where is this coming from?

After 13 years of working with Perl, and the better part of a decade interviewing more developers for Perl jobs than I care to remember, I've noticed an alarming trend. The majority of people I've spoken with either:

1. haven't had the opportunity to use Perl for large projects due to constraints placed on them by their employers, or
2. haven't taken the opportunity to use Perl for large projects because they view the language as one only suitable for small monolithic scripts and glue code.

If you've ever conducted jobs interviews, you've likely run across countless candidates who fall into the first category. They once wrote a Perl script to convert a CSV file to XML, but "don't get to use Perl much" because every place they've worked has been a /java|ruby|python/c(?:\+\+)?/ shop. For each tool or technology you listed in the job posting, in addition to Perl, the number of these candidates increased exponentially. They assumed $other_listed_tool was the primary development environment, and that Perl was merely required for occasional data munging. In short, Perl is a tool that many use a little, and few use a lot.

The latter category - those who undervalue Perl - is a little harder to spot in the wild if you're working mostly in Perl and hiring for such a position. Who is going to be foolish enough to walk into an interview and deride the development team's chosen tool? It happens, but is rare. We recently had a gentleman come in and take every opportunity to preach the virtues of Ruby and Rails, and denigrate Perl as a "mess" and a "nightmare" for anything but small scripts. This continued after he even admitted, when asked, he has no knowledge of Catalyst, Moose, or other modern tools and best practices. This situation is rare though.

Typically to find these Perl naysayers, you have to travel outside your immediate sphere of influence (unless you're already working in a role other than Perl development). I spent 5 years working for a large university and interviewed a great many students looking for practical experience in various undergrad and graduate student jobs involving work on some large, OO Perl software. In all, I probably talked with between 100 and 150 students over the years (university student jobs have a high turnover).

It did not take long to notice a trend. Not only did the majority of students have no significant experience with Perl (which was expected), they had no desire to learn it and questioned the wisdom of choosing it for anything but the simplest of scripts. My own curiosity had me digging and probing with each of these encounters, especially after the umpteenth time hearing that backend systems should always be written in Java or C++, all modern web software should be done with Ruby and RoR, and other generalities that leave Perl by the wayside.

When pressed why they felt this way, a common reason emerged: Perl is old, and fading fast. These young, budding technologists had come under the impression that few people are making improvements in the Perl ecosystem. They worried it would be foolish to board a sinking ship, and that their time would be better spent with the shiny and trendy (Ruby) and the pillars of "enterprise" (Java and C++).

It was during this time that I started becoming increasingly concerned with the future of Perl, and began to seriously contemplate the power of perception.

Perception begets reality

This is an admittedly subjective and highly experiential take on things, from my own little corner of the world. For every each developer I have seen turn away from Perl, there may be only 1, or as many as 10,000 others. Either way it is a loss, it concerns me, and it should concern you.

Each developer who chooses not to consider Perl as a serious contender for large-scale software development actually results in a bigger loss to the community. Each loss contributes to the perception of a shrinking community, which in turn drives away even more potential Perl users. The perception becomes self-fulfilling.

You might be asking "why should I care?" You use Perl. You work with Perl. Your friends and coworkers use Perl. None of this affects you.

I push back on this assertion and challenge you to examine the bigger picture. A growing negative perception of Perl, or worse, a shrinking a pool of Perl developers has two devastating consequences.

First, with each passing year there will be fewer and fewer experienced and highly qualified developers to hire. Empty seats on your team will remain vacant for longer, or will be filled with less qualified hires.

Second, and more importantly, as the perception of Perl as a tool for large-scale software development diminishes, so too do the number of companies and departments willing to undertake new development with it. The opportunity to write in a great language, all day every day, dries up.

Perhaps I am being too pessimistic, or putting undue weight on my own unique experiences. Perhaps my concerns are founded, but the time it takes for them to be realized will exceed the lifetime of Perl 5, and Perl 6 will renew and reinvigorate the community. I concede that either or both of these may be the case.

Regardless, this is just context for my plan and proposal (part 3). Whether or not you agree with my assessments, I hope you can agree to this:

We need to correct the negative perceptions of Perl and attract more new developers to our community.

What does this have to do with documentation?

I believe there are two general ways to improve peoples' perceptions of Perl:

• Active public relations
• Passive changes to the appearance of Perl and its community

I see a lot of momentum in the public relations realm. Initiatives like distributing Perl flyers and the Iron Man blogging challenge (and countless others) can have some amazing results. I fully support these worthwhile efforts, and hope to jump in and help in whatever ways I can.

What I see lacking, though, and have begun investing significant time and effort into, is a similar renewal of documentation efforts - or the passive aspect of Perl's reinvigoration. Looking at perldocs and perusing search.cpan.org is how new (and even experienced) Perl programmers spend a sizable chunk of their time. When somebody comes and sees Unix man-style API documentation with little context or explanation, finds few tie-ins between libraries, frameworks and modules, and has difficulties finding the answer to "how do I...." on CPAN, this leaves a bad taste.

We have great documentation; Don't get me wrong. And a wealth of it. But I think we can do better. Much better.

I'll hash out specific ways our current documentation practices might be improved in Part 2, The Power of Shine, and Harnessing the Web (coming soon), and propose a possible new future for Perl and CPAN documentation in Part 3, An Evolution in Perl and CPAN Documentation (coming soon).