CPAN Archives

CPAN6 Is Here

Read this article on Rakudo.Party

If you've been following Rakudo's development since first language release on Christmas, 2015, you might've heard of numerous people working to bring CPAN support to Rakudo Perl 6.

Good news! It's finally here in usable form and you should start using it!

Let's talk about all the moving parts and how to upload your dists to CPAN.

The Moving Parts and Status Report

All of the heavy lifting has been done awhile back, during Perl Toolchain Summit and other times. I wasn't present for it to know the details, but to catch up you could join #perl6-toolchain chat and talk to humans or read the channel log. PAUSE/CPAN support for Perl 6 dists was implemented and zef module installer was trained to check for CPAN dists as well as our GitHub/GitLab-based ecosystem (called "p6c").

The only bit that was left missing is a front-end to browse available CPAN dists. There is a team who wished to take metacpan.org's codebase and modify it for Rakudo dists. I'm told that project is currently "stalled but not dead".

That's unfortunate, however, earlier this week, modules.perl6.org was taught to handle CPAN dists, so—hooray!—we finally have some sort of a front-end for CPAN dists. If you only want to see CPAN dists in search results, you can use from:cpan search qualifier (just like you can use from:github and from:gitlab ones).

GitHub/GitLab dists URLs still direct to repos, but CPAN dists have a file browser that lets you see what files are up in the dist. The file browser also renders README.md/README.markdown markdown readme files.

The viewer doesn't have all the bells and whistles of metacpan.org and doesn't (yet) render POD6, but it's certainly useable. The person who implemented this viewer will be busy preparing 6.d language release in the near future and won't have the time to make additional improvements to the CPAN dist viewer. So… you're invited to contribute and make it better!

Why Upload to CPAN

CPAN has many mirrors ensuring module installation is not affected whenever GitHub (a single website) has issues. The uploaded dists are also immutable and stay there forever (barring special deletion requests, even deleted dists remain available on BackPAN). This means people are more likely to trust these dists for use in their larger projects that need dependable dependencies. Lastly… it's what the cool kids use!

How to Upload to CPAN

Here's the process for how you can get your dists to CPAN. If these dists are currently listed in our p6c ecosystem, both p6c and CPAN versions will appear on modules.perl6.org, and you're encouraged to remove the p6c version. Some of the described tools are brand-new and others are brand-old, created before Rakudo existed, so treat this guide as part information and part invitation to improve the tools.

Step 1: Get a PAUSE account

PAUSE stands for "The [Perl programming] Authors Upload Server", it's located at pause.perl.org, and it's a site you use to upload dists to CPAN.

Go to request PAUSE account page and subscribe for an account. The "desired ID" field is for your PAUSE ID, and it's currently used as "author" field on modules.perl6.org. For example, mine is ZOFFIX.

I had my account for over a decade, so my memory is a bit fuzzy, but I think you'll need to wait for a human to approve and create your account—it's not instantaneous.

Step 2: Make a Dist Archive

You can manually create a tarball or a zip archive. I don't have all the details on which files you're supposed to have in them; you can take a look at other CPAN dists to see what they're doing or…

Use App::Mi6 module! It's possible you were already using it to create dists, in which case you're in luck, as you can just run mi6 dist to make a dist archive.

I rolled my dists by hand and wrote all the docs in README.md, so when I gave mi6 dist a whirl, it replaced my README.md with emptiness because I wasn't using any POD6—something (currently) to watch out for.

Step 3: Upload Your Dist

The first option is to upload manually: log into pause.perl.org, then go to Upload a file to CPAN, be sure to select Perl6 in the select input and then upload either via an uploaded file or a URL.

The second option is to use App::Mi6's mi6 upload command.

Shortly after the upload, you'll get an email about whether your upload succeeded (you can also see emails on nntp.perl.org). Make sure you have a META6.json file in your dist and that the dist version you're uploading is higher than the currently uploaded version. Those are the most common upload errors.

Step 4: Relax and Wait

If you're on IRC, in about 10 minutes after your upload, our buggable robot will announce it:

<buggable> New CPAN upload: Number-Denominate-1.001001.tar.gz by ZOFFIX
    https://www.cpan.org/authors/id/Z/ZO/ZOFFIX/Perl6/Number-Denominate-1.001001.tar.gz

In about 2 hours, the dist will also appear on modules.perl6.org. Its updater is started in a cron job on 20th and 40th minute of the hour (unless a job is already running) and it takes about 2 hours to finish each run.

Step 5: Celebrate with the Appropriate Amount of Fun

That's about it to the process. I foresee more tools will be created in the future to make the process even easier than it is today. If you have any questions or issues, just talk to a human or a robot on our #perl6 IRC channel.

Conclusion

CPAN support for Rakudo Perl 6 dists is now usably here. You're encouraged to upload your dists to CPAN, to grow a more dependable ecosystem. You're also invited to improve and create tooling that manages and displays CPAN uploads.

-Ofun

Perl 6: The Next Node.js, Done Right

If you've been a programmer for more than a couple of years, then you remember when Node.js first appeared. Some folks were raging about how insanely fast their ecosystem was growing, while others confusedly shrugged, wondering why anyone would want to use a language made for scripting browsers for server side work.

I can easily dismiss Node's large ecosystem of reinvented wheels, but the underlying point of why it got popular is valid: the fewer languages you need for an app, the fewer (or less experienced) programmers you have to hire, the cheaper it is to make your app. The self-proclaimed "HTML programmer" who was copy-pasting JS snippets from blogs and forums, now of all a sudden became a full-stack Web developer. But is using a made-for-browsers language for all-purpose work the right approach?

If you follow news or social media, today you definitely saw an article that invariably had a stock photo of someone playing Jenga. To sum up: lawyers pressured a developer to rename an NPM package. Developer refused. Lawyers came to NPM admins, who obliged to do so. The enraged developer then unpublished all of their NPM packages and the huge swath of the Interweb that depended on those packages chaotically crumbled and sufficient mocking of the whole system ensued.

The whole ordeal made me think of how such a situation might have played out in the Perl community. Would PAUSE admins remove an indexed module due to a request from lawyers, even when the author is responsive? Knowing personalities of a few of them, I doubt so. A communication channel among the involved parties would've been established instead.

And were a critical package actually deleted by someone, we have BackPAN, GitPAN, and dozens of slow-refreshing mirrors. We wouldn't need to take NPM's "an unprecedented step" to restore order to chaos. The system would've worked fine for awhile, and the breakage would be restricted to telling people to specify an actual package URL to the cpanm client. Not to mention, the NPM package that caused most of the breakage was tiny and some question why that little bit of code was even a dependency in the first place.

Going back to ecosystems. Browsing NPM, I can barely find packages with documentation, let alone tests for them; and I'm not the only one questioning quality. The Perl community, on the other hand, is a community of comprehensive tests. The quality of packages without tests is viewed with inherent suspicion and I've participated in many group mockings of packages that had no tests. Not to mention things like CPANTS Kwalitee Game that has "has tests" metric.

So what does my rambling have to do with Perl 6? The answer is the Perl 6 JavaScript Backend. Lead by Paweł Murias, the project was recently accepted for a grant and is well on its way to being usable.

You write Perl 6 code for both your backend and your frontend. The Perl 6 meant for the frontend gets compiled to JavaScript, so it can run in any browser that hasn't yet implemented native Perl 6 scripting in its core (c'mon, I can dream!). This is a complete inversion of the Node's paradigm: instead of a limited browser language made all-purpose, an all-purpose language's subset is made available in browsers. Throw into the mix the maturity and test-oriented culture of Perl, and I think you've got Node.js, Done Right.

So is the Jenga of Node starting to tumble down? Are we starting to realize the fad popularity of a language isn't the sole reason why your company should use it? Is there a successor ready to replace Node? Right now, I can only speculate, but in a couple of years, I'm sure Perl 6 will be the shiny new tool of a full-stack Web developer.

Bit Rot Thursday

Part 1: There is a Problem

I don't think I'd have to look for long for someone who'd agree that writing new code is much more fun that fixing bugs in old one. A cool new idea gets written up, while older code is still lacking tests. A new module gets shipped, while there's still that API improvement proposal from 6 months ago in the other. And while you're drafting a design document for the Next Awesome Thing, the rest of your code is being slowly consumed by bit rot.

Having written 250–300 Perl 5 modules and now 32 Perl 6 modules and other ideas, I'm more than aware of what it feels like to be leaving a decaying pile of code in your wake. The problems I notice are these:

  • Unfixed bugs
  • Lack of compresensive tests
  • Lack of documentation
  • Bad documentation (too wordy; incorrect; partial)
  • Unimplemented new features, even if the proposal for them was approved
  • Partial implementation (an FTP client that can only download, for example)

I can think of several reasons why it might be hard to find motivation to take care of bit rot, all of which are perfectly valid:

  • You're "too busy." You spent 8 hours at work, hacking on arcane code, and when you come home you want to relax and play Warframe and not spend 3 more hours hacking on arcane code. Your next Killer App is more important than this module you wrote 5 years ago because you were bored. Your interests changed: you no longer do web development with Perl 5 and instead are hacking on Artificial Intelligence software with Perl 6. And maybe you just have too many projects in the first place.
  • You don't know what needs to be done. Sure, CPAN Testers send you emails when something's broken, but it's easy to put off until the mythical "tomorrow." Your documentation is missing and users want new features, but you forgot that was brought up a few months ago. And what about Issues that go stale without any plan of action?
  • It's too hard. You set out to do a task and you implemented A, B, and C. Now, someone came up to you and said your thing is lacking X. Problem is, you don't have adequate knowledge and experience to implement X. And learning it requires both time and interest. Another example: users are reporting a nasty bug, but you can rarely reproduce it, and when you do, you still have no idea why it occurs.

And while entropy is a tough foe to conquer, I think at least acknowledging there is a problem is a good stepping stone to finding a workable solution.

Part 2: There is a Solution

I toyed with the idea of dedicating a special day to deal with just these sort of problems for a while. Today, I decided to commit myself to it, and I invite anyone willing, to do so themselves and address bit rot in their own software. The plan is simple:

Every Thursday is a "Bit Rot Thursday." You find some time to address bit rot and just do it. Why Thursday? I figure Friday, Saturday, and Sunday are all about relaxing; Monday, Tuesday, Wednesday are a beginning of the week people tend to "hate"; and Thursday is in the sweet spot: do work just before the weekend and feel good about yourself when the weekend comes. Since monotony is boring, you get one Thursday every month that you can "legally" skip and do nothing. Keep in mind, Bit Rot Thursday is not only about dealing with current bit rot, but doing preventative care as well. Let's see how we can accomplish those goals:

Toss It in the Bin

Before you rush off to fix a bug in your lolcat_phrase_generator_5000.pl, ask yourself: is this script or module still needed? You may feel attached to this "web framework" you wrote 8 years ago, but nobody—even you any more—uses it, so implementing HTTP/2 in it is likely a waste of time.

The modules you delete off CPAN will still be available on BackPAN, so don't be shy. You only have this much time in a day, and it's necessary to regularly throw clutter out. This is quite equivalent to accumulating useless trinkets in your garage, in case you'll need them some day. Just toss it in the bin.

Give It Away

If you don't think your software deserves such a harsh fate as being wiped from the face of Earth, consider putting it up for adoption. You can use blogs, social media, and IRC to advertise that you're looking for a new maintainer who'd be interested in fixing issues. On CPAN, you can transfer the module to user ADOPTME, to enlist your module as adoptable by interested parties. Perl 6 has its own version of the idea: place the META file into the SHELTER and then remove your module from the Ecosystem.

Delegate

Instead of parting with your goods completely, delegate. The Mojolicious project is a good example of how to seek out volunteers: Issue labels future and help wanted are used to indicate work to be done and social media is used to advertise and find volunteers willing to do the required work.

Did someone submit a report for a bug? Ask the submitter whether they'd be able to fix it. Did someone request a new feature? Suggest you'd accept a patch. Include details on what you think might be causing the bug and outline the details of how a feature can be implemented. Even if that person is unable to help, you'll have a plan of action already in writing.

Find and Review Reports

Go through RT tickets, GitHub Issues, your notes and reminders. When you log in to RT, the Bugs in My Distributions panel is third on the left. On GitHub, click the Issues link in top, center of the page and then change author: to user: in the search box. This will show you all open Issues in your code. You can use user: more than once, if you wish to also include issues from GitHub organizations you belong to. As an example, here's a search query that shows both Issues in my code and in Perl 6 organization: is:open is:issue user:zoffixznet user:perl6

Get a MetaCPAN account and try out the Dashboard MetaCPAN Lab to see the number of open issues and test failures in all your modules: https://metacpan.org/lab/dashboard.

Categorize the issues using tags and labels. Merge duplicates. Close anything that's irrelevant, can't or won't be fixed. If the conversation died out, just close the ticket with a request to reopen it if anyone has the same issue. There's no reason for tickets to be left open for years.

Evaluate Unreported Things

Go to CPAN Testers and browse by Author to your Author page (middle of page, third section). You can then use the Preferences panel on the left to generate a link to show failures only. Bookmark it. Examine any failures and see if you can fix them. When you upload new modules, if their tests are failing, you should be getting an email from CPAN Testers informing you there is a problem. If you're not getting those, check your settings and your spam folder.

Use Devel::Cover to evaluate how well your tests cover your code. If you use Dist::Zilla, there's Dist::Zilla::App::Command::cover that adds cover command to dzil.

To ensure your documentation is complete, add Pod::Coverage to your author tests (that is, place it in xt/ directory rather than t/ so it doesn't run during user's installation). Completeness is one thing, but quality is another. Examine your extant documentation. Try to reduce the amount of words in it, while still retaining information. Use examples! A good, clear example can shave off a whole paragraph of dense technical text. There's nothing wrong with actively inquiring your users about the quality of the documentation. Do people frequently ask a particular question? Creating a FAQ is one way to go, but it's often an indicator your actual documentation can be improved in that area.

In Perl 6, Test::META lets you find problems in your META file and Pod::Coverage distribution includes Test::Coverage module to check completeness of your docs.

Prevent Bit Rot

So far, our Bit Rot Thursday sounds mundane and boring, dealing with tedious problems. However, you can actively work against those problems appearing in the first place.

When you start a new project, do you write down a clear definition of the problem you're attempting to solve? A sure-fire way to waste time is to spend several days adding a ton of configuration options and features that you don't have any use for, simply because you never defined a concrete problem to solve. Worse: you continue to support and maintain all those features for years to come!

When you lay down the first line of code, do you have some form of a "spec" that details the bits and pieces of your project and how they're supposed to interact with each other? I'll leave it to other sources to argue for cost savings of such an approach, but I can tell you that when you realize JSON and not PDF is better suited for the output of your program, rewriting two paragraphs of text in the spec is much simpler than rewriting several subroutines, methods, and tests.

A design spec also makes it much easier to create both tests and user documentation for your project. The test suite is the spec translated into a computer language. The docs are the spec with internal details removed and code examples added.

Speaking of tests, it's often handy to write them first, before your actual module or script. The tests define the problem you're attempting to solve. Failing tests indicate what bits you haven't solved yet. You can also very easily track your progress, as you can see how many tests still fail. Since this approach generates a lot of output on each test run, you may find Test::Most's DIE_ON_FAIL useful. You can include it in code or use DIE_ON_FAIL=1 environmental variable and this will cause the test suite to stop at the first failure.

So spend your Bit Rot Thursday making templates for specs and tests and outlining plans for what sort of documents need to be prepared for the types of projects you create. It's also a good idea to draft up guidelines for contributors, to make it clearer what sort of contributions you're looking for. You'll spend less time explaining your standards in pull requests!

Growing Yourself

Who is more likely to produce a wall that'll crumble in a year: an experienced bricklayer or a novice? Writing code certainly gives you practice, but you need theory too, as well as to keep yourself up to date with the world. You can write a million lines of code that uses CGI.pm and memorize every word in its documentation, but in 2016 you'll still be unemployable in the field of Web Development if that's all you can do. Technologies change. Standards change. Practices change. We should change too.

And so, it's possible to spend the Bit Rot Thursday away from the keyboard. On a train, plane, bus, or boat, with your phone or tablet, reading programming blogs and articles. At a bar with your fellow programmers, exchanging ideas. In a park, on a sunny day, with a programming book in your hand.

And as you get better, so will your code...

About Zoffix Znet

user-pic I blog about Perl.