Olaf Alders

Ab(Using) the MetaCPAN API for Fun and Profit v2013

2013-06-03T18:00:39Z

I'll be speaking about using (and abusing) the MetaCPAN API on day 2 of YAPC::NA. If you think you might be attending (or even if you're not), I thought I'd post some links to some helpful info. No, you don't need to do any prep for this talk, but for those who are interested, this is just some background info that will help you get your head around MetaCPAN.

This talk will touch on and expand on some of the points I discussed last year, so you could have a quick look at last year's slides. I've also put together some sample code for tomorrow's talk. If you use Carton, you can follow the instructions in the top-level README to get started in just a couple of minutes.

On a barely related note, iCPAN 2.1.0 is now in the app store. I used it on the flight down to prepare for my talk. Seemed to work well for my purposes. :)

Sifting Through the CPAN

2013-03-09T03:32:01Z

Some interesting stuff has been said today on The Four Major Problems with CPAN, Don't release experiments to CPAN and CPAN is for experimentation and I hope that never changes.

If you haven't read these posts yet, I encourage you to do so. They've all got interesting things to say.

(As an aside, let me say that I think PAUSE authors could probably participate more on PrePAN. I've posted a few things there and found it quite helpful. Even if your module is already on CPAN and you just want feedback on it, post something on PrePAN. I've found the discussion to be of a good quality and actually quite positive.)

One of the proposals which was being discussed was not (immediately) uploading experiments to CPAN. Now, even if we were to propose people uploading less modules (which, to be clear, I'm not proposing), we still have many years worth of good, less good and kind of crazy code to sift through. What this tells me is not that CPAN is full of junk, deprecated modules and weird stuff, but that we need better search tools to deal with the code we already have. The web on the whole is full of stuff of questionable quality, but modern search engines generally do a decent job of cutting through the noise. We should also be able to cut through the noise. This is one of the goals of MetaCPAN, but we haven't quite gotten there.

Right from the beginning, we've wanted to implement some system of tagging, which would allow interested parties to impose their own views on what a module is or is not. Imagine tags like "deprecated", "author unresponsive", "author deceased", "best practices", "needs love", "looking for maint/co-maint". The list goes on. You just need to use your imagination. These sorts of tags could a) help improve search results and b) help humans sift through the results when they do get them. There's a certain amount of heuristics which can be applied in the parsing and searching phase, but really I think we just need humans to have more influence on the outcome of results and tags are a great way to do that.

Now, where is this tagging of which I speak? Well, for now it's just a glimmer in our eyes. My hope was that last year we could round up a GSoC student to take on the tagging project, but with TPF not being one of the accepted organizations in 2012, it didn't happen for us. There is still hope for GSoC 2013, which has not yet begun accepting applications. There is also hope that outside the scope of a program like GSoC someone will step forward and "just do it". Along those lines, I actually think this would be a good candidate for TPF grant. I've even considered applying for the grant myself, but I'd much rather if it were done by someone with a schedule that's a little more flexible than mine currently is.

We don't need to be stuck with trying to figure out which module is best for which task. That's a problem which we, to a great degree, have the ability to solve. It's a layer on top of the CPAN that we can provide. We just need to do it.

MetaCPAN is Moving Today (Dec 31, 2012)

2012-12-31T06:08:56Z

This is just a quick note to let everyone know that MetaCPAN is moving to a new hosting sponsor today (Dec 31st). Our old hosting relationship with speedchilli.com has unfortunately come to an end. We've been quite grateful for and more than satisfied with their sponsorship over the last 1.5 years or so. Having the hardware and bandwidth which speedchilli freely made available to us was instrumental in the early days. (Initially everything was running off a tiny instance in the Rackspace cloud. It wasn't much, but it was all we could afford after passing the hat around). Having speedchill sponsor us allowed everyone who contributes to MetaCPAN create something truly special.

We'll have an announcement in the near future, to formally thank our excellent, new hosting sponsor. At this point I just want to say that there is a lot of work going on behind the scenes to move everything over to the new machines. The bulk of the actual move has been taken on by Moritz Onken, who was able to jump in over the holidays to just get it done. I should also thank Leo Lapworth for moving our deployment to Puppet and for nagging everyone to make sure it was working. This was done well before we knew that we'd need a new hosting sponsor, but it sure makes re-deploying a lot easier. I'd also like to thank Clinton Gormley, Florian Ragwitz and Randy Stauner for their contributions over the last little while. There are a lot of people working in the background to keep MetaCPAN running smoothly. I certainly haven't mentioned all of them here, but we're grateful for all of their efforts.

Part of the new deployment requires a full re-index of both CPAN and BackPAN, so there may be some dists which have not been indexed at the time when we switch over the DNS. So, please be patient if some search results aren't returning exactly what you want. That will be fixed as the indexing completes.

Have you added your Perl project to 24pullrequests.com?

2012-12-01T04:02:41Z

24pullrequests.com takes the spirit of the Advent coding calendar and puts a new spin on it. Try to send one pull request per day for the next 24 days and 24pullrequests.com will help you track your progress. At the time of writing, 1,224 devs have already signed up.

Some Perl projects are already represented, but there's still room for more. It would be good to see some more Perl projects on the list for participating Perl devs who are looking for something to contribute to.

Introducing HTTP::CookieMonster

2012-09-19T16:55:47Z

If you've ever had the pleasure of poking around in your WWW::Mechanize or LWP::UserAgent cookie_jar, you'll know it's not an entirely painless process. It's certainly not impossible, but it feels a bit like jumping through hoops. The cookie_jar functionality in LWP::UserAgent and the modules which inherit from it is provided by HTTP::Cookies. Before I go any further, I'd like to thanks Gisle Aas for HTTP::Cookies, which is a very important bit of code. This isn't a complaint about HTTP::Cookies, but rather an attempt to make it even more accessible.

Before we do anything with cookies, let's make one request to ensure we have some cookies in our cookie_jar.

    use WWW::Mechanize;
    my $mech = WWW::Mechanize->new;
    $mech->get('http://www.nytimes.com');

nytimes.com sets a couple of cookies by default and they are now in your cookie_jar. You can now get a list of your cookies (as objects) with just a "use" statement and 1 additional line of code.

    use HTTP::CookieMonster qw( cookies );
    my @cookies = cookies( $mech->cookie_jar );

Now you can iterate over your cookies. Available methods are documented at https://metacpan.org/module/HTTP::CookieMonster::Cookie.

    foreach my $cookie ( @cookies ) {
        print $cookie->key, $cookie->val, $cookie->expires, "\n";
    }

Maybe you just want to get one particular cookie without having to loop over an array. The NYT website serves an "RMID" cookie.

    my $cookie = cookies( $mech->cookie_jar, 'RMID' );

There's one caveat here. It's quite possible that you could have multiple cookies in your cookie jar with the same name, since there's nothing to stop many different domains (or subdomains) from serving up cookies with the same name. If you're not sure, get an array of cookies instead.

    my @cookies = cookies( $mech->cookie_jar, 'RMID' );

If you get more than one cookie, you'll need to figure out which one you really want. Calling cookies($cookie_jar, 'cookie_name') in scalar context will return the first matching cookie. Calling it in list context will return all cookies in the order in which HTTP::Cookies provides them.

Now, let's say you want to mangle your cookies. That's easy to do via the OO interface.

    use HTTP::CookieMonster;
    my $monster = HTTP::CookieMonster->new( $mech->cookie_jar );
    my $cookie = $monster->get_cookie('cookie_name');
    print $cookie->val;

    $cookie->val('random stuff');

    $monster->set_cookie( $cookie );

    # now fetch a page using the mangled cookie

    $mech->get( $url );

Now that we can easily read our cookies, we can also easily test them:

    use Test::Most;
    ok( $cookie->secure, "this is a secure cookie" );

It's also simple to add a new cookie to the jar:

    use HTTP::CookieMonster::Cookie;
    my $cookie = HTTP::CookieMonster::Cookie->new
       key       => 'cookie-name',
       val       => 'cookie-val',
       path      => '/',
       domain    => '.somedomain.org',
       path_spec => 1,
       secure    => 0,
       expires   => 1376081877
    );

    $monster->set_cookie( $cookie );

    $mech->get( $url );

HTTP::CookieMonster will re-examine your cookie_jar every time you call it. This means that it can keep up with changes to the cookie_jar which happen outside of your code. Just think of it as a simple wrapper around HTTP::Cookies. The interface is not yet set in stone, but I think it's quite useable already. Feedback is most welcome.

Using Plack::Middleware::CSRFBlock and jQuery to deal with Cross Site Request Forgery

2012-07-19T20:00:41Z

At $work, our flagship application was recently audited for potential security issues. One of the items which raised a red flag was the fact that we weren't dealing with the threat of CSRF (Cross Site Request Forgery). The solution which we decided to implement was to add a CSRF token to all POST requests. This token should only be known to the app and the end user. Passing it along with a POST request gives some measure of assurance that a POST by the user is intentional and so can help to reduce the risk of CSRF.

On the Perl side of things, it was actually easy to implement. Since the app is deployed under Plack, CSRF tokens could easily be implemented with Plack::Middleware::CSRFBlock, albeit a slightly modified version (which I still need to contact the author about). This module parses outgoing HTML which is produced by your app. If it finds a form which can POST, it creates a hidden form input field inside the form and inserts a CSRF token into it.

So, that takes care of your traditional forms in HTML, but it doesn't detect your fancy jQuery Ajaxy goodness. It turns out, that's fairly easy to handle in one place as well:

        var token = 'some_random_secret';
        // some defaults for all ajax calls
        $.ajaxSetup({
            cache: false,
            data:  {'csrf_token': token}
        });

And with that, we re-submitted ourselves to the Burp Suite scanning software, hoping for a clean bill of health. Again, it was not meant to be. This time, the Burp Suite pointed out that the CSRF token is not very secret when it is attached to the params of a GET request. It can show up in browser history, web server logs, etc. That is sub-optimal to be sure. As far as fixing this goes, I was still looking for a solution to fix this in one place. The app is big enough on JS that I wasn't about to scour for every GET and fix this manually. It turns out that jQuery has a solution for this as well, courtesy of StackOverflow:

        var token = 'some_random_secret';
        // some defaults for all ajax calls
        $.ajaxSetup({
            cache: false
        });

        $.ajaxPrefilter(function (options, originalOptions, jqXHR) {

            // do not send data for GET/PUT/DELETE

            if(originalOptions.type !== 'POST' || options.type !== 'POST') {

                return;

            }

            options.data = $.param($.extend(originalOptions.data, { csrf_token: token }));

        });

This allows us to mangle only the params of POST requests, leaving GET, PUT and DELETE untouched. That's basically it. There were still some additional hacks we had to put into place in order to get the tokens to work with 3rd party plugins (like tinyMCE), but for the most part this was a very clean and simple solution. It didn't clutter the app with a lot of extra code (Perl or JS) which will need to be maintained. If you can fix something without seeing a big bump in lines of code, people will be more inclined to thank you down the line.

I should add the caveat that using Plack::Middleware to solve this particular problem may not scale well for your particular case. The app in question doesn't deal with high volumes of traffic, so the tradeoff of some extra computation versus a comprehensive HTML audit made it a very tidy solution for us.

iCPAN 2.0.0 now Available in the App Store

2012-05-24T05:19:59Z

I'm happy to announce that iCPAN 2.0.0 is now in the app store. If you're not familiar with it, iCPAN is a free iOS app which allows you to browse CPAN Pod on your iPhone/iPod/iPad. The previous version of iCPAN was released in November of 2010, so this has been a long time in coming.

If only CPAN had a web service...

Originally, my hope was to release frequent iCPAN updates with new Pod, bug fixes and improvements. However, I was running into a lot of problems with getting good coverage of CPAN modules. There was (not yet) a web service which you could query to find out what is currently indexed in CPAN or where you can find all the Pod in an arbitrary distribution. I had been rolling my own solution with the help of some CPAN modules, but my coverage was only around 60,000 documents, which was poor. After discussing this with some friends at PerlMongers in Toronto, I started hacking on an API to provide the Pod as well as some metadata. The API soon had a name: MetaCPAN. Working on MetaCPAN was so much fun and so productive that consequently most of my open source efforts were directed toward this project. This all took place around the time of the last iCPAN update, but I still had hopes to keep iCPAN reasonably up to date. After all, it only took about 6 weeks of evenings and weekends to get a proof of concept up and running for MetaCPAN. I would have plenty of time to keep iCPAN running as well.

How did this break?

Things may have stayed reasonably on track, but an iOS update broke iCPAN along the way. After many hours of debugging, I was just not able to find the root of the issue, so I decided to rewrite iCPAN as a Universal app (one with a native iPhone and iPad UI). I started working on the iPad, but my personal resources were dwindling. The number of developers working on iCPAN had been cut in half (from 2 to 1) and the number of kids in our family had doubled (from 1 to 2). Now with a newborn in the home, I didn't have the time or energy to finish off the app. Any spare time I did have available was generally spent making some small contributions to MetaCPAN where I could.

Any day now...

By July of 2011 things had settled a bit and I was back at it, spending a lot of late nights finishing up the iPad UI. I was optimistic that I'd be able to release iCPAN soon. However, as I soon discovered, the consistent late nights were quite productive in the short term, but the sleep deficit over the long term made me generally unproductive during the day. I eventually had to concede that my brain doesn't function well on little sleep, so I gave up a lot of the late nights and put iCPAN on the shelf. It was nice to have my brain back.

A few months ago, I got the bug to finish things up. It had really been bothering me that I had been telling people "this thing is almost done" for so many months. I also saw the excellent CPAN Sidekick in the Android app store and I was starting to feel a bit embarrassed about not having iCPAN working. Bad (but fair) reviews for iCPAN were piling up. I buckled down and spent any available free time on getting this done. It turned out to be vastly more work than I had anticipated or budgeted, but I did get the app to a releasable state about 1 week ago when I submitted it to the app store.

The Post-Mortem

I've learned a lot along the way. I've learned Objective-C, XCode, Interface Builder and iTunes Connect. I've learned about managing certs for releasing apps, how
to distribute an app via an Ad Hoc Distribution (TestFlightApp in my case) and how to keep the iPhone and iPad UI in the same app while making sure that the logic
uses shared classes. It has also made me appreciate Perl all that much, in particular CPAN, Moose, DBIx::Class, built-in regex support and the ability to express
so much in Perl with very little code. I've learned Apple's Core Data. I know how to optimize an SQLite database with 80K rows in a table so that it runs smoothly on a phone. (I could write an entire post on that topic alone).

iCPAN 2.0.0 has just over 80,000 documents available, which is a huge improvement over the previous version. It has been downloaded over 1,300 times in the last 1.5 years. If you do like it, please take a moment to review it in the app store or watch it on Github. If you don't like it, please contact me or, even better, send me a pull request. :)

MetaCPAN at the QA Hackathon

2012-04-09T02:47:47Z

One week ago, I happily had the opportunity to be at the QA Hackathon in Paris. In the past I had been vaguely aware that the hackathon exists and I had some shadowy idea of what goes on at such a thing, but I just never considered getting involved. I didn't think it was very much related to the sorts of things I work on. Happily, it turns out that I was wrong.

First off, thanks!

Before I even talk about the code, I need to thank the organizers and the sponsors. Philippe Bruhat (BooK), Laurent Boivin (elbeho) and Les Mongueurs de Perl did an absolutely fantastic job keeping a large group of people housed, fed and occupied. The only responsibility I had to take for myself was booking a flight and finding my way from the airport to the hotel. Beyond that, pretty much everything was taken care of. The venue (which was really cool) was maybe a 5 minute walk from the hotel. The dinners I attended were great. At the hackathon itself there was an impressive supply of food (including fresh fruit) pretty much always available. Your needs were met. :)

This, of course, would not have been possible on this scale without the help of many generous sponsors. It's encouraging to see that companies (and individuals) see the value in this sort of work. Please note that I am lifting this list of sponsors directly from David Golden's Hackathon post:

The City of Science and Industry, Diabolo.com, Dijkmat, DuckDuckGo, Dyn, Freeside Internet Services, Hedera Technology, Jaguar Network, Mongueurs de Perl, Shadowcat Systems Limited, SPLIO, TECLIB', Weborama, and $foo Magazine

These people made individual donations: Martin Evans, Mark Keating, Prakash Kailasa, Neil Bowers, 加藤敦 (Ktat), Karen Pauley, Chad Davis, Franck Cuny, 近藤嘉雪, Tomohiro Hosaka, Syohei Yoshida, 牧大輔 (lestrrat), and Laurent Boivin

Some of the highlights for MetaCPAN

A very short chat with Andreas Koenig set off a chain of events that is seeing 2packages.details updated much more frequently than it was up until now. This means that newly uploaded modules will be searchable on MetaCPAN much more quickly than they have been over the last year or so. This is a major improvement and one that I'm particularly happy about. We still have some work to do on MetaCPAN to take advantage of the more frequent updates, but it's excellent to have them.
Florian Ragwitz (rafl) and Matthew Horsfall (wolfsage) created a new /distribution endpoint to the MetaCPAN API. This endpoint currently contains only a summary of the RT bug counts. In future, Github issues and other things which are not specific to a versioned distribution will also be found here.
Michael Peters (wonko) fixed a bug in the indexer which affected about 800 packages. He also wrote a command line script we can use to analyze which modules are missing from the index and why that might be. This is going to be a very helpful diagnostic tool and one that we, frankly, needed a long time ago. We're very happy to have this now.
Nicholas Perez (nperez) wrote MetaCPAN::API::Tiny, which is basically an implementation of MetaCPAN::API
After David Golden (dagolden) mentioned that a MetaCPAN command line interface would be helpful, Chris Nehren (apeiron) began working on it. I'm not sure why nobody thought to create one in the past, but this is going to be another helpful tool to come out the hackathon.
I spent some of my time fixing bugs and adding a bit of new stuff, but most of my time was spent looking over existing pull requests as well as dealing with pull requests from the hackathon itself. I was also able to spend time with new MetaCPAN hackers, getting them started with their own local deployments as well as working on the codebase. I didn't get to write as much code as I had planned to, but spending time accepting patches and working with developers who are new to the project is an ideal situation to be in.

A New MetaCPAN Logo

As part of the weekend, we were also able to announce Raul Matei as the winner of the MetaCPAN logo contest, which EPO so kindly provided the prize money for. We went live with the new logo immediately, which looks great and takes care of one more missing piece of the puzzle.

Wrapping Up

We now have a number of new hackers who are armed with enough knowledge to bend the MetaCPAN code to their will. Together we fixed bugs, improved the test suite, worked on new functionality, dreamed up even cooler things to do and generally got a lot of good things done.

For MetaCPAN, the hackathon feels like a great success. I'm so pleased to have been a part of it. I got to meet lots of great people, hang out in a beautiful city and focus my daytime hours on MetaCPAN without any distractions. I've also got a much clearer understanding of what the QA Hackathon is all about it and I see how valuable it really is to get all of these hackers in the same room together. It's so much easier when you can walk up to some and *actually* poke them for information rather than having to wait until they check IRC, email, a bug queue etc. That in itself is a big win.

++ Goes on Hiatus

2011-12-30T06:08:05Z

If you saw @kraih's recent MetaCPAN tweet, you'll know that MetaCPAN's ++ feature has been gamed. Now, we were aware there was some potential for gaming. Initially you needed a PAUSE id to be able to ++, but this had an unexpected side effect in that there were some requests for PAUSE accounts with the justification of "I'd like to be able to ++ on MetaCPAN". Because this placed an additional burden on the already busy PAUSE admins, we were asked to remove this requirement.

The result of this was that the bar was now much lower for someone who wanted to create a MetaCPAN account and begin using the ++ feature. People behaved and used the feature as it was meant to be used. It was helpful, a great addition to the site and it led to a fun little leaderboard where you could get a glimpse of what people in the Perl community care about in general. You could also view recent ++ activity, which could give you an indication of what is trending "right now". So far, so good. The code was written in good faith and the community used it in that same spirit.

Earlier this week someone start to game the system, apparently creating fake Github accounts, creating MetaCPAN accounts with that Github login and then boosting first Dancer and the Mojolicious in the leaderboard. I don't know if it was the same person and I really don't care to do the research. By later today the bot was adding ++ data for modules which didn't even exist -- a bug we weren't aware of. It was getting to the point of silliness and the ++ feature had at best been rendered useless and at worst become somewhat of an embarrassment.

So, earlier this evening, I turned off the ++ feature on the MetaCPAN front end. I'm not going to pontificate on whether the bot behaviour is good (points out an obvious weakness in the code) or if it's bad behaviour (the exploit was run without an appropriate issue being opened first). What I am going to say is that a feature which was very helpful to many people has been now taken offline. A lot of work goes into this stuff and some of that work has been rendered useless for the time being. I can't give any kind of estimate as to when this might come back online. The limiting factors are a) finding an appropriate way of limiting abuse and b) finding a volunteer to sink the necessary hours into it.

So, I'd like to extend my congratulations to the person(s) who sunk the time into defeating the ++ system, but who conversely can't be bothered to provide a patch to fix the hole. You've accomplished what you set out to do and now somebody else gets to clean up after you. Your parents must be so proud.

Not a designer? Get involved with the MetaCPAN logo contest anyway!

2011-12-20T04:26:21Z

The MetaCPAN logo contest is now in full swing, thanks in no small part to FLORA for his blood, sweat and tears in organizing it and also to the Enlightened Perl Organization, which fully funded this contest with astonishing speed.

Just to recap, the contest rules are posted at contest.metacpan.org and the entries are being posted at entries.contest.metacpan.org.

Now, you don't need to be a designer to get involved. Here are a couple of ways you can help out:

1) Get the word out. Let your designer friends know there is $400 up for grabs (not to mention eternal glory). If you're active on Twitter, please tweet about it as well. Send an email to your local PerlMongers group etc.

2) Comment on the entries. Because there is no limit on the number of entries per contestant, the logo submission process is allowed to be iterative. Constructive criticism can lead to authors resubmitting tweaked versions of their logos, which can only be good for the contest.

As aside, anyone who has contributed to MetaCPAN will be able to vote on submissions. If you'd like to vote, it's not too late to get a pull request in to MetaCPAN. Fix an issue, send a documentation patch, etc. There are lots of place to help. If you're unsure of where to begin, say hello on #metacpan and someone will be sure to point you in the right direction.

Happy Birthday, MetaCPAN!

2011-11-03T03:55:19Z

It was exactly 1 year ago today (Nov 3, 2010) that I registered the metacpan.org domain name. Over the course of this year, the project has gone from a couple of guys writing some code to a true community effort. MetaCPAN is steadily being adopted by Perl developers around the globe and it is steadily gaining in functionality.

Together we've created a free, open, collaborative project which makes new tools available to CPAN lovers everywhere. The API has spawned some cool and very useful projects. The web site has spawned some much needed functionality in the form of diffs, syntax highlighting, author profiles etc.

What you may not know is that MetaCPAN is now set up in such a way that all resources (servers, DNS, email, domain registration, etc) can be configured by a handful of core devs. This means that if any one (or several) core devs drop out of the picture, the project can continue on as before. There is no one person who can act as a blocker to the rest of the contributors and anything which might be considered controversial requires consensus before it gets pushed to the master branch. So far, I think this has yielded good results.

For example, here's the latest roundup, which does not include numerous bug fixes and other tweaks by PERLER, among others:

Sept 6: PLU adds a contributors leaderboard

Dyn officially comes on board as our DNS sponsor

Sept 9: RWSTAUNER adds CPANRatings links to /release and /module pages

Sept 12: KENTNL adds identi.ca to author profiles

Sept 15: JROBINSON adds flattr to author profiles

DRAEGTUN adds hackernews to author profiles

Sept 19: KENTNL adds geekli.st to author profiles

Sept 27: LLAP edits learn.perl.org links to point at MetaCPAN

Oct 14: MIYAGAWA updates cpanminus to support the following syntax: "cpanm --metacpan" (code courtesy of TOKUHIROM)

Oct 16: TOKUHIROM posts his slides from his YAPC::Asia talk on MetaCPAN.

Oct 17: KENTNL comes up with a nice little hack to employ caching in MetaCPAN::API calls

Oct 18: AZAWAWI integrates MetaCPAN into Padre

Oct 19: AZAWAWI adds the ability for Padre to fetch Pod directly from MetaCPAN with the option of inserting the SYNOPSIS directly into your document

Oct 20: The video of TOKUHIROM's MetaCPAN talk is posted. That is a quick turnaround on posting video. :)

Oct 21: I add a chronologically sorted list of distributions to get a ++ on MetaCPAN. (A ++ leaderboard is also made available).

Oct 26: Dimitar Petrov now has mcpan.org online. This means converting links from search.cpan.org to MetaCPAN is as simple as adding the letter "m": search.mcpan.org

RWSTAUNER adds syntax highlighting for .c, .h and .xss See https://metacpan.org/source/DOY/Moose-2.0205/mop.c

Oct 28: RWSTAUNER adds syntax highlighting to Changes files. See https://metacpan.org/source/BRICAS/CPAN-Changes-0.18/Changes

Objective-C for Perl Hackers

2011-09-30T03:26:58Z

I've been messing around with Objective-C off and on for a while now. Whenever I take a break and come back to it, I find myself having to look up some of the same syntax over and over. Some examples would be:

Perl:



$foo =~ s{bar}{baz}g;

Objective-C:



foo = [foo stringByReplacingOccurrencesOfString:@"bar" withString:@"baz"];

Perl:



$list{'foo'} = 'baz';

Objective-C:



[list setObject:baz forKey:@"foo"];

One thing in particular that gets me is the assigning a value to a dictionary, like in the example above. Naming the value *before* the key seems counter-intuitive to me. Luckily Xcode is great with autocompletion, which makes a lot of this easier, but I still find I need reference materials which are more Perlish. To make things easier for myself, I've set up a wiki to map Perl syntax to Objective-C where it makes sense.

Feel free to edit/update/correct the wiki where you see fit. I make no claims of expertise here, but I do find myself referring back to this list and adding to it as I write my own Objective-C code.

https://github.com/oalders/objective-c-for-perl-hackers/wiki.

MetaCPAN August Roundup

2011-09-01T05:37:26Z

August was another busy month for MetaCPAN, so I think a recap is in order. I'll proceed chronologically.

2011-08-01

Gabor Szabo included MetaCPAN in his first issue of the Perl Weekly. It's an excellent aggregation of all Perl things blog and newsworthy. I can tell you just from the time involved in a MetaCPAN recap that Gabor is spending a huge amount of time on his weekly newsletters. I follow a lot of blog posts, but I find his weekly recap helps me find interesting stuff I may have missed.

2011-08-03

Have you every blown away a lot of data you didn't have a backup for? I know I have. It was client data and it was ugly. A lot of us have been there and precisely this happened to MetaCPAN. In our rush to get cool features online, we put off taking snapshots of all the cool data Perl hackers were adding to the system. On this particular day, it came back to bite us. So, what was effectively a hard reset forced us to re-evaluate, re-index and put a proper backup strategy in effect. You could say our new slogan is "The New MetaCPAN: Now with Backups!".

If you added any info (author or +1) before Aug 3 and haven't logged in since, you'll need to clear your MetaCPAN cookies, reauthenticate and start from scratch. It's sub-optimal, to be sure, but lessons have been learned, more hard disks have been installed and we're working on sponsorship to get a second server online which we can fail over to.

2011-08-04

There had been some complaints about not being able to diff on MetaCPAN, so on Aug 4th Mo put a little something together to do just that. It looks great, works well and may have been some small compensation for the events of the previous day. :)

On the same day RJBS announced that his advent calendars now link to MetaCPAN rather than search.cpan.org!

2011-08-08

On this day SARTAK added wishlist functionality to MetaCPAN. If you find the PayPal button too mercenary, you may appreciate the wishlist as a way of letting others thank you for your hard work. Log in to add this to your author profiles.

2011-08-10

On this day YANICK added klout.com to the author profiles.

2011-08-16

On this day AYOUNG added Ohloh.net to author profiles and I added github-meets-cpan.com to the profiles as well.

2011-08-17

On this date mo gave a MetaCPAN lightning talk at YAPC::EU, complete with impressive graphics.

2011-08-18

On this day DOHERTY rebranded the +1 to ++ to make it more Perlish and less Googlesque.

2011-08-19

This date marks the intial Github commit of wata's excellent CPANHelper Chrome helper extension. I've been making use of this extension every day and I can tell you it's very cool. I highly recommend it. Install it now.

On this date I also became aware that TOKUHIROM is slated to give a MetaCPAN talk at YAPC::ASIA. Kudos to him for submitting the proposal. He has done a lot of work with the API and I hope he posts his slides. :)

2011-08-22

Around this date, I added links to Changelogs on MetaCPAN at the release level. This was a much-requested feature.

2011-08-23

On this date Mo added "reverse dependency" listings.

2011-08-24

VTI came up with another gem on Aug 24th. PerlResume.org is an excellent example of the kind of cool stuff you can do with the MetaCPAN API. Look yourself up and see how your Perl Résumé stacks up against those of other authors.

2011-08-29

Around this time I became aware of the fact that p3rl.org now points at MetaCPAN. Try it: http://p3rl.org/shit

August also marks the wrap-up for Mo's Google Summer of Code project, which was a huge success. Over the coming weeks we'll be putting a road map together, to make plans for going forward without the same GSoC resources at our disposal. We've got a really solid foundation to work on going forward, so we're feeling quite good about how the project is progressing.

In other news, we're working to bring a new sponsor on board this week and I'll have an update on that once that has been set up.

My apologies to any contributions I may have inadvertently missed in this roundup. It wasn't intentional.

If you'd like to get involved, stop by #metacpan in irc.perl.org, follow us on Twitter or just comment on this post.

Kickstarting the Gitpan

2011-08-31T04:28:01Z

If you've played with Gitpan at all, you probably have an idea of how cool this project is. The idea behind Gitpan is to take all of CPAN and make it available as Git repositories which are tagged by release. It's a great idea. It makes it trivial for Github (and Git users in general) to create patches for modules which don't already exist in a public Git repo. It's also useful in the case where the original Git repositories for modules on CPAN may no longer easily be found online.

Now, if you've followed the Gitpan project, you probably also know that it is not currently up to date, which is a shame. I contacted Schwern about this recently and he was kind enough to create and tag some issues which are blockers to the resumption of Gitpan.

If anyone out there has some interest in getting this project back online, there's enough info in these issues to get you started. There may well also be some resources in the MetaCPAN API which could speed the work along as well. If Gitpan gets back up and running it would be one more resource we could link to from metacpan.org and could likely lead to some more interesting mashups down the line.

If you do feel inclined to do some heavy lifting on this, please stop by #metacpan so that we can give you some guidance as to how best to integrate MetaCPAN with Gitpan.

MetaCPAN Undergoes "Unplanned Re-index"

2011-08-03T17:02:50Z

Earlier today a some dev work on MetaCPAN had the unfortunate side effect of dropping the ElasticSearch index. This has effectively forced us to re-index the site from scratch. What makes this particularly painful is that we've lost the author data (including +1) which so many Perl devs have lovingly created over the past few weeks.

The short story is that minicpan should be indexed within a few hours. The rest of CPAN and BackPAN will likely be in the next day or so.

Because we've been looking at adding new features and haven't spent time on a proper backup strategy, we're in a suboptimal place right now. The good news is that this didn't happen several weeks from now with much more author data. Also, we're now forced to find a workable backup solution as our first priority.

I think now would be a good time to put out a call for volunteers. If you're inclined to help dig in with some sysadmin-minded tasks or just want to help with the overall MetaCPAN ecosystem, we could really use your help. The interest in the project is great, but so is the number of open issues, which grows by the day.

The more devs we have who can dig in and get something done, the closer we'll be to getting a bullet-proof system online which we can deploy and recommend with confidence.

Now might be a good time to remind folks that we are indeed still in beta, so there is some allowance for unplanned downtime etc. We're doing our best with the resources we have. If you'd like to help speed the progress, your contributions will be warmly welcomed.

Please drop in at #metacpan on irc.perl.org with kudos or complaints.