July 2011 Archives

Now you need LWP::Protocol::https

Gisle split out HTTPS support from libwww-perl into LWP::Protocol::https earlier this year when I wasn't paying attention. I needed HTTPS support for one of my Perl 5.14 applications and I wasn't reading the error message closely because I assumed it was business as usual with Crypt::SSLeay. Previously, I just installed that module and everything worked. Now I have to install LWP::Protocol::https to get everything to work. If I haven't done that, I get the error:

 501 Protocol scheme 'https' is not supported (LWP::Protocol::https not installed) <URL:https://www.example.com>

That's not the end of the story though, because the fancy new stuff is a bit more strict with the SSL stuff. For libwww-perl-5.837 and earlier, hostname checking was off by default. Now it's on by default. I can't just connect to any HTTPS server. By default, LWP::UserAgent wants to verify the certificates and so on. That can be a problem if the Certificate Authority root certificate isn't around:

500 Can't connect to pause.perl.org:443 (certificate verify failed)
Content-Type: text/plain
Client-Date: Sun, 24 Jul 2011 11:18:59 GMT
Client-Warning: Internal response

Can't connect to pause.perl.org:443 (certificate verify failed)

LWP::Protocol::https::Socket: SSL connect attempt failed with unknown errorerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed at /usr/local/perls/perl-5.14.1/lib/site_perl/5.14.1/LWP/Protocol/http.pm line 51.

Coincidentally, Phred just mentioned this in Google Chrome fails at pause.perl.org at the same time that I was updating some applications for the latest LWP.

I could get around the verification by setting the PERL_LWP_SSL_VERIFY_HOSTNAMES environment to 0, or setting the SSL options before I connect. This is still documented in LWP::UserAgent, though, so if you are looking in LWP::Protocol::https, you'll miss it:

 $ua = LWP::UserAgent->new( ... );
 $ua->ssl_opts( verify_hostnames => 0 );  # not so nice

That's not really all that pleasing though. I should just get the root certificate for CACert.org, which PAUSE uses. Once I have that, I just have to tell the user-agent where it is:

 $ua->ssl_opts( SSL_ca_file => '/path/to/root.crt' );

If you want to use the certificate authority root certificates that come with Mozilla, you can use Mozilla::CA, which comes with the certificates and a function to figure out where you installed them:

 $ua->ssl_opts( SSL_ca_file => Mozilla::CA::SSL_ca_file() );

I like the Mozilla::CA approach since I can get what I need as a module dependency. So, I created CACertOrg-CA with the root certificate that I need.

What should be in core?

How should we figure out what should be in core? Florian floated some ideas for removing some crufty modules from Perl 5.16. Some of these I don't care about, some of these I never knew about, and at least one I'd like to see in the standard distribution:

  • Text::Soundex
  • File::CheckTree
  • Text::Abbrev
  • Dumpvalue
  • autouse
  • NEXT

There are several camps in this little battle, and people tend to stay in their camp. I think all of them have at least threads of validity.

  • We're stuck with everything already in there because we don't know who depends…

What language do you prefer for your Perl reading?

I have a new survey, mostly because I had to buy a month of SurveyMonkey to download the data from my last survey, Which Perl are you using?.

What language do you prefer for Perl documentation?

Also, because I can and I want to get the most out of my $22, I also have the survey What should we survey?

Tom's Unicode Scripts So Life is Easier

I packaged Tom Christiansen's various Unicode scripts into the Unicode-Tussle distribution to make them easier to distribute and install ahead of his Perl Unicode Essentials tutorial at OSCON.

Tussle, or "Tom's Unicode Scripts So Life is Easier", is a collection of some of the utilities that Tom created to explore various Unicode things. If you've read some of his Unicode treatises, you've probably seen him uʍop əpᴉ…

Rethinking smart matching

Ricardo wants to fix smart matching, which is horribly broken and always has been, although we're just starting to realize how bad it really is. He reduces the table to just a few operations:

$a      $b              Meaning
======= =======         ======================
Any     undef           ! defined $a
Any     ~~-overloaded   invokes the ~~ overload on the object, $a as arg
Any     Regexp, qr-OL   $a =~ $b
Any     CodeRef         $b->($a)
Any     Any             fatal

It's just as easy to not use the smart match operator in these cases, and it removes the cases that genuinely could have made conditionals easier. It also leaves behind the idea that a smart match can decide on its own how to get the answer. For instance, a smart match might short-circuit, might hand off the work to another machine, or parallelize it. That is, most of the power of smart matching is in container types. With Ricardo's table, I'd rather see smart matching completely removed.

I'd like to bring back some of what we left behind with Perl 5.10.1, and eliminate the cases that are causing all of the problems. My vision of smart matching is one that simplifies common questions we like to ask in a conditional. As such, if a smart match wouldn't simplify the conditional, I want to remove that feature.

  • Remove the numish stuff. Let Perl compare things as strings in all cases. If you want to compare numbers, don't use smart matching.
  • Don't smart match recursively. Only the top level smart matches.
  • Bring back commutativity.
  • Remove Object and CodeRef features. Use methods for that.

In my fantasy table, these types refer to the low-level types, although you can still use references. An array reference still counts as Array, a hash reference as a hash, and so on. The order of operands does not matter:

$a      $b              Meaning
======= =======         ======================
Scalar  Regexp          Scalar matches Regexp
Scalar  Array           Scalar is an element of Array
Scalar  Hash            Scalar is a key of Hash
Scalar  Scalar          The string values are the same
Array   Regexp          At least on element of Array matches Regex
Array   Array           Arrays are the same. Reference elements point to the same data.
Array   Hash            Every element of Array is a key in Hash
Hash   Regexp           At least on key of Hash matches Regex
Hash   Hash             Hashes are the same. Reference elements point to the same data.
                        everything else is an error, as soon as Perl reduces it

For example, with scalar and array operands, the current behavior distributes the ~~ over all elements:

  if( $scalar ~~ $array ) { ... }

  if( $scalar ~~ $array[0] or $scalar ~~ $array[1] ... $scalar ~~ $array[N] ) { ... }

This leads to some odd behavior is an element of @array is a reference since Perl must then go back to the chart to see what to do. Instead, I want to see those turned into string comparisons:

  if( $scalar ~~ $array ) { ... }

  if( $scalar eq $array[0] or $scalar eq $array[1] ... $scalar or $array[N] ) { ... }

A possible short circuit is much better than our alternatives:

  if( grep { $scalar eq $array[$_] } @array ) { ... }

  use List::Util qw(first);
  if( first { $scalar eq $array[$_] } @array ) { ... }

But, we might not have to choose one way. People are looking at removing smart matching from the core to put it into a pragma, possibly lexical, which allows everyone to be able to write their own smart match engine (although I predict virtually no one will).

I also realize it's basically too late for any changes. My fundamental goal is to not confuse people. Changing smart matching to the best feature it could ever hope to be is going to do that, no matter how good it is. We already have two versions. We don't need a third. It might be time for this to go the way of pseudohashes.

Indexing MooseX::Declare modules

As a PAUSE admin, I ran into a new problem today. The URI::Dispatch module did not index because it has no package statements and has no provides in META.yml.

This leads me to a couple of questions, for which I invite you to answer:

  • Just how hard should the PAUSE indexer work to discover namespaces when you work hard to hide them? Remember, at the moment it's only MooseX::Declare, but it might be many other modules later.
  • What are other MooseX::Declare people doing to have their stuff indexed?

I'm also thinking about this for my BackPAN indexing bits, which uses my module Module::Extract::Namespaces. That's a PPI-based module, which I guess I now have to mutate to understand Moose. I haven't thought too much about that, so I'm not sure how I'm going to do that.

About brian d foy

user-pic I'm the author of Mastering Perl, and the co-author of Learning Perl (6th Edition), Intermediate Perl, Programming Perl (4th Edition) and Effective Perl Programming (2nd Edition).