Now you need LWP::Protocol::https
Gisle split out HTTPS support from libwww-perl into LWP::Protocol::https earlier this year when I wasn't paying attention. I needed HTTPS support for one of my Perl 5.14 applications and I wasn't reading the error message closely because I assumed it was business as usual with Crypt::SSLeay. Previously, I just installed that module and everything worked. Now I have to install LWP::Protocol::https to get everything to work. If I haven't done that, I get the error:
501 Protocol scheme 'https' is not supported (LWP::Protocol::https not installed) <URL:https://www.example.com>
That's not the end of the story though, because the fancy new stuff is a bit more strict with the SSL stuff. For libwww-perl-5.837 and earlier, hostname checking was off by default. Now it's on by default. I can't just connect to any HTTPS server. By default, LWP::UserAgent wants to verify the certificates and so on. That can be a problem if the Certificate Authority root certificate isn't around:
500 Can't connect to pause.perl.org:443 (certificate verify failed)
Content-Type: text/plain
Client-Date: Sun, 24 Jul 2011 11:18:59 GMT
Client-Warning: Internal response
Can't connect to pause.perl.org:443 (certificate verify failed)
LWP::Protocol::https::Socket: SSL connect attempt failed with unknown errorerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed at /usr/local/perls/perl-5.14.1/lib/site_perl/5.14.1/LWP/Protocol/http.pm line 51.
Coincidentally, Phred just mentioned this in Google Chrome fails at pause.perl.org at the same time that I was updating some applications for the latest LWP.
I could get around the verification by setting the PERL_LWP_SSL_VERIFY_HOSTNAMES environment to 0, or setting the SSL options before I connect. This is still documented in LWP::UserAgent, though, so if you are looking in LWP::Protocol::https, you'll miss it:
$ua = LWP::UserAgent->new( ... );
$ua->ssl_opts( verify_hostnames => 0 ); # not so nice
That's not really all that pleasing though. I should just get the root certificate for CACert.org, which PAUSE uses. Once I have that, I just have to tell the user-agent where it is:
$ua->ssl_opts( SSL_ca_file => '/path/to/root.crt' );
If you want to use the certificate authority root certificates that come with Mozilla, you can use Mozilla::CA, which comes with the certificates and a function to figure out where you installed them:
$ua->ssl_opts( SSL_ca_file => Mozilla::CA::SSL_ca_file() );
I like the Mozilla::CA approach since I can get what I need as a module dependency. So, I created CACertOrg-CA with the root certificate that I need.
I'm the author of Mastering Perl, and the co-author of Learning Perl (6th Edition), Intermediate Perl, Programming Perl (4th Edition) and Effective Perl Programming (2nd Edition).
Thanks for your notes.
As I can see on russian-spaking forums this issue already affected few users and was quickly solved thanks to your article.
Hi, I am a beginner Perl programmer.
I have a problem of using LWP to send https request, and it's quite annoying!
I am using ubuntu, newest,
perl newest, 5.14
I only want to send a simple https request to facebook graph API, and it's quite simple:
my $ua = LWP::UserAgent->new(ssl_opts => { verify_hostname => 1 });
my $url = "https://graph.facebook.com/xxx";
my $res = $ua->get($url);
However, I always get errors:
LWP will support https URLs if the LWP::Protocol::https module
is installed.
OK, I installed the https module by using "cpan -fi LWP::Protocol::https" and it showed OK.
Run the program still same errors.
I can only say .... Stupid LWP!!!
Anyone can help!!
Really useful, thanks. The 'quick and dirty' way has a typo though:
s/verify_hostnames/verify_hostname/
$ua->ssl_opts( verify_hostname => 0 );