Randy Stauner

Shebangs with perlbrew aliases and EUMM (and without local::lib)

2011-09-04T23:22:27Z

I wanted to comment on kevin spencer's perlbrew intro but my blabbering grew so long I chose to make a separate post instead.

shebangs

I used to use the #!/usr/bin/env perl trick for shebangs until (when reading about packaging executables with dists) I learned that ExtUtils::MakeMaker (and Module::Build) will automatically rewrite a full-path shebang to point to the interpreter used during install, but it specifically does not do this with the env trick.

I inadvertently combined 'env perl' with local::lib once and found that the script (reachable from my local::lib bin/) would die because I was trying to run a script in my path with a perl that didn't have that module installed (I must have switched perls in the meantime).

If the perl that installed the module is embedded in the script then calling it from the command line makes sure that it will be run by a perl that has the module installed (and has the same version that corresponds to the script using it). So it seems better to me to use the full path and let EUMM rewrite it.

tl;dr: use perlbrew alias to denote "current" version

I typically use one perl as my main perl and use a perlbrew alias to point to it:

perlbrew alias create perl-5.14.1 local

Then I can use

#!/home/rando/perl5/perlbrew/perls/local/bin/perl

for the shebang in all of my scripts.

Yeah, it's a little long, but I tab complete it or dump an env var rather than typing it out. This way any scripts that are part of dists I release will get rewritten by EUMM upon install and any scripts that are just for my personal use work with my "current" favorite perl.

When 5.14.2 is released I will install it, reinstall the modules I use, and change my perlbrew alias to point to it.

Sometimes I also symlink /usr/local/bin/perl to my perlbrew alias if I'm the only one on the machine or if perlbrew is in a location that is available to everyone (like /opt).

Then you can use the very common

#!/usr/local/bin/perl

which is shorter and easier to remember.

perlbrew and cpan modules

As a side note I do a perlbrew switch and use the cpan client to install modules directly into the perl lib dir (with the default installdirs site option) and I only use local::lib if I need to test a module with my system-provided perl.

That way (with modules installed relative to the perl that installed them) when you perlbrew switch if you haven't installed a module it's script won't even be found in your path.

If anyone has any points of contention or better suggestions, please share.

Maintenance on Text::VimColor by Geoff Richards (GEOFFR)

2011-07-15T22:30:56Z

I'm interested in maintaining Text::VimColor authored by Geoff Richards (GEOFFR).

Does anybody know Geoff?

He hasn't uploaded to CPAN since 2006.

I emailed him a few months ago (using 3 different addresses from his CPAN page) but never heard back.

I asked on IRC and it sounded like a couple of guys knew him but hadn't heard from him in a long time.

So this is my last attempt before emailing the module list and asking for comaint status.

Thanks!

cpan client tries to install non-existent version

2011-07-12T00:19:13Z

Ran into an issue today and wanted to share in case anybody else hits the problem.

Using the good-old cpan client I found myself unable to install a module. It took me a while to track down the problem...

I had CPAN::SQLite installed and that appears to have a bug (which I filed as RT 69415):

If a greater version of a dist is removed from the CPAN, CPAN::SQLite will fail to reindex the "current" (lesser) version, which means the cpan client will spiral into a 404 loop and give up.

So if you try to install a module and cpan spews 404 errors for a different version than you're expecting, open up ~/.cpan/cpandb.sql with sqlite3 and delete that record with something like:

DELETE FROM dists WHERE dist_name = '?'

Weird perl bug fixed [7c2d9d0]

2011-07-09T23:40:53Z

My previous post about a weird do-block bug I stumbled upon has been fixed as of 7c2d9d0, the day after I posted. I don't know if anybody saw this post or if it was a coincidence, but thanks!

I love the comment:

Fix context propagation below return()

A.k.a. "RT #38809 strikes back".

Perl bug? Weird combination of do, local, and something else?

2011-06-26T21:33:28Z

UPDATE: This perl bug has been fixed

I was writing some very simple unit tests last night and was baffled when one didn't work. The code was very simple, something I thought I had done hundreds of times:

return do { local $/; $fh->getline }

But I stumbled upon some weird voodoo... It's extra baffling to me considering this recent post/bug discovery by Brian D Foy.

It's not the same thing, but it seems similar to me. Unfortunately the commit that fixed Brian's bug 93548 does not fix this one.

I refactored the problem down to a pretty small sub, and am further baffled by the minor variations that cause the problem to go away.

Here is the script and the output from various perls... Interestingly this test appears to include another bug that was fixed somewhere between 5.10 and 5.12...

# original weirdness: same result changing local to my and if(){} to bare {} sub weird { my ($c) = @_; if( ref $c ){ return (ref $c eq 'SCALAR' ? 't' : do { local $/; 'do'; }); } return 'r'; } # fix: no local, no my sub no_vars_in_do { my ($c) = @_; if( ref $c ){ return (ref $c eq 'SCALAR' ? 't' : do { 'do'; }); } return 'r'; } # fix: no superfluous return after the if block (which is true for all of our tests) sub no_extra_return { my ($c) = @_; if( ref $c ){ return (ref $c eq 'SCALAR' ? 't' : do { local $/; 'do'; }); } } # fix: both if(true){} or bare {} cause problems, works without block sub no_enclosing_block { my ($c) = @_; return (ref $c eq 'SCALAR' ? 't' : do { local $/; 'do'; }); return 'r'; } # fix: use constant true/false values instead of ref $c sub any_bool { my ($c) = @_; if( 1 ){ return ( 0 ? 't' : do { local $/; 'do'; }); } return 'r'; } my ($i, @subs) = qw( 0 no_vars_in_do no_extra_return no_enclosing_block weird ); printf "# perl %vd\n1..%d\n", $^V, 2 * @subs + 1; # only run this sub once since we can't reach the 't' outcome t(any_bool => any_arg => 1, 'do'); for my $s ( @subs ){ t($s, SCALAR => \'s' , 't' ); t($s, other => ['s'], 'do'); } sub t { my ($sub, $name, $val, $exp) = @_; my $r = &$sub( $val ); my $result = ((my $ok = $r eq $exp) ? '' : 'not ') . 'ok'; printf "%s %d # %*s: %10s => '%s' %5s\n", $result, ++$i, 22 - length($result) - int($i/10), $sub, $name, $r, $ok?'':'!'; } __END__ # perl 5.8.3 # perl 5.10.1 1..9 not ok 1 # any_bool: any_arg => '' ! ok 2 # no_vars_in_do: SCALAR => 't' ok 3 # no_vars_in_do: other => 'do' ok 4 # no_extra_return: SCALAR => 't' ok 5 # no_extra_return: other => 'do' ok 6 # no_enclosing_block: SCALAR => 't' ok 7 # no_enclosing_block: other => 'do' ok 8 # weird: SCALAR => 't' not ok 9 # weird: other => '' ! # perl 5.12.3 # perl 5.14.0 # perl 5.15.0 (blead) (c08f093) 1..9 ok 1 # any_bool: any_arg => 'do' ok 2 # no_vars_in_do: SCALAR => 't' ok 3 # no_vars_in_do: other => 'do' ok 4 # no_extra_return: SCALAR => 't' ok 5 # no_extra_return: other => 'do' ok 6 # no_enclosing_block: SCALAR => 't' ok 7 # no_enclosing_block: other => 'do' ok 8 # weird: SCALAR => 't' not ok 9 # weird: other => '' !

Can anybody make sense of this? Am I missing something? Or is this a legitimate perl bug?

UPDATE: I should add that, in my original function, I added a warning to the getline method on the object so that I could tell that it was in fact getting called (inside the do-block), but somehow undef was still being returned in the end.

Exploratory one-liners with less typing

2011-06-22T23:21:04Z

Here's one for the "stupid shell tricks" category:

I made myself a shell alias today to simplify the times when i want a quick view of how something works in perl:

alias perl1='perl -MData::Dumper -MYAML::Any -MClass::Autouse=:superloader -E "sub D(\$){ say Dumper(shift) } sub Y(\$){ say Dump(shift) }"'

I'm not quite sure about the name, and I may add more helper functions, but a handy alias like this one allows me to type this at the shell:

perl1 -E 'D [File::Spec->path]'

Instead of

perl -MData::Dumper -MFile::Spec -E 'say Dumper( [File::Spec->path] )'

I don't have to type out -MData::Dumper and I can use D instead of say Dumper()

Plus, thanks to Class::Autouse and it's :superloader import tag I don't have to type out Module::Name ("File::Spec" for example) multiple times just to get a quick glance at it's output.

I haven't decided yet if I want to just append " -E" to the alias. That might get in the way if I want the "-n" or "-p" flags or multiple -E's or something...

Anyway... thought that was handy; Thought I'd share.

releasing to [remote] minicpan (preferably with Dist::Zilla)

2011-06-11T17:53:02Z

Traditionally at $work we do a git clone of our various repos and run the projects straight from the working directory, but lately I've been moving towards the good habit of packaging small related components into their own dists ("I \N{HEAVY BLACK HEART} Dist::Zilla"), so the desire for a real company darkpan has been increasing.

I have a combination minicpan/darkpan that (mostly) works right now using:

One advantage of using CPAN::Mini::Inject::Server is that you can run it as a designated user and anybody who has access to the server can post to it.

I got it running with this app.psgi:

use CGI::Application::Emulate::PSGI;
use CPAN::Mini::Inject::Server::Dispatch;

CGI::Application::Emulate::PSGI->handler(sub {
  CPAN::Mini::Inject::Server::Dispatch->dispatch();
});

I tried CGI::Application::PSGI and CGI::Application::Dispatch::PSGI but wasn't able to get either of them working right (according to their synopses). Third time's a charm. Thanks, CGI::Application::Emulate::PSGI.

So I added the Inject plugin to my $work Dist::Zilla bundle, set remote_server to "http://localhost:7990/" (arbitrary port) and set a fake author_id.

I telecommute, so I spend most of my work day with an open ssh connection to the office, so adding another port-forward to my present list was no trouble. Obviously, your authentication method will be up to you.

If you don't use Dist::Zilla, CPAN::Mini::Inject::Remote has a command line script (and an API) that you should be able to leverage for your situation.

the issue (if you don't update)

The only problem I have with this setup is an outstanding issue with CPAN::Mini::Inject: injecting newer versions of your modules doesn't remove older versions from the list, so cpanminus will find the first (oldest) version in the 02packages file and use that one. The workaround is to update your minicpan mirror and then re-inject (you don't have to re-add).

UPDATE (fixed)

CPAN::Mini::Inject 0.30 was released this morning which fixes the aforementioned issue. Thanks go to Christian Walde and Paul Driver. Hooray!

a new breed

I'm very interested to see if chromatic's new Darkpan module will be a more interesting or practical solution in this or other scenarios. It may prove easier to keep 2 different PANs (one mini, one dark) up to date separately, and just add an extra --mirror argument to cpanm. I'll have to play with that this week and I'll try to pass along any feedback.

A big thanks to all the authors of all the modules that work together to make this happen. CPAN really is the happiest place on earth.

one-liner for separating iCal events by category

2011-06-07T17:33:58Z

Attempting to move a friend from Outlook to Google Calendar, we exported the calendar to an iCal file.

I wanted to create separate Calendars for each event category.

I searched cpan for modules to read the file and found iCal::Parser, Text::vFile, and Text::vFile::asData.

I really didn't need to parse the data, because then I would have to figure out how to unparse it, and i didn't want the data to get messed up in the process.

I was definitely justified: simply doing iCal::Parser->new->parse($file) put my CPU at 3.5 load for 2+ minutes, whereas this one-liner takes a fraction of a second:

perl  -MIO::File -nE 'if(/BEGIN:VEVENT/){ warn("event not ended: $e") if defined $e; $e = $_ } elsif(/END:VEVENT/){ $e .= $_; $e =~ /^CATEGORIES:\s*(.+)\s*$/m; ($n=$1||"")=~s/\W+/_/g; ($fh{$n} ||= IO::File->new("cal-$n.txt", "w"))->print($e); $e = undef; } else { (defined($e) ? $e : $v) .= $_ } END { print $v }' outlook.ics > leftover.txt

Then a bit of verification:

$ expr `wc -l < outlook.ics` - `wc -l < leftover.txt`
16604

$ cat cal* | wc -l
16604

$ diff <(grep -h CATEGORIES cal* | sort | uniq -c) <(grep -h CATEGORIES outlook.ics | sort | uniq -c)

Then I split leftover.txt manually because I don't know that much about the iCal format and don't know what other possibilities there might be, but the file I had was pretty straightforward... there was a 20 line header, and 1 line tail ("END:VCALENDAR"), and everything in between was events.

$ for i in cal*; { cat head.txt $i tail.txt > icals/${i%.txt}.ics; }

Now I have separate calendar files for each category. Now, if only Google Calendar would let me upload them all at once and name them like I name my files... oh well. I've gained enough already, I can do this part manually.

CPAN::Reporter fails to send through Metabase with new LWP 6

2011-03-09T14:43:24Z

I tweeted something last night knowing full well that few people see my tweets and most of those that do aren't programmers. So I decided to post here in case anyone else experienced the same problem.

I installed the new LWP (v6) last night via cpan.

The next time I tried to install a module my cpan client barfed red errors (see below). CPAN::Reporter was erroring when trying to send test results via Metabase.

I updated all SSL modules (cpan> upgrade /SSL/) and installed Mozilla::CA as suggested in the LWP Changes (and a tweet by miyagawa).

No luck.

I noticed something else in the LWP Changes file:

For https://... default to verified connections with require IO::Socket::SSL
and Mozilla::CA modules to be installed.  Old behaviour can be requested by
setting the PERL_LWP_SSL_VERIFY_HOSTNAME environment variable to 0.  The
LWP::UserAgent got new ssl_opts method to control this as well.

so I added this to my .bashrc and all is well again:

alias cpan='env PERL_LWP_SSL_VERIFY_HOSTNAME=0 cpan'

Not exactly a solution, but I'm ok with it for now.

(of course elsewhere in my .bashrc I have

alias sudo='sudo '

so that when I do sudo cpan it expands aliases.

So is this a module problem or is it a problem with cpantesters' certificate?

Here are the errors I saw in case anyone is interested (after installing LWP 6 I tried to reinstall the old 5.837 before I found a fix for the problem):

CPAN::Reporter: Test result is 'pass', All tests successful.
CPAN::Reporter: preparing a CPAN Testers report for libwww-perl-5.837
CPAN::Reporter: sending test report with 'pass' via Metabase
CPAN::Reporter: Test::Reporter: error from 'Test::Reporter::Transport::Metabase:'
fact submission failed: Can't connect to metabase.cpantesters.org:443 (certificate verify failed) at /usr/local/share/perl/5.10.1/Metabase/Client/Simple.pm line 111
        Metabase::Client::Simple::submit_fact('Metabase::Client::Simple=HASH(0x94572a8)', 'CPAN::Testers::Report=HASH(0xa4d2778)') called at /usr/local/share/perl/5.10.1/Test/Reporter/Transport/Metabase.pm line 132
        Test::Reporter::Transport::Metabase::send('Test::Reporter::Transport::Metabase=HASH(0xa332920)', 'Test::Reporter=HASH(0x9eb3ce8)') called at /usr/local/share/perl/5.10.1/Test/Reporter.pm line 279
        eval {...} called at /usr/local/share/perl/5.10.1/Test/Reporter.pm line 279
        Test::Reporter::send('Test::Reporter=HASH(0x9eb3ce8)') called at /usr/local/share/perl/5.10.1/CPAN/Reporter.pm line 503
        CPAN::Reporter::_dispatch_report('HASH(0x9ec5c70)') called at /usr/local/share/perl/5.10.1/CPAN/Reporter.pm line 117
        CPAN::Reporter::grade_test('CPAN::Distribution=HASH(0x9153068)', '/usr/bin/make test', 'ARRAY(0x9f08748)', 0) called at /usr/local/share/perl/5.10.1/CPAN/Reporter.pm line 223
        CPAN::Reporter::test('CPAN::Distribution=HASH(0x9153068)', '/usr/bin/make test') called at /usr/local/share/perl/5.10.1/CPAN/Distribution.pm line 3242
        CPAN::Distribution::test('CPAN::Distribution=HASH(0x9153068)') called at /usr/local/share/perl/5.10.1/CPAN/Distribution.pm line 3464
        CPAN::Distribution::install('CPAN::Distribution=HASH(0x9153068)') called at /usr/local/share/perl/5.10.1/CPAN/Shell.pm line 1796
        CPAN::Shell::rematein('CPAN::Shell', 'install', 'GAAS/libwww-perl-5.837.tar.gz') called at /usr/local/share/perl/5.10.1/CPAN/Shell.pm line 1976
        CPAN::Shell::__ANON__('CPAN::Shell', 'GAAS/libwww-perl-5.837.tar.gz') called at /usr/local/share/perl/5.10.1/App/Cpan.pm line 459
        App::Cpan::__ANON__('GAAS/libwww-perl-5.837.tar.gz') called at /usr/local/share/perl/5.10.1/App/Cpan.pm line 468
        App::Cpan::_default('ARRAY(0x857d800)', 'HASH(0x8d68a58)') called at /usr/local/share/perl/5.10.1/App/Cpan.pm line 386
        App::Cpan::run('App::Cpan', 'GAAS/libwww-perl-5.837.tar.gz') called at /usr/local/bin/cpan line 11