June 2011 Archives

Perl bug? Weird combination of do, local, and something else?

By Randy Stauner on June 26, 2011 3:33 PM

UPDATE: This perl bug has been fixed

I was writing some very simple unit tests last night and was baffled when one didn't work. The code was very simple, something I thought I had done hundreds of times:

return do { local $/; $fh->getline }

But I stumbled upon some weird voodoo... It's extra baffling to me considering this recent post/bug discovery by Brian D Foy.

It's not the same thing, but it seems similar to me. Unfortunately the commit that fixed Brian's bug 93548 does not fix this one.

I refactored the problem down to a pretty small sub, and am further baffled by the minor variations that cause the problem to go away.

Here is the script and the output from various perls... Interestingly this test appears to include another bug that was fixed somewhere between 5.10 and 5.12...

# original weirdness: same result changing local to my and if(){} to bare {} sub weird { my ($c) = @_; if( ref $c ){ return (ref $c eq 'SCALAR' ? 't' : do { local $/; 'do'; }); } return 'r'; } # fix: no local, no my sub no_vars_in_do { my ($c) = @_; if( ref $c ){ return (ref $c eq 'SCALAR' ? 't' : do { 'do'; }); } return 'r'; } # fix: no superfluous return after the if block (which is true for all of our tests) sub no_extra_return { my ($c) = @_; if( ref $c ){ return (ref $c eq 'SCALAR' ? 't' : do { local $/; 'do'; }); } } # fix: both if(true){} or bare {} cause problems, works without block sub no_enclosing_block { my ($c) = @_; return (ref $c eq 'SCALAR' ? 't' : do { local $/; 'do'; }); return 'r'; } # fix: use constant true/false values instead of ref $c sub any_bool { my ($c) = @_; if( 1 ){ return ( 0 ? 't' : do { local $/; 'do'; }); } return 'r'; } my ($i, @subs) = qw( 0 no_vars_in_do no_extra_return no_enclosing_block weird ); printf "# perl %vd\n1..%d\n", $^V, 2 * @subs + 1; # only run this sub once since we can't reach the 't' outcome t(any_bool => any_arg => 1, 'do'); for my $s ( @subs ){ t($s, SCALAR => \'s' , 't' ); t($s, other => ['s'], 'do'); } sub t { my ($sub, $name, $val, $exp) = @_; my $r = &$sub( $val ); my $result = ((my $ok = $r eq $exp) ? '' : 'not ') . 'ok'; printf "%s %d # %*s: %10s => '%s' %5s\n", $result, ++$i, 22 - length($result) - int($i/10), $sub, $name, $r, $ok?'':'!'; } __END__ # perl 5.8.3 # perl 5.10.1 1..9 not ok 1 # any_bool: any_arg => '' ! ok 2 # no_vars_in_do: SCALAR => 't' ok 3 # no_vars_in_do: other => 'do' ok 4 # no_extra_return: SCALAR => 't' ok 5 # no_extra_return: other => 'do' ok 6 # no_enclosing_block: SCALAR => 't' ok 7 # no_enclosing_block: other => 'do' ok 8 # weird: SCALAR => 't' not ok 9 # weird: other => '' ! # perl 5.12.3 # perl 5.14.0 # perl 5.15.0 (blead) (c08f093) 1..9 ok 1 # any_bool: any_arg => 'do' ok 2 # no_vars_in_do: SCALAR => 't' ok 3 # no_vars_in_do: other => 'do' ok 4 # no_extra_return: SCALAR => 't' ok 5 # no_extra_return: other => 'do' ok 6 # no_enclosing_block: SCALAR => 't' ok 7 # no_enclosing_block: other => 'do' ok 8 # weird: SCALAR => 't' not ok 9 # weird: other => '' !

Can anybody make sense of this? Am I missing something? Or is this a legitimate perl bug?

UPDATE: I should add that, in my original function, I added a warning to the getline method on the object so that I could tell that it was in fact getting called (inside the do-block), but somehow undef was still being returned in the end.

1 comment

Exploratory one-liners with less typing

By Randy Stauner on June 22, 2011 5:21 PM

Here's one for the "stupid shell tricks" category:

I made myself a shell alias today to simplify the times when i want a quick view of how something works in perl:

alias perl1='perl -MData::Dumper -MYAML::Any -MClass::Autouse=:superloader -E "sub D(\$){ say Dumper(shift) } sub Y(\$){ say Dump(shift) }"'

I'm not quite sure about the name, and I may add more helper functions, but a handy alias like this one allows me to type this at the shell:

perl1 -E 'D [File::Spec->path]'

Instead of

perl -MData::Dumper -MFile::Spec -E 'say Dumper( [File::Spec->path] )'

I don't have to type out -MData::Dumper and I can use D instead of say Dumper()

Plus, thanks to Class::Autouse and it's :superloader import tag I don't have to type out Module::Name ("File::Spec" for example) multiple times just to get a quick glance at it's output.

I haven't decided yet if I want to just append " -E" to the alias. That might get in the way if I want the "-n" or "-p" flags or multiple -E's or something...

Anyway... thought that was handy; Thought I'd share.

9 comments

releasing to [remote] minicpan (preferably with Dist::Zilla)

By Randy Stauner on June 11, 2011 11:53 AM

Traditionally at $work we do a git clone of our various repos and run the projects straight from the working directory, but lately I've been moving towards the good habit of packaging small related components into their own dists ("I \N{HEAVY BLACK HEART} Dist::Zilla"), so the desire for a real company darkpan has been increasing.

I have a combination minicpan/darkpan that (mostly) works right now using:

One advantage of using CPAN::Mini::Inject::Server is that you can run it as a designated user and anybody who has access to the server can post to it.

I got it running with this app.psgi:

use CGI::Application::Emulate::PSGI;
use CPAN::Mini::Inject::Server::Dispatch;

CGI::Application::Emulate::PSGI->handler(sub {
  CPAN::Mini::Inject::Server::Dispatch->dispatch();
});

I tried CGI::Application::PSGI and CGI::Application::Dispatch::PSGI but wasn't able to get either of them working right (according to their synopses). Third time's a charm. Thanks, CGI::Application::Emulate::PSGI.

So I added the Inject plugin to my $work Dist::Zilla bundle, set remote_server to "http://localhost:7990/" (arbitrary port) and set a fake author_id.

I telecommute, so I spend most of my work day with an open ssh connection to the office, so adding another port-forward to my present list was no trouble. Obviously, your authentication method will be up to you.

If you don't use Dist::Zilla, CPAN::Mini::Inject::Remote has a command line script (and an API) that you should be able to leverage for your situation.

the issue (if you don't update)

The only problem I have with this setup is an outstanding issue with CPAN::Mini::Inject: injecting newer versions of your modules doesn't remove older versions from the list, so cpanminus will find the first (oldest) version in the 02packages file and use that one. The workaround is to update your minicpan mirror and then re-inject (you don't have to re-add).

UPDATE (fixed)

CPAN::Mini::Inject 0.30 was released this morning which fixes the aforementioned issue. Thanks go to Christian Walde and Paul Driver. Hooray!

a new breed

I'm very interested to see if chromatic's new Darkpan module will be a more interesting or practical solution in this or other scenarios. It may prove easier to keep 2 different PANs (one mini, one dark) up to date separately, and just add an extra --mirror argument to cpanm. I'll have to play with that this week and I'll try to pass along any feedback.

A big thanks to all the authors of all the modules that work together to make this happen. CPAN really is the happiest place on earth.

5 comments

one-liner for separating iCal events by category

By Randy Stauner on June 7, 2011 11:33 AM

Attempting to move a friend from Outlook to Google Calendar, we exported the calendar to an iCal file.

I wanted to create separate Calendars for each event category.

I searched cpan for modules to read the file and found iCal::Parser, Text::vFile, and Text::vFile::asData.

I really didn't need to parse the data, because then I would have to figure out how to unparse it, and i didn't want the data to get messed up in the process.

I was definitely justified: simply doing iCal::Parser->new->parse($file) put my CPU at 3.5 load for 2+ minutes, whereas this one-liner takes a fraction of a second:

perl  -MIO::File -nE 'if(/BEGIN:VEVENT/){ warn("event not ended: $e") if defined $e; $e = $_ } elsif(/END:VEVENT/){ $e .= $_; $e =~ /^CATEGORIES:\s*(.+)\s*$/m; ($n=$1||"")=~s/\W+/_/g; ($fh{$n} ||= IO::File->new("cal-$n.txt", "w"))->print($e); $e = undef; } else { (defined($e) ? $e : $v) .= $_ } END { print $v }' outlook.ics > leftover.txt

Then a bit of verification:

$ expr `wc -l < outlook.ics` - `wc -l < leftover.txt`
16604

$ cat cal* | wc -l
16604

$ diff <(grep -h CATEGORIES cal* | sort | uniq -c) <(grep -h CATEGORIES outlook.ics | sort | uniq -c)

Then I split leftover.txt manually because I don't know that much about the iCal format and don't know what other possibilities there might be, but the file I had was pretty straightforward... there was a 20 line header, and 1 line tail ("END:VCALENDAR"), and everything in between was events.

$ for i in cal*; { cat head.txt $i tail.txt > icals/${i%.txt}.ics; }

Now I have separate calendar files for each category. Now, if only Google Calendar would let me upload them all at once and name them like I name my files... oh well. I've gained enough already, I can do this part manually.