Extracting exported variables from a module

Here is a routine from a module I made to extract the variables from a module so they can be put into Perl documentation:

=head2 extract_vars

    extract_vars ('Moby', \%vars);

Extract all the exported variables from the specified module, so if
Moby exports C<$data_dir> then 

    $vars{data_dir} = $Moby::data_dir

This examines C<@EXPORT_OK> in Moby to get the list of variables, and
also evaluates Moby. It assumes that it is being run from a
F<make-pod.pl> which is situated such that Moby is in F<lib/Moby.pm>.

Thus this is strongly dependent on a specific file layout.

Use

    extract_vars ($pm, \%vars, verbose => 1);

for debugging.

=cut

sub extract_vars
{
    my ($module, $vars, %options) = @_;
    if ($module =~ m!/!) {
        warn "$module looks wrong; use \$info->{colon} not \$info->{pm}";
    }
    my $verbose = $options{verbose};
    if ($verbose) {
        print "Module is $module.\n";
    }
    my $evals = "use lib \"$Bin/lib\";use $module ':all';";

    if ($verbose) {
        print "Evaling '$evals'.\n";
    }
    eval $evals;
    if ($verbose) {
        print "Getting exports from $module.\n";
    }
    my @exports = get_exports ($module);
    for my $var (@exports) {
        if ($var =~ /\$(.*)/) {
            if ($verbose) {
                print "Adding $var to variables.\n";
            }
            my $nodollar = $1;
            $vars->{$nodollar} = eval "\$$module::$nodollar";
        }
        elsif ($var =~ /((\@|\%)(.*))/) {
            my $variable = $1;
            my $sigil = $2;
            my $nosigil = $3;
            my $export = "\\$sigil$module::$nosigil";
            if ($verbose) {
                print "Adding $variable as $export.\n";

            }
            $vars->{$nosigil} = eval $export;
        }
    }
}

The documentation is generated from a script called "make-pod.pl" in the top directory using the template toolkit.

The routine "get_exports" here looks like this:

# Helper to get exported variables.

sub get_exports
{
    my ($module) = @_;
    my @exports;
    eval "use lib \"$Bin/lib\";use $module ':all';\@exports = \@${module}::EXPORT_OK";
    if ($@) {
    croak $@;
    }
    return @exports;
}

An alternative to HTML::Tagset

HTML::Tagset is a popular module on CPAN and has nearly nine thousand descendants in the "River of CPAN". Unfortunately its last version was in March 2008, nearly ten years ago, so it doesn't feature the HTML5 tags yet. There has been some discussion on the HTML tag set bug tracker but so far that has not come to fruition. As an alternative to HTML::Tagset, I made the module HTML::Valid::Tagset, which incorporates the data tables of the "Tidy HTML5" project.

It's partly based on HTML::Tagset's interface, so it can substitute for some of that module's functionality, and it also has specific tag sets for each version of HTML, including HTML5.

Having the Tidy HTML5 data tables means I was also able to add a function attributes which returns a list of valid attributes for a tag, and tag_attr_ok, which returns true or false depending on whether a particular tag can take a particular attribute. This is part of a larger distribution called HTML::Valid, based on the Tidy HTML5 project.

Why I recommend using the "++" system of Metacpan

A few days ago I bought some Christmas tree lights from amazon.co.jp, the Japanese version of Amazon:

https://www.amazon.co.jp/gp/product/B076WTG77B/ref=oh_aui_detailpage_o00_s00?ie=UTF8&psc=1

One of the words in the description of the item is パーティスマスツリー, which seems to be "party-mas-tree". You might think that is some kind of Japanese neologism, but there's no such word as "パーティスマスツリー" in Japanese. A google search just brings you back to Amazon.co.jp.

https://encrypted.google.com/search?hl=en&q=%22%E3%83%91%E3%83%BC%E3%83%86%E3%…

Cygwin interesting problems

According to this test my module JSON::Parse wasn't working on Cygwin. I have a Windows computer with Cygwin installed, so I thought I would try to compile the module myself. The first problem I encountered was that the cpan shell command didn't work. It would print odd-looking errors and hang up at the following prompt:

What approach do you want?  (Choose 'local::lib', 'sudo' or 'manual')

So I decided to download the module myself with wget and compile it. The next problem was that I hadn't got wget in my cygwin, so I had to install that. After downloading and untarring the file, I tried the usual

perl Makefile.PL

then "make", and got a rather baffling error

fatal error: EXTERN.h: No such file or directory

I was able to find some assistance at this site. It turned out that the C compiler gcc was not included in my cygwin, and the gcc on my path was actually the Strawberry Perl version, so I installed the Cygwin gcc.

Next, the following problem occurred.

$ make
rm -f blib/arch/auto/JSON/Parse/Parse.dll 
g++ --shared -Wl,--enable-auto-import -Wl,--export-all-symbols -Wl,--enable-auto-image-base -fstack-protector-strong Json3.o -o blib/arch/auto/JSON/Parse/Parse.dll \
/usr/lib/perl5/5.22/x86_64-cygwin-threads/CORE/cygperl5_22.dll \

g++.exe: error:
/usr/lib/perl5/5.22/x86_64-cygwin-threads/CORE/cygperl5_22.dll: No
such file or directory

I spent quite a while searching around for "cygperl5_22.dll" and related things before I noticed that the g++ in my path was pointing at the Strawberry Perl one. I had only installed gcc, because I had assumed that would be enough for my non-C++ module, but it seems to use g++ regardless of whther the code is C or C++. I went back and installed g++-gcc from the Cygwin installer (setup64.exe, see cygwin.org). Finally, with the Cygwin g++, it compiled and built the module, and I was able to reproduce the issue that the tester had demonstrated and come up with a possible solution.

Unfortunately I'm not really sure why the original problem only happened on Cygwin. There is some sort of memory management issue which I don't understand properly.

"inf" doesn't work everywhere, apparently

I've just got this CPAN Testers report which says that my module Compress::Huffman had an error due to $minkey not being set to any value on line 176. Tracing this back, it seems to be due to the use of the word 'inf' on line 166, then this being not converted to infinity but to zero for some reason, and $minkey never getting set.

I don't remember where I got the idea to use 'inf' for infinity, but I'm switching over to the method given here of using 9**9**9 instead.

Incidentally, for those interested in the Huffman algorithm, $minkey here is the least probable key, and its probability is $min, which is set to the value of infinity at the start so that every key will have a lower value than it.