<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
    <title>Leon Timmermans</title>
    <link rel="alternate" type="text/html" href="https://blogs.perl.org/users/leon_timmermans/" />
    <link rel="self" type="application/atom+xml" href="https://blogs.perl.org/users/leon_timmermans/atom.xml" />
    <id>tag:blogs.perl.org,2009-11-03:/users/leon_timmermans//150</id>
    <updated>2025-07-13T13:12:28Z</updated>
    <subtitle>A blog about the Perl programming language</subtitle>
    <generator uri="http://www.sixapart.com/movabletype/">Movable Type Pro 4.38</generator>

<entry>
    <title>XS the easy way</title>
    <link rel="alternate" type="text/html" href="https://blogs.perl.org/users/leon_timmermans/2025/07/xs-the-easy-way.html" />
    <id>tag:blogs.perl.org,2025:/users/leon_timmermans//150.11809</id>

    <published>2025-07-13T13:08:12Z</published>
    <updated>2025-07-13T13:12:28Z</updated>

    <summary>XS has a reputation for being hard to access and I think it&apos;s a shame because I don&apos;t think it has to be: it&apos;s mostly that the Perl API is hard. What if you offload as much logic as possible...</summary>
    <author>
        <name>Leon Timmermans</name>
        <uri>http://codespeaks.blogspot.com/</uri>
    </author>
    
    <category term="perl" label="perl" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="xs" label="xs" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="https://blogs.perl.org/users/leon_timmermans/">
        <![CDATA[<p>XS has a reputation for being hard to access and I think it's a shame because I don't think it has to be: it's mostly that the Perl API is hard. <strong>What if you offload as much logic as possible to perl, and use XS only to expose functions to perl?</strong> That would be much easier for a casual XS writer who doesn't know anything about Perl's internals.</p>

<p>So, in this example I will write a simple XS module for a real-life API, in this case POSIX real-time semaphores. This allows you to synchronize data between different processes or threads. At its core the API is this:</p>

<pre><code>sem_t* sem_open(const char* path, int open_flags, mode_t mode, unsigned int value);
int sem_close(sem_t *sem);
int sem_unlink(const char* path);

int sem_wait(sem_t *sem);
int sem_trywait(sem_t *sem);
int sem_post(sem_t *sem);
</code></pre>
]]>
        <![CDATA[<p>What's the simplest way to expose that through XS? That would look something like this:</p>

<pre><code>#define PERL_NO_GET_CONTEXT
#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"
</code></pre>

<!---->

<pre><code>#include &lt;semaphore.h&gt;
</code></pre>

<!---->

<pre><code>MODULE = POSIX::Sem                PACKAGE = POSIX::Sem
</code></pre>

<!---->

<pre><code>TYPEMAP: &lt;&lt;END
sem_t*    T_PTR
mode_t    T_IV
END
</code></pre>

<!---->

<pre><code>sem_t *sem_open(const char* path, int open_flags, mode_t mode, unsigned int value)
</code></pre>

<!---->

<pre><code>int sem_close(sem_t *sem)
</code></pre>

<!---->

<pre><code>int sem_unlink(const char* path)
</code></pre>

<!---->

<pre><code>int sem_wait(sem_t *sem)
</code></pre>

<!---->

<pre><code>int sem_trywait(sem_t *sem)
</code></pre>

<!---->

<pre><code>int sem_post(sem_t *sem)
</code></pre>

<p>Let me explain. An XS file always starts with a C section, followed by one or more XS sections declared using the <code>MODULE</code> keyword.</p>

<p>Our C section is fairly short and obvious, I'm first importing all the Perl bits that we need, this is a boilerplate that every XS module will start with. Then we include the API we want to wrap (<code>semaphore.h</code>).</p>

<p>Then we declare we want to declare the <code>POSIX::Sem</code> XS module. This contains a typemap and a list of functions.</p>

<p>The typemap declares which conversion template should be used for specific types; it will already be predefined for standard types such as <code>int</code> and <code>char*</code>, but we do need to tell it that <code>sem_t*</code> should be treated as a basic pointer (<code>T_PTR</code>), and <code>mode_t</code> as an integer (<code>T_IV</code>). Typemaps are really useful for making your XS code simpler, but sadly they're rather underdocumented.</p>

<p>After that, all we have to do is declare the functions in XS as they're declared in C. All the code needed to wrap the functions up and export them into Perl will be automatically generated. </p>

<p>Now we just need some Perl code to wrap it up:</p>

<pre><code>package POSIX::Sem;
our $VERSION = '0.001';
</code></pre>

<!---->

<pre><code>use 5.036;
</code></pre>

<!---->

<pre><code>use XSLoader;
XSLoader::load(__PACKAGE__, $VERSION);
</code></pre>

<!---->

<pre><code>use Errno 'EAGAIN';
</code></pre>

<!---->

<pre><code>sub open($class, $path, $open_flags, $mode = 0600, $value = 1) {
    my $sem_ptr = sem_open($path, $open_flags, $mode, $value);
    die "Could not open $path: $!" if not $sem_ptr;
    return bless \$sem_ptr, $class;
}
</code></pre>

<!---->

<pre><code>sub DESTROY($self) {
    sem_close($$self);
}
</code></pre>

<!---->

<pre><code>sub post($self) {
    die "Could not post: $!" if sem_post($$self) &lt; 0;
}
</code></pre>

<!---->

<pre><code>sub wait($self) {
    die "Could not wait: $!" if sem_wait($$self) &lt; 0;
}
</code></pre>

<!---->

<pre><code>sub trywait($self) {
    if (sem_trywait($$self) &lt; 0) {
        return undef if $! == EAGAIN;
        die "Could not trywait: $!";
    }
    return 1;
}
</code></pre>

<!---->

<pre><code>sub unlink($class, $path) {
    die "Could not unlink: $!" if sem_unlink($path) &lt; 0;
}
</code></pre>

<!---->

<pre><code>1;
</code></pre>

<p>Here we use XSLoader to load the XS module and import the functions from the library. I could have chosen to leave it at that, but I chose to wrap it up in  a nice object-oriented API by storing the pointer that we get from <code>sem_open</code> in a scalar reference so I can bless it.. Other than that all the Perl code does is turn any error into an exception. The resulting API can be used something like this:</p>

<pre><code>use 5.036;
</code></pre>

<!---->

<pre><code>use POSIX::Sem;
use Fcntl 'O_CREAT';
</code></pre>

<!---->

<pre><code>my $sem = POSIX::Sem-&gt;open("foo", O_CREAT);
</code></pre>

<!---->

<pre><code>$sem-&gt;post;
$sem-&gt;wait;
</code></pre>

<p>That's all folks!</p>
]]>
    </content>
</entry>

<entry>
    <title>A different Perl Toolchain Summit</title>
    <link rel="alternate" type="text/html" href="https://blogs.perl.org/users/leon_timmermans/2025/05/a-different-perl-toolchain-summit.html" />
    <id>tag:blogs.perl.org,2025:/users/leon_timmermans//150.11755</id>

    <published>2025-05-11T13:05:20Z</published>
    <updated>2025-05-20T03:19:47Z</updated>

    <summary>A week ago I attended the 2025 PTS. For me it was a different PTS than the previous ones. Firstly because it was my first PTS without Abe Timmerman. He was a regular in both the PTS (as maintainer of...</summary>
    <author>
        <name>Leon Timmermans</name>
        <uri>http://codespeaks.blogspot.com/</uri>
    </author>
    
    <category term="perl" label="perl" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="pts" label="PTS" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="pts2025" label="PTS 2025" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="https://blogs.perl.org/users/leon_timmermans/">
        <![CDATA[<p>A week ago I attended the 2025 PTS. For me it was a different PTS than the previous ones.</p>

<p>Firstly because it was my first PTS without <a href="https://github.com/abeltje">Abe Timmerman</a>. He was a regular in both the PTS (as maintainer of Test::Smoke), and of the Amsterdam Perl Mongers. In fact the last time I saw him was on our flight back to Amsterdam after the PTS in Lisbon last year. He was greatly missed.</p>

<p>Secondly, because of a question that Book asked at the very beginning of the PTS: how often we had been to the PTS before. I was one of the few who had attended more than 10 of them. Combined with the fact that several other regular attendees couldn't make it that meant that this PTS I spent more time than ever on helping others with various issues.</p>
]]>
        <![CDATA[<p>I also spent quite a bit of time in discussions. Most obviously on the future of the CPAN Security group that I joined last year. I also spent time giving feedback to the new CPAN Testers group, and talked with a bunch of people to get feedback on my idea to write a new CPAN client.</p>

<p>But obviously I also did a lot of programming work. I synced up a year's worth of updates on <a href="https://metacpan.org/pod/ExtUtils::ParseXS">ExtUtils::ParseXS</a> to CPAN (more changes are coming up from David Mitchell after 5.42). I released a new <a href="https://metacpan.org/pod/ExtUtils::Builder::Compiler">ExtUtils::Builder::Compiler</a> (now working with Microsoft's compiler thanks to Mithaldu helping me out). I made <a href="https://metacpan.org/pod/ExtUtils::Manifest">ExtUtils::Manifest</a> compatible with <a href="https://metacpan.org/pod/Test2::Harness">Test2::Harness</a>/yath after someone (Breno?) pointed out it wasn't. I released a new <a href="https://metacpan.org/pod/App::ModuleBuildTiny">App::ModuleBuildTiny</a> with better configuration options after I sat down with Paul Evans to observe his release process. I fixed a minor issue in <a href="https://metacpan.org/pod/Dist::Zilla::Plugin::ModuleBuildTiny">Dist::Zilla::Plugin::ModuleBuildTiny</a> on older perls that Julien had pointed out to me. I worked on making <a href="https://metacpan.org/pod/Module::Build::Tiny">Module::Build::Tiny</a>'s XS support more compatible with Devel::Cover after Paul Johnson pointed out that combination didn't work. I worked on making <a href="https://metacpan.org/pod/Test::Harness">Test::Harness</a> more asynchronous/parallel. I released a new <a href="https://metacpan.org/pod/Software::License">Software::License</a> and worked on improving PAUSE's password handling. And I updated <a href="https://metacpan.org/pod/experimental">experimental.pm</a> for 5.42.</p>

<p>I spent a bunch of time investigating how to get TLS support into core. This is much more complicated than one might hope; partially because Net::SSLeay is a 29 years old large XS module (OpenSSL forked from SSLeay in 1998&#133;), and partially because OpenSSL itself deeply depends on Perl in its build system. This is looking like it will be my next big project.</p>

<p>And I scared everyone by tripping on a ramp that got treacherously slippy in the rain. I didn't break anything. I think.</p>

<p>All in all I had a really useful PTS, even if I mostly did completely different things from what I had been initially planning.</p>

<p>All of this wouldn't be possible without our wonderful organizers (Daniel, Philippe, Laurent, Tina and Breno), as well as our sponsors:</p>

<h2>Monetary Sponsors</h2>

<p><a href="https://www.booking.com/">Booking.com</a>, <a href="https://www.webpros.com/">WebPros</a>, 
<a href="https://www.cosmoshop.de/">CosmoShop</a>, <a href="https://datensegler.at/">Datensegler</a>, 
<a href="https://opencagedata.com">OpenCage</a>, <a href="https://www.suse.com/">SUSE</a>, 
<a href="https://www.simplelists.com/">Simplelists Ltd</a>, <a href="https://www.ctrlo.com/">Ctrl O Ltd</a>, 
<a href="https://www.findus-internet-opac.de/">Findus Internet-OPAC</a>, <a href="https://www.plusw.de/">plusW GmbH</a></p>

<h2>In-kind sponsors</h2>

<p><a href="https://www.grantstreet.com/">Grant Street Group</a>, <a href="https://www.fastmail.com/">Fastmail</a>, 
<a href="https://en.shift2.nl/">shift2</a>, <a href="https://www.oleeo.com/">Oleeo</a>, <a href="https://ferki.it/">Ferenc Erki</a></p>

<h2>Community Sponsors</h2>

<p><a href="https://www.perlfoundation.org/">The Perl and Raku Foundation</a>, <a href="https://japan.perlassociation.org/">Japan Perl Association</a>, 
Harald Joerg, Alexandros Karelas (<a href="https://www.perlmodules.net/">PerlModules.net</a>),  Matthew Persico,
Michele Beltrame (<a href="https://www.blendgroup.it/">Sigmafin</a>), Rob Hall, Joel Roth, Richard Leach, 
Jonathan Kean, Richard Loveland, Bojan Ramsa</p>
]]>
    </content>
</entry>

<entry>
    <title>An introduction to App::ModuleBuildTiny part 2: authoring</title>
    <link rel="alternate" type="text/html" href="https://blogs.perl.org/users/leon_timmermans/2025/03/an-introduction-to-appmodulebuildtiny-part-2-authoring.html" />
    <id>tag:blogs.perl.org,2025:/users/leon_timmermans//150.11718</id>

    <published>2025-03-21T22:09:37Z</published>
    <updated>2025-03-21T23:10:31Z</updated>

    <summary>Now that we have set up our mbtiny configuration in the previous post, we can actually use it. Minting a new distribution Minting a distribution is trivial once you&#8217;ve completed the setup. It&#8217;s typically just a matter of calling mbtiny...</summary>
    <author>
        <name>Leon Timmermans</name>
        <uri>http://codespeaks.blogspot.com/</uri>
    </author>
    
    <category term="authoring" label="authoring" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="modulebuildtiny" label="modulebuildtiny" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="perl" label="perl" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="https://blogs.perl.org/users/leon_timmermans/">
        <![CDATA[<p>Now that we <a href="https://blogs.perl.org/users/leon_timmermans/2025/03/an-introduction-to-appmodulebuildtiny-part-1-setting-things-up.html">have set up</a> our mbtiny configuration in the previous post, we can actually use it.</p>

<h1>Minting a new distribution</h1>

<p>Minting a distribution is trivial once you&#8217;ve completed the setup. It&#8217;s typically just a matter of calling <code>mbtiny mint Foo::Bar</code>. If needed you can override the global configuration at minting time (e.g.  <code>mbtiny mint Foo::Bar --license BSD</code>).</p>

<h1>Converting an existing distribution</h1>

<p>You can also convert an existing distribution to App::ModuleBuildTiny. In most cases that requires just two things:</p>
]]>
        <![CDATA[<ol>
<li><p>Delete the pre-existing Makefile.PL / Build.PL</p></li>
<li><p>run <code>mbtiny config copy</code>, to copy the global configuration into the local project.</p></li>
</ol>

<h1>Prereqs</h1>

<p>Prerequisites can be handled in two ways. If the <code>auto_scan</code> option is enabled, dependencies will be automatically detected, usually that is enough. Alternatively, you can set them explicitly prereqs.yml or a cpanfile. E.g.</p>

<pre class="yaml">

# prereqs.yml

runtime:
  requires:
    perl: '5.012'
    Path::Tiny: 0
  recommends:
    Term::ReadLine::Gnu: 0
test:
  requires:
    Test::More: '0.88'

</pre>

<p>It should be noted that App::ModuleBuild::Tiny will automatically inject a configure dependency on the build tool (usually <a href="https://metacpan.org/pod/Module::Build::Tiny">Module::Build::Tiny</a>).</p>

<h1>Main subcommands</h1>

<p>App::ModuleBuildTiny mostly tries to stay out of your way. Mostly you can just develop your module the way you&#8217;re used to, you just need 2-3 commands to start to become productive:</p>

<h2>regenerate</h2>

<p>Typically this is run right before the release of a new version. It will regenerate all files that need regeneration. It&#8217;s no coincidence that almost all of the previously mentioned configuration is about this subcommand, it truly is the heart of the application. What it does exactly depends on your configuration but in a typical case it would:</p>

<ul>
<li><p>Bump the version in the module files</p></li>
<li><p>Scan for prerequisites</p></li>
<li><p>Scan the pod for the license / author</p></li>
<li><p>Gather all metadata.</p></li>
<li><p>Write Build.PL, META files, LICENSE, MANIFEST, etc&#133;</p></li>
<li><p>Update Changes file for the new version.</p></li>
<li><p>Commit all of the above to git</p></li>
</ul>

<p>If new to the tool you might want to do a <code>git show</code> before releasing, to better understand what it does, but in general it shouldn&#8217;t do anything surprising.</p>

<h2>upload</h2>

<p>This will do exactly what it advertises: creates a tarball for the distribution and uploads that to CPAN. Before doing that however it will do a series of preflight checks such as</p>

<ul>
<li><p>Check if the Changes file contains entries for this release.</p></li>
<li><p>Check if the main module has an abstract in its pod.</p></li>
<li><p>Check if your module has a valid version that matches the one in the meta file.</p></li>
<li><p>Check if the desired version isn&#8217;t already tagged.</p></li>
<li><p>Check if all tests pass.</p></li>
</ul>

<p>After these tests it will ask for final confirmation and then uploads to PAUSE.</p>

<h2>test</h2>

<p>In simple modules, you don&#8217;t actually need this subcommand, you can just use <code>prove -l</code> instead. That said, mbtiny test does two additional things that make it useful:</p>

<ol>
<li><p>It does a fresh and full distribution build, allowing one to test XS modules or generated modules.</p></li>
<li><p>It will run author tests (e.g. xt/pod-syntax.t), and with <code>--release</code> will also run release tests.</p></li>
</ol>

<h1>Secondary subcommands</h1>

<p>mbtiny provides much more than just the above three commands. The full list is <a href="https://metacpan.org/dist/App-ModuleBuildTiny/view/script/mbtiny">documented</a> in the mbtiny script but some honorable mentions are:</p>

<h2>listdeps</h2>

<p>This lists the dependencies. This is particularly useful after a regeneration with auto-scan, to double check if the list of dependencies is what you expect it to be.</p>

<h2>config</h2>

<p>This allows you to inspect and update the configuration of the local distribution.  You may need this to change your configuration, or to update it if configuration options have been added.</p>

<h1>Choosing install tool</h1>

<p>By default it will use Module::Build::Tiny, as its name probably had already given away. For the vast majority of people this will be the right choice as it can handle pure-perl and simple xs modules just fine. But if you need to use <a href="https://metacpan.org/pod/Dist::Build">Dist::Build</a> instead that&#8217;s easy: it will automatically switch if it sees a <code>planner/</code> directory (that is what Dist::Build uses for its build scripts). More about that in the next blogpost!</p>
]]>
    </content>
</entry>

<entry>
    <title>An introduction to App::ModuleBuildTiny part 1: setting things up</title>
    <link rel="alternate" type="text/html" href="https://blogs.perl.org/users/leon_timmermans/2025/03/an-introduction-to-appmodulebuildtiny-part-1-setting-things-up.html" />
    <id>tag:blogs.perl.org,2025:/users/leon_timmermans//150.11713</id>

    <published>2025-03-19T23:17:11Z</published>
    <updated>2025-03-19T23:20:43Z</updated>

    <summary>App::ModuleBuildTiny is a relatively new authoring tool. It aims to be a relatively lightweight (at least to some other tools like Dist::Zilla) and newbie friendly tool. It supports two install tools: Module::Build::Tiny (obviously what it was originally designed for) and...</summary>
    <author>
        <name>Leon Timmermans</name>
        <uri>http://codespeaks.blogspot.com/</uri>
    </author>
    
    <category term="perlauthoringmodulebuildtiny" label="perl authoring modulebuildtiny" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="https://blogs.perl.org/users/leon_timmermans/">
        <![CDATA[<p><a href="https://metacpan.org/pod/App::ModuleBuildTiny">App::ModuleBuildTiny</a> is a relatively new authoring tool. It aims to be a relatively lightweight (at least to some other tools like Dist::Zilla) and newbie friendly tool. It supports two install tools: <a href="https://metacpan.org/pod/Module::Build::Tiny">Module::Build::Tiny</a> (obviously what it was originally designed for) and <a href="https://metacpan.org/pod/Dist::Build">Dist::Build</a>; it does not support ExtUtils::MakeMaker or Module::Build.</p>
]]>
        <![CDATA[<h1>Installing</h1>

<p>Installing is just a matter of installing the App::ModuleBuildTiny distribution using your favorite CPAN client.</p>

<h1>A minimal setup</h1>

<p>Before minting, it's advisable to have run <code>mbtiny setup</code> at least once, to set up a general configuration. It's not strictly required, but it will set a default configuration so you don't have to answer these same questions every time you mint a new distribution. If you don't care about the details, <code>mbtiny setup minimal</code> will have you set up in a minute with two unavoidable licensing related questions:</p>

<ul>
<li>What is the author's name?</li>
<li>What is the author's email?</li>
</ul>

<p>If you want a simple setup, that's all you need and you can actually stop reading the rest of this post.</p>

<h1>A detailed setup</h1>

<p>If you want more control, you can use <code>mbtiny setup all</code> to get all configuration questions. All questions have usable defaults. It adds one more legal question to the previous two.</p>

<h3>What license do you want to use?</h3>

<p>This defaults to <code>Perl_5</code>, the same license as Perl uses. If you've never given licenses much thought this is probably the best option for a perl module because this is what most of CPAN uses.</p>

<h2>Writeback questions</h2>

<p>These questions decide which files are written back to your filesystem/repository.</p>

<h3>Do you want to write your build files to your filesystem?</h3>

<p>If enabled, this will write <code>Build.PL,</code> <code>META.json,</code> <code>META.yml</code>, and <code>MANIFEST</code> files to disk when regenerating. If not they'll only be in the tarball. Enabling this is generally recommended as it makes CI workflows easier and enhances reproducibility.</p>

<h3>Do you want to write your LICENSE file to your filesystem?</h3>

<p>If enabled, it will generate and write a LICENSE file on regeneration (though usually it doesn't change after minting). This is usually what you want.</p>

<h3>Do you want to write your README file to your filesystem?</h3>

<p>If true this will generate a README file from the main module's pod. Unless you'd rather write your own README file you probably want to enable this.</p>

<h2>Automation questions</h2>

<p>These questions enable some extra magic that make mbtiny a little easier to use, but you don't actually need any of these if you don't want to.</p>

<h3>Do you want mbtiny to automatically handle git for you?</h3>

<p>This integrates automatic git handling. If enabled this will turn on flags on three commands</p>

<ul>
<li>On mbtiny mint it will set --init-git to initialize a new git repository.</li>
<li>On mbtiny regenerate it will set --commit to commit the generated changes.</li>
<li>On mbtiny upload it will enable --tag and --push to tag and push the current commit.</li>
</ul>

<p>I would generally recommend enabling this unless you're not using git. All of these options can be overridden the usual way (e.g. mbtiny mint Foo --no-init-git).</p>

<h3>Do you want mbtiny to automatically bump on regenerate for you?</h3>

<p>If enabled this will turn on the --bump option to mbtiny regenerate by default. In my experience bumping is the only time when you want to regenerate, but maybe someone wants to use a different workflow from mine.</p>

<h3>Do you want mbtiny to automatically scan dependencies for you?</h3>

<p>If enabled this will turn on the --scan option to mbtiny regenerate by default. This is similar to the [AutoPrereqs] plugin in Dist::Zilla. If you don't like this you can disable it and manually set your prerequisites using a cpanfile or a preqreqs.yml file.</p>

<h3>Do you want mbtiny to automatically add a repository link to the metadata</h3>

<p>This will try to automatically detect the right repository link for your distribution. Currently only github is well supported, patches for other git providers are welcome.</p>

<h3>Do you want mbtiny to automatically add a bug tracker link to the metadata</h3>

<p>This will try to automatically detect the right bug tracker for your distribution. Currently only github is well supported, patches for other git providers are welcome.</p>

<h1>Changing things later</h1>

<p>If you want to change things later, you call always call <code>mbtiny setup all</code> to go through the same questions again. Or use the setup set mode like <code>mbtiny setup set auto_scan false</code>.</p>
]]>
    </content>
</entry>

<entry>
    <title>A deep dive into the Perl type systems</title>
    <link rel="alternate" type="text/html" href="https://blogs.perl.org/users/leon_timmermans/2025/02/a-deep-dive-into-the-perl-type-systems.html" />
    <id>tag:blogs.perl.org,2025:/users/leon_timmermans//150.11675</id>

    <published>2025-02-01T21:32:26Z</published>
    <updated>2025-02-02T16:07:04Z</updated>

    <summary>People usually don&apos;t think about Perl&apos;s type system. Some would even mistakenly claim it doesn&apos;t have one. It is, however, a most unusual one that doesn&apos;t really look like anything else. What is a type anyway? And what is a...</summary>
    <author>
        <name>Leon Timmermans</name>
        <uri>http://codespeaks.blogspot.com/</uri>
    </author>
    
    <category term="perl" label="perl" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="type" label="type" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="types" label="types" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="https://blogs.perl.org/users/leon_timmermans/">
        <![CDATA[<p>People usually don't think about Perl's type system. Some would even mistakenly claim it doesn't have one. It is, however, a most unusual one that doesn't really look like anything else.</p>

<p>What is a type anyway? And what is a type system? I'm not going to precisely define it here, that's for academics, but generally speaking a type is a fundamental property of a variable or value that determines what operations can and can not be done with it and what invariants it must hold. In a strong type system it is a stable trait: it can't change over the lifetime of the value/variable.</p>

<p>In some type systems containers are typed (such as C) and values really don't exist separate from containers. In other typed systems containers are typeless but values are typed (e.g. Python, Javascript, &#133;). There are languages where both values and containers are typed (e.g. Java, C#), typically this means that the container constrains the values in it.</p>

<p>Contrary to what you might expect, Perl has the latter sort of typesystem, but with a twist.</p>
]]>
        <![CDATA[<h1>The Perl static type system</h1>

<p>When you got up this morning you might not have expected to read that Perl is a statically typed language, but here we are. To be precise it is a closed, strong and statically typed language. There are exactly 7 types, all variables/values are exactly one of them, and they can never change into another type. Though only 5 of them are directly accessible so one could argue there are only 5 true types.</p>

<h2>Scalar</h2>

<p>Typically identified by the sigil <code>$</code>, this is the most famous type as it does most of the work. Almost all operations in the language work on scalars, it provides an enormous flexibility.</p>

<h2>Arrays</h2>

<p>Arrays are identified by the <code>@</code> sigil. They're an ordered series of scalars, and support various operations typical of arrays such as indexing using the <code>[]</code> postfix operator, <code>push</code>, <code>pop</code>, <code>splice</code>, &#133; that all do what anyone would expect from them.</p>

<h2>Hashes</h2>

<p>Hashes (also known as maps, dictionaries or associative arrays in other languages) are identified by the <code>%</code> sigil. They're an unordered mapping of string to scalars. Its values can be accessed using the <code>{}</code> postfix operator, and it supports the expected operations for a mapping: <code>keys</code>, <code>values</code>, <code>delete</code>, <code>exists</code>, etc&#133;. Like arrays they work more or less the same as they do in other dynamic languages.</p>

<h2>Subs</h2>

<p>Also known as functions. They have a sigil (<code>&amp;</code>), but they are omitted when defining them (<code>sub foo {}</code>) and usually also when calling the function (<code>foo(1)</code> versus <code>&amp;foo(1)</code>). These values can't be used to access these values directly; the only thing you can really do with it is to call it. Instead you might encounter these values as a reference to a sub.</p>

<h2>Globs</h2>

<p>The last type to actually have a sigil (<code>*</code>) are globs. This is probably the most confusing type as it's unique to Perl. A glob contains its own name, as well as one of each other sigiled types  (scalar, array, hash, sub) and two opaque values: io and format. The symbol table is essentially a hash of globs. So a package variable <code>@Foo::bar</code> can also be accessed as <code>@{ *Foo::bar{ARRAY} }</code>. Much of this is vestigial behavior from perl version 4 and older, and unless you're hacking the symbol table you should have no need of messing with them.</p>

<h2>IOs</h2>

<p>These contain file handles. They have no sigil and can only really be used opaquely through their glob (or via a reference, but no one does that). All operations on them are typically done through that glob (or a reference to such a glob). They support various operations that you'd expect from a file handle (e.g. <code>readline</code>, <code>printf</code>, &#133;).</p>

<h2>Formats</h2>

<p>Formats are another unique type in Perl. They're a special kind of sub that is attached to an IO via the glob, used via the format keyword. You're probably better off knowing as little as possible about them.</p>

<h2>Common operations</h2>

<p>All sigiled types can be used with the referencing operator <code>\</code> (e.g. <code>my $foo = \&amp;function</code>). All accessible types can be used in both scalar and list context (e.g. <code>scalar(@array)</code> will return the number of entries in the array), and <code>@array = %hash</code> will return a list of alternating keys and values); some would consider that type weakness. All accessible types can be used as arguments to functions (though that usually means list context). Other than that there are no operations common across all types.</p>

<h1>The dynamic type system</h1>

<p><em>Of course, none of the above is what you were thinking of when I first said type system</em>. What you were probably thinking about was types of scalars. Once again, this actually works differently from other systems.</p>

<p>Fundamentally though, there are only 7 operations you can do on scalars, all the others can be composed of them.</p>

<h3>Primitives</h3>

<p>Firstly there are the three primitive conversions:</p>

<ul>
<li>Get an integer value.</li>
<li>Get a floating point value.</li>
<li>Get a string value. This value might be internally latin1 or utf8, this should not matter to the language; when it does that is known as The Unicode Bug.</li>
</ul>

<p><strong>All scalars are usable as integers, floats and strings, this is a fundamental trait of the scalar type</strong>. If any scalar is lacking any of these, the missing primitive will be generated from one of the others (e.g. float from int, int from string). It might give a warning because it's unhappy about those conversions (e.g. «<em>Argument "a" isn't numeric in addition</em>»), but it will continue nonetheless.</p>

<p>This is why perl has two sets of comparison operators (e.g. <code>==</code> versus <code>eq</code>), and a concatenation operator (<code>.</code>) separate from addition (<code>+</code>). Perl puts that polymorphism in the operators instead of the values. When people complain that perl is weakly typed they usually complain about this triple valuedness. I'd argue it's not weakly typed, just unusually typed.</p>

<h3>Assignment</h3>

<p>Any scalar can be assigned to. This replaces the value entirely. In reality of course there are various mutating operations for efficiency reasons, but conceptually the model doesn't actually require mutators; creating a new value and assigning it to the old should always be equivalent to mutating the old.</p>

<h3>Definedness</h3>

<p>Any scalar can be either defined or undefined. An undefined is a value that doesn't have any primitive value. As such, you can do nearly nothing without it warning warnings, other than checking definedness.</p>

<h3>Package lookup</h3>

<p>Package lookup what's used by method invocation(<code>-&gt;</code> and the <code>isa</code> operator (since perl 5.34)) to determine what class should be used for method calls on a scalar.</p>

<h3>Dereferencing</h3>

<p>Dereference it. There is a dereferencing operator for each of the 5 sigils (e.g. <code>@{ ... }</code>; values may support any subset of it (including all or none).</p>

<h3>That's it</h3>

<p>Literally all scalar operations in Perl can be composed out of combinations of these. E.g. numeric operations like <code>+</code> will numify their operands and create a new numeric scalar. An operation like <code>print</code> will take the string value and print that to the handle.</p>

<h2>Subtypes</h2>

<p>As I said before, Perl is polymorphic on operators not on values; it will generally use the appropriate value for that operation. That said, some values consistently behave differently from other values, and can be thought of as a subtype for that reason. Any value is strictly one of these three subtypes, and it can't change its subtype without an assigning/mutating operation to that variable.</p>

<h3>References</h3>

<p>A references are indirections to a value of one of the 7 basic types of the language, or in recent perls potentially other opaque types (e.g. regexes or the new object representation in <a href="https://perldoc.perl.org/perlclass">feature class</a>). They will numify to an integer representation of their address (e.g. <code>0x57b6255ae580</code>) and stringify to something derived from that (e.g. <code>"ARRAY(0x57b6255ae580)"</code>). They are always defined, and how they invoke and dereference is dependent on the value they refer to.</p>

<p>Only the dereferencing operator matching their referent's type is allowed, this ensures type-safety; trying to use any of the other 4 will throw an exception. Dereferencing isn't allowed for types that don't have a sigil (IO, format and opaque types).</p>

<p>Method invocation on references is only allowed when its referent is blessed (a lot of documentation suggests it's a trait of the reference itself, this is incorrect). Any type of value can be blessed (even formats, though some would call that cursed instead). A blessing is literally nothing more than tagging a value as belonging with a specific class. On method invocation, the method is looked up using that class as its starting point.</p>

<p>Blessed references can support operator overloading for both primitive operations and dereferencing. </p>

<h3>Fake globs</h3>

<p>Fake globs are essentially a vestigial feature from perl4, I don't even think they're even documented. They work sort of like references to globs, except they will accept all dereferencing operators (and use the appropriate glob slot them for). It will stringify to the glob's full sigiled name (e.g. <code>"*Foo::bar"</code>) and are always defined. I think you can work with Perl for decades without ever encountering or needing any of these, but they exist in weird corners.</p>

<h3>Other</h3>

<p>Anything else is other. Other values may contain any combination of the three slots: integer, float and string (e.g. (<code>1.0</code>, <code>"1"</code>)). Usually the values logically match with each other, but that does not necessarily have to be the case (most famously in <code>$!</code> they do not). An undefined value is any value that has none of these three set; it's equivalent to (<code>0</code>, <code>""</code>) except it will <a href="https://perldoc.perl.org/functions/warn">warn</a> when used as either of them. These are the only values for which the defined operator returns false.</p>

<p>If <code>use strict</code> (or actually <code>use strict 'refs'</code>) is not enabled, these values can be used with any of the dereferencing operators for symbolic dereferencing (e.g. <code>@{ "Foo::Bar" }</code>) using their string value. When strict is enabled (they should be in 99.9% of all code) symbolic lookup is forbidden.</p>

<h1>The dynamic type system, level two</h1>

<p><em>Of course, none of the above is what you were thinking of when I first said type system</em>. What you actually want to think about is objects.</p>

<p>One of the issues with calling this a type system is that it's actually mutable, which suggests it's a weak type system at best, but you can just easily argue it's part of the value instead of the type. We treat it as if it's a strong type because people don't actually rebless objects; the only exceptions I know of are cases where an object is reblessed into a subtype (e.g. upgrading an <code>IO::Socket</code> to <code>IO::Socket::SSL</code>), which is just sensible enough to not break people's Liskov expectations.</p>

<p>Fundamentally Perl's OO-by-blessing is one of the most minimal OO systems that has ever been invented (though). With just blessing and the method call operator it gives us useful types; but it doesn't really give much a system. </p>
]]>
    </content>
</entry>

<entry>
    <title>Using peppers with Crypt::Passphrase</title>
    <link rel="alternate" type="text/html" href="https://blogs.perl.org/users/leon_timmermans/2024/02/using-peppers-with-cryptpassphrase.html" />
    <id>tag:blogs.perl.org,2024:/users/leon_timmermans//150.11368</id>

    <published>2024-02-11T17:50:41Z</published>
    <updated>2024-03-25T21:26:55Z</updated>

    <summary>Crypt::Passphrase Crypt::Passphrase is a module for managing passwords. It allows you to separate policy and mechanism, meaning that the code that polices authorization doesn&#8217;t have to know anything about what algorithms are used behind the screen, and vice-versa; thus making...</summary>
    <author>
        <name>Leon Timmermans</name>
        <uri>http://codespeaks.blogspot.com/</uri>
    </author>
    
    
    <content type="html" xml:lang="en" xml:base="https://blogs.perl.org/users/leon_timmermans/">
        <![CDATA[<h1>Crypt::Passphrase</h1>

<p>Crypt::Passphrase is a module for managing passwords. It allows you to separate policy and mechanism, meaning that the code that polices authorization doesn&#8217;t have to know anything about what algorithms are used behind the screen, and vice-versa; thus making for a <a href="https://en.wikipedia.org/wiki/Cryptographic_agility">cryptographically agile</a> system.</p>

<p>It&#8217;s not only handling the technical details of password hashes for you but also it deals with a variety of schemes. It&#8217;s especially useful for transitioning between them.</p>

<p>A configuration might look like this (Koha):</p>
]]>
        <![CDATA[<pre class="perl">
my $auth = Crypt::Passphrase->new(
    encoder => {
        module => 'BCrypt',
        cost   => 8,
    },
    validators => [ 'MD5::Base64' ],
);
</pre>

<p>Using it might look like this:</p>

<pre>
if (!$auth->verify_password($password, $hash)) {
    die "Invalid password";
}
elsif ($auth->needs_rehash($hash)) {
    my $new_hash = $auth->hash_password($password);
    ...
}
</pre>

<p>It supports a variety of algorithms, but argon2 and bcrypt are by far the most popular ones. That said, it can do much more that that: it can do peppers for you.</p>

<h1>Why Peppers</h1>

<p>The function of peppers is to protect leaked passwords, especially the bad ones. Password hashes try making brute-force attacks so expensive that attackers won&#8217;t even bother, but in the end they can&#8217;t really protect passwords from a dictionary attack. The key-space of bad passwords is so small that you can&#8217;t really prevent that.</p>

<p>When you add a pepper, that means an attacker needs to brute-force both password and pepper, but because the pepper doesn&#8217;t need to be memorized by a human it can actually be a piece of high-entropy (e.g. a 16 or 32 byte chunk of good randomness). That would make it well outside of reach of any brute force attack for sheer <a href="https://www.reddit.com/r/explainlikeimfive/comments/22udcp/comment/cgqh4eo/?utm_source=reddit&amp;utm_medium=web2x&amp;context=3">physical reasons</a>.</p>

<p>The most important thing you must understand about peppers is that like passwords the security they provide hinges entirely on their secrecy. If that secrecy is compromised they don&#8217;t do anything for security. If you remember nothing else of this blog post, please remember that.</p>

<h1>Mapping Peppers</h1>

<p>The first thing you&#8217;d probably notice about my modules is that you don&#8217;t pass it a pepper, but a map of peppers. This is an essential quality of the system that a lot of naive pepper implementations are lacking. Peppers are keys, and all keys must be rotatable. Like passwords, you need to be able to change them if they may have been compromised. By using a map and adding the identifier in the metadata section of the hash, you can rotate in a new key while still able to check old ones. This gives the system the agility it needs to &#133;</p>

<h1>Style of peppers</h1>

<p>The second thing you might notice is that I provide two very different styles of peppering; using some sort of MAC before the password hash, and using symmetric encryption after the password hash (e.g. <a href="https://metacpan.org/pod/Crypt::Passphrase::Argon2::AES">Crypt::Passphrase::Argon2::AES</a> and <a href="https://metacpan.org/pod/Crypt::Passphrase::Bcrypt::AES">Crypt::Passphrase::Bcrypt::AES</a>). The former approach appears to be more common out in the wild, but that latter is by far the better one. Firstly because its security is easily provable (it hinges only on symmetric encryption, not on an unusual combination of constructs), but secondly because it allows for easy re-peppering without needing the user&#8217;s password to recompute the password hash inside of it (essentially just decrypting with the old key and encrypting with the new one). For that reason I would strongly recommend the latter approach.</p>

<h1>Using peppers</h1>

<p>It can be as simple as this:</p>

<pre>
my $auth = Crypt::Passphrase->new(
    encoder => {
        module => 'Argon2::AES',
        peppers => \%peppers,
    },
);
</pre>

<p>All you really need to do is change the module name and pass in the peppers. The hardest part of it is probably securely storing the peppers. There are many tools to help you with this (e.g. <a href="https://www.vaultproject.io/">vault</a>, <a href="https://github.com/bitnami-labs/sealed-secrets">sealed secrets</a>, and/or my own <a href="https://metacpan.org/dist/Mojolicious-Plugin-Credentials">Mojolicious::Plugin::Credentials</a>). How to best do this really depends on your setup.</p>

<p>Arguably the best option is using a hardware security module (e.g. CP::Argon2::HSM), but few people has a hardware security module laying around (good ones are rather expensive, though you might convince your TPM2 to function as one).</p>

<h1>TL;DR</h1>

<p>Using peppers doesn&#8217;t have to be that hard. If you have an appropriate credentials store, you can easily add it to your application and enhance the security of your passwords. Maybe you too should give <code>Crypt::Passphrase::Argon2::AES</code> a try.</p>
]]>
    </content>
</entry>

<entry>
    <title>My 2023 in Perl</title>
    <link rel="alternate" type="text/html" href="https://blogs.perl.org/users/leon_timmermans/2024/01/my-2023-in-perl.html" />
    <id>tag:blogs.perl.org,2024:/users/leon_timmermans//150.11348</id>

    <published>2024-01-23T17:30:02Z</published>
    <updated>2024-01-23T17:51:40Z</updated>

    <summary>2023 was a rather productive year for me on CPAN. Aided by taking some time off I managed to release a whopping 18 new modules. Passwords Crypt::HSM Crypt::Passphrase::Argon2::AES Crypt::Passphrase::Argon2::HSM Crypt::Passphrase::Linux Crypt::Passphrase::Pepper::HSM Crypt::Passphrase::Yescrypt Crypt::Yescrypt DBIx::Class::CryptColumn Mojolicious::Plugin::Passphrase Half of my new modules...</summary>
    <author>
        <name>Leon Timmermans</name>
        <uri>http://codespeaks.blogspot.com/</uri>
    </author>
    
    <category term="perl2023" label="perl 2023" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="https://blogs.perl.org/users/leon_timmermans/">
        <![CDATA[<p>2023 was a rather productive year for me on CPAN. Aided by taking some time off I managed to release a whopping 18 new modules.</p>

<h1>Passwords</h1>

<ul>
<li><a href="https://metacpan.org/pod/Crypt::HSM">Crypt::HSM</a></li>
<li><a href="https://metacpan.org/pod/Crypt::Passphrase::Argon2::AES">Crypt::Passphrase::Argon2::AES</a></li>
<li><a href="https://metacpan.org/pod/Crypt::Passphrase::Argon2::HSM">Crypt::Passphrase::Argon2::HSM</a></li>
<li><a href="https://metacpan.org/pod/Crypt::Passphrase::Linux">Crypt::Passphrase::Linux</a></li>
<li><a href="https://metacpan.org/pod/Crypt::Passphrase::Pepper::HSM">Crypt::Passphrase::Pepper::HSM</a></li>
<li><a href="https://metacpan.org/pod/Crypt::Passphrase::Yescrypt">Crypt::Passphrase::Yescrypt</a></li>
<li><a href="https://metacpan.org/pod/Crypt::Yescrypt">Crypt::Yescrypt</a></li>
<li><a href="https://metacpan.org/pod/DBIx::Class::CryptColumn">DBIx::Class::CryptColumn</a></li>
<li><a href="https://metacpan.org/pod/Mojolicious::Plugin::Passphrase">Mojolicious::Plugin::Passphrase</a></li>
</ul>

<p>Half of my new modules were related to my password framework <code>Crypt::Passphrase</code>. To be honest most of them are either small (± 100 LOC) glue two or three other pieces of code together. And then there was <code>Crypt::HSM</code>, a PKCS11 interface (to use cryptographic hardware without exposing cryptographic keys) that was probably more work (2600 LOC of XS) than the others combined.</p>

<p>Most of this was with the aim to add peppering support to <code>Crypt::Passphrase</code>, a subject extensive enough that I should probably dedicate a separate blogpost to it.</p>
]]>
        <![CDATA[<h1>Typemaps</h1>

<ul>
<li><a href="https://metacpan.org/pod/App::typemap">App::typemap</a></li>
<li><a href="https://metacpan.org/pod/Dist::Zilla::Plugin::Typemap">Dist::Zilla::Plugin::Typemap</a></li>
<li><a href="https://metacpan.org/pod/ExtUtils::Typemaps::Magic">ExtUtils::Typemaps::Magic</a></li>
</ul>

<p><code>ExtUtils::Typemaps::Magic</code> contains a set of typemaps that help me write XS based objects. In particular the MagicExt typemap allows me to write thread-safe objects (in my particular case: refcounted), which no built-in typemap does. <code>App::typemap</code> helps one integrate typemap bundles into your local typemap file, and <code>Dist::Zilla::Plugin::Typemap</code> does the same for <code>dzil</code>.</p>

<h1>Toolchain specs</h1>

<ul>
<li><a href="https://metacpan.org/pod/CPAN::API::BuildPL">CPAN::API::BuildPL</a></li>
<li><a href="https://metacpan.org/pod/CPAN::Static">CPAN::Static</a></li>
</ul>

<p>I finally got around to publishing two pieces of toolchain that had been in the pipeline for years. <code>CPAN::Static</code> contains a specification and reference implementation for static installation of modules in CPAN clients. For 90% of all dists, ExtUtils::MakeMaker and Module::Build are an overkill and all they really need is to copy some files and run tests. </p>

<p><code>CPAN::API::BuildPL</code>, a specification for Build.PL implementations was mostly written by David Golden but never got published, but now CPAN::Static depends on it so was published alongside with it.</p>

<h1>Other</h1>

<ul>
<li><a href="https://metacpan.org/pod/Magic::Check">Magic::Check</a></li>
<li><a href="https://metacpan.org/pod/Magic::Coerce">Magic::Coerce</a></li>
</ul>

<p>These two modules add a little typing to Perl. <code>Magic::Check</code> implements runtime (type) checking on a variable, and <code>Magic::Coerce</code> implements coercers. They're both really more low-level backend modules that beg for a wrapper with a better syntax that I haven't come up with yet.</p>

<ul>
<li><a href="https://metacpan.org/pod/Thread::GoChannel">Thread::GoChannel</a></li>
</ul>

<p>This module brings Thread::CSP style channels to threads.pm as an alternative to <code>Thread::Queue</code>. As its name indicates, its semantics are close to that of Go channels, instead of the more asynchronous behavior of <code>Thread::Queue</code>.</p>

<ul>
<li><a href="https://metacpan.org/pod/Syntax::Infix::Smartmatch">Syntax::Infix::Smartmatch</a></li>
</ul>

<p>This is an implementation of a simpler and more predictable kind of smartmatching than the one that comes with core. It's intended to be usable even if smartmatching gets removed from core itself.</p>

<h1>Conclusion</h1>

<p>I had a productive year, and some pretty good leads to move forward this year. I'm looking forward to it.</p>
]]>
    </content>
</entry>

<entry>
    <title>Some tricks for prettier xs</title>
    <link rel="alternate" type="text/html" href="https://blogs.perl.org/users/leon_timmermans/2022/02/some-tricks-for-prettier-xs.html" />
    <id>tag:blogs.perl.org,2022:/users/leon_timmermans//150.10705</id>

    <published>2022-02-20T11:52:01Z</published>
    <updated>2022-02-20T14:35:56Z</updated>

    <summary>XS has a reputation of being ugly and cumbersome, but in my experience, it doesn&apos;t have to be. Let&apos;s take for example this snippet from my Thread::Csp::Promise class: MODULE = Thread::Csp PACKAGE = Thread::Csp::Promise PREFIX = promise_ SV* promise_get(Promise* promise)...</summary>
    <author>
        <name>Leon Timmermans</name>
        <uri>http://codespeaks.blogspot.com/</uri>
    </author>
    
    <category term="xs" label="xs" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="https://blogs.perl.org/users/leon_timmermans/">
        <![CDATA[<p>XS has a reputation of being ugly and cumbersome, but in my experience, it doesn't have to be. Let's take for example this snippet from my <code>Thread::Csp::Promise</code> class:</p>

<pre>
MODULE = Thread::Csp PACKAGE = Thread::Csp::Promise PREFIX = promise_

<p>SV* promise_get(Promise* promise)</p>

<p>bool promise_is_finished(Promise* promise)</p>

<p>SV* promise_get_notifier(Promise* promise)<br />
</pre></p>]]>
        <![CDATA[<p>How did I write XS with so little code/boilerplate? By using XS the way it was originally intended: to glue Perl and C together, not to implement any behavior.</p>

<h2>No CODE</h2>
A lot of people seem to think you need a <code>CODE</code> block in your XS functions, but often you don't. For example 
<pre>
SV* promise_get(Promise* promise)
</pre>
is actually equivalent to 
<pre class=code>
SV* promise_get(Promise* promise)
CODE:
    RETVAL = promise_get(promise);
OUTPUT:
    RETVAL
</pre>
By giving the `promise_get` function the right shape and name, I don't need to write any of that.

<p>This doesn't only mean less code (which is always good), it also means that it's much easier to split a large amount of code into multiple files, as doing this in C is much easier than doing it in XS (e.g. DBI.xs is 5700 lines). This aids in making your project more maintainable.</p>

<h2>No K&R</h2>
The second thing you may notice is that I'm declaring the types of the arguments in ANSI style (within the parentheses), not the common K&R style like:
<pre>
SV*
promise_get(promise)
    Promise* promise;
</pre>
The author of perlxs and perlxstut was clearly fond of K&R style, and everyone seems to have copied it from the documentation, but ANSI style is far more familiar to most people, and less repetitive. While the K&R style can do a few things ANSI style can't (e.g. with regards to custom conversions), it's very uncommon to need any of that.

<h2>Typemaps</h2>
To convert from arguments from Perl values to C values, and vice-versa for the return values, I used type maps. 

<pre>
TYPEMAP
Promise* T_PROMISE

<p>INPUT<br />
T_PROMISE<br />
        $var = sv_to_promise($arg)</p>

<p>OUTPUT<br />
T_PROMISE<br />
        $arg = promise_to_sv($var);<br />
</pre></p>

<p>Using these templates, you don't need the XS or the individual functions to worry about type conversions for the most common argument types.</p>

<h2>Prefix</h2>
All the XS packages in my module are defined with a prefix:
<pre>
MODULE = Thread::Csp PACKAGE = Thread::Csp::Promise PREFIX = promise_
</pre>
That way I can namespace my C functions to all start with `promise_`, but on the Perl side <code>promise_get</code> will be a sub called <code>get</code> (in the package <code>Thread::Csp::Promise</code>).

<h2>C_ARGS</h2>
I still had one function where the default glue code didn't quite cut it, because that had a slurpy argument so I couldn't map arguments 1-on-1. Instead of using a `CODE` block here, I used the <code>C_ARGS</code> to override only how the arguments are passed from Perl to C, without overriding any of the rest of the code generation. And then I defined a helper that turns the arguments on the stack into an array that is passed to the function.
<pre class=code>
Promise* thread_spawn(SV* class, SV* module, SV* function, ...)
        C_ARGS:
                slurp_arguments(1)
</pre>]]>
    </content>
</entry>

<entry>
    <title>My new modules in 2021</title>
    <link rel="alternate" type="text/html" href="https://blogs.perl.org/users/leon_timmermans/2021/12/my-new-modules-in-2021.html" />
    <id>tag:blogs.perl.org,2021:/users/leon_timmermans//150.10609</id>

    <published>2021-12-29T14:29:16Z</published>
    <updated>2021-12-29T14:49:27Z</updated>

    <summary>Perl I had a reasonably productive year, releasing several modules that I think/hope are useful for the wider ecosystem. Crypt::Passphrase This module manages the passwords in a cryptographically agile manner. That means that it can not only verify passwords using...</summary>
    <author>
        <name>Leon Timmermans</name>
        <uri>http://codespeaks.blogspot.com/</uri>
    </author>
    
    <category term="perl" label="perl" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="raku" label="raku" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="https://blogs.perl.org/users/leon_timmermans/">
        <![CDATA[<h1>Perl</h1>

<p>I had a reasonably productive year, releasing several modules that I think/hope are useful for the wider ecosystem.</p>

<h2>Crypt::Passphrase</h2>

<p>This module manages the passwords in a cryptographically agile manner. That means that it can not only verify passwords using different ciphers, but it also aids in gradually upgrading passwords hashed with an outdated cipher (or outdated settings) to the current one; for example when you want to upgrade from bcrypt to argon2. Password hashing is both a rather common form of cryptography, and one that is more subject to change than others; you should probably reevaluate your password handling every couple of years. With this module, you can initiate such a transition with a simple configuration change.</p>

<p>This also includes a number of extension distributions (e.g. <code>Crypt::Passphrase::Argon2</code>, <code>Crypt::Passphrase::Bcrypt</code>, etc&#133;), and one new backend module (<code>Crypt::Bcrypt</code>)</p>

<h2>Thread::Csp</h2>

<p>My most ambitious project of the year by far. It's actually been in the making for a decade, full of lessons learned in my <a href="https://metacpan.org/pod/threads::lite">previous attempt</a>. Thread::Csp is a new threading library (build on ithreads primitives, but not on threads.pm and doesn't clone whole interpreters); it is based on <a href="https://en.wikipedia.org/wiki/Communicating_sequential_processes">Communicating Sequential Processes</a> (hence the name), the same model that Go uses (in particular for its channels).</p>

<p>I firmly believe share-nothing message-passing models of multi-threading are the overlap between what is useful and what is realistically possible given the current interpreter.</p>

<h2>autocroak</h2>

<p>This is essentially an autodie replacement with one important difference: it's based on opcode overrides instead of function overrides. This means not only that it interacts better with other pragmas, but also that it can support keywords that can not easily be overriden (such as <code>print</code> and <code>system</code>). It should also give less weird edge-cases than autodie.</p>

<h1>Raku</h1>

<p>I didn't produce as much Raku code this year, most of my Raku energy went into writing a series of blog posts that eventually I made a <a href="https://www.youtube.com/watch?v=elalwvfmYgk">conference presentation</a> instead.</p>

<h2>Crypt::Passphrase</h2>

<p>This was a port of the previously mentioned Perl module. It doesn't quite have the backend ecosystem that its big brother has, but given that there's a lot less legacy software in Raku that's not all that much of a problem.</p>

<h2>Net::MQTT</h2>

<p>A friend complained about the lack of MQTT support in Raku, and binary protocols just happen to be something I have a lot of experience with, so I implemented an MQTT client. While arguably this is the least useful module of the bunch, it was the most fun to write. Raku's typesystem and integrated event loop made this experience a lot smoother than they would have been in other languages.</p>
]]>
        

    </content>
</entry>

<entry>
    <title>The Witch and the Witch-hunt</title>
    <link rel="alternate" type="text/html" href="https://blogs.perl.org/users/leon_timmermans/2021/05/the-witch-and-the-witch-hunt.html" />
    <id>tag:blogs.perl.org,2021:/users/leon_timmermans//150.10236</id>

    <published>2021-05-23T20:51:52Z</published>
    <updated>2024-05-12T20:45:00Z</updated>

    <summary>A lot has been said about the recent CAT report and updates. It feels to me like we&apos;re not getting anywhere because the critical matters aren&apos;t being addressed....</summary>
    <author>
        <name>Leon Timmermans</name>
        <uri>http://codespeaks.blogspot.com/</uri>
    </author>
    
    <category term="cat" label="cat" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="tpf" label="tpf" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="https://blogs.perl.org/users/leon_timmermans/">
        <![CDATA[<p>A lot has been said about the recent CAT report and updates. It feels to me like we're not getting anywhere because the critical matters aren't being addressed.</p>
]]>
        <![CDATA[<h1>Division</h1>

<p>Right now there are two groups of people with opinions on this matter.</p>

<p>One group is appalled by the original report, because they have a number of serious concerns with the report. There was</p>

<ul>
<li>A truth-seeking process that was haphazard at best, in ways that are obvious to anyone with the slightest bit of knowledge of what happened</li>
<li>An expeditionary power-grab despite explicit requests not to do so</li>
<li>A punishment that was clearly not proportional to the incidents described</li>
<li>A willingness to cherry pick people from one side of a conflict where both sides have behaved in ways they really shouldn't have.</li>
<li>An apparent willingness to punish people who associate with Subject</li>
</ul>

<p>Combined this means that people fear the CAT because this is exactly the sort of behavior that can easily result in innocent people being banned.</p>

<p>The other group was relieved that someone they have known to be toxic is finally being removed from the community. Most have had so many negative experiences with him that they'll readily believe any further accusations in his general direction without need for further evidence. Others genuinely don't care anymore how the sausage is made as long as he's eliminated from the community.</p>

<p><strong>These different worldviews make it almost impossible for people to talk about the issue at hand, because they're talking past each other</strong>. Almost any discussion on the subject quickly devolves to bickering between people saying "How can you defend this toxic person" versus people saying "how can you defend this miscarriage of justice". For a lot of people it becomes a "you're either with us or against us" type of issue. Without splitting these conversations, we can't actually meet each other eye-to-eye. <strong>One can admit that what happened here was a cockup without denying that it tries to deal with an actual issue</strong>.</p>

<h1>Accountability</h1>

<p>Simply put, it rather appears like the CAT is firmly in the second camp. Everything that happens makes sense if they already believed him to be toxic and this incident was an opportunity to kick him out for once and for all. I'm not saying this was a conspiracy or some such; I'm suggesting that they were sufficiently biased that they got sloppy in dealing with this incident, they were entirely caught off guard by opposition to what they had done. The thing is, <strong>the CAT should not be doing a witch hunt, even if we know the target to actually be a witch</strong>.</p>

<p>The CAT's (draft) charter says <em>"the CAT must be trusted and viewed as consistent and impartial"</em> and <em>"to maintain the trust of the community, the CAT must make its processes and actions transparent while not sacrificing privacy"</em> but right now a large segment of the community doesn't trust them anymore because they have failed to do exactly those things. Despite all their good intentions, the CAT's actions actively worked against those intentions by focusing on the "easy win" and made the situation more difficult the next time action needs to be taken.</p>

<p><strong>The CAT is supposed to enforce accountability in our community, but it can not credibly and effectively do that if it is not accountable itself</strong>. What TPF should have done IMHO is pull the report and let someone else redo the entire thing, but it's probably too late for that now. What they can still do to put our community on a path out of this conflict is for them to:</p>

<ol>
<li>acknowledge what they did wrong</li>
<li>answer the question "how could this have happened?"</li>
<li>apologize for it</li>
</ol>

<p>but from my conversations with them over the past five weeks it rather looks like they intend to just move on without doing any of those things.</p>

<h1>Closure</h1>

<p>We've been infighting for a full year now, for a brief moment between the PSC's <code>use v7</code> announcement and the CAT's report it seemed we might finally get some peace. The CAT clearly underestimated just how divisive this action would be, and more division is the very last thing our community needs right now. This is the thing that upsets me the most of all; I remember a few weeks ago telling myself "finally we can put all this drama behind ourselves", I even wrote a blog post with that perspective and quite the opposite has happened.</p>

<p>I am tired of conflict and very disappointed.</p>
]]>
    </content>
</entry>

<entry>
    <title>Perl7 is a fork of values</title>
    <link rel="alternate" type="text/html" href="https://blogs.perl.org/users/leon_timmermans/2020/08/perl7-is-a-fork-of-values.html" />
    <id>tag:blogs.perl.org,2020:/users/leon_timmermans//150.9836</id>

    <published>2020-08-01T11:28:46Z</published>
    <updated>2020-08-01T21:29:51Z</updated>

    <summary>Before reading this, you should watch this video where Bryan Cantrill explains a value-conflict between Joyent and Node.js, I believe we have a similar problem. In it he defines a list of project values: All these values are important -...</summary>
    <author>
        <name>Leon Timmermans</name>
        <uri>http://codespeaks.blogspot.com/</uri>
    </author>
    
    <category term="fork" label="fork" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="perl7" label="perl7" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="https://blogs.perl.org/users/leon_timmermans/">
        <![CDATA[<p>Before reading this, you should watch <a href="https://vimeo.com/230142234">this video</a> where Bryan Cantrill explains a value-conflict between Joyent and Node.js, I believe we have a similar problem.</p>

<p>In it he defines a list of project values:</p>

<p><img src="https://image.slidesharecdn.com/nodesummit2017-170726223246/95/platform-as-reflection-of-values-joyent-nodejs-and-beyond-3-638.jpg" alt="values" title="" /></p>

<p>All these values are important - but they are in tension. In the end one has to choose between them.</p>

<p>Perl's has traditionally prioritized certain values over these others, and in my experience these are:</p>

<ul>
<li>Expressiveness</li>
<li>Extensibility</li>
<li>Stability</li>
</ul>
]]>
        <![CDATA[<p>Expressiveness is probably the most obvious one.</p>

<p>Extensibility is probably less obvious, probably because it was less of a concious choice, but feels like the right pick for a language that has several OO frameworks and custom keywords.</p>

<p>Stability, in particular backwards compatibility, is thoroughly embedded in our <a href="https://perldoc.perl.org/perlpolicy.html">policy document</a>:</p>

<blockquote>
  <p>Lately, ignoring or actively opposing compatibility with earlier
versions of Perl has come into vogue. Sometimes, a change is proposed
which wants to usurp syntax which previously had another meaning.
Sometimes, a change wants to improve previously-crazy semantics.</p>

<p>Down this road lies madness.</p>

<p>...</p>

<p>When in doubt, caution dictates that we will favor backward
compatibility.</p>

<p>...</p>

<p>Using a lexical pragma to enable or disable legacy behavior should be
considered when appropriate, and in the absence of any pragma legacy
behavior should be enabled.</p>

<p>...</p>

<p>No matter how frustrating these unintentional features may be to us
as we continue to improve Perl, these unintentional features often
deserve our protection. It is very important that existing software
written in Perl continue to work correctly.</p>
</blockquote>

<p>More than any other major scripting language, we value keeping code working. Where other similar languages (especially Python) are breaking relatively common constructs regularly, we generally tried to limit that to the margins (though there's certainly some breakage in any major release).</p>

<p>That doesn't mean all subcommunities share exactly the same values though. I'm involved in the toolchain, and in the toolchain we have very specific values:</p>

<ul>
<li>Robustness</li>
<li>Portability</li>
<li>Stability</li>
</ul>

<p>These are the values of sysadmins. Environments where working things have to keep working.</p>

<p>Whereas for example the Mojo community generally seems to prioritize</p>

<ul>
<li>Approachability</li>
<li>Expressiveness</li>
<li>Velocity</li>
</ul>

<p>These are the values of modern web development, where change is the only constant.</p>

<p>And mostly, that difference is fine. It helps a lot if a community's values overlap with the language values, but different communities can have different values without biting each other.</p>

<p>That said, Perl has been having an internal conflict over its values and where to take the language itself. This tension has existed for several years now, and is focused primarily around stability. The primary axis of tension is approachability versus stability.</p>

<p>Simply put, should new features and defaults be guarded by a version or feature guard (e.g. <code>use v5.34</code> or <code>use v7</code>) (stability), or should they be enabled by default in the next perl version (approachability). <strong>7.0 doesn't aim to bring new features, it doesn't enable us to do anything that isn't possible without it</strong> (other than not writing that guard). <strong>Instead, it aims to change perl culture as we know it. The whole point of perl7 is radically choosing approachability over stability</strong>.</p>

<p>The crucial thing to realize here is that that means that perl7 is not just a fork of the interpreter, <strong>it is also a fork of our community and our ecosystem</strong>. To some extent that fork can be postponed until perl8 drops perl5 compatibility, but given this new course it is inevitable. Some will join this brave new world, and some will not.</p>

<p>To make this fork of values complete, even the values of governance are completely different. Where perl5 had perl5-porters, a mailing list that was open to the entire community (and historically perhaps a bit too open), perl7 has a steering committee whose membership is invite-only and that only posts summaries of its activities to p5p.</p>

<p>And while everyone is wondering where perl7 is going, the other crucial question is where perl5 is going; will it stop where it is now (the current official plan), will there be a 5.34 (something I have repeated argued for because it makes no sense for the sunsetting release to have experimental features, and is lacking a perl5 executable out the box), will perl5 development continue as it did before? This is something that isn't talked about much and I'm not sure yet what will happen, but I am pretty sure that decision shouldn't be taken by the people who don't want to use it.</p>

<p>I don't know where we're going. I'm not even sure if this forking is good or bad in the long run (it could be good if managed well, but so far it isn't). And that terrifies me.</p>
]]>
    </content>
</entry>

<entry>
    <title>Perl 7, not quite getting better yet</title>
    <link rel="alternate" type="text/html" href="https://blogs.perl.org/users/leon_timmermans/2020/06/not-quite-getting-better-yet.html" />
    <id>tag:blogs.perl.org,2020:/users/leon_timmermans//150.9799</id>

    <published>2020-06-24T20:50:30Z</published>
    <updated>2020-06-24T21:16:10Z</updated>

    <summary> Hegel remarks somewhere that all great world-historic facts and personages appear, so to speak, twice. He forgot to add: the first time as tragedy, the second time as farce. The Eighteenth Brumaire of Louis Napoleon - Karl Marx Sawyer...</summary>
    <author>
        <name>Leon Timmermans</name>
        <uri>http://codespeaks.blogspot.com/</uri>
    </author>
    
    <category term="perl7perl5" label="perl7 perl5" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="https://blogs.perl.org/users/leon_timmermans/">
        <![CDATA[<blockquote>
  <p><em>Hegel remarks somewhere that all great world-historic facts and personages appear, so to speak, twice. He forgot to add: the first time as tragedy, the second time as farce.</em></p>

<p>The Eighteenth Brumaire of Louis Napoleon - Karl Marx</p>
</blockquote>

<p>Sawyer just announced his plans for perl 7. And while Perl 7 sounds like a lovely language, I do see a number of issues:</p>

<h1>Cohabitation / Forking</h1>

<p>The proposal is presented as a linear progress, I don't believe this is realistic. This would be fork much like the python 3 transition is (which also wanted to be a simple linear progression). As we all know, they're currently in year 12 of a 5 year transition.</p>

<p>There are several problems here. CPAN as an ecosystem is the one that is given most attention to (not without reason; it is without doubt the most important collection of Perl code), but it's not even the biggest problem.</p>

<p>The biggest problem is that <code>/usr/bin/perl</code> is infrastructure. We can't do breaking changes to its basic functionality for the same reason that shell and awk can't. Too many things in too many places are dependent on it, from system administration scripts to bioinformatics workflows to build systems (e.g. autotools, postgresql) and many more.</p>

<p>And this change is vastly breaking. Enabling strict and disabling prototypes (to make way for signatures) will break vast amounts of code, especially in the scripting domain of perl.
It's quite telling that 12 years after python3 was released <code>/usr/bin/python</code> isn't a python3 by default on any of the big distributions (Ubuntu, Debian, Fedora, Red Hat, OpenSuse); and arguably python is less entrenched than perl is. I don't believe that <code>/usr/bin/perl</code> will ever be perl7. That means that perl7 can only meaningfully exist if it's set up to coexist alongside of perl5 for a very long time. And that actually comes with a number of challenges that may not seem obvious at first (e.g. colliding script names and man pages).</p>

<p>Releasing a Perl7 will not erase perl5. Perl5 will in all likelihood remain the Perl that's available on any platform regardless of how successful perl7 will be.</p>

<h1>Perl 8 and beyond</h1>

<p>Major version transitions are costly, and often traumatic (Perl 6 and Python 3 being obvious examples). Communities also take a lot of time catching up with them (again, see above examples); at least a decade if not more.</p>

<p>A big, breaking release is something a mature programming language can only do once per decade or so; anything else would result in two transitions going on at the same time. We shouldn't even be thinking about a perl8 this decade, let alone a perl9. If we are to do a perl7, we must get it right the first time. And I don't think this plan is quite getting it right.
And quite frankly, I can't imagine any reason for wanting to do a big breaking release if we'd do 7 right.</p>

<h1>We are not ready</h1>

<p>The current plan is essentially <em>Enable all non-controversial features by default</em>, and I don't think that that is the best we can do. There are a lot of features that haven't been implemented before because they don't make much sense in a minor release (in particularly the kind that removes syntax like <code>no feature 'indirect'</code>). Releasing it now will force a perl8 relatively soon, and that would be undesirable for all the reasons stated above.</p>

<h1>Manpower</h1>

<p>We have been failing at shipping non-experimental signatures for more than half a decade now, why would we be able to ship a perl 7? The most significant new feature that made it out of experimental in the past half decade was postfix dereferencing, and while welcome it's not quite a game changer.</p>

<p>Sadly, the most convincing reason not to go through with this may very well be "we may not be able to". I think we need to figure out what problems we can resolve before deciding to actually go forward with this.</p>

<h1>Timescale</h1>

<p>There's just no way we can do all of the above before the end of the year for a variety of reasons. Not only because it will require adaptations on the perl5 side to enable cohabitation, but also because we will need to sort out a lot of details. Trying to rush is likely to result in a failure, and is not something we can afford. I can't imagine any way of successfully doing this that doesn't involve releasing a v5.34 first (and possibly more).</p>

<h1>Then don't upgrade?</h1>

<p>The <em>if you don't want your code to break then don't upgrade</em> argument is rather assuming users have a firm control over which perl they are running. This is generally true for million line perl web applications, but this is not true for system perl.</p>

<p>If our objective is to limit ourselves to perlbrew/perlbuild/etc&#133;, many of my objections become moot. But I don't think that should be the target, I think that would exclude a wide range of applications. So no, I don't think saying "then don't upgrade" really solves that problem. We may be able to postpone the problem, but it won't go away by itself.</p>

<h1>The absentee/maintainer dichotomy</h1>

<p>I do not recognize this distinction at all. Just because I actively maintain my stuff doesn't mean I want to be dealing with other people breaking my code. If I wanted to deal with the whims of a platform breaking trivial things I'd be programming python.</p>

<p>Associating not wanting the language to break with diminishing use of the language is perplexing to me. Perl is a language of which a lot has been written already, and relative to that past popularity not all that much new code is being written. Quite a lot of Perl strongholds are attributable to Perl not breaking, it's uncertain if the pain of this process will be worth the gain.</p>

<p>There's also this suggestion that people who care about backwards compatibility contribute less to the language. This isn't actually explained further but it seems like a rather bold statement to me.</p>

<h1>Whom are we serving?</h1>

<p>Perl has many different types of users, with many different needs. This is inherent to a language that tries to be useful at 1 line and at 1 million lines.</p>

<p>The argument that has been made in the keynote suggests that the only reason why one would use "old-style Perl" is because you've abandoned your code, and I don't think that is true. Many best practices that are essential when writing large applications are not nearly as valuable in a small script; it would be outright silly to suggest one-liners need strict.</p>

<p>The changes that are proposed are largely serving the manipulexity end of the spectrum. And this is an important user base, but it's not our only user base. For the whipuptitude end of the spectrum, the scripters, this represents their code breaking without them getting anything in return. That is the priority that is being chosen here.</p>

<h1>Bad code</h1>

<p>I believe this "bad pattern" rhetoric is flawed. Ultimately the only good code is working code, and the only bad code is code that doesn't get the job done. What I hear being described as bad code is actually merely ugly code. And this transition can break stuff for people, and breaking code is bad, whereas ugly code is only a problem to me if it ends up on my plate.</p>

<p>How did we get into this brave new world where one calls judgment on users and deplatforming the ones deemed bad?</p>

<p>This reminds me of a bioinformatician I met at a recent TPC. Was their code strict? No. Did it get the job done? Yes. Why would they care if we in the echo chamber approve of her code, they have more important things to do, like curing ovarian cancer. In my book, they got their priorities straight.</p>

<h1>Is this really worth it?</h1>

<p>This seems like a lot of pain, just to avoid having to type <code>use v5.32</code>. The real problem of course is that that doesn't only not enable warnings (which we can easily fix for 5.34), but it also doesn't enable signatures (probably the recent feature people care most about). If we can make <code>use v5.34</code> do those two things, <em>I don't think I need a perl7</em>, even if I understand why some other people feel they do want it. Boilerplate may be annoying, but one line of boilerplate in every file is way more tolerable to me than the pain of a fork.</p>

<p>This was Jesse Vincent's vision <a href="https://www.slideshare.net/obrajesse/perl-516-and-beyond">9 years ago</a>, and I still think this is the right trade-off for a platform like Perl.</p>
]]>
        

    </content>
</entry>

<entry>
    <title>Smartmatch in 5.27.7</title>
    <link rel="alternate" type="text/html" href="https://blogs.perl.org/users/leon_timmermans/2017/12/smartmatch-in-5277.html" />
    <id>tag:blogs.perl.org,2017:/users/leon_timmermans//150.8388</id>

    <published>2017-12-23T13:24:34Z</published>
    <updated>2017-12-24T21:41:09Z</updated>

    <summary>What happened? In the latest development release of perl, smartmatch changed quite a bit. Almost everything you believed about smartmatching is now wrong No really, everything. All previous rules are gone except a single one: you can smartmatch against any...</summary>
    <author>
        <name>Leon Timmermans</name>
        <uri>http://codespeaks.blogspot.com/</uri>
    </author>
    
    <category term="p5p" label="p5p" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="https://blogs.perl.org/users/leon_timmermans/">
        <![CDATA[<h1>What happened?</h1>

<p>In the latest development release of perl, smartmatch changed quite a bit.</p>

<h2>Almost everything you believed about smartmatching is now wrong</h2>

<p>No really, everything. All previous rules are gone except a single one: you can smartmatch against any object that overloads smartmatching (the only "objects" that overload them out of the box are <code>qr//</code> regexps).</p>

<p>Matching against a scalar value? Gone. Matching against a list of values? Also gone.</p>

<h2><code>when</code> is no more.</h2>

<p>The <code>when</code> keyword is gone, split into two keywords: <code>whereis</code> and <code>whereso</code>; one smartmatches the value against the current subject and the other does a simple boolean check much like <code>if</code>. I'll let you guess which is which. This split is for good reasons (when sometimes does one and sometimes the other, sometimes depending on things like optimizations), but that doesn't make this any more intuitive.</p>

<h2><code>use 5.010</code>/<code>use 5.028</code> won't guard you from this.</h2>

<p>It would have been possible to support both behaviors, because the old behavior is already using feature.pm. In fact one could even enable old and new style <code>when</code> at the same time in a scope without problems. None of this was done though.</p>

<h1>My suggestions</h1>

<h2>new smartmatch should be more useful.</h2>

<p>Right now one can't do anything with it without a helper library (like my <a href="https://metacpan.org/pod/Smart::Match">Smart::Match</a>). That's just silly.</p>

<p>The insanity of old style matching was that the overloads depended on both operands, this gave rise to hard to predict behavior, but that doesn't mean one can't define useful behavior that only depends on the right side that follows the most common use-cases. In particular matching scalars stringwise, and making <code>$foo ~~ ["bar", "baz" ]</code> mean <code>$foo in [ "bar", "baz" ]</code>.</p>

<h2>This should be opt-in</h2>

<p>Despite retroactively adding warnings the feature is experimental, it has become a widely used feature. This change is breaking a (yet unknown but significant) number of CPAN modules, and likely much more code in darkpan. Breaking this unless strictly necessary is dumb.</p>

<p>And it isn't necessary. We can easily only enable the new behavior when asked for. That way we can improve smartmatching without breaking a decade worth of smartmatching code.</p>

<h2>We need better words</h2>

<p><code>whereso</code> and <code>whereis</code> are way too confusing. I'm not sure what that would look like but this just doesn't cut it.</p>

<h2>Above all, we need a better process</h2>

<p>Somehow, p5p made a fundamental breaking change to the language without even trying to involve the wider community. This blogpost shouldn't have been the first time the wider community hears about it. And we need that wider community IMHO because no one on p5p (myself included) has the kind of language design talent that's required to do the sort of thing we did here. I don't know what the solution would look like exactly, but I like this process even less than I like the outcome so far. I must admit I'm somewhat jealous of Python's PEP process, though I'm not sure that would work without a language designer to guide it.</p>
]]>
        

    </content>
</entry>

<entry>
    <title>File::Slurp is broken and wrong</title>
    <link rel="alternate" type="text/html" href="https://blogs.perl.org/users/leon_timmermans/2015/08/fileslurp-is-broken-and-wrong.html" />
    <id>tag:blogs.perl.org,2015:/users/leon_timmermans//150.6984</id>

    <published>2015-08-18T11:54:05Z</published>
    <updated>2015-08-18T12:49:57Z</updated>

    <summary>If you are using File::Slurp, you should possibly reconsider. Basically, there are three reasons to do so; It is wrong in a lot of cases. File::Slurp predates IO layers, and as such doesn&apos;t take them into account well. A few...</summary>
    <author>
        <name>Leon Timmermans</name>
        <uri>http://codespeaks.blogspot.com/</uri>
    </author>
    
    
    <content type="html" xml:lang="en" xml:base="https://blogs.perl.org/users/leon_timmermans/">
        <![CDATA[<p>If you are using File::Slurp, you should possibly reconsider. Basically, there are three reasons to do so;</p>

<h1>It is wrong in a lot of cases.</h1>

<p>File::Slurp predates IO layers, and as such doesn't take them into account well. A few years ago, after some complaints, an attempt was done to make it handle encodings. This was nothing short of being wrong.</p>

<p>The best known bug in this area is <a href="https://rt.cpan.org/Ticket/Display.html?id=83126">#83126</a>, which means that <code>:encoding()</code> layers are always interpreted as <code>:utf8</code>. This not only means that UTF-8 encoded text is not validated (which can be a security risk), but also that files in other encodings (such as UTF-16) will be read as UTF-8, which surely will give an incorrect result.</p>

<p>Likewise it's not handling <code>:crlf</code> correctly, in particular explicitly asking for <code>:crlf</code> will always disable it, even on Windows.</p>

<p>Basically, it's doing all binmodes wrong except the one you shouldn't be using anyway (<code>:utf8</code>), and you should pretty much always be using a binmode, so there's no way to win really.</p>

<h1>The interface is poorly huffmanized.</h1>

<p>Huffmanization is the process of making commonly used operations shorter. File::Slurp is failing to huffmanize in the unicode world of 2015. Text files are usually UTF-8 nowadays, which in File::Slurp would typically be <code>read_file($filename, binmode =&gt; ':raw:utf8')</code>. The shortest option, <code>read_file($filename)</code>, does something that most people don't really want anymore: latin-1 encoded files with platforms specific line-endings.</p>

<p>This is mainly the fault of perl itself (backwards compatibility is a PITA), but a library can work around this to make the programmers life easier.</p>

<h1>It is poorly maintained</h1>

<p>The critical bug mentioned above has been known for about two years, yet the author hasn't even bothered to respond to it, let alone fix it. There hasn't been a release in 4 years despite an increasingly long <a href="https://rt.cpan.org/Public/Dist/Display.html?Name=File-Slurp">list of issues</a>. Worst yet, this isn't the first time such a thing happens; before his last maintenance surge in the spring of 2011 the author was also missing-in-action for years. This negligence is inexcusable for a module that is so commonly depended upon.</p>

<h1>Recommendations</h1>

<p>Instead of File::Slurp, I recommend you use one of these modules depending on your needs:</p>

<p>If your needs are minimal, I'd recommend my <a href="https://metacpan.org/pod/File::Slurper">File::Slurper</a>. It provides correct, fast and easy to use slurping and spewing functions.</p>

<p>If your needs are average (which is the case for most people), I'd recommend <a href="https://metacpan.org/pod/Path::Tiny">Path::Tiny</a>. This provides a well-balanced set of functions for dealing with file paths and contents.</p>

<p>If you want to go for maximal overkill, try <a href="https://metacpan.org/pod/IO::All">IO::All</a>. It will do everything you can imagine and more. </p>
]]>
        

    </content>
</entry>

<entry>
    <title>QA Hackathon 2013</title>
    <link rel="alternate" type="text/html" href="https://blogs.perl.org/users/leon_timmermans/2013/04/qa-hackathon-2013.html" />
    <id>tag:blogs.perl.org,2013:/users/leon_timmermans//150.4580</id>

    <published>2013-04-15T22:46:00Z</published>
    <updated>2024-05-18T22:38:34Z</updated>

    <summary>Pre-hackathon: We Tux Day 1 My first day was largely spent analyzing and fixing bugs in the Module::Build::Tiny toolchain, and some Lancaster consensus discussions on various toolchain pieces. This was a very useful day, that ended with some wonderful food....</summary>
    <author>
        <name>Leon Timmermans</name>
        <uri>http://codespeaks.blogspot.com/</uri>
    </author>
    
    <category term="qah" label="QAH" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="qah2013" label="QAH 2013" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="https://blogs.perl.org/users/leon_timmermans/">
        <![CDATA[<h1>Pre-hackathon:</h1>

<p>We Tux</p>

<h1>Day 1</h1>

<p>My first day was largely spent analyzing and fixing bugs in the Module::Build::Tiny toolchain, and some Lancaster consensus discussions on various toolchain pieces. This was a very useful day, that ended with some wonderful food.</p>

<h1>Day 2</h1>

<p>Spent the first part of the day fixing bugs I made the day before, then helped out with other people's issues. Didn't participate in all consensus discussions as PAUSE isn't really my thing. </p>

<h1>Day 3</h1>

<p>Only </p>
]]>
        

    </content>
</entry>

</feed>
