Joe McMahon

I've been working on WhiteHat's new sourcecode scanner, and one of the things we need to do is transmit snippets of source code around inside of XML. Since many XML parseres are notoriously sensitive to non-ASCII characters, even in CDATA, we decided that the simplest thing to do was Base64-encode them. This shouldn't be a problem, right?

Well, no. Turns out that MIME::Base64 is very picky itself about encoding things properly. If you hand it UTF-8, it screeches, "Characters with more than 8 bits?! Heavens!" and falls over. There's an incantation noted it its documentation that will let you handle it, but it didn't, at least for me, work all that dependably.

A little poking around turned up Encode::Encoder, which works like a charm, and is very clean:

    use Encode::Encoder qw(encoder);
    use Encode::Base64;

    my $encoded = encoder($unencoded)->base64;
    my $decoded = encoder($encoded)->utf8('base64');

Pretty sweet. Only one thing to note, which is probably unique to my application: the result returned from an encoder()->base64() call (and the corresponding ->utf8() call) is an object, not a string. If you're going to want to hand those off to the standard JSON module, be aware that it does not like this, and will die with the error encountered object 'Point=HASH(0x40017288)', but neither allow_blessed nor convert_blessed. It's easy enough to get around this: just interpolate the output from the base64() method into a double-quoted string. This invokes the object's qq() overload and gives you a plain old string:

    $actual_string = qq($encoded);

If you want to do that all in one operation without a separate variable, you can use this trick to interpolate an evaluated expression into a string:

    $actual_string = qq<@{[encoder($unencoded)->base64]}>;

That's evaluate the expression, make it the only element in an anonymous array, then dereference that array in the string to force the value of the element to be interpolated.