Term::ReadLine::Gnu Unicode Hell

I must be doing something wrong. Surely. I have a REST service. I get data using HTTP::Tiny and use JSON::Tiny to decode it.

If I try to print to the Term::ReadLine::Gnu OUT filehandle, I get double encoded strings, like this:


If I try to binmode it to utf-8, things get worst, with triple encoded strings:


Resolved it decoding (Encode::decode) from UTF-8 and using the internal Perl character representation. It worked.

The problem was when I tried to feed a pre-defined input line to Term::ReadLine::Gnu. Result looked something like this:

my $newGloss = $self->{term}->readline(">> ", 
    decode("utf-8", decode("utf-8", $gloss->{gloss})));

my $x = encode("utf-8", $newGloss);
if (length($x) && $x ne $gloss->{gloss}) {

  my $ans = $self->post("gloss/$offset", 
                gloss => decode("utf-8",$newGloss));
  if ($ans) {
      // ...	

else {
  $self->INFO("Nothing changed!");

If someone knows any good solution, I am grateful!


is it possible that you are not decoding your json as it comes in? In JSON::Tiny you probably should be using decode_json rather than from_json (which you might be, but without seeing the code I don't know).

Leave a comment

About Alberto Simões

user-pic I blog about Perl. D'uh!