Ed J

Version 0.01 of JSON::Transform is now on CPAN. It lets you express transformations of JSON-able data (i.e. data that is only hashes, arrays, simple scalars plus booleans) concisely and declaratively, without writing any other code.

Locations within the data structure are expressed using JSON Pointer (RFC 6901), and there are both user and automatically-system-set variables available that can be interpolated to make such locations be computed.

What is this for?

Ben Bullock poses an excellent question in the comments: "What kind of jobs would we use this for?". My answer is that this provides another tool for separating out a common class of concerns, to reshape data. This is part of my ambition to eliminate a lot of code that people routinely write. By porting this module to other languages, that will make my libraries more widely usable.

More immediately, I will be enhancing this module to the point where I will be able to replace the 572 lines of code in SQL::Translator::Parser::OpenAPI with just a "transformation", which ought to be a few dozen lines, and will instantly become as portable as this module itself.

The immediate steps towards this will be to add array and object/hash literals, and both user-defined and system-provided functions, largely for string operations.

Commenting problem

When I try to reply, I get "Your session has expired. Please sign in again to comment." despite it also saying above that I'm signed in, and despite me signing out and in again - if anyone can help me fix this, comment or drop me a line!

Examples

A few example transformations, taken from the tests. The data is in Perl format.

Array identity

"" <@ [ $V ]

Input AND output: [ 1, 'a', 4 ]

Hash identity

"" <% { $K:$V }

Input AND output: { a => {k=>'va'}, b => {k=>'vb'} }

Hash move to new

"/c" << "/a"

Input: { a => {k=>'va'}, b => {k=>'vb'} }

Output: { c => {k=>'va'}, b => {k=>'vb'} }

Hash keys

"" <% [ $K ]

Input: { a => {k=>'va'}, b => {k=>'vb'} }

Output: [ qw(a b) ]

Array of hashes to single hash

"" <@ { "/$K/id":$V#`id` }

Input: [{id=>'a', k=>'va'}, {id=>'b', k=>'vb'}]

Output: { a => {k=>'va'}, b => {k=>'vb'} }

Complete reshape using variables

$a <- "/a"
$b <- "/b"
"" <- $EA
"/0" <- $b
"/1" <- $a
-- this is a comment!
"/2" <- $EA

Input: { a => {k=>'va'}, b => {k=>'vb'} }

Output: [ {k=>'vb'}, {k=>'va'}, [] ]

Slightly more complex reshape using object literals

Prompted by Mike T's comment, version 0.02 has added array and object literals, together with the capability to apply a JSON pointer to an existing value:

"" <- "/Time Series (Daily)" <% [ .{ `date`: $K, `close`: $V<"/4. close" } ]

Input:

{
  "Meta Data" => {},
  "Time Series (Daily)" => {
    "2018-10-26" => { "1. open" => "", "4. close" => "106.9600" },
    "2018-10-25" => { "1. open" => "", "4. close" => "108.3000" },
  }
}

Output:

[
  { 'close' => '108.3000', 'date' => '2018-10-25' },
  { 'close' => '106.9600', 'date' => '2018-10-26' },
]

Command-line example

There is also a command-line script supplied, jt:

$ echo '{"a": 1, "b": 2}' | jt '"" <% [ $K ]'
# prints: ["a","b"]

Within Perl example

That same transformation, from within Perl. While this is obviously a trivial example, it shows my thought on how one should use this module to make your own library. I generally think, to paraphrase Einstein, one should have things as configurable as possible, and no more so. Therefore, rather than having the transformation description got externally, it is completely acceptable and indeed idiomatic to have it hardcoded within your library.

package MyKeys;
use strict;
use warnings;
use JSON::Transform qw(parse_transform);

my $transform = '"" <% [ $K ]';

my $transform_func = parse_transform($transform);

sub get_keys {

  my ($data) = @_;

  $transform_func->($data);

}

package main;

use JSON::MaybeXS;

print encode_json(MyKeys::get_keys(decode_json(join '', <>)));

Acknowledgements

Dave Lambley (on CPAN: DLAMBLEY) made excellent suggestions that I hope has made this post clearer and more illustrative.

Thanks also to all the commenters.