byterock

I decided I might as well get busy with CloudFront and at least get most of my real world scripts written.

At the moment I am getting 400 errors such as 'InvalidArgument' or 'InvalidOrigin' on the Delete and Create actions as I do not have the proper config on the AWS end for the Creates and for the Deletes as I do not have anything on my AWS account to delete.

Reading though the API documentation is seems there is quite the procedure to actually do some of the actions, for example to invoke the DeleteStreamingDistribution action you have to follow a six pre-steps all of which must pass. So I guess I can forget a quick run on this API

So the plan is to get all the real world scripts written up and then go though the full CRUD actions for each and get them working with a good generated test case for each.

After a few hours of mind-numbingly boring cut and pasting and the odd edit I was ready to really get cracking and the first action I started to look at was 'CreateCloudFrontOriginAccessIdentity '

Now this I know is working as I have played with it before but this time out I am going to have a much closer look at it.

Well when I run it I do get a Status 201 but my returned class is

 bless( {

        'ETag' => 'E2J612BD0LRDHQ',

        'Location' => 'https://cloudfront.amazonaws.com/2019-03-26/origin-access-

identity/cloudfront/E3D5Y5RWA05QO1',

        'CloudFrontOriginAccessIdentity' => bless( {

                'Id' => '',

                'CloudFrontOriginAccessIdentityConfig' => bless( {

                                'CallerReference' => '',

                                'Comment' => ''

                        },

                 'Paws::CloudFront::CloudFrontOriginAccessIdentityConfig' ),

                        'S3CanonicalUserId' => ''

                }, 'Paws::CloudFront::CloudFrontOriginAccessIdentity' ),

                '_request_id' => '3b880846-30c6-11ea-a755-f94346075f98'

         }, 'Paws::CloudFront::CreateCloudFrontOriginAccessIdentityResult' );

which is missing most of the attriobute values as my returned XML is

<?xml version="1.0"?>
<CloudFrontOriginAccessIdentity xmlns="http://cloudfront.amazonaws.com/doc/2019-03-26/">
  <Id>E3D5Y5RWA05QO1</Id>
  <S3CanonicalUserId>84f47125a87a...1</S3CanonicalUserId>
  <CloudFrontOriginAccessIdentityConfig>
    <CallerReference>some test here</CallerReference>
    <Comment>This is Mr Poopy Buthole Calling</Comment>
  </CloudFrontOriginAccessIdentityConfig>
</CloudFrontOriginAccessIdentity>

Ok back into got old RestXMLResponse.pm to see what is being dropped. My first hunch was the XML was not being parsed correctly and adding a little debugging my suspicions where correct

{

    'Location' => {

        'Id' => 'E3D5Y5RWA05QO1',

        'S3CanonicalUserId' => '84f47125a87a26e...632e70ffcd71',

        'xmlns' => 'http://cloudfront.amazonaws.com/doc/2019-03-26/',

        'CloudFrontOriginAccessIdentityConfig' => {

                        'Comment' => 'This is Mr Poopy Buthole calling',

                        'CallerReference' => 'some test here'

                        }

    }

};

so something is out of whack between the class and the returned XML. Well first place to look is in the Boto.

 "CreateCloudFrontOriginAccessIdentityResult":{
      "type":"structure",
      …
    "payload":"CloudFrontOriginAccessIdentity"
    },

so the payload of that is 'CloudFrontOriginAccessIdentity' but in the generated class we have

package Paws::CloudFront::CreateCloudFrontOriginAccessIdentityResult;
  use Moose;
…
    use MooseX::ClassAttribute;
    class_has _payload => (is => 'ro', default => 'Location');
1;

So there is the problem. Into the callresult_class.tt template to correct that.

It did not take me long to find my mistake. Seem the other day when I fixed the template I did not notice in the boto that sometimes the name of the param and the payload may not match up. So to fix that I just did this edit;

  [%- IF (shape.members.$param_name.streaming == 1) %], traits => ['ParamInBody'][% stream_param = param_name %][% END %]
-- [%- IF (shape.payload) %] [% has_payload=param_name %] [% END %]
++  [%- IF (shape.payload) %] [% has_payload=shape.payload %] [% END %]
   [%- IF (c.required_in_shape(shape,param_name)) %], required => 1[% END %]);

and on a recompile I get what I want.

 bless( {
    '_request_id' => 'e7719890-30ca-11ea-a584-9ba090fa6cd5',
    'ETag' => 'E2J612BD0LRDHQ',
    'CloudFrontOriginAccessIdentity' => bless( {
                'CloudFrontOriginAccessIdentityConfig' => bless( {
                        'CallerReference' => 'some test here',
                        'Comment' => 'This is Mr Poopy Buthole calling'
                },
                 'Paws::CloudFront::CloudFrontOriginAccessIdentityConfig' ),
    'S3CanonicalUserId' => '84...f',
    'Id' => 'E3D5Y5RWA05QO1'
    }, 'Paws::CloudFront::CloudFrontOriginAccessIdentity' ),
    'Location' => 'https://cloudfront.amazonaws.com/2019-03-26/origin-access-
identity/cloudfront/E3D5Y5RWA05QO1'
    }, 'Paws::CloudFront::CreateCloudFrontOriginAccessIdentityResult' );

A good Start for today. But I really want to do things right with this round of changes so I also checked my generated test and I found there was a bug there I should fix; I was getting this

headers:
  x-amz-request-id: ~

so I am not setting that correctly. A little poking about I noticed there are a few ways the AWS returns the request id so I updated my test generator to reflect this.

I next notice that my request tests where not generating correctly as well as I was getting this

---
CloudFrontOriginAccessIdentityConfig: !!perl/hash:Paws::CloudFront::CloudFrontOriginAccessIdentityConfig
  CallerReference: some test here
  Comment: This is Mr Poopy Buthole calling

This one took a little while to figure out but in the end the fix was very simple; The problem was in with the way I was gnerating the above YAML. I was simply coercing my caller class into a hash like this;

    my $call_params = {%$call};

The problem with the CloudFront API is there are a large number of embedded objects in most of the call classes and my simple coerce will not work correctly embedded objects.

Fortunately someone must of blundered into this before because in the 'Paws::CloudFront ' class there is a 'to_hash' sub that converts a call into a hash and I can access it directly as I pass that in on the $service param. All I need is is this little patch;

++  my $call_params = $service->to_hash($call);
--  my $call_params = {%$call};

and now I get

---
CloudFrontOriginAccessIdentityConfig:
  CallerReference: some test here
  Comment: This is Mr Poopy Buthole calling

I am going to check that new test and when I tried to run it I get;

can't locate Paws/CLOUDFRONT.pm

Another little bug; Seem I need the class name vs the service name. Easy enough;

  my @services =  split("::",ref($service));
  my $test_hash = {
        call    => $call->_api_call,
        service => $services[1],
…

and now my 09_request test passes

ok 1 - Call CloudFront->CreateCloudFrontOriginAccessIdentity from t/09_requests
/cloudfront-createcloudfrontoriginaccessidentity.request
ok 2 - Got content eq from request
ok 3 - Got method eq from request
ok 4 - Got Param->key: content-md5 eq from request
ok 5 - Got Param->key: content-type eq from request
ok 6 - Got Param->key: host eq from request
ok 7 - Got Param->key: x-amz-content-sha256 eq from request
ok 8 - Got Param->key: CloudFrontOriginAccessIdentityConfig.CallerReference eq from request
ok 9 - Got Param->key: CloudFrontOriginAccessIdentityConfig.Comment eq from request
ok 10 - Have https://cloudfront.amazonaws.com/2019-03-26/origin-access-identity/cloudfront in the URL
ok 11 - Have /2019-03-26/origin-access-identity/cloudfront in the URL
ok 12 - Have /2019-03-26/origin-access-identity/cloudfront in the URI
1..12


ok 1 - Call CloudFront->CreateCloudFrontOriginAccessIdentity from t/10_responses/cloudfront-createcloudfrontoriginaccessidentity.response
ok 2 - Got CloudFrontOriginAccessIdentity.CloudFrontOriginAccessIdentityConfig.CallerReference eq some test here from result
ok 3 - Got CloudFrontOriginAccessIdentity.CloudFrontOriginAccessIdentityConfig.Comment eq This is Mr Poopy Buthole calling from result
ok 4 - Got CloudFrontOriginAccessIdentity.Id eq E3D5Y5RWA05QO1 from result
ok 5 - Got CloudFrontOriginAccessIdentity.S3CanonicalUserId eq 84f47125a87a26ea5ba42f3be65fbefebdb7440d82e7d27907c52c969ac4f6c05ef03046db8cd6f74dab632e70ffcd71 from result
ok 6 - Got ETag eq E2J612BD0LRDHQ from result
ok 7 - Got Location eq https://cloudfront.amazonaws.com/2019-03-26/origin-access-identity/cloudfront/E3D5Y5RWA05QO1 from result
ok 8 - Got _request_id eq 5459351b-bb65-414e-9696-df581ea8b373 from result
1..8

so now onto the next call.