byterock

he first order of the day was to clean up all the debugging code I peppered across my perl trying to find an answer to my 'Failed Signature' bug.

This did take a while and in the end the changes that I am sticking with are;

  if ( my $xml_body = $self->_to_xml_body($call) ) {

        $request->content($xml_body);

++      $request->header( 'content-type' => 'application/xml');  #this is an XML interface so it should have this header

     }

--    $self->sign($request);
++   if (ref($self) eq "Paws::S3Control"){ #calls out to S3Control need the Account ID
++      $self->sign($request,$call->AccountId());
++   }
++   else {
++      $self->sign($request);
++   }

      return $request;

in RestXmlCaller.pm and in 'S3V4Signature.pm'

   sub sign {

--    my ($self, $request) = @_;

++    my ($self, $request,$account_id) = @_;

     $request->header( Date => $request->header('X-Amz-Date') // strftime( '%Y%m%dT%H%M%SZ', gmtime ) );

--    $request->header(

--        'Host' => $self->endpoint->default_port == $self->endpoint->port

--        ? $self->endpoint->host

--        : $self->endpoint->host_port);

--    if ($self->session_token) {

++    my $url = $request->url();

++    $url =~ s/s3-control/$account_id\.s3-control/g;

++    $request->url($url);

++       $request->header(

++        'host' => $self->endpoint->default_port == $self->endpoint->port

++        ? $account_id.".".$self->endpoint->host

: $account_id.".".$self->endpoint->host_port);

Well now that I have all the debugging code additions cleaned up it is time to take a quick review of the changes I made to get 'S3Control' 'CreateJob 'working.

First in RestXmlCaller.pm I added it the request header 'content-type' = XML and a new call out to the sign sub of 'Paws::Net::S3V4Signature' based on the S3Control class that now passes in the account_id.

In the sign sub of 'Paws::Net::S3V4Signature'' I added in the account id into the host header

At some point, more than likely after I finish off all the XML API calls, I will have to go back and re-factor the code as there really is very little difference in the two signature call roles S3V4Signature.pm and S3Signature.

I am going to hold off for now because I am not sure how many other changes have to go in to get the other XML APIs working

After all of the above I re-ran the test suite and still got 100% pass so I was able to move forward and tackel the next problem this error;

<Code>InvalidRequest</Code><Message>Missing role arn</Message>

I first checked the XML that is I was sending vs the AWS documentation and I confirmed it is missing from the XML I was generating but I did include it in the 'parameters' that where being passed along.

So I checked Boto to make sure I did not overlook anything new

"RoleArn":{

"shape":"IAMRoleArn ",

"documentation":"<p>The Amazon Resource Name (ARN) for the Identity and Access Management (IAM) Role that batch operations will use to execute this job's operation on each object in the manifest.</p>"

} 

No nothing there I haven't seen before. It simply points to a string.

"IAMRoleArn":{
"type":"string",
"max":2048,
"min":1
}, 

So the problem is not in the templates or the auto-generated classes it must be in the XML generation subs of ' RestXmlCaller.pm'

So I dove into the XML subs string and found after a little playing around there is a fall though for all the top level attributes that do not have a trait. For example in the 'CreateJob' class attributes like this one;

 has RoleArn => (is => 'ro', isa => 'Str' , required => 1);

will fall though the XML sub and not get added to the XML so I fixed that with this;

  else {

                $xml .= sprintf '<%s>%s</%s>',$attribute->name,$attribute_value,$attribute->name;

            }

but that was a problem as there where other attributes on the class like these;

 has _location => (is => 'ro', isa => 'Str',  default => 'CreateJobRequest');

  has _xmlNamespace => (is => 'ro', isa => 'Str',  default => 'http://awss3control.amazonaws.com/doc/2018-08-20/'

and these non XML attrubtes cause invalid XML errors like this;

<_location>CreateJobRequest</_location><_xmlNamespace>http://awss3control.amazonaws.com/doc/2018-08-20/</_xmlNamespace></CreateJobRequest>

so I will have to get rid of that. The best way to acomplish this is with yet another trait. With this new trait I can check is see if an attribute is part of the API.

After playing about a little I found it would be easier to mark the few attributes I do not want then all the ones I want. So with this new trait;

package Paws::API::Attribute::Trait::isLocal;
  use Moose::Role;
  use Moose::Util;
  Moose::Util::meta_attribute_alias('isLocal');

I added it to the templates

[%- IF operation.input.locationName -%]
  has _location => (is => 'ro', isa => 'Str',  default => '[% operation.input.locationName %]', traits => ['isLocal']);
[%- END %]
[%- IF operation.input.xmlNamespace %]
  has _xmlNamespace => (is => 'ro', isa => 'Str',  default => '[% operation.input.xmlNamespace.uri %]',traits => ['isLocal'] );
[%- END %]

and at the end of my XML subs I just add in;

--else {
elsif (!$attribute->does('Paws::API::Attribute::Trait::IsLocal')) {

Now a quick recompile and I have removed those nasty extra XML attributes.

Alas the story is not over yet. Once I was done I re-ran my test suite and got some 25 errors. Seems there are Quite a few calls in S3 that are effected by the above change as the XML no longer matches up. I know have extra XML nodes popping up all over the place.

I was thinking I could just change the tests to match the new output but before I tried that I figured it was a good idea give one of my real world tests a go to make sure the S3 API would allow the extra Nodes.

It didn't

so back into my code.

Well I know that only the s3Contol class have the '_location' at the class level so maybe a different tack is need here.

I played about for a while and just create a temp var $xml_extra that I just use in the fall though and then tack onto the end like this

sub _to_xml_body {
    my ( $self, $call ) = @_;
    my $xml = '';
    my $xml_extra = '';
…

 }

            elsif (!$attribute->does('Paws::API::Attribute::Trait::IsLocal')) {

                $xml_extra .= sprintf '<%s>%s</%s>',$attribute->name,$attribute_value,$attribute->name;

        }

…

if ($call->can('_location')){

      my $location = $call->_location();

      $xml .= $xml_extra;

…

and all my test are passing both for S3Contgrol and S3.

Time to move on again.