Paws XXV (A break though)

By byterock on November 15, 2019 9:00 PM

Well today’s post marks a tuning point in Paws me thinks, at least in the S3 name-space. I have been spending quite allot of time trying to get this action to work;

PutBucketAcl

It is the most nasty bit of AWS code I have come across so far. First one needs to send this XML up to AWS;

<AccessControlPolicy xmlns="http://s3.amazonaws.com/doc/2006-03-01/"> <Owner> <ID>852b113e7a2f25102679df27bb0ae12b3f85be6BucketOwnerCanonicalUserID</ID> <DisplayName>OwnerDisplayName</DisplayName> </Owner> <AccessControlList> <Grant> <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser"> <ID>852b113e7a2f25102679df27bb0ae12b3f85be6BucketOwnerCanonicalUserID</ID> <DisplayName>OwnerDisplayName</DisplayName> </Grantee> <Permission>FULL_CONTROL</Permission> </Grant>

You will notice that in the 'Grantee' tag there are some XML attributes!!. Sure enough it is in boto JSON


 "Type":{
          "shape":"Type",
          "documentation":"<p>Type of grantee</p>",
          "locationName":"xsi:type",
          "xmlAttribute":true
        },

I did a quick change in all of boto and the above is the only space you will find it.

Really??

In the hundreds of XML requests across all services they had to leave just one and only one XML attribute in there.!!

Thanks again AWS!!

That is really just a side story which I will get back to later in in this post. What I was the real breakthrough fix I found was figure out why the addition to boto;


"Grants":{
       "type":"list",
+         "flattened":true,
       "member":{
         "shape":"Grant",
         "locationName":"Grant"

did not fix this problem I was having.

Whithout this change I wold get the following

<AccessControlList> <Grants>

which gave me my very familiar Invalid XML error.

After I added in the 'flattened' I was getting this;

<AccessControlPolicy xmlns="http://s3.amazonaws.com/doc/2006-03-01/"> <Grant>

which was not correct either.

I can tell you I spent a good deal of time trying to figure this one out with god know how many iterations and incantations I tried of both boto and RestXmlCaller.pm.

Until I noticed this;


has Grants => (is => 'ro', isa => 'ArrayRef[Paws::S3::Grant]', request_name => 'Grant', request_name => 'AccessControlList', traits => ['NameInRequest','NameInRequest']);

'NameInRequest' is in there twice??

Could this be a bug I introduced with all my changes??

Well I had a look in a clean version of Paws 41 and found the above bug at least nine times in eight separate files so it is not my bug;

Phew!!

Well I went back to boto and had a look at the JSON but I did not really find anything obvious all I knew I was getting two request_name traits.

At least with 'AccessContollList' and 'Grant' where present in the ' request_name' as that is what I needed in the XML.

All I really needed to do was figure out how to change the second trait to another trait that I can use.

Well first I added in a new trait to API.pm


+package Paws::API::Attribute::Trait::ListNameInRequest;
+  use Moose::Role;
+  use Moose::Util;
+  Moose::Util::meta_attribute_alias('ListNameInRequest');
has list_request_name => (is => 'ro', isa => 'Str');

and next a change to my default/object.tt


+ [%- IF (member.type == 'list' and member.member.locationName.defined) %][% traits.push('NameInRequest','ListNameInRequest') %], request_name => '[% member.member.locationName %]', list_request_name => '[% shape.members.${param_name}.locationName %]'
[%- ELSE %]
…
++ [END]

which will produce this in my generated pm file.


package Paws::S3::AccessControlPolicy;
  use Moose;
  has Grants => (is => 'ro', isa => 'ArrayRef[Paws::S3::Grant]', request_name => 'Grant', list_request_name => 'AccessControlList'

and that produces the correct XML;

<AccessControlList> <Grant> <Grantee

Hooray!!

For the XML attribute I am not going to bother to put the patches here as it really is just a one off. All I did was create another trait so I can check to see if the incoming attribute is an "xmlAttribute" and I then use that in the template to map out a new moose attribute so in the end my 'Grantee' pm looks like this;


package Paws::S3::Grantee;
  use Moose;
  has DisplayName => (is => 'ro', isa => 'Str');  has EmailAddress => (is => 'ro', isa => 'Str');  
  has ID => (is => 'ro', isa => 'Str');  
  has Type => (is => 'ro', isa => 'Str', xml_attribute_name => 'xsi:type'
, traits => ['XMLAtribute'], required => 1);  
  has URI => (is => 'ro', isa => 'Str' );

sub _xml_attributes {

            return ['Type'];

}

it was a simple matter of just coding for the above in RestXmlCaller.pm.

Though is did take me quite a while to figure the finer parts of the patch. The patch will work for any point where AWS decides to add in an XML attribute.

I is a little overkill but better than the kludge I found in the JAVA code I disassembled from a JAR file, but that would be telling.

The break though part is that 'ListNameInRequest' as I think that will solve a number of other bugs and perhaps I will not have to change boto after all.

Now back to some coding!