Paws XXVI (The Big Clean)

By byterock on November 17, 2019 9:00 PM

So time to hold on a bit and go back and and see if my latest changes have broken anything or for that matter fixed something that was already broken?

So back to my tests script and and I got my first fail on

'PutBucketCors'

and surprise it was a bit of Mooso poop.


You must pass a package name and it cannot be blessed at /wwwveh/lib/x86_64-linux-thread-multi/Moose/Meta/Attribute.pm line 867
        Moose::Meta::Attribute::get_value('Moose::Meta::Class::__ANON__::SERIAL::12=HASH(0x4ad5358)', 'http://www.example.com') called at /home/scolesj/aws-sdk-perl/

So I checked the generated 'PutBucketCors,pm' file with one from a clean .41 build and there was no changes there so it must be someplace else?

Well lots of debugging later I found it here


 } elsif ($attribute->type_constraint eq 'ArrayRef[Str|Undef]') {
    my $location = $attribute->request_name;
--  $xml .= ( join '', map { sprintf '<%s%s>%s</%s>',$location,  $self->_to_xml_attributes($attribute->get_value($_)), $_, $location } @{ $attribute->get_value($value) });
–  $xml .= ( join '', map {sprintf '<%s>%s</%s>',$location,  $_, $location } @{ $attribute->get_value($value) });

I guess I was a little over zealous with my XML attribute add in. Didn't need that there as a string or an undef can never have an attribute hence the bad call to object error.

My next bug was with

PutBucketMetricsConfiguration

The forever with me 'Invalid XML' as now I am trying to post this;

<MetricsConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"> <Filter> <And> <Tag> <Tag> <Value>high</Value> <Key>priority</Key>

so that tagging problem back again;

Looking at the botocore changes I made for that one, there was the usual


{Bucket}?metrics" 
{Bucket}?metrics&id={$Id}"

change to fix the URL/URI issues. There was also this change


--"Tag":{
--"shape":"Tag",
++"Tags":{
++"shape":"TagSet",
"documentation":"<p>The tag used when evaluating a metrics filter.</p>"
},

and


"TagSet":{
"type":"list",
+flattened":true,
"member":{
"shape":"Tag",
"locationName":"Tag"

I took a chance and reversed the two tag changes and after a compile I got the same error so it must be a problem someplace else.

I had a look at the output of the auto generated code of Paws::S3::MetricsFilter; and I see I have gone back to


 has Tag => (is => 'ro', isa => 'Paws::S3::Tag');

but in 'Paws::S3::MetricsAndOperator' I see I have;


 has Tags => (is => 'ro', isa => 'ArrayRef[Paws::S3::Tag]', request_name => 'Tag', list_request_name => 'Tag' , traits => ['NameInRequest','ListNameInRequest']);

Good I can work with that I need only change the RestXmlCaller.pm a little as all I need to is drop the extra Tag tag if the two are the same


    my $location = $attribute->request_name();
    my $list_name =  $attribute->list_request_name();
    $xml .=  ( join '', map { sprintf '<%s%s>%s</%s>',$location,   $self->_to_xml_attributes($attribute->get_value($_)), $sel
f->_to_xml($_), $location } @{ $attribute->get_value($value) } );
--  $xml ="<$list_name>$xml</$list_name>";
++  $xml ="<$list_name>$xml</$list_name>"
++   if ($location ne $list_name);

and now it works

><Filter>
<And>
<Prefix>documents</Prefix>
<Tag>
<Key>priority</Key
><Value>high</Value>
</Tag>
...

Yes!! The change from PAWS XXV is paying off!!

The next one to fail was the one that started all of this

RestoreObject

With my recent changes I now getting my old friend 'Invalid XML' as I was posting this;

<S3> <Prefix>lhf/insp/attachments/assigned</Prefix> <Tagging> <> <Tag> <Value>1</Value> <Key>Restored From Glacier</Key> </Tag> </> </Tagging>

Hmm odd an empty tag. Oh well back I go into 'RestXmlCaller.pm'.

again a very simple fix;


  $xml ="<$list_name>$xml</$list_name>"
 --  if ( $location ne $list_name);
 ++  if ( $list_name and $location ne $list_name);

and this time I get this

<S3>
<Prefix>lhf/insp/attachments/assigned</Prefix>
<Tagging>
<Tag>
<Value>1</Value>
<Key>Restored From Glacier</Key>
</Tag>
</>
</Tagging>

But that is wrong as well as the doc says

<Tagging> <TagSet> <Tag>

I had a look in 'Paws::S3::Tagging' and I have


has TagSet => (is => 'ro', isa => 'ArrayRef[Paws::S3::Tag]', request_name => 'Tag', list_request_name => '', traits => ['NameInRequest','ListNameInRequest'], required => 1);

So not a change in RestXmlCaller.pm I will role that back and try to fix the template.

I fixed up the template again this time accounting for that empty case in the template with this change;


-  [%- IF (member.type == 'list' and member.member.locationName.defined) %][% traits.push('NameInRequest','ListNameInRequest') %], request_name => '[% member.member.locationName %]', list_request_name => '[% shape.members.${param_name}.locationName %]'
-  [%- ELSE %]
+  [%- IF (member.type == 'list' and member.member.locationName.defined) %][% traits.push('NameInRequest','ListNameInRequest') %], request_name => '[% member.member.locationName %]'
+      [%- IF (shape.members.${param_name}.locationName) %], list_request_name => '[% shape.members.${param_name}.locationName %]'
+         [%- ELSE %]
+            , list_request_name => '[% param_name %]'
+         [% END %]
[%- ELSE %]

and after chasing my tail for a good two hours, as I had a typo in one of my params and when that was corrected I got the expected results.

I did not find any more bugs in my test scripts so I ran the full test suite a got only one fail;

not ok 10044 - Call S3->GetBucketAcl from t/10_responses/s3-get-bucket-acl.response

I had a look in the response class file for that one the new one 'AccessControlPolicy.pm' and found this;


has Type => (is => 'ro', isa => 'Str', xml_attribute_name => 'xsi:type', traits => ['XMLAtribute'], required => 1);  has URI => (is => 'ro', isa => 'Str');

Ok I am not correctly parsing out this XML;

<Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser">

as I am getting this


  {
    'Grantee' => {
           'URI' => 'http://acs.amazonaws.com/groups/global/AllUsers',
           'xmlns:xsi' => 'http://www.w3.org/2001/XMLSchema-instance',
           'xsi:type' => 'Group'
     },
     'Permission' => 'READ'

}, when I parse the incomming XML. What I have to do is make my parsed XML look like this;


  {
   'Grantee' => {
          'URI' => 'http://acs.amazonaws.com/groups/global/AllUsers',
          'Type' => 'Group'
    },
   'Permission' => 'READ'

But I think I can set that up with this patch to RestXMLResponse.pm.


--  if ( $meta->does('ParamInStatus')){
++  if ($meta->does("XMLAtribute")){
++    $args{ $key } =  $result->{$meta->xml_attribute_name()};
++  }
      elsif (

and now I get;

ok 10044 - Call S3->GetBucketAcl from t/10_responses/s3-get-bucket-acl.response ok 10045 - Got Grants.0.Grantee.DisplayName eq CustomersName@amazon.com from result ok 10046 - Got Grants.0.Grantee.ID eq 75aa57f09aa0c8caeab4f8c24e99d10f8e7faeebf76c078efc7c6caea54ba06a from result ok 10047 - Got Grants.0.Permission eq FULL_CONTROL from result ok 10048 - Got Owner.DisplayName eq CustomersName@amazon.com from result ok 10049 - Got Owner.ID eq 75aa57f09aa0c8caeab4f8c24e99d10f8e7faeebf76c078efc7c6caea54ba06a from result

and I think that is it for S3 at least for today.