byterock

Well the saw-buck edition of Paws proved to be a real tough one a least for me. I my last post I was just thinking of fixing all the S3 calls that do not have a test I was just going to start with all the delete calls but I decided to rater take a functional approach and tackle the calls in that order so I started out with the 'Analytics' calls

• DeleteBucketAnalyticsConfiguration
• GetBucketAnalyticsConfiguration
• ListBucketAnalyticsConfigurations
• PutBucketAnalyticsConfiguration

The first one I was going to play with was the 'PutBucketAnalyticsConfiguration'. This is where I ran into a little bit of a sticky wicket.

I noticed that I could easily write up a 10_resposne test but no where did I see in the test case that would test how the data is being called?? There are a few S3 call tests in 05_service_calls.t which look like this

my $s3 = $aws->service('S3');

$request = $s3->ListObjects(

  Bucket => 'test_bucket',

  Prefix => 'A/Prefix',

  MaxKeys => 5

);

 

$test_params = {

  prefix => 'A/Prefix',

 'max-keys' => 5,

};

like($request->uri, qr/list-type=2/, 'Found list-type=2 in the URI');

request_has_params($test_params, $request);

Which is ok for the very simple ones but my call is much more complex as the calls can be a huge XML like this;

<AnalyticsConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
   <Id>string</Id>
   <Filter>
      <And>
         <Prefix>string</Prefix>
         <Tag>
            <Key>string</Key>
            <Value>string</Value>
         </Tag>
         ...
      </And>
      <Prefix>string</Prefix>
      <Tag>
         <Key>string</Key>
         <Value>string</Value>
      </Tag>
   </Filter>
   <StorageClassAnalysis>
      <DataExport>
         <Destination>
            <S3BucketDestination>
               <Bucket>string</Bucket>
               <BucketAccountId>string</BucketAccountId>
               <Format>string</Format>
               <Prefix>string</Prefix>
            </S3BucketDestination>
         </Destination>
         <OutputSchemaVersion>string</OutputSchemaVersion>
      </DataExport>
   </StorageClassAnalysis>
</AnalyticsConfiguration>

plus two params on the URI?

It would be nice to know I am sending the correct XML and my params are correct? I guess I could discombobulate the 10_respose.t test and make one to test the call pattern as well?? Well a little out of scope for this post. I might come back to it though.

I did not have any analytic properties on my AWS S3 buckets so I figured I might as well add one with a simple script and at lease see what the response is.

The script was simple enough;

my $Output = $s3->PutBucketAnalyticsConfiguration
(
  AnalyticsConfiguration => {
    Id                   => 'test1333',
    'StorageClassAnalysis' =>  {
                              'DataExport' => {
                                 'Destination' =>  {
                                   'S3BucketDestination' =>  {
                                        'Prefix' => 'Test',
                                        'Bucket' => 'arn:aws:s3:::cargotel-logs',
                                         'Format' => 'CSV'
                                        },
                                    },
                                    'OutputSchemaVersion' => 'V_1'
                                 },
                                 },
                     },
  Bucket        => 'dev.cargotel.paws',
  Id            => 'test1333',
);

but, here comes the very frustrating part. The above script was just giving me a 400 response

<Error>
  <Code>MalformedXML</Code>
  <Message>The XML you provided was
 not well-formed or did not validate against
 our published schema
  </Message>
</Error>'

I am not going to put down a play by play of the two or three hundred test runs with modification I tried as I do not think anyone would read it and I will not server any useful purpose, I will just cut to the chase;

After a very very long time debugging I finally stumbled upon it. A very exact reading of the API

I found this;

PUT /?analytics&Id=Id HTTP/1.1
Host: Bucket.s3.amazonaws.com

I checked what I was getting out of my request object;

'uri' => '/dev.cargotel.paws?id=test1333',
'url' => 'https://s3.amazonaws.com/dev.cargotel.paws?id=test1333'

and that should look like this;

'uri' => '/?analytics&Id=test1333',
'url' => 'https://s3.amazonaws.com/dev.cargotel.paws?analytics&Id=test1333'

So something was amiss. I spent another while trying to figure out what was going but I eventully tried this change to boto

       "name":"PutBucketAnalyticsConfiguration",
       "http":{
         "method":"PUT",
-        "requestUri":"/{Bucket}?analytics"
+        "requestUri":"/{Bucket}?analytics&id={Id}"
       },
       "input":{"shape":"PutBucketAnalyticsConfigurationRequest"},
       "documentation":"<p>Sets an analytics configuration for the bucket (specified by the analytics configuration ID).</p>"
@@ -2332,8 +2332,8 @@
         "Id":{
           "shape":"AnalyticsId",
           "documentation":"<p>The ID that identifies the analytics configuration.</p>",
-          "location":"querystring",
-          "locationName":"id"
+          "location":"uri",
+          "locationName":"Id"
         }

What the above does is create this URI

'uri' => '/?analytics&Id=test1333',

but I was still getting a '400' error darn.

Then I remember a similar error I once have well back in the last century when XML was all the rage.

The 'did not validate against our published schema' was telling me that my XML was being validated before the server processed it and looking this up, in a very dusty JAVA SOAP book I had handy, it seems this form of validation is a two way street.

That means I had to provide the 'server validator' the schema I want it to validate against. I then had a quick look at the S3 API doc and there it is in plain view;

<?xml version="1.0" encoding="UTF-8"?>
<AnalyticsConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">

I was just sending

<?xml version="1.0" encoding="UTF-8"?>
<AnalyticsConfiguration >

I need to add the XML schema!!!

Looking in the PAWS code I did not find any reference to add in the XMLNS but I did check boto and the code is there for that;

        "AnalyticsConfiguration":{
          "shape":"AnalyticsConfiguration",
          "documentation":"<p>The configuration and any analyses for the analytics filter.</p>",
          "locationName":"AnalyticsConfiguration",
          "xmlNamespace":{"uri":"http://s3.amazonaws.com/doc/2006-03-01/"}
        }

Now how to get that into my Classes?

Oh well that is another post.