Egad not more PAWs posts :(

Well back on my PAWS run again. This one might be a rather short series as I am really just looking at one Action in the Kinesis API 'SubscribeToShard'. There is an open bug for this one up on github and one I think I can fix up fairly eaisy.

First things first, a little word on Kinesis. Well in short it touted as a very scalable real time data-stream thingy that sings dances and basically makes you line much better. Myself I do not havea use for it but it is part of the system and there is a bug so in I go.

I first had to set things up on the AWS server side with some permission etc the usualal srtuff I also had to run a number of command top build up my Kineses system to a point where I can actually use the 'SubscribeToShard'

I too by now my standard path I created a number of small scripts one each command I was using. Will I eventually had to make my way through 10 actions to get set up. Nothing lost though as for each action I played with I created a test case with ''. A good thing as no function tests exists that I can find in PAWS.

Anyway the fist thing I noticed (thanks to Jess who pointed it out) was the boto for the eventual output for this action used something I had not seen before. "eventstream":true

Well if I have learned one thing in my PAWS play- dates it is there are very few one-offs in botot. So I had a snooped about and found it in two others S3 which is mentioned in the bug but also in Pinpoint (one that I actually use).

The second thing I have learned it it is far better to work with boto thatn against it. I have seen similar things in the past so the first thing I need to do is look at the 'Paws::Kinesis::SubscribeToShardEventStream' package and see what I have.

package Paws::Kinesis::SubscribeToShardEventStream;
  use Moose;
  has InternalFailureException => (is => 'ro', isa => 'Paws::Kinesis::InternalFailureException'  );
  has KMSAccessDeniedException => (is => 'ro', isa => 'Paws::Kinesis::KMSAccessDeniedException'  );
  has KMSDisabledException => (is => 'ro', isa => 'Paws::Kinesis::KMSDisabledException'  );
  has KMSInvalidStateException => (is => 'ro', isa => 'Paws::Kinesis::KMSInvalidStateException'  );
  has KMSNotFoundException => (is => 'ro', isa => 'Paws::Kinesis::KMSNotFoundException'  );
  has KMSOptInRequired => (is => 'ro', isa => 'Paws::Kinesis::KMSOptInRequired'  );
  has KMSThrottlingException => (is => 'ro', isa => 'Paws::Kinesis::KMSThrottlingException'  );
  has ResourceInUseException => (is => 'ro', isa => 'Paws::Kinesis::ResourceInUseException'  );
  has ResourceNotFoundException => (is => 'ro', isa => 'Paws::Kinesis::ResourceNotFoundException'  );
  has SubscribeToShardEvent => (is => 'ro', isa => 'Paws::Kinesis::SubscribeToShardEvent'  , required => 1);


well nothing to tell me this is an event stream. If you have been folloing along the next thing I will have to do is play with the templates to add in the value I want. I really only need to tell that this call is an event streram and I have done that many times before with the good old Moose attribute 'class_has'

Seeing that this is an end item my best place to add in my code is in the 'default/' and this is what I did

[% END -%]
++[%- IF (shape.eventstream) %]
++  class_has _event_stream => (is => 'ro', default => 1);
++[%- END %]
[% iclass=shape; INCLUDE '' %]
The neat thing this should also work for the other two APIs but we will see later. Frist a quick recompile;

carton exec builder-bin/ --classes botocore/botocore/data/kinesis/2013-12-02/service-2.json

and now my class has this;

 has SubscribeToShardEvent => (is => 'ro', isa => 'Paws::Kinesis::SubscribeToShardEvent', required => 1);

class_has _event_stream => (is => 'ro', default => 1);

So now to use it?

Kinesis uses the 'JsonCaller' and 'JsonResponse' modules so I will be working on those ones but in this post I am really only inserted in 'Response'.

So after a quick poke about in 'lib/Paws/Net/', fortunately I am fairly familiar with this module, I added in this warning near the end of the 'response_to_object' sub;

sub response_to_object {
    return Paws::API::Response->new(_request_id => $request_id) if (not $returns);
++  warn("Does this do an evnet stream =".$ret_class->does('_event_stream'));
    my $unserialized_struct = $self->unserialize_response( $response );

Now just to run my script and lets see if I am on the right track?

Eventually, after about 5 minutes my script returned a response, this was expected though as the API doc says it keeps the Event Stream open only for that long. The response did return a '200' but what I got back was gobbledygook the even the boys over at CuriousMarc would have a hard time decrypting.

It looked soimething like this

 '_content' => 'r`▒▒

but I did see a successful line of debuggin as well

Does this do an evnet stream =1

So a few good things

  1. I am getting succes back from AWS

  2. There is some JSON in the stuff I amd getting back from AWS

  3. my little patch will let me handle the wacked out content

I did take a very close look at was returned and I noticed that the 'content-type' was something called 'application/' So It looks like I have some research to do. Really going to have to put my thinking Pom on for this one


Leave a comment

About byterock

user-pic Long time Perl guy, a few CPAN mods allot of work on DBD::Oracle and a few YAPC presentations