Mojolicious Archives

Non-blocking Mojolicious apps are even easier now!

Hopefully by now you have seen that Mojolicious is a great way to build your non-blocking web (or even non-web) application. Mojolicious come with all kinds of non-blocking functionality out of the box. For more about that see my blog series on the topic. This post is an aside to show you the cool things happening in Mojolicious lately designed to make writing non-blocking apps easier.

Mojolicious is known for fast development and clean APIs. Mojo was that child with lots of excitement and energy, doing new and cool things, providing new and cool functionality, and yes, changing its mind on occasion. But Mojo is growing up and settling down a little bit. It recently went to its first conference and professional training. And it’s starting a family too!

Mojo is starting to feel more grown up, and grown-ups have responsibilities. To borrow one of Perl’s catch phrases, this more mature Mojo knows that it is not good enough anymore to just make things possible, it’s time to make them easy.

MojoConf 2014 Recap

Last week I had the joy to attend the first (and certainly not the last) MojoConf in Oslo Norway. It was an incredible experience! First and foremost I want to thank the Oslo Perl Mongers, the organisation and execution of the conference was first rate! I also want to thank Jan Henning Thorsen (batman), who graciously offered to host me. We had such productive conversations over evening congacs, both Perl and otherwise.

I will admit now, that had wondered if the community was large enough to support an international conference. I am quite happy to say that my fears were unfounded. We had attendees from all over the world, including the USA, France, Greece, Israel, the UK, Germany and others I’m forgetting I’m sure.

Glen Hinkle (tempire) gave a professional training, which was sold-out! When companies (and even a few individuals) are willing to pay real money for training, it goes a long way to prove that Mojolicious is the world-class framework that we know it is.

I was also impressed (as Glen was) at the talk given by Richard Elberger (riche) in which he touts his company and the success it is having replacing an enterprise Java application with a Mojolicious one. He shared the success of the transition, detailing how everyone from the developers to the development managers to the stakeholders have been impressed at the ease and power of Mojolicious and Perl! That is so encouraging to hear! He even had all those enterprisey words that I try my best to ignore at my $dayjob, but what a feeling to hear them spoken about Mojolicious!

From a personal standpoint, it was so much fun to meet all these wonderful people and programmers who I have interacted with online for so long. I love that I can hear Sebastian’s laugh when he writes “haha” on IRC now! I can see him and Marcus and Salve and Vincent and Nils and Jan Henning and Nicolas and Glen and Alexander and Dotan and Richard and all the others, as they were, sitting around their laptops at the hackathon, rather than just names scrolling along a chat window.

So now the conference is over, but my excitement for Mojolicious is still peaking. I got so many ideas and so much motivation last week that I’m almost not sure where to put my attention first. I can’t wait for the next MojoConf, it can’t come soon enough!

See the presentation videos on the MojoConf YouTube channel!

Writing Non-Blocking Applications with Mojolicious: Part 3

This the the third part in an on-going series about using Mojolicious to write non-blockging applications (with an eye towards the web, obviously). In part 1 I demonstrated the how it can improve the number of requests/clients served when the application uses high-latency backends (in that case a database). In part 2 I showed how each request can be sped up when that request needs multiple resources from a high-latency service (e.g. external web services).

In each, I showed a blocking example, then a non-blocking example. I then gave the usual warning that you had to use a Mojolicious server for the nonblocking version. While its true that you need a Mojolicious server to get the benefits of the nonblocking architecture, in this post I will show how with a little care in construction, you can build your application so that it will run correctly on any supported server and the nonblocking benefits will be evident where possible.

A lighter nopaste, using Mojolicious

This post announces the release of Mojolicious::Command::nopaste, a clone of App::Nopaste using the Mojolicious toolkit. I wrote it mostly as an example of using Mojolicious and its command system, but it has the side benefit of having a much lighter dependency chain than the venerable original (I already have the only required dependency on every box I use).

It also has a few bugfixes that the nopaste author either hasn’t tripped on, or had the time to fix (mostly in the Clipboard interaction on Linux) (sartak if you want to ping me I’d be happy to work with you on it). It has a slightly different list of services, including the very fun MathB.in but lacking Gist for now since the OAuth is something I don’t want to deal with yet. Any other services that people are interested in may be contributed via a PR or even a decent api spec.

Enjoy!

Writing Non-Blocking Applications with Mojolicious: Part 2

Last time, I showed you how to write non-blocking (web) applications using Mojolicious. In those examples, each action only had to perform one non-blocking action. In this article I’m going to take things a little further, introducing you to Mojo::IOLoop::Delay. I will use a delay object to perform multiple simultaneous non-blocking actions and wait until they complete before moving on, all without blocking the server for other requests.

Writing Non-Blocking Applications with Mojolicious: Part 1

One question I often hear is “Why should I chose Mojolicious versus one of the other major Perl web frameworks?” While I have many answers to that question, I personally believe the most important difference is that Mojolicious is designed to be non-blocking. Many of you will have heard of Node.js. The reason that Node is popular is that it is designed to be non-blocking. By writing your webapp in a non-blocking style using a non-blocking framework, you can often build a faster and smarter application, requiring fewer servers to handle the same amount of traffic. Although Perl has several web frameworks, only one was written with non-blocking design in mind: Mojolicious.

To demonstrate a non-blocking application, I am going to write a simple pastebin using Mojolcious and Mango, a non-blocking MongoDB library (from the same developers as Mojo).

The Templates

Before I dive into the server-side code, let’s look at the templates which will form the view of the application. Mojolicious has its own templating engine which is a thin layer over normal Perl syntax.

@@ layouts/basic.html.ep

<!DOCTYPE html>
<html>
  <head>
    <title><%= title =%></title>
    %= stylesheet '//netdna.bootstrapcdn.com/bootstrap/3.0.0/css/bootstrap.min.css'
  </head>
  <body>
    <div class="container">
      <%= content =%>
    </div>
  </body>
</html>

@@ show.html.ep

% title $doc->{title};
% layout 'basic';

%= stylesheet begin
pre.prettyprint {
  background-color:inherit;
  border:none;
}
% end
%= tag h1 => $doc->{title}
%= tag div => class => 'well' => begin
  %= tag pre => class => 'prettyprint' => begin
    <%= $doc->{content} =%>
  % end
% end
%= javascript 'https://google-code-prettify.googlecode.com/svn/loader/run_prettify.js'

@@ submit.html.ep

% title 'Paste your content';
% layout 'basic';

%= form_for '/' => role => form => method => POST => begin
  %= tag div => class => 'form-group' => begin
    %= tag label => for => 'title' => 'Title'
    %= text_field 'title', class => 'form-control'
  % end
  %= tag div => class => 'form-group' => begin
    %= tag label => for => 'content' => 'Paste Content'
    %= text_area  'content', class => 'form-control'
  % end
  %= submit_button 'Paste' => class => 'btn btn-primary'
% end

The first template defines a basic html layout which will be applied to any other template that requests it. The contents of the requesting template will be inserted at the content directive. The show and submit templates (both of which request the basic layout) are the views that will be seen when a user wants to display a paste or create new one. Notice that in the show template, a variable $doc is seen. This (template lexical) variable will be created by the controller using the stash command later. Any functions you see are called helpers in Mojolicious parlance and are more accurately curried controller methods. Some style is applied by using Twitter’s Bootstrap and Google’s Prettify from CDN sites.

These templates will be used in both versions of my following code; they may be inserted as-is in the __DATA__ section of the following files or else placed separately in a directory named templates.

The Blocking Form

Since most people are comfortable with writing web applications using a blocking style, I will first present my paste application in this way.

#!/usr/bin/env perl

use Mojolicious::Lite;
use Mango;
use Mango::BSON 'bson_oid';

helper mango  => sub { state $mango = Mango->new($ENV{PASTEDB}) };
helper pastes => sub { shift->mango->db->collection('pastes') };

get '/' => 'submit';

post '/' => sub {
  my $self = shift;
  my $title = $self->param('title') || 'Untitled';
  my $content = $self->param('content')
    or return $self->redirect_to('/');
  my $doc = {
    title   => $title,
    content => $content,
  };
  my $oid = $self->pastes->save($doc);
  $self->redirect_to( show => id => "$oid" ); 
};

get '/:id' => sub {
  my $self = shift;
  my $id = bson_oid $self->stash('id');
  my $doc = $self->pastes->find_one({ _id => $id })
    or return $self->redirect_to('/');
  $self->stash( doc => $doc );
} => 'show';

app->start;

After importing the necessary libraries (importing Mojo turns on strict, warnings and utf8 and all v5.10 features), I build some helpers of my own. The mango helper will connect to a Mongo instance whose uri I will specify in an environment variable (yes I could put it in a configuration file, but I had to draw the line on this example somewhere :-) ). I have a helper which will return an instance of the collection (read: table) which will store the paste information. By default Mango will generate a unique document ID, which for our purposes, we will use as our page identifier.

Once the helpers have been created, three routes are defined and connected to controller callbacks (the lite version of controller methods), whose purposes should be rather self explanatory. One quick thing I note is that since I don’t explicitly called render, the controller will automatically render the template with the same name, where the name is defined in lite syntax by a string after the controller callback. I could have as used the render method, but this is more concise.

Running the application (seen in its completeness here, don’t forget the database env var!) will start the server. This application may be run as-is under CGI, any PSGI server, or using Mojolicious’ built-in servers. The application should run as you expect, however it has a major drawback! Any time any client causes the app to make a request to the database, all clients must wait for it to respond before the server may serve the next client. Meanwhile the server is sitting idle waiting for a response. Seems inefficient doesn’t it?!

The Non-Blocking Form

I can now present a very similar application, but which has a couple tweaks to prevent database calls from blocking the application.

#!/usr/bin/env perl

use Mojolicious::Lite;
use Mango;
use Mango::BSON 'bson_oid';

helper mango  => sub { state $mango = Mango->new($ENV{PASTEDB}) };
helper pastes => sub { shift->mango->db->collection('pastes') };

get '/' => 'submit';

post '/' => sub {
  my $self = shift;
  my $title = $self->param('title') || 'Untitled';
  my $content = $self->param('content')
    or return $self->redirect_to('/');
  my $doc = {
    title   => $title,
    content => $content,
  };
  $self->render_later;
  $self->pastes->save($doc, sub {
    my ($coll, $err, $oid) = @_;
    $self->redirect_to( show => id => "$oid" ); 
  });
};

get '/:id' => sub {
  my $self = shift;
  my $id = bson_oid $self->stash('id');
  $self->render_later;
  $self->pastes->find_one({ _id => $id }, sub {
    my ($coll, $err, $doc) = @_;
    return $self->redirect_to('/') if ( $err or not $doc );
    $self->render( show => doc => $doc );
  });
} => 'show';

app->start;

This new code only differs by a few lines, but they are important ones! First, before I make a non-blocking call, I need to call render_later to prevent the automatic rendering I mentioned above; the server cannot render when it reaches the end of the action method because it doesn’t have the data yet!

    title   => $title,
    content => $content,
  };
-  my $oid = $self->pastes->save($doc);
-  $self->redirect_to( show => id => "$oid" ); 
+  $self->render_later;
+  $self->pastes->save($doc, sub {
+    my ($coll, $err, $oid) = @_;
+    $self->redirect_to( show => id => "$oid" ); 
+  });
};

In the POST handler, you can see that I call the same save method on the collection, however, I don’t just pass the document, I also pass a subroutine reference (a callback) which will get called when the database finishes inserting the data. When a client posts the new paste to the server, it will wait until the database finishes storing the document before being redirected to view the document. Unlike the preview version of the code however, while waiting to serve this client, the server is able to move on to serve other clients which are waiting for other data!

get '/:id' => sub {
  my $self = shift;
  my $id = bson_oid $self->stash('id');
-  my $doc = $self->pastes->find_one({ _id => $id })
-    or return $self->redirect_to('/');
-  $self->stash( doc => $doc );
+  $self->render_later;
+  $self->pastes->find_one({ _id => $id }, sub {
+    my ($coll, $err, $doc) = @_;
+    return $self->redirect_to('/') if ( $err or not $doc );
+    $self->render( show => doc => $doc );
+  });
} => 'show';

Similar changes are made to the show controller. Once again, I call render_later and then move my post-database call logic into a callback. This time, the server will make a request of the database to get the relevant document, but then proceeds to serve other clients until the database responds. When the response comes back, the server invokes the callback and the client sees the requested page.

One small drawback is that now you must deploy using Mojolicious’ built-in servers. No need to fear however, they are very capable.

Ok this is cool but …

Yes it seem like a little more code, but the payoff comes next. If you are running your database on the same computer as the server, you will see little or no difference in performance. However, perhaps you want to use an off-site database host like MongoHQ. Using the tool wrk you can clearly see the benefit of rewriting your blocking application:

$ PASTEDB=mongodb://demo:pass@linus.mongohq.com:10025/MangoTest hypnotoad blocking_paste.pl
$ wrk -t 10 -c 10 -d 1m http://localhost:8080/0
Running 1m test @ http://localhost:8080/0
  10 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   286.87ms  331.49ms   1.45s    90.25%
    Req/Sec     5.91      4.32    22.00     81.12%
  3745 requests in 1.00m, 2.83MB read
Requests/sec:     62.41
Transfer/sec:     48.32KB
$ PASTEDB=mongodb://demo:pass@linus.mongohq.com:10025/MangoTest hypnotoad -s blocking_paste.pl

… in a non-blocking style:

$ PASTEDB=mongodb://demo:pass@linus.mongohq.com:10025/MangoTest hypnotoad nonblocking_paste.pl
$ wrk -t 10 -c 10 -d 1m http://localhost:8080/0
Running 1m test @ http://localhost:8080/0
  10 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    59.41ms   18.69ms 365.66ms   97.45%
    Req/Sec    16.73      2.19    21.00     75.56%
  10290 requests in 1.00m, 7.78MB read
Requests/sec:    171.49
Transfer/sec:    132.77KB
$ PASTEDB=mongodb://demo:pass@linus.mongohq.com:10025/MangoTest hypnotoad -s nonblocking_paste.pl

Please keep in mind, this is only a toy case on a free database and an older computer. That said, it is clear that there is a marked improvement in transfer and requests handled. Try it yourself, the code for both scripts is available here.

Conclusion

Mojolicious is a very powerful web framework which makes hard things easy. This is especially true for non-blocking code. I hope this post is the first in a series on writing non-blocking webapps (and perhaps even more general apps), using Mojolicious. Happy Perling!

P.S. see Mojo::IOLoop::Delay for even more ways that the Mojolicious tool suite (as I like to call it) makes writing non-blocking code easier.

This post is the first in an on-going series, continued in part 2

Some code ports to Mojolicious, just for fun.

Today is a relatively minor holiday in the US, but I had work off all the same. I found myself experimenting (when I probably should have been working on my YAPC::Brazil talk :-/). Thanks to today’s PerlWeekly (you are a subscriber right??), I found out about an interesting post by Johnny Moreno which creates a tiny json service backend using Perl. Of course it uses CGI.pm to do so, which made me curious what a Mojolcious port would look like.

Anyone miss Mojolicious' memorize helper?

Mojolicious’s 4.0 release came with lots of shiny features but it also came with a lot of housecleaning. One of the old things swept away was the memorize helper, which would cache a part of a template and prevent its repeated evaluation. Do you miss it, as some users undoubtedly do, or else does this helper sound useful to you? Then read on, because its back and better than ever!

My "Mojolicious Introduction" now updated for 4.0

On Feb 28, 2013 I gave a talk to Chicago.pm about Mojolicious. I called it an introduction, but I really wanted to show some of the features that sets Mojolicious apart. Because of this, the talk moves very fast. It hits routing and responses quickly, hits testing often, on all the way to well-tested non-blocking websocket examples.

I promised to get my slides up afterwards but life (i.e. my doctoral thesis) got in the way. Now with the release of Mojolicious 4.0 I thought I would take the opportunity to right a wrong and get the slides up; so here they are: http://mojolicious-introduction.herokuapp.com/!

The talk is itself a Mojolicious app, the source of which is available from on GitHub. Not only are all the code snippets shown in the talk included, not only do they all run, but they are actually what is rendered by the talk (DRY++), so what you see is what you get! Please leave any feedback and ask any questions. I may not see the responses here, so feel free to ping me elsewhere if needed.

Mojolicious 4.0 is coming soon!

As a newer member of the Mojolicious Core Development Team, I am more than usually excited for a Mojolicious release. This is because the next major release, version 4.0, is set to ship very soon! For those of you who don’t know, Mojolicious is a modern Perl web framework which is lightweight and easy to get started learning and using, while containing features that are cutting-edge. It’s asynchronous/non-blocking to the core, websockets work out of the box, comes with built-in DOM/JSON/UserAgent, etc etc.

Our fearless leader Sebastian Riedel (aka sri) will no doubt post a message with all the details when it ships. In the meantime, I want to share a little story of how community interaction, even at the StackOverflow level, can lead to innovation and enhancement of major projects like Mojolicious!

About Joel Berger

user-pic As I delve into the deeper Perl magic I like to share what I can.