February 2014 Archives

Running a non-blocking command in Mojolicious

Mojolicious is an awesome framework designed from the ground up to be non-blocking. During web development it is sometimes desirable to run a command and do something with the results. However, with methods such as "system" and "open" the command will block concurrent requests.

A non-blocking process execution is possible with background processes; however, the process must be managed and if the UserAgent needs updating, then long-polling will need to be utilized.

Below is a complete Mojolicious::Lite program that does this. In words, the app generates a utility script on-the-fly that a) sleeps; b) writes to a state file; and c) sleeps at the the end. This means that our state file will be around while the program is running. This mimics several use-cases, for example creating a thumbnail.

The state file can be inspected using events provided by Mojo::IOLoop::ProcBackground. The alive event is a heartbeat for the process and the dead event happens when the process stops.


my $proc = $self->stash->{_proc} = Mojo::IOLoop::ProcBackground->new;

# Every so often we get a heartbeat from the background process
$proc->on(alive => sub {
    my ($proc) = @_;

    # ...
});

# When the process terminates, we get this event
$proc->on(dead => sub {
    my ($proc) = @_;

    # ...

    $self->finish;   # The GET request is done now
});

# Start our process
$proc->run([$^X, $script, $statefile]);
Here is the full example:
C:\>perl poll.pl
[Thu Feb 13 22:37:51 2014] [info] Listening at "http://*:5555".
Server available at http://127.0.0.1:5555.
[Thu Feb 13 22:37:55 2014] [debug] GET "/run".
[Thu Feb 13 22:37:55 2014] [debug] Routing to a callback.
[Thu Feb 13 22:37:55 2014] [debug] 200 OK (0.001246s, 802.568/s).
[Thu Feb 13 22:38:08 2014] [debug] Done: C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\done.3676
[Thu Feb 13 22:38:08 2014] [debug] Finished

...

C:\Documents and Settings\Administrator>mojo get "http://127.0.0.1:5555/run"
Starting...
Done C:\Documents and Settings\Administrator>

The code was tested in OS X and Win XP.


use Mojolicious::Lite;

use Mojo::IOLoop::ProcBackground;

use File::Temp;
use File::Spec;
use Proc::Background;

any '/run' => sub {
        my $self = shift;

        # Setup our request to take a while
        Mojo::IOLoop->stream($self->tx->connection)->timeout(30);
        $self->render_later;

        $self->on(finish => sub {
            $self->app->log->debug("Finished");
        });

        # We want the UserAgent to see something as soon as possible
        $self->res->code(200);
        $self->res->headers->content_type('text/html');
        $self->write_chunk("Starting...
\n"); # This is our utility script that will run in the background my $tmp = File::Temp->new(UNLINK => 0, SUFFIX => '.pl'); my $statefile = $self->stash->{_statefile} = File::Spec->catfile(File::Spec->tmpdir, "done"); print($tmp 'sleep(10); $f="$ARGV[0].$$"; open($fh, ">", $f); sleep(3)'); my $script = $tmp->filename; undef($tmp); # Thanks CPAN.. :) The magic happens in Proc::Background my $proc = $self->stash->{_proc} = Mojo::IOLoop::ProcBackground->new; # Every so often we get a heartbeat from the background process $proc->on(alive => sub { my ($proc) = @_; my $pid = $proc->proc->pid; my $statefile = $self->stash->{_statefile} . ".$pid"; if (-f $statefile) { $self->write_chunk("Done"); $proc->unsubscribe("alive"); } }); # When the process terminates, we get this event $proc->on(dead => sub { my ($proc) = @_; my $pid = $proc->proc->pid; my $statefile = $self->stash->{_statefile} . ".$pid"; $self->app->log->debug("Done: $statefile"); $self->finish; }); # Start our process $proc->run([$^X, $script, $statefile]); }; # Run the app push(@ARGV, 'daemon', '-l', 'http://*:5555') unless @ARGV; app->log->level("debug"); app->secrets(["I Knos# you!!"]); app->start;

It should be noted that there is more than one way to run a process. For example, Mojo::IOLoop::ReadWriteFork can be used to follow the process STDOUT/STDERR; and Mojo::IOLoop::ProcBackground simply runs a command in the background and allows for job control; and Mojo::IOLoop::ForkCall can be used to execute arbitrary perl in an async fashion (you can run a command, as well) and have programmatic control over the output.

In summary, one allows for STDOUT/STDERR events; one simply runs a command; and another allows for arbitrary perl to be utilized.

About Brian Medley

user-pic I blog about Perl.