Getting a progress report from a running program

Imagine this scenario: you ran a program to do some (potentially) long-running task. The program has a --verbose or --debug command-line option to print progress report to stdout and/or log file. This time you forgot to issue the command-line option and 10 minutes into it you wonder how the program is doing. You could start over by terminating the program and run it again. But it would be nice if we can get the running program to report its progress.

We can of course. One of the ways is to use Unix signals. We can use either SIGUSR1 or SIGUSR2, which are reserved for application purposes. Here's a sample program:

#!/usr/bin/env perl

use 5.010;
use strict;
use warnings;

use Progress::Any;

my $progress = Progress::Any->get_indicator(task => '');
my $verbose = @ARGV && $ARGV[0] eq '--verbose';

$SIG{USR1} = sub {
    if ($progress->target) {
        say sprintf("task progress: %d/%d (%d secs remaining)",
                    $progress->pos, $progress->target, $progress->remaining);
    } else {
        say "task not running";
    }
};

my $end = int(20+100*rand());
$progress->target($end);
for (1..$end) {
    sleep 2;
    $progress->update(pos => $_);
    if ($verbose) {
        say sprintf("(verbose) task progress: %d/%d (%d secs remaining)",
                $_, $end, $progress->remaining);
    }
}
$progress->finish;
say "Finished";

If we invoke the program with --verbose, it will nicely report its progress on the stdout:

%  ./program --verbose
(verbose) task progress: 1/47 (0 secs remaining)
(verbose) task progress: 2/47 (45 secs remaining)
(verbose) task progress: 3/47 (58 secs remaining)
...

However, if we left out the --verbose option, it will not output anything until finished. However, we can send a signal to it and it will print something like:

$ kill -USR1 %1
task progress: 2/70 (125 secs remaining)
% kill -USR1 %1
task progress: 10/70 (110 secs remaining)

The good ol' dd program also employs the same trick. If you send SIGUSR1 to it, it will print I/O statistics like how much data it has copied and the transfer speed.

Aside from the above technique, the HUP signal is also traditionally used. For example, if verbosity/logging-level is controlled from a config file, the HUP signal can be used to make our program re-read the config file and re-set the level. And thus we can temporarily increase logging level without having to start over or drop connections.

3 Comments

I often use SIGQUIT for this purpose, because by default it's even bound to a key on the terminal - Ctrl-\. This means you can send it to a foreground process without having to kill or pkill from another terminal.

Steve Lembark had a really good presentation about this sort of idea. I think he gave it at a Frozen Perl, but that site has been hijacked it looks like: http://frozen-perl.org

Another nice approach is a proc-like filesystem within your software. Currently we are using named sockets in our event-driven software, so we can then use socat to get the current state.

$ socat proc/state -
task progress: 2/70 (125 secs remaining)
$ _

I would prefer a real proc-like-filesystem to interact with cat & echo, but i dont get this running ...

Leave a comment

About Steven Haryanto

user-pic A programmer (mostly Perl 5 nowadays). My CPAN ID: SHARYANTO. I'm sedusedan on perlmonks. My twitter is stevenharyanto (but I don't tweet much). Follow me on github: sharyanto.