Of course I still use Perl

I usually implement "progress bar" in a command-line application via logging. For example:

use Log::Any::App qw($log);
$log->info("Starting ...");
for my $i (0..@items-1) {
    my $item = $items[$i];
    $log->infof("(%d/%d) Processing item %s ...", $i+1, ~~@items, $item);
    do_stuff($item);
}
$log->info("Finished");

Log::Any::App already gives me an easy way to output those log lines to screen and/or file. I can turn on the log by giving VERBOSE=1 or --verbose to my program. The log lines are printed with a timestamp, so I can guess how long a run will last.

Lately, however, I've been a bit annoyed by the long output of my gitbunch script. I currently have over 330 repos in my ~/repos, and syncing those to/from laptop always produces pages of output which I need to review by scrolling up, to see whether some repos failed to be synced.

Turning off verbose output (the green lines in the screenshot above) is also bad because the script will spend much time seemingly standing still. I usually run this script before having to rush out of the office, so I need to know just how many more minutes the script will run.

So, earlier today I finally separated normal logging and progress reporting, actually something which has been on my mind for quite a while. The two actions have real differences anyway, and explicitly saying one and the other is desirable. Progress indicators usually are more transient, do not need to be saved to log file, and can also be displayed more "creatively" rather than simply dumped linearly to the screen.

Looking at the available progress bar modules on CPAN and comparing with several Ruby progress bar libraries left me unsatisfied. I want something like Log::Any for progress indicators, not something that is specifically tied to command-line or terminal. I envision progress indicators over email, SMS, IM, twitter, or whatnot. The progress outputting functionality should be split from progress updating.

The result: Progress::Any. The first draft is already on CPAN and used by gitbunch. Two output modules have been written: LogAny (for outputting progress as log lines using Log::Any) and TermProgressBar (outputting progress as bar, using Term::ProgressBar). If no output module is specified, the default is Null output.

The code now looks something like this:

use Log::Any::App qw($log);
use Progress::Any qw($progress);
$progress->set_target(target => ~~@items);
for my $i (0..@items-1) {
    my $item = $items[$i];
    $progress->update(message => "Processing $item ...");
    do_stuff($item);
}
$progress->finish; # just making sure that we end with 100%

To assign an output:

Progress::Any->set_output(output => Progress::Any::Output::TermProgressBar->new);

Because Term::ProgressBar doesn't support displaying a message inside a progress bar, the messages in update() don't get displayed. But some other output, like LogAny can and do display the messages.

The output selection can be done from other parts of your code, making your core code free from visual configuration of progress indicators. Multiple outputs is also supported. Normal logging to screen can still be done. My CLI framework Perinci::CmdLine has even been updated to not interfere the bar display when logging to screen. Sample output of gitbunch program after using Progress::Any:

Reverting to outputting progress to Log::Any, as previously, is a matter of changing a single line of code (but this is something which is not implemented in Perinci::CmdLine yet, maybe if needed later).

Progress::Any has (or will have) other features too, like multiple indicators (for different tasks) and hierarchiecal progress (subtask, after finishing, will increment their parent task's progress). The output modules will let you configure the visual aspects, like, for TermProgressBar: how wide the bar is, the characters, the time/date format, and so on. Progress::Any itself will let you configure things like levels, maximum update rate, and possibly a few others. I'm still reviewing the API. Named parameters via hash is used throughout to be flexible and extensible, especially since the API and code are still young with some features lacking. Comments welcome.

I have also been contemplating about an event+notification API. It takes input from logging, progress indicator, explicit notify calls from applications, and even emails (like cron reports) and other sources (e.g. service up/down events, events from OS, external data from website, whatever), filters/summarizes/digests/remixes them as well as stores them in a central database, then notifies users. But that is really a subject of another blog post...