Shlomi Fish

The Tip

perlopentut gives several options for opening a file for read and write, and I opted to use '+>' for Maniac Downloader, but as it turned out, it caused the existing file to be clobbered (= its length to be set to zero and all of its contents deleted).

After asking on Freenode's #perl channel, we reached this solution for opening a filehandle so that: 1) it will be opened for read/write. 2) it can open both an existing file or create a new file that does not exist. 3) If the file exists, its contents will be retained, and it won't be clobbered. and 4) it will overwrite existing contents, and not append everything at the end of the file.

The code is as follows:

sub _open_fh_for_read_write_without_clobbering
{
    my ($path, $url_basename) = @_;

    # open with '+>:raw' will clobber the file.
    # On the other hand, open with '+<:raw' won't create a new file if it
    # does not exist.
    # So we have to restort to this.
    #
    # For more information, see: http://perldoc.perl.org/perlopentut.html
    {
        open my $fh_temp, '+>>:raw', $path
            or die "Cannot open '$path' for temp-creation. $!";
        close($fh_temp);
    }
    open my $fh, "+<:raw", $path
        or die "${url_basename}: $!";

    return $fh;
}

( You can freely reuse this code under your choice of the MIT/X11 License or the Public Domain/CC0. )

The $url_basename is a relic of the roots of Maniac Downloader. Using that file-handle I used sysseek and syswrite, and it worked nicely.

I should note that mauke suggested using this instead:

use File::Open qw(fsysopen);
my $fh = fsysopen($path, 'rw');
binmode $fh;

More about Maniac Downloader

I investigated why Maniac Downloader misdownloaded several ".iso" files after being interrupted and resumed, and it turned out to have been caused by the fact that it clobbered the temporary buffer files, due to the reason I mentioned above. This was fixed in version 0.0.5.

Next I tried to benchmark it against aria2 and GNU wget in trying to download a file from a local mirror. Using ten connections for both aria2 and Maniac Downloader, I saw that aria2 and wget took about 30 minutes to download the file, while Maniac Downloader downloaded it in less than 13 minutes. However, at one point the ManDown downloading completely stopped (displaying 0 KB/s), and I had to Ctrl+C it and run it again. I guess now I'll need to investigate how to cancel the HTTP/HTTPS downloads and start them again if they stalled.