atoomic

I wanted to share with you a new experimental project which could allow you to mock file tests also known as -X.

All started while we initiate a larger discussion about testing, and decided to give a size to our unit tests. Using sizes like the ones used for t-shirt: Small, Medium, Large...

The definition for each size might differ and evolve.
But at one point we were considering that a small test should not have any interactions with the file system....

So how could you test such a function, without any interactions of the filesystem...

sub mycode {

   return unless -e "/somewhere";

   return 1 if -d _;

   return 2 if -f _;

   return 3;

}

Several ideas came in mind, but all of them involved refactoring the code differently so you can mock stuff while testing....

But isn't the concept of designing your code for testing a bad habit? All the most if it requires adding tons of boilerplate and artificial helpers around your code... making it larger and more complex to maintain. Should not the tests adjust and provide framework helping for testing.

Unfortunately, you cannot natively mock calls as '-e', '-f', '-d', ....
But the good news is that each of these checks corresponds to a specific Perl OP.

Which mean that using XS code, you can replace the OP for any of these FileChecks with one XS function. And Overload::FileCheck tries to provide you a way to setup only a Pure Perl hook for any of them.

overload::substr from Paul Evans shows how it is possible to mock one Perl OP.

Inspired by overload::substr concept, I came up after some discussions between Tony Cook, and Todd Rinaldo... with a prototype allowing to mock any FT OPs as defined in pp_sys.c with a Pure Perl function.

The module Overload::FileCheck (already available on CPAN) is very experimental and mainly exist as a proof of concept.

Several limitations/caveats exist when using it... I do not think this module should be used outside of testing purpose. (some might argue it should not be used at all, but in this case, consider it is as educational to learn how one can mock a Perl OP)

Overload::FileCheck

You can mock, one or more checks (at import time, or run time) using Overload::FileCheck like this:

use Overload::FileCheck '-e' => \&my_dash_e, -f => sub { 1 }, ':check';
 
# example of your own callback function to mock -e
# when returning
#  0: the test is false
#  1: the test is true
# -1: you want to use the answer from Perl itself :-)
 
sub dash_e {
    my ($file_or_handle) = @_;
 
    # return true on -e for this specific file
    return CHECK_IS_TRUE if $file_or_handle eq '/this/file/is/not/there/but/act/like/if/it/was';
 
    # claim that /tmp is not available even if it exists
    return CHECK_IS_FALSE if $file_or_handle eq '/tmp';
 
    # delegate the answer to the Perl CORE -e OP
    #   as we do not want to control these files
    return FALLBACK_TO_REAL_OP;
}

use Overload::FileCheck q{:all};
 
my @exist     = qw{cherry banana apple};
my @not_there = qw{not-there missing-file};
 
mock_all_file_checks( \&my_custom_check );
 
sub my_custom_check {
    my ( $check, $f ) = @_;
 
    if ( $check eq 'e' || $check eq 'f' ) {
        return CHECK_IS_TRUE  if grep { $_ eq $f } @exist;
        return CHECK_IS_FALSE if grep { $_ eq $f } @not_there;
    }
 
    return CHECK_IS_FALSE if $check eq 'd' && grep { $_ eq $f } @exist;
 
    # fallback to the original Perl OP
    return FALLBACK_TO_REAL_OP;
}

Mocking all file checks is nice and could work with some easy cases but in truth most of these file checks would need a full stat (or lstat) call to be able to answer to the question correctly. If possible, the Perl FileCheck OPs worked on a cached stat struct (PL_statcache and other special global variables like: PL_laststatval, PL_laststype and PL_statname).

The next idea was then to only mock 'stat' and 'stat' and let the magic happen to overload/define all the other file checks.

This is what you can do using mock_from_stat

# setup at import time

use Overload::FileCheck '-from-stat' => \&mock_stat_from_sys, qw{:check :stat};

 

# or set it later at run time

# mock_all_from_stat( \&my_stat );

 

sub mock_stat_from_sys {

 

    my ( $stat_or_lstat, $f ) = @_;

    # $f can be a file or a file handle

 

    # $stat_or_lstat would be set to 'stat' or 'lstat' depending

    #   if it's a 'stat' or 'lstat' call

 

    if ( defined $f && $f eq 'mocked.file' ) {    # "<<$f is mocked>>"

        return [                                  # return a fake stat output (regular file)

            64769,      69887159,   33188, 1, 0, 0, 0, 13,

            1539928982, 1539716940, 1539716940,

            4096,       8

        ];

 

        # you can also use some helpers to return a fake stat entry

        return stat_as_file() if $f eq 'something';

        return stat_as_directory( uid => 0, gid => 'root' ) if $f eq 'fake.dir';

        return stat_as_file( perms => 0755 ) if $f eq 'touch.file.0755';

 

        return [] if $f eq 'missing';                                # if the file is missing


        return FALLBACK_TO_REAL_OP;

    }

 

    # let Perl answer the stat question for you

    return FALLBACK_TO_REAL_OP;

}

# ...

 

# later in your code

if ( -e 'mocked.file' && -f _ && !-d _ ) {

    print "# This file looks real...\n";

}

 

Such a mocking comes with several limitations:

• keep in mind this is design for Unit Test purpose
  
• right now mocked OPs cannot make the difference between an empty string and undef as answer
  
• Code loaded/interpreted before mocking a file check, would not take benefit of Overload::FileCheck.
  
• -B and -T are using heuristics... not possible to guess from a fake stat entry... alternatively you should provide a custom mock functions for them
  
• ...
  

One cool thing you can do with this module is trace all file check calls from a codebase.
By providing a hook function that would always fallback to the regular Perl OP (using FALLBACK_TO_REAL_OP), and either print to a file or to any other output (stdout or stderr, ... )

use Carp;
use Overload::FileCheck q{:all};
 
mock_all_file_checks( \&my_custom_check );
# can also consider something like
# mock_all_from_stat( \&my_custom_check );
 
sub my_custom_check {
    my ( $check, $f ) = @_;
 
    local $Carp::CarpLevel = 2;    # do not display Overload::FileCheck stack
    printf( "# %-10s called from %s", "-$check '$f'", Carp::longmess() );
 
    # always fallback to the original Perl OP
    return FALLBACK_TO_REAL_OP;
}
 

This module is the first step of something bigger that Todd Rinaldo is currently working on providing even more flexibility when mocking files in a more common way via Test::MockFile

Let me know if you think Overload::FileCheck is valuable, or if you plan using it for testing purposes or building other tools on top of it, like Todd is doing with Test::MockFile.

thanks for reading
nicolas