WTF: $Test::Builder::Level is invalid
Someone started using Test::Pretty and now I can't run tests. I get:
GraphViz2::Marpa: Start ... # $Test::Builder::Level is invalid. Testing library you are using is broken. : 2 at /home/ron/perl5/perlbrew/perls/perl-5.18.2/lib/site_perl/5.18.2/Test/Version.pm line 134.
Googling didn't help.
I've run cpanm on a range of modules:
o Test::Pretty
o Test::More
o Test::Version
o Test::Exception
o TryCatch
Any ideas?
I try to write all code in Perl, but find I end up writing in bash, CSS, HTML, JS, and SQL, and doing database design, just to get anything done...
You already tried hard to locate or isolate the problem?
What exactly are you doing?
Test::Pretty also exists as App::Prove::Plugin, thus can come via prove (.proverc, ENV, -Pretty ...).
Not sure if that applies, but at least it looks suspiciously.
Just saw [2] and [1] (which has to do with $Test::Builder::Level).
[2] says: "CPAN test modules might — in their own tests — be looking for specific output and start breaking. Such tests are inherently fragile, and should be changed to use a TAP parser instead of parsing literal text, but in the meantime, if your favorite test class fails its tests, this Test::More change might be the reason."
[1] https://github.com/Test-More/test-more/pull/396/files
[2] http://www.dagolden.com/index.php/2374/the-next-testmore-might-break-fragile-test-modules/
It's Test::Pretty with Test::Version in combination which is broken.
This is reproducable.
Who is the someone using Test::Pretty? Ron, how are you starting your tests?
$many x $thanx for the replies.
For each of my 8 modules which use Marpa::R2, I do:
local $CWD = $dir_name;
($stdout, $stderr) = capture{system 'prove', 't'};
Then I parse the output with:
if ($line =~ /^Files=.+Tests=(\d+),/)
Looking for a line like:
Files=3, Tests=88, 2 wallclock secs ( 0.04 usr 0.02 sys + 2.15 cusr 0.04 csys = 2.25 CPU)
And no, this is not the problem :-).
The first module, Graph::Easy::Marpa, works. It does not use Test::Version. All its tests use Test::More.
The second module, GraphVaz2::Marpa, fails. It uses Test::Version. All its tests use Test::More, except of course t/version.t, which uses Test::More and Test::Version.
Googling suggested Test::Pretty was involved. As I say, I don't use it (knowingly).
Time passes....
I've eliminated t/version.t and Test::Version from the 2 (out of the 8) modules which used it.
I'm re-running the bulk test code now.
Waiting, waiting.....
Success!
Marpa::R2 V 2.085008
Counts: Tests: 14. Modules: 8. Passes: 8. Fails: 0
Duration: 1 minute and 37 seconds
(The note 'Tests: 14' completely mis-represents the total # of tests run. That's the number of t/*.t files all up. There are actually hundreds of tests).
Helmut Wollmersdorfer: Congratulations. I'm deeply grateful for your advice.
Please take the rest of the day off - and tell your boss I said it was OK!
For each of my 8 modules which use Marpa::R2, I do:
local $CWD = $dir_name;
($stdout, $stderr) = capture{system 'prove', 't'};
Then I parse the output with:
if ($line =~ /^Files=.+Tests=(\d+),/)
Looking for a line like:
Files=3, Tests=88, 2 wallclock secs ( 0.04 usr 0.02 sys + 2.15 cusr 0.04 csys = 2.25 CPU)
Please for the love of all kittens, use Test::Harness to run your tests and
then ask it for the result, rather than manually parsing the text output.