| Home » Tutorials » Writing test harness |
Leif Eriksen
OK, so you've managed to write your first Perl module, and you've managed to install it so you can write 'use YourModule;' in your scripts. Now you want to provide it to your friends or colleagues, but you also want to make sure its pretty well tested before that.
How can you test the functions and methods in your new module ?
One thing I always tell myself, when wondering how to do something in Perl, is 'You are not the first.'
What this means is that whatever the issue I have come across and am struggling with, someone else has almost certainly struggled, and solved, it before me. And it seems to be in the nature of Perl programmers to provide that solution to others, either through an addition to Perl itself, a CPAN module, a newsgroup posting or on a community website such as this.
And when it comes to testing, Perl has extremely strong support for testing scripts and modules, through the builtin Test::More and ExtUtils::MakeMaker facilities.
To use these facilities you write files containing test cases, in a particular way, and place those files in a particular location.
Lets say you developed a module Monger.pm, that provides functions and utilities to emulate a Perl programmer (see Perl Mongers). You have developed this module on a *nix machine, in a directory ~/workspace/www. Lets make this directory our working directory. Hence Monger.pm has the path of ~/workspace/www/Monger.pm.
Here is a sample Perl module called Monger.pm that you can use as you work
your way through this tutorial:
package Monger;
use strict;
use Carp;
our $AUTOLOAD;
sub AUTOLOAD {
my ($self, @args) = @_;
my ($accessor, $attr) = ($AUTOLOAD =~ m/^.*?::([GS]et)(.*)$/)
or croak "only [GS]et<attribute> supported :: $AUTOLOAD\n";
exists $self->{$attr}
or croak "no such attribute $attr";
unless ($accessor eq "Set") {
return $self->{$attr};
}
return ($self->{$attr} = $args[0]);
}
sub new {
my ($class, %attributes) = @_;
return bless \%attributes, $class;
}
sub program {
return;
}
sub DESTROY {
}
1;
Lets start to get our testing organised. I'll show you the conventions Perl expects in testing. By following these conventions, you get to leverage the powerful builtin support provided with Perl.
In the working directory, we make a directory called 't' i.e. ~/workspace/www/t.
You will place your test cases in a file in this t directory, called Monger.t. The prefix of the filename (the 'Monger' part) isn't important, but the path (the t/ path) and the suffix (the '.t' part) are important. By convention, Perl expects test cases to be in 'dot-T' files in a directory called 't'. e.g. ~/workspace/www/t/Monger.t
By following this convention, you get to leverage some useful automations provided by Test::More and ExtUtils::MakeMaker
Test::More is the backbone of writing test cases for your modules.
In the t/Monger.t file we will use the facilities provided by Test::More to test the functions and methods of Monger.pm.
A 'dot-T' file is effectively a normal Perl script, just the suffix is different. So t/Monger.t starts like most other Perl scripts
#!/usr/bin/perl -w use strict; use Test::More qw(no_plan); ...
Test::More normally expects to be told how many tests are in the 'dot-T' file in question. But as we are actively developing our test cases, we'll use the 'no_plan' directive let it know there is no count available.
So on to writing an actual test case.
Test::More provides a number of functions to test your modules public interface., the general form of which is method(<param1>, <param2>, "comment").
That is, they take 3 parameters, 2 compulsory and one optional. They can be loosely described as
method(<actual value>, <expected value>, [<description>]);
You put a call to one of your modules functions or methods in the <actual value> position, the value you expect that function to return in the next position, and a nice description of what the test is supposed to do in the third position if you want.
For example ...
is(factorial(0), 0, "boundary case");
The factorial function is called with the parameter 0. We expect that call to return 0. Test::More's is() compares the two values for equality, and returns true or false. False comparisons generate a noisy message.
The power of Test::More is that it keeps a record of what passed and failed, and gives this information to you in a report after running your test cases.
So lets show what a simple t/Monger.t file could look like :-
#!/usr/bin/perl -w
use strict;
use Test::More qw(no_plan);
BEGIN {
use_ok('Monger');
}
can_ok('Monger', ('new'));
my %attributes = (Height => 'average',
Swagger => 'slight',
Geekiness => 'extreme'
);
my $monger = Monger->new(%attributes);
isa_ok($monger, 'Monger');
foreach my $attribute (keys %attributes){
my $accessor = 'Get' . $attribute;
is($monger->$accessor(),
$attributes{$attribute}, "Get$attribute");
$accessor = 'Set' . $attribute;
is($monger->$accessor($attribute),
$attribute, "Set$attribute");
}
can_ok($monger, ('program'));
is($monger->program(), undef);
So lets break this down into the individual tests.
BEGIN {
use_ok('Monger');
}
This tests that we can successfully do a 'use Monger;'. It actually does do the same as a use, but just returns a simple OK/NOK status. We have to wrap it in a BEGIN {} block so that the test occurs at compile time, because this is when the 'use ...' statements in a script get executed.
can_ok('Monger', ('new'));
Here we check that the package/module Monger (or one of its parent modules, via the '@ISA ...' or 'use base ...' mechanisms) can find a method called 'new'. Maybe you inherited the method from another module, and you want to check that the inheritance is correct. Also, say one day the author of the parent module changes the method name from 'new' to 'create' - this test catches that change immediately (or rather immediately you run your test cases again). Note that methods whose implementation is provided by an AUTOLOAD() function can't be found via a can_ok..
isa_ok($monger, 'Monger');
OK, so the preceding few lines of code returned an instance of the Monger class - or did it ? This test checks the constructor works as expected, with the attributes supplied.
The foreach loop tests the 'accessor' methods of the class, calling GetHeight(), SetHeight etc, and checking the return value via the is() call. In this case, the values returned are those in the hash passed to the constructor.
can_ok($monger, ('program'));
This is very similar to the preceding 'can_ok' call, but the first parameter is an instance variable, rather than a class/package name. So the test is 'can this specific instance of Monger call a method program() ?'
is($monger->program(), undef);
The Monger::program() method is expected to return undef. Test::More knows how to compare undef's for equality.
There are many other useful test methods in Test::More, check the perldoc. Note that some methods have a negative flavour too - is/isnt, like/unlike etc.
Now because 'dot-T' files are just normal Perl scripts, you can run them as such -
perl t/Monger.t
If we actually do this, we'll get a report from Test::More like this :-
[le6303@itdevtst www]$ perl t/Monger.t
ok 1 - use Monger;
ok 2 - Monger->can('new')
ok 3 - The object isa Monger
ok 4 - GetHeight
ok 5 - SetHeight
ok 6 - GetSwagger
ok 7 - SetSwagger
ok 8 - GetGeekiness
ok 9 - SetGeekiness
ok 10 - Monger->can('program')
ok 11
1..11
Which indicates a successful test run. Let's break the code, by having the Set<attribute>() methods in Monger.pm prefix the return value with "ERR". To
do this, change line 21 in Monger.pm from:
return ($self->{$attr} = $args[0]);
to:
return "ERR" . ($self->{$attr} = $args[0]);
The resulting test output is :-
[le6303@itdevtst www]$ perl t/Monger.t
ok 1 - use Monger;
ok 2 - Monger->can('new')
ok 3 - The object isa Monger
ok 4 - GetHeight
not ok 5 - SetHeight
# Failed test (t/Monger.t at line 30)
# got: 'ERRHeight'
# expected: 'Height'
ok 6 - GetSwagger
not ok 7 - SetSwagger
# Failed test (t/Monger.t at line 30)
# got: 'ERRSwagger'
# expected: 'Swagger'
ok 8 - GetGeekiness
not ok 9 - SetGeekiness
# Failed test (t/Monger.t at line 30)
# got: 'ERRGeekiness'
# expected: 'Geekiness'
ok 10 - Monger->can('program')
ok 11
1..11
# Looks like you failed 3 tests of 11.
Looking at the line "not ok 5 - SetHeight", the "SetHeight" word comes from the comment parameter to is().
You also see the actual and expected values that disagreed in the test.
Now this is all well and good, and we can use this to test all the modules we write from now on, but lets say you wrote dozens of modules, and you also wrote dozens of 'dot-T' files to test them all. Running them all individually via 'perl t/<file>.t' can get pretty boring and labourious. Wouldn't it be good if we could come up with a way to do this automatically?
Again remembering our mantra 'You are not the first', Perl already provides just this mechanism through the facilities of the module ExtUtils::MakeMaker. This module is one of the oldest in Perl, and it shows its age in its slightly clunky interface, but for writing and running test files, very little work is required.
If we write a file called ~/workspace/www/Makefile.PL that contains just these two lines:
use ExtUtils::MakeMaker; WriteMakefile;
then automated running of test file (and LOTS more besides) is available.
What WriteMakefile does is ... write a makefile. In order to get Makefile.PL to generate this makefile, we run it like a normal Perl script, (you should be
in the www directory when you do this, and you must have a copy of Monger.pm in the same
directory for this to work):-
perl Makefile.PL
If we examine ~/workspace/www, we now see we have a new file 'Makefile', generated by ExtUtils::MakeMaker::WriteMakefile().
A makefile is a series of instructions about how to construct and run components of a software system. In Perl's case, it provides a number of 'targets' we can run. The target we are most interested in is the 'test' target. To run that,
first undo the "ERR" change we made at line 21 of Monger.pm so that once
again it looks like:
return ($self->{$attr} = $args[0]);
and then we type:
make test
In this case we get the following output :-
[le6303@itdevtst www]$ make test cp Monger.pm blib/lib/Monger.pm PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t t/Monger....ok All tests successful. Files=1, Tests=11, 0 wallclock secs ( 0.10 cusr + 0.00 csys = 0.10 CPU)
Lets break that down...
cp Monger.pm blib/lib/Monger.pm
The make file issues a command to copy our module to a 'staging area'. If you had a number of modules, they would be copied here. The modules in the staging area are the ones that get compiled, run etc.
PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
This complex line give a number of instructions to the Perl executable. Firstly it sets an environmental variable PERL_DL_NONLAZY=1, (you don't have to care what it does, so skip to the end of this sentence if you want) which forces dynamic libraries loaded by Perl to resolve their symbols now, rather than trust them to resolve at runtime. If you don't understand what that means, that's OK. Next is the path and name of the Perl executable to run. This is passed the remaining text as parameters to Perl. The first tells Perl to load the module "ExtUtils::Command::MM". The "-e" parameter tells Perl that the next parameter is a Perl script to run. The parameters after the "-e" tell Perl to run the test_harness() function (from the ExtUtils::Command::MM package), with the parameters (, 'blib/lib', 'blib/arch') (the verbose flag and the paths to the staging areas), on all the *.t files in the t directory. Simple!!
The result is that if we have lots of modules and test cases, they all get run - here's an example of an extensive test run of some other software not shown here:
[le6303@itdevtst hpa.perllibs]$ make test Skip blib/lib/HPA/LT_Utils/t/rules9.test (unchanged) Skip blib/lib/HPA/t/CSVHFile.dat (unchanged) ... PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" ./Class/t/*.t ./HPA/t/*.t ./HPA/LT_Utils/t/*.t ./Number/t/*.t ./Class/t/Phrasebook...........ok ./HPA/LT_Utils/t/Checkpoint....ok ./HPA/LT_Utils/t/Document......ok ./HPA/LT_Utils/t/File..........ok ./HPA/LT_Utils/t/FileFail......ok ./HPA/LT_Utils/t/Rules.........ok ./HPA/LT_Utils/t/Settings......ok ./HPA/t/AnyFile................ok ./HPA/t/AnyRecord..............ok ./HPA/t/csv_viewer.............ok ./HPA/t/CSVHFile...............ok ./HPA/t/Errors.................ok ./HPA/t/MultiFile..............ok ./Number/t/Spell...............ok ./Number/t/Spell_English.......ok All tests successful. Files=15, Tests=992, 9 wallclock secs ( 5.21 cusr + 0.40 csys = 5.61 CPU)
You can see that we had 'make test' pass 4 different directories for test_harness() to run against. That is controlled by options in the Makefile.PL file. Read the ExtUtils::MakeMaker perldoc to see how that is done.
ExtUtils::MakeMaker generates a makefile with a whole slew of useful targets - install, dist, clean - again see the perldoc.
So there you go, a fairly comprehensive introduction to getting a test harness up and running in Perl. Hopefully it will help you write solid, robust, reliable Perl modules.
perldoc Test::More
perldoc perlnewmod
perldoc ExtUtils::MakeMaker
Leif Eriksen