Running the tests
All quattor projects have unit tests. These are run via the mvn test
command, and run the perl
unit tests under the src/test/perl
subdirectory.
Maven runs the tests via prove
(the command line tool from TAP::Harness
), and the required
include paths are already set, except for any dependencies.
prove
only runs files with a .t
suffix; any other files are ignored. This allows you to create
any helper modules for the unit tests under the src/test/perl
directory (src/test/perl
is added to the perl @INC
via the prove
command line).
The tests run the modules that are preprocessed by mvn
in the target/lib/perl
directory.
The preprocessing does e.g. the templating of the ${author}
. It also means that any perl messages
that have a line number in them, are from the templated files (so there’s some offset to take into account).
If you want to run a single unit test, add -Dunittest=name_of_test.t
to the mvn
command line.
Container-based tests
Some repositories include a Dockerfile
to run the tests (for example configuration-modules-core
). This makes
it pretty easy to get a test environment. The following examples are using podman instead of docker on a
recent Fedora. They should be run from the root directory of the repository:
- First build the container:
podman build -t quattor_testing -f Dockerfile .
This uses the current ‘master’ of
template-library-core
so you need to rebuild the container if you need a newer version. - To run all tests using the current (changed) repository:
podman run -it --user=$(id -ur):$(id -gr) --userns=keep-id -v $PWD:/quattor_test:z localhost/quattor_testing
The
--user
and--userns
options are to make sure the test run as your user inside the container. To run specific test, you can pass theMVN_ARGS
variable:-e MVN_ARGS="-Dunittest=00-tqu.t -pl ncm-systemd"
.
Logging
If you want more logging, you can set -Dprove.args=-v
(the verbose
flag of prove
)
TODO: document the QUATTOR_TEST_ variables
Wrapper scripts
TODO: document / distribute [bash_functions][mvn_bash_functions] [mvn_bash_functions]: https://github.com/quattor/quattor.github.com/issues/146#issuecomment-144681288
mvnprove.pl
TODO: run the unit tests without maven and using -d
for logging control and support more than one selected unittest
Dependencies
The main issue with the unittests is the large number of dependencies that are required by all the various repositories.
Basic non-perl:
maven
panc
(>= 10.2
)
Optional non-perl:
rpm-build
(for building rpms)
Perl dependencies:
- number of quattor tools (
LC
,CAF
,CCM
) - perl modules required for runtime
- perl modules required for testing
The quattor test framework Test::Quattor
is controlled through maven (this is the build-profile
in the pom.xml
).
TODO: add list from build_all_repos
Note: If your development environment is managed by Quattor, you can use the Quattor list to add all required dependencies
Bootstrap yum-based system
The build_all_repos
script tries to build all rpms for most quattor repositories.
Repositories that are not build include panc
and the template repositories.
One of it’s features is that it resolves (or tries to) all dependencies (currently) using yum
.
(sudo
rights are required for yum
and repoquery
; it is NOT recommended to run this as root)
Perl dependencies that can’t be installed via yum, are installed from CPAN
(using cpanm
).
As it runs all unit tests, the result of running the script is an environment where you can start developing.
By default, build_all_repos
installs everything in a quattordev
subdirectory, and has subdirectories
repos
all the quattor Git repos, with the remoteupstream
set.- you need to add your fork as the remote
origin
.
- you need to add your fork as the remote
install
contains the quattor repositories from unpacked tarballs and any dependencies installed viaCPAN
rpms
contains all build rpms- a number of log files with e.g. the installed dependencies
Running the script takes a while to complete. Best run it in a screen
session and redirect the output to a logfile.
(Best to check the sudo
command upfront, as it can prompt for passsword, and thus block the script).
TODO: check the sudo timeout (a.k.a when do the credentials expire).
TODO: use the source $DEST/env.sh
to set initial PERL5LIB
Writing perl unittests
Test::More
The basic test module
ok($bool, "message");
tests if$bool
is true- use
ok(! $bool, "message");
to test if$bool
is false
- use
is($a, $value, "message")
tests if$a
is$value
- for comparing hash and array, use
is_deeply
to compare referencesis_deeply(\%hash, {expected => 'value'}, "message");
is_deeply(\@array, [qw(value1 value2)], "message");
- for comparing hash and array, use
isa_ok($instance, "Instance::Class::Name", "message")
tests if$instance
is an instance of classInstance::Class::Name
like("text", qr{regexp}, "message");
tests if"text"
matches the compiled regular expressionqr{regexp}
- complex texts can be tested with
Test::Quattor::RegexpTest
TODO add reference
- complex texts can be tested with
Test::MockModule
You can mock almost any module using Test::MockModule
including the code you are working on.
E.g. by defining
my $mock = Test::MockModule->new('NCM::Component::mycomponent');
$mock->mock('do_something', sub () {return [qw(1 2 3)]} );
you have mocked the do_something
method of the mycomponent
component.
If you then initialise the component, any calls to the do_something
method will
return the reference to the arrayref [1, 2, 3]
.
Test::Quattor
provides a mocked version of a number of CAF
modules and their methods,
there is typically no need to not mock these yourself.
Caveat: there are a few perl builtins that can’t be mocked, esp. tests like -f $filename
.
TODO: CAF::Check
will address all this
If you want to test and mock this sort of calls, it is best if you define a short private method in your code like
sub _test_file {
my ($self, $filename) = @_;
return -f $filename;
}
and use $self->_test_file($some_file)
in the code. This can then be mocked as e.g.
my $mock = Test::MockModule->new('NCM::Component::myothercomponent');
$mock->mock('_test_file', sub () {
my ($self,$fn) = @_;
my $ans = 0; # does not exist
if ($fn eq "/some/path") {
# more logic
$ans = 1;
}
return $ans;
}
Test::Quattor
Quattor has it’s own set of methods to help testing and mocking called Test::Quattor
.
Compiled profiles
To test using a profile, you create the object template under src/test/resources
(e.g. src/test/resources/foo.pan
).
In the test, you then prepare a compiled and ready-to-use EDG::WP4::CCM::Configuration
instance during the load of Test::Quattor
use Test::Quattor qw(foo);
and to get the configuration instance you then do
my $cfg = get_config_for_profile('foo');
e.g. to be used
use Test::Quattor qw(foo);
use NCM::Component::mycomponent;
my $cmp = NCM::Component::mycomponent->new('mycomponent');
my $cfg = get_config_for_profile('foo');
$cmp->Configure($cfg);
ok(!exists($cmp->{ERROR}), "Configure succeeds with any error logged");
All methods of Test::Quattor
are exported, so no need to use the load for that (in fact, you can’t use load for that).
Caveat: reusing the object template for different tests can lead to race conditions (and thus failures), try to use unique profiles to avoid this buggy behaviour.
CAF::Process
-
set_command_status
,set_desired_output
,set_desired_err
mock the status, stdout and stderr of futureCAF::Process
callset_desired_output("/usr/bin/command", "expected output");
-
get_command
use to test if aCAF::Process
with exact command line was called (and returns theCAF::Process
instance).ok(get_command("/usr/bin/someexecutable -l -s"), "Command was called");
or if you need to access the instance
my $procinstance = get_command("/usr/bin/someexecutable -l -s");
-
command_history_ok
,command_history_reset
test ordered execution ofCAF::Process
instancescommand_history_reset(); ok(command_history_ok(['pattern1','pattern2']), "message");
-
additional logging environment variables
QUATTOR_TEST_LOG_CMD=1
log each command that is run via CAF::ProcessQUATTOR_TEST_LOG_CMD_MISSING=1
reports that a process wanted output but non was set via theset_desired_output
CAF::FileWriter (Editor,Reader)
-
get_file
returns the instance that opened/modified a filemy $fh = get_file("/some/path");
-
set_file_contents
set the content a futureCAF::File*
instance will seeset_file_contents("/some/path", $text);
-
set_caf_file_close_diff
mock theCAF::File*->close()
return behaviour to return a true value if the content changed. By default, this is false, andclose()
will return some garbage.set_caf_file_close_diff(1);
TODO: why is this not default true?