Integrating tests into Kaya modules

This functionality was added to Kaya in version 0.5.2.

Kaya supports build-time testing of functions in modules using the %test compiler directive.

Simple tests

module Example;

Bool isOdd(Int i) {
    return (i%2)==1;
}

%test isOdd_true1 = isOdd(1);
%test isOdd_true2 = isOdd(-1);
%test isOdd_false1 = !isOdd(0);
%test isOdd_false2 = !isOdd(2);
%test isOdd_false3 = !isOdd(-2);

The %test directive defines a test as a name, and then an expression that returns Bool. The test passes if the expression is true, and fails if the expression is false.

Test names must be alphanumeric and begin with a letter or underscore.

To run the tests, pass the -test parameter to kayac.

$ kayac -test Example.k
Compiling module Example
Test 1 (Example::isOdd_true1) success
Test 2 (Example::isOdd_true2) success
Test 3 (Example::isOdd_false1) success
Test 4 (Example::isOdd_false2) success
Test 5 (Example::isOdd_false3) success

All tests pass

If a test fails, the output looks like this:

Compiling module Example
Test 1 (Example::isOdd_true1) success
Test 2 (Example::isOdd_true2) success
Test 3 (Example::isOdd_false1) FAILURE: Unspecified Assertion Failure
Test 4 (Example::isOdd_false2) success
Test 5 (Example::isOdd_false3) success

1 test failure

Complex tests

When using complex data types or functions, it may be necessary to construct a test larger than a single expression. The test syntax can be used to define a larger test function. Test functions have type Void, and pass if no uncaught Exceptions are thrown.

%test regex1 {
    regex = compile("a([Bb])c");
    m = match(regex,"__abc__");
    case m of {
        matches(ms,b,a) -> assert(size(ms)==1,"Unexpected number of matches");
	assert(b=="__","Unexpected string before match");
	| noMatch -> assert(false,"No match found");
    }
}

The assert function can be used to throw an Exception if a Bool expression is false. The optional second parameter allows a message to be specified, which will appear in the compiler output. Compiling the test above will give the following output:

Compiling module Example
Test 1 (Example::regex1) FAILURE: Unexpected number of matches

1 test failure

If an Exception is thrown other than assert's AssertionFailure, then the name of the Exception will be reported in the test output.

Test 6 (Example::throw1) FAILURE: Uncaught exception Array::OutOfBounds

To test expected failures of functions, the following construction can be useful:

%test test_failure {
    try {
        fn("Bad Parameter");
        assert(false,"Failed to throw ExpectedException");
    } catch(ExpectedException) {}
}

Selectively running tests

Using the -tests option to kayac allows tests to be run selectively. -tests takes either a number or a string. If a string is specified, only tests with a name containing that string will be run. For example:

$ kayac -tests true Example.k
Compiling module Example
Test 1 (Example::isOdd_true1) success
Test 2 (Example::isOdd_true2) success

All tests pass

If the option is numeric, only tests with a priority equal to or lower than that number will be run. Tests by default have a zero priority number, but a higher one may be specified by adding it between the %test directive and the test name (e.g. %test 3 test18 { ... }). This can be useful for large modules with slow test cases to just test the most important parts in routine builds, or if the test requires a particular environment (for example, a database server or user interaction).