Bird's eye view of unit testing

Key activities in software development include building, debugging, and testing. Testing is the process of verifying that a software fulfills it specified requirements. It is also a measure of software quality. Testing helps reduce the cost of software development and ownership.

The software requirements specification of a software must set benchmarks, list features, and these features should serve as a basis for testing the software. Ask yourself whether the software fulfills each and every requirement as it is described. When writing the requirements and specifications, it is important to ensure that the wording is clear and unambiguous, each requirement is testable, and most importantly the requirements must fulfill the needs of the customer.

There are many different forms of testing but the three important ones which would be discussed here are unit testing, integration testing, and regression testing. The purpose is only to provide a comparative understanding of the three.

Unit Testing
Unit testing involves isolating smallest testable pieces of code and then testing them for correctness. It only verifies that the small piece of code is doing what it is supposed to do.

Integration Testing
Integration testing is the logical next step after unit testing. It tests whether two or more units do what they are supposed to do when used together. For example, we have verified that the wheel is working properly with unit testing and we have verified that the steering wheel is working correctly. Now we need to verify whether wheel responds correctly to the changes made to the steering wheel.

Regression Testing
Regression tests should be run whenever a software's implementation is modified. It is a series of tests designed to see whether something has broken as a result of the change in implementation. Unit tests greatly simplify regression test since they could be re-run at any time.

Debugging with inserted code

Suppose you are debugging and you wish to verify whether a variable in the following code is empty and an array before using it:

$var2 = array();
// ... some code here followed by
$var2[] = 'adam';

Several possible problems could arise for between two the two lines of code by the code which would be in between. We know that on the last line, we expect $var2 to be an empty array. One way to verify would be to insert the following code which would only complain when $var2 is not what we expected it to be.

$var2 = array();
// ... some code    
// testing whether variable is empty and whether it is still an array
if (!is_array($var2)) { print 'error: $var2 is not an array'; }
if (!empty($var2)) { print 'error: $var2 is not empty'; }
$var2[] = 'adam';

A more elegant way to do the same would be to write a function for this as follows:

$var2 = array();
// ... some code    
// it is a good idea to test whether is empty and whether it is still an array   
assertTrue(is_array($var2), '$var2 is not an array');
assertTrue(empty($var2), '$var2 is not empty');
$var2[] = 'adam';

function assertTrue($condition, $msg) {
    if (!$condition) {
        print "Assertion failed: {$msg}";
    }
}

Once again, the system complains only when something goes wrong.

Naturally, the next thing which would cross your mind is that there should be a library or framework for this. In fact, there are many frameworks but the two most commonly used ones are PHPUnit and SimpleTest.

SimpleTest

SimpleTest is a PHP unit testing and web testing framework. The complete API documentation is available at http://simpletest.org/api/. If you are using Eclipse, a SimpleTest plugin is also provided on this site. SimpleTest has been integrated into Elgg 1.7 and onwards, Moodle, and Drupal.

Installing SimpleTest
Simply download the latest stable release. Unzip it into your working directory
$ tar xzvf simpletest_1.0.1.tar.gz

SimpleTest Example

require_once('simpletest/autorun.php');

class MyExample extends UnitTestCase {   
    function myex() {
        $var2 = array();        
        $this->assertTrue(is_array($var2));    
    }
}

output

OK?
Test cases run: 0/1, Passes: 0, Failures: 0, Exceptions: 0

This example also involves inserting testing code into the original code. In reality, unit tests are written in separate files and these files are not ported into production environments.

Unit Testing in Elgg

Do NOT do this on production environments!!

Elgg 1.7 and above includes PHP SimpleTest framework. Here you will see how to unit test in Elgg.

First of all, log into your dev environment with admin access. Go to:
Admin > Site Administration and change "Turn off debug mode (recommended)" to "Display errors and warnings".
Then we create a new plugin. This example would have the following structure:

|- mod 
|     |- utest 
|     |     |- manifest.xml
|     |     |- start.php
|     |     |- tests 
|     |     |     |- utest.php

manifest.xml

<?xml version="1.0" encoding="UTF-8"?>
<plugin_manifest>
<field key="author" value="Nazim Rahman" />
<field key="version" value="1.0" />
<field key="description" value="unit testing example" />
<field key="website" value="" />
<field key="copyright" value="" />
<field key="licence" value="GNU Public License version 2" />
<field key="elgg_version" value="2011071212" />
</plugin_manifest>

start.php

function utest_init() {
    register_plugin_hook('unit_test', 'system', 'utest_unit_tests');
}
function utest_unit_tests($hook, $type, $value, $params) {
    global $CONFIG;
    $value[] = $CONFIG->path . 'mod/utest/tests/utest.php';
    return $value;
}
function utest_function1() {
    $var = 1;
    if ($var == 1) { return true; } else { return false; }
}
register_elgg_event_handler('init','system','utest_init');

tests/utest.php

class UtestUnitTest extends ElggCoreUnitTest {
    public function testFunction1() {
        $value = utest_function1();
        $this->assertTrue($value);
    }
}

To run this unit test, you need of course enable the plugin utest. Then go to:

Admin > System Diagnostics > Execute All

Note that start.php itself does not contain any debugging code or unit testing code. The unit testing code is in separate file inside tests directory. This directory is not pushed to production.

The beauty of this solution is that you can test all your plugins in dev with one click even years after writing them.

SimpleTest list of assert functions

assertTrue($x)
Fail if $x is false

assertFalse($x)
Fail if $x is true

assertNull($x)
Fail if $x is set

assertNotNull($x)
Fail if $x not set

assertIsA($x, $t)
Fail if $x is not the class or type $t

assertNotA($x, $t)
Fail if $x is of the class or type $t

assertEqual($x, $y)
Fail if $x == $y is false

assertNotEqual($x, $y)
Fail if $x == $y is true

assertWithinMargin($x, $y, $m)
Fail if abs($x - $y) < $m is false

assertOutsideMargin($x, $y, $m)
Fail if abs($x - $y) < $m is true

assertIdentical($x, $y)
Fail if $x == $y is false or a type mismatch

assertNotIdentical($x, $y)
Fail if $x == $y is true and types match

assertReference($x, $y)
Fail unless $x and $y are the same variable

assertClone($x, $y)
Fail unless $x and $y are identical copies

assertPattern($p, $x)
Fail unless the regex $p matches $x

assertNoPattern($p, $x)
Fail if the regex $p matches $x

expectError($x)
Swallows any upcoming matching error

assert($e)
Fail on failed expectation object $e