Build Your Own Testing Framework. Part 3

Alex Fedorov Aug 14th, 2016 1:55 pm build-your-own-testing-framework, javascript, tdd, test-driven-development, testing, xunit

Welcome back to the new issue of “Build Your Own Testing Framework” series! Today we are going to unit-test runTestSuite function of our testing framework. Currently, only its happy path is implicitly tested via every test of the system. Additionally, this function’s unhappy paths are, in fact, untestable at the moment.

This article is the third one of the series “Build Your Own Testing Framework”, so make sure to stick around for next parts! All articles of these series can be found here.

Shall we get started?

TDD #6: Continuous Integration Training

Alex Fedorov Aug 10th, 2016 7:35 pm continuous-integration, kata, screencast, tdd, tdd-screencasts, test-driven-development

Test-Driven Development Screencast #6: Continuous Integration Training

(Recommended to watch full-screen and HD: 720p or more)

List of all TDD Screencasts can be found here.

Summary

Today we are going to discuss Continuous Integration, what it is and what it isn’t. We will define best Check-Out/Check-In cycle. We will explore ways to train our Continuous Integration skills. And we will do some live coding on Kata with special restriction, that allows one to train such skills. Then we will draw a bottom line.

Thanks for watching! If you liked it, please share this article on social networks and follow me on twitter: @tdd_fellow.

If you have any questions or feedback for me, don’t hesitate to reach me out on Twitter: @tdd_fellow.

Parallel Change Refactoring

Alex Fedorov Jul 31st, 2016 9:57 am clean-code, pseudo-code, refactoring, software-craftsmanship

Parallel Change is the refactoring technique, that allows implementing backward-incompatible changes to an API in a safe manner. It consists of 3 steps:

Expand - add new functionality as an extension of the interface (e.g.: add new methods), instead of editing signatures of existing interfaces
Migrate - mark (or log a warning) existing interface as deprecated and give time to the clients of the interface to migrate to the new interface. This might be as simple as changing your own code-base one client of this interface at a time in case when you do not have 3rd party clients of these interfaces
Contract - once all clients have migrated remove old interfaces

This technique is tremendously useful when you have 3rd party clients for your API (open-source library, SaaS with REST API, etc). Also, it is as useful for normal application development because it allows such breaking changes to never break your test suite. It allows deploying your code in the middle of the refactoring, in fact, every few minutes (Continuous Integration + Continuous Deployment).

This article contains code examples in a pseudo-code.

Build Your Own Testing Framework. Part 2

Alex Fedorov Jul 23rd, 2016 2:52 pm build-your-own-testing-framework, javascript, tdd, test-driven-development, testing, xunit

Today we are going to unit-test existing functionality of our own testing framework, that we have test-driven with FizzBuzzKata in the previous part.

This needs to be done, since currently only happy paths are implicitly tested via FizzBuzzKata test suite, i.e.: when all the tests pass. The following unhappy paths are not tested at the moment:

when assertTrue fails, it throws an error,
when assertEqual fails, it throws an error,
when the test fails, it renders the error and fails the whole test suite run, i.e.: non-zero exit code.

This article is the second one of the series “Build Your Own Testing Framework”, so make sure to stick around for next parts! All articles of these series can be found here: http://www.tddfellow.com/blog/categories/build-your-own-testing-framework/.

Shall we get the ball rolling?

Testing `assertTrue(condition, message)`

Let’s create a test suite for assertTrue with the first test for the case, when assertTrue succeeds:

// test/AssertTrueTest.js
var runTestSuite = require("../src/TestingFramework");

runTestSuite(function (t) {
    this.testSuccess = function () {
        t.assertTrue(true);
    };
});

If we run this test suite, it should not fail:

/usr/local/bin/node AssertTrueTest.js

Process finished with exit code 0

Let’s check if that test actually is testing anything by trying to break the implementation of assertTrue:

// src/TestingFramework.js

assertTrue: function (condition, message) {
    if (condition) {    // <- this condition was inverted
        throw new Error(...);
    }
},

And if we run it, we get the expected error:

/usr/local/bin/node AssertTrueTest.js
/path/to/project/src/TestingFramework.js:4
            throw new Error(message || "Expected to be true, but got false");
            ^

Error: Expected to be true, but got false
    .. stack trace ..

Process finished with exit code 1

OK, it fails as expected, now we should undo our breaking change in the implementation and run the test suite again and it should pass:

// src/TestingFramework.js

assertTrue: function (condition, message) {
    if (!condition) {    // <- the breaking change here have been undone
        throw new Error(...);
    }
},

/usr/local/bin/node AssertTrueTest.js

Process finished with exit code 0

Now, let’s write a new test for the case, when assertTrue fails:

// test/AssertTrueTest.js

this.testFailure = function () {
    try {
        t.assertTrue(false);
    } catch (error) {
        t.assertEqual("Expected to be true, but got false", error.message);
    }
};

And if we run the test suite, it passes. If I was confident in the previous test, that I know why it didn’t fail, here I feel a bit uncomfortable about writing these 5 lines of test code and never seeing them fail. So let’s break the code once again!

// src/TestingFramework.js

assertTrue: function (condition, message) {
    if (!condition) {
        throw new Error(message || "oops");   // <- we have changed error message here
    }
},

And if we run tests, we get an expected failure:

/usr/local/bin/node AssertTrueTest.js
/path/to/project/src/TestingFramework.js:4
            throw new Error(message || "oops");
            ^

Error: Expected to equal Expected to be true, but got false, but got: oops
    .. stack trace ..

Process finished with exit code 1

And if we undo our breaking change in the implementation the test should pass:

// src/TestingFramework.js

assertTrue: function (condition, message) {
    if (!condition) {
        throw new Error(message || "Expected to be true, but got false");
        //                         ^ we have restored correct message ^
    }
},

/usr/local/bin/node AssertTrueTest.js

Process finished with exit code 0

And the last test for assertTrue to test the custom failure message:

// test/AssertTrueTest.js

this.testCustomFailureMessage = function () {
    try {
        t.assertTrue(false, "it is not true!");
    } catch (error) {
        t.assertEqual("it is not true!", error.message);
    }
};

If we run the test suite, it passes:

/usr/local/bin/node AssertTrueTest.js

Process finished with exit code 0

Even that I am confident enough, that this try { ... } catch (..) { ... } construction does the right thing, let’s be diligent about it and break the implementation of that functionality and see this test fail:

// src/TestingFramework.js

assertTrue: function (condition, message) {
    if (!condition) {
        throw new Error("Expected to be true, but got false");
        //              ^ 'message || ' was removed here   ^
    }
},

If we run the test suite:

/usr/local/bin/node AssertTrueTest.js
/path/to/project/src/TestingFramework.js:4
            throw new Error("Expected to be true, but got false");
            ^

Error: Expected to be true, but got false
    .. stack trace ..

Process finished with exit code 1

It fails, but this change breaks assertEqual too so that we don’t see any meaningful error message now. We can figure out if that is an expected failure or not by inspecting the stack trace:

Error: Expected to be true, but got false
    at Object.assertions.assertTrue (/path/to/project/src/TestingFramework.js:4:19)
    at Object.assertions.assertEqual (/path/to/project/src/TestingFramework.js:9:14)
    at testCustomFailureMessage (/path/to/project/test/AssertTrueTest.js:20:15)
    at runTestSuite (/path/to/project/src/TestingFramework.js:21:32)
    at Object.<anonymous> (/path/to/project/test/AssertTrueTest.js:3:1)
    at Module._compile (module.js:409:26)
    at Object.Module._extensions..js (module.js:416:10)
    at Module.load (module.js:343:32)
    at Function.Module._load (module.js:300:12)
    at Function.Module.runMain (module.js:441:10)

and precisely this line is the most interesting one:

at testCustomFailureMessage (/path/to/project/test/AssertTrueTest.js:20:15)

If we open the code at this stack trace frame, we will see:

t.assertEqual("it is not true!", error.message);

Great, this is exactly what we expected. Undoing the breaking change and running the test suite again:

// src/TestingFramework.js

assertTrue: function (condition, message) {
    if (!condition) {
        throw new Error(message || "Expected to be true, but got false");
        //              ^ 'message || ' was inserted here again       ^
    }
},

/usr/local/bin/node AssertTrueTest.js

Process finished with exit code 0

Interestingly enough, I know how to break the code now and all the tests will pass:

First, let’s extract local variable in assertTrue:

// src/TestingFramework.js

assertTrue: function (condition, message) {
    var errorMessage = message || "Expected to be true, but got false";

    if (!condition) {
        throw new Error(errorMessage);
    }
},

And that still passes its tests:

/usr/local/bin/node AssertTrueTest.js

Process finished with exit code 0

Now let’s expand message || into the proper if statement:

var errorMessage = "Expected to be true, but got false";

if (message) {
    errorMessage = message;
}

And that still passes its tests:

/usr/local/bin/node AssertTrueTest.js

Process finished with exit code 0

Now, what if I were to replace errorMessage = message; with errorMessage = "it is not true!"; inside of the if statement:

if (message) {
    errorMessage = "it is not true!";
}

The test still passes!

/usr/local/bin/node AssertTrueTest.js

Process finished with exit code 0

This is rather interesting. It seems, that we will have to make sure, that message parameter actually gets passed to new Error(...). We can do that by applying the Triangulation technique focusing on message parameter:

// test/AssertTrueTest.js

this.testCustomFailureMessage_withOtherMessage = function () {
    try {
        t.assertTrue(false, "should be true");
    } catch (error) {
        t.assertEqual("should be true", error.message);
    }
};

And if we run the test suite, we get our expected failure:

/usr/local/bin/node AssertTrueTest.js
/path/to/project/src/TestingFramework.js:10
            throw new Error(errorMessage);
            ^

Error: it is not true!
    ...
    at testCustomFailureMessage_withOtherMessage (/path/to/project/test/AssertTrueTest.js:28:15)
    ...

Process finished with exit code 1

And if we look at the stack trace frame, it points us to the correct line of code:

t.assertEqual("should be true", error.message);

And we can now fix the test by fixing the implementation:

// src/TestingFramework.js

// inside of `assertTrue` function:
if (message) {
    errorMessage = message;   // <- here we actually have to use 'message' now
}

And when we run the test suite, it passes:

/usr/local/bin/node AssertTrueTest.js

Process finished with exit code 0

I think we have all required tests for assertTrue now. This is great! And have you spotted the duplication already?

Refactoring `AssertTrueTest`

The duplication that is present here is our try { ... } catch (..) { ... } construct. Let’s make it a completely duplicate code by extracting couple of local variables:

// first, let's extract the Action Under the Test variable:
var action = function () { t.assertTrue(false); };

try {
    action();
} catch (error) {
    t.assertEqual("Expected to be true, but got false", error.message);
}

// next, let's extract the Expected Error Message:
var expectedMessage = "Expected to be true, but got false";

try {
    action();
} catch (error) {
    t.assertEqual(expectedMessage, error.message);
}

If we apply the same refactoring for all tests in the AssertTrueTest suite, we will see the duplication:

try {
    action();
} catch (error) {
    t.assertEqual(expectedMessage, error.message);
}

Let’s give that operation a name and extract is as a function: assertThrow(expectedMessage, action):

function assertThrow(expectedMessage, action) {
    try {
        action();
    } catch (error) {
        t.assertEqual(expectedMessage, error.message);
    }
}

Use this function in all tests and inline all the extracted local variables:

this.testFailure = function () {
    assertThrow("Expected to be true, but got false", function () {
        t.assertTrue(false);
    });
};

this.testCustomFailureMessage = function () {
    assertThrow("it is not true!", function () {
        t.assertTrue(false, "it is not true!");
    });
};

this.testCustomFailureMessage_withOtherMessage = function () {
    assertThrow("should be true", function () {
        t.assertTrue(false, "should be true");
    });
};

Function assertThrow seems to be useful for any test, not just AssertTrueTest suite. Let’s move it to the assertions object. We will be doing that by using Parallel Change technique:

Create new functionality first;
Step by step migrate all calls to use new functionality instead of old one;
Once old functionality is not used, remove it.

The advantage of that method is that it consists of very small steps, that can be executed with confidence and each such small step never leaves the user in a red state (failing tests or compile/parsing errors).

Let’s see how this one can be applied here:

1. Create new functionality first - we can do it by copying the assertThrow function to assertions object:

// src/TestingFramework.js

var assertions = {
    // ...
    assertThrow: function (expectedMessage, action) {
        try {
            action();
        } catch (error) {
            // `t` needs to be changed to `this` here
            this.assertEqual(expectedMessage, error.message);
        }
    }
};

The tests should still pass:

/usr/local/bin/node AssertTrueTest.js

Process finished with exit code 0

2. Step by step migrate all calls to use new functionality instead of old one - we do it by calling assertThrow on t (our assertions object) in the test suite. Since we still haven’t removed the old assertThrow function, we can do that one function call at the time and the tests will always be green:

this.testFailure = function () {
    t.assertThrow("Expected to be true, but got false", function () {
// ^ 't.' added here ^
        t.assertTrue(false);
    });
};

// run tests and they still pass.

this.testCustomFailureMessage = function () {
    t.assertThrow("it is not true!", function () {
// ^ 't.' added here ^
        t.assertTrue(false, "it is not true!");
    });
};

// run tests and they still pass.

this.testCustomFailureMessage_withOtherMessage = function () {
    t.assertThrow("should be true", function () {
// ^ 't.' added here ^
        t.assertTrue(false, "should be true");
    });
};

3. Once old functionality is not used, remove it. - now we can remove our assertThrow function defined inside of the AssertTrueTest suite and run tests:

/usr/local/bin/node AssertTrueTest.js

Process finished with exit code 0

And they pass. Let’s see the complete AssertTrueTest suite again:

// test/AssertTrueTest.js
var runTestSuite = require("../src/TestingFramework");

runTestSuite(function (t) {
    this.testSuccess = function () {
        t.assertTrue(true);
    };

    this.testFailure = function () {
        t.assertThrow("Expected to be true, but got false", function () {
            t.assertTrue(false);
        });
    };

    this.testCustomFailureMessage = function () {
        t.assertThrow("it is not true!", function () {
            t.assertTrue(false, "it is not true!");
        });
    };

    this.testCustomFailureMessage_withOtherMessage = function () {
        t.assertThrow("should be true", function () {
            t.assertTrue(false, "should be true");
        });
    };
});

The only problem here is that we are relying on the untested assertThrow assertion here. Let’s unit-test it.

Testing `assertThrow(expectedMessage, action)`

Let’s create a new test suite with first test when assertThrow succeeds:

// test/AssertThrowTest.js
var runTestSuite = require("../src/TestingFramework");

runTestSuite(function (t) {
    this.testSuccess = function () {
        assertThrow("an error message", function () {
            throw new Error("an error message");
        });
    };
});

And the test should pass:

/usr/local/bin/node AssertThrowTest.js

Process finished with exit code 0

The code can be broken without test failure by not using expectedMessage parameter:

// src/TestingFramework.js

assertThrow: function (expectedMessage, action) {
    try {
        action();
    } catch (error) {
        this.assertEqual("an error message", error.message);
        // ^ here 'expectedMessage' was changed to constant ^
    }
}

Let’s triangulate the code to make sure expectedMessage is used correctly by adding a new test:

// test/AssertThrowTest.js

this.testSuccess_withDifferentExpectedMessage = function () {
    t.assertThrow("a different error message", function () {
        throw new Error("a different error message");
    });
};

And that test fails as expected:

/usr/local/bin/node AssertThrowTest.js
/path/to/project/src/TestingFramework.js:10
            throw new Error(errorMessage);
            ^

Error: Expected to equal an error message, but got: a different error message

Process finished with exit code 1

Undoing the breaking change (Mutation) will make the test pass:

/usr/local/bin/node AssertThrowTest.js

Process finished with exit code 0

Next test is to make sure, that assertThrow is actually comparing actual and expected error messages correctly:

// test/AssertThrowTest.js

this.testFailure = function () {
    t.assertThrow("Expected to equal an error message, but got: a different error message", function () {
        t.assertThrow("an error message", function () {
            throw new Error("a different error message");
        });
    });
};

And it passes. The last test, that assertThrow needs is the case, when action() is not throwing any error. In that case assertThrow should fail:

// test/AssertThrowTest.js

this.testFailure_whenActionDoesNotThrow = function () {
    t.assertThrow("Expected to throw an error, but nothing was thrown", function () {
        t.assertThrow("an error message", function () {
            // does nothing
        });
    });
};

Oh, and that test is passing! We clearly don’t have that functionality yet. We have to add another test without usage of outer t.assertThrow to make sure that we get a test failure:

this.testThrows_whenActionDoesNotThrow = function () {
    var hasThrown = false;

    try {
        t.assertThrow("an error message", function () {
            // does nothing
        });
    } catch (error) {
        hasThrown = true;
    }

    t.assertTrue(hasThrown, "it should have thrown");
};

And if we run our tests we get the expected failure:

/usr/local/bin/node AssertThrowTest.js
/path/to/project/src/TestingFramework.js:10
            throw new Error(errorMessage);
            ^

Error: it should have thrown

Process finished with exit code 1

We can fix that by verifying, that catch block is executed in the assertThrow function:

// src/TestingFramework.js

assertThrow: function (expectedMessage, action) {
    var hasThrown = false;  // <- we initialize hasThrown here

    try {
        action();
    } catch (error) {
        hasThrown = true;   // <- and we set it to true in the catch block
        this.assertEqual(expectedMessage, error.message);
    }

    this.assertTrue(hasThrown);  // <- and we check that it is true
}

And now if we run the test suite, the original test that we were trying to write fails as expected:

/usr/local/bin/node AssertThrowTest.js
/path/to/project/src/TestingFramework.js:10
            throw new Error(errorMessage);
            ^

Error: Expected to equal Expected to throw an error, but nothing was thrown, but got: Expected to be true, but got false

Process finished with exit code 1

Now we need to fix the custom message provided to assertTrue call inside of assertThrow:

// src/TestingFramework.js

// inside of assertThrow:
this.assertTrue(hasThrown, "Expected to throw an error, but nothing was thrown");

And the test pass:

/usr/local/bin/node AssertThrowTest.js

Process finished with exit code 0

Great, I think we are done with testing the assertThrow function. Let’s see the whole AssertThrowTest suite:

// test/AssertThrowTest.js
var runTestSuite = require("../src/TestingFramework");

runTestSuite(function (t) {
    this.testSuccess = function () {
        t.assertThrow("an error message", function () {
            throw new Error("an error message");
        });
    };

    this.testSuccess_withDifferentExpectedMessage = function () {
        t.assertThrow("a different error message", function () {
            throw new Error("a different error message");
        });
    };

    this.testFailure = function () {
        t.assertThrow("Expected to equal an error message, but got: a different error message", function () {
            t.assertThrow("an error message", function () {
                throw new Error("a different error message");
            });
        });
    };

    this.testFailure_whenActionDoesNotThrow = function () {
        t.assertThrow("Expected to throw an error, but nothing was thrown", function () {
            t.assertThrow("an error message", function () {
                // does nothing
            });
        });
    };

    this.testThrows_whenActionDoesNotThrow = function () {
        var hasThrown = false;

        try {
            t.assertThrow("an error message", function () {
                // does nothing
            });
        } catch (error) {
            hasThrown = true;
        }

        t.assertTrue(hasThrown, "it should have thrown");
    };
});

Last one for today is assertEqual:

Testing `assertEqual(expected, actual)`

Let’s create a test suite with the first test, when assertEqual succeeds:

// test/AssertEqualTest.js
var runTestSuite = require("../src/TestingFramework");

runTestSuite(function (t) {
    this.testSuccess = function () {
        t.assertEqual(42, 42);
    };
});

This test passes, and it can be broken without test failure by always comparing to 42:

// src/TestingFramework.js

assertEqual: function (expected, actual) {
    this.assertTrue(
        42 == actual,   // <- here 'expected' is replaced with constant
        "Expected to equal " + expected + ", but got: " + actual
    );
},

Triangulation to fix that:

// test
this.testSuccess_whenExpectedIsDifferent = function () {
    t.assertEqual(29, 29);
};

// Error: Expected to equal 29, but got: 29

// fix implementation:
expected == actual,   // <- here 'expected' is restored

// and the test passes

Next mutation that does not break any tests looks this way:

// src/TestingFramework.js

assertEqual: function (expected, actual) {
    this.assertTrue(
        expected == actual,
        // "Expected to equal " + expected + ", but got: " + actual
        "oops"    // <- replace error message
    );
},

Let’s add the test to protect from this kind of mutation:

this.testFailure = function () {
    t.assertThrow("Expected to equal 42, but got: 29", function () {
        t.assertEqual(42, 29);
    });
};

This fails with Error: oops, because assertThrow uses assertEqual. Stack trace shows, that the failure is happening here:

// src/TestingFramework.js in assertThrow
this.assertEqual(expectedMessage, error.message);

So that is the expected failure. We can fix it by always providing the message "Expected to equal 42, but got: 29":

// src/TestingFramework.js in assertions
assertEqual: function (expected, actual) {
    this.assertTrue(
        expected == actual,
        "Expected to equal 42, but got: 29"
    // ^ here the exact constant is used ^
    );
},

This needs some more triangulation:

this.testFailure_withDifferentExpectedAndActual = function () {
    t.assertThrow("Expected to equal 94, but got: 1027", function () {
        t.assertEqual(94, 1027);
    });
};

// Error: Expected to equal 42, but got: 29

// fix:
this.assertTrue(
    expected == actual,
    "Expected to equal " + expected + ", but got: " + actual
);

I think we are done now with testing the assertEqual function. The AssertEqualTest suite is looking like that now:

// test/AssertEqualTest.js
var runTestSuite = require("../src/TestingFramework");

runTestSuite(function (t) {
    this.testSuccess = function () {
        t.assertEqual(42, 42);
    };

    this.testSuccess_whenExpectedIsDifferent = function () {
        t.assertEqual(29, 29);
    };

    this.testFailure = function () {
        t.assertThrow("Expected to equal 42, but got: 29", function () {
            t.assertEqual(42, 29);
        });
    };

    this.testFailure_withDifferentExpectedAndActual = function () {
        t.assertThrow("Expected to equal 94, but got: 1027", function () {
            t.assertEqual(94, 1027);
        });
    };
});

Final `assertions` object after all the testing

// src/TestingFramework.js
var assertions = {
    assertTrue: function (condition, message) {
        var errorMessage = "Expected to be true, but got false";

        if (message) {
            errorMessage = message;
        }

        if (!condition) {
            throw new Error(errorMessage);
        }
    },

    assertEqual: function (expected, actual) {
        this.assertTrue(
            expected == actual,
            "Expected to equal " + expected + ", but got: " + actual
        );
    },

    assertThrow: function (expectedMessage, action) {
        var hasThrown = false;

        try {
            action();
        } catch (error) {
            hasThrown = true;
            this.assertEqual(expectedMessage, error.message);
        }

        this.assertTrue(
            hasThrown,
            "Expected to throw an error, but nothing was thrown"
        );
    }
};

Bottom Line

Congratulations! We have completely unit-tested our assertions assertTrue and assertEqual. This resulted in natural emergence of the new assertion - assertThrow. We have unit-tested it too!

Additionally, we have practiced usage of Mutational Testing and Triangulation Technique to detect missing test cases and derive them from the code. Also, we have slightly touched the Parallel Change refactoring technique - we will see more of that in the future in these series.

The code is available on Github: https://github.com/waterlink/BuildYourOwnTestingFrameworkPart2

Now that we have unit-tested some basic assertions, we should unit-test our testing framework test runner: runTestSuite function. This will be covered in next series of “Build Your Own Testing Framework”. Stay tuned!

Thanks!

Thank you for reading, my dear reader. If you liked it, please share this article on social networks and follow me on twitter: @tdd_fellow.

If you have any questions or feedback for me, don’t hesitate to reach me out on Twitter: @tdd_fellow.

Build Your Own Testing Framework

Alex Fedorov Jul 15th, 2016 8:00 am build-your-own-testing-framework, javascript, tdd, test-driven-development, testing, xunit

Today we are going to test-drive the testing framework without any external testing framework. This will be done through test-driving a simple kata (FizzBuzzKata). For example:

every time we expect a test to fail and it doesn’t, this is a failing test for our testing framework, that we will be fixing,
every time we expect a test to pass and it doesn’t, this is another failing test for our testing framework, that we will be fixing.

For practical reasons, today we are going to use concrete programming language instead of pseudo-code - javascript. Except for small details, that we will point out, the techniques shown here are language-agnostic.

This article is only first one of the series “Build Your Own Testing Framework”, so make sure to stick around for next parts! All articles of these series can be found here: http://www.tddfellow.com/blog/categories/build-your-own-testing-framework/.

Shall we begin?

FizzBuzzKata

Given the number,

return Fizz when the number is divisible by 3,
return Buzz when the number is divisible by 5,
return FizzBuzz when the number is divisible by 3 and 5,
return string representation of number otherwise.

Writing your first test

How do we write our first test, when we don’t have a testing framework and we want to create one? - It seems, that we have to design how the test should like in our brand new testing framework.

I personally, would go with the xUnit-like design, since it is relatively simple. Given this, we might write our first test and it will look something like that:

// test/FizzBuzzKataTest.js

function FizzBuzzKataTest() {
    this.testNormalNumberIsReturned = function() {
        this.assertTrue("1" === fizzBuzz(1));
    };
}

This test should fail, because function fizzBuzz is not defined, but it doesn’t fail, since function testNormalNumberIsReturned is never called. In fact, the object with FizzBuzzKataTest is never being created.

Easiest way to solve that:

// test/FizzBuzzKataTest.js

function FizzBuzzKataTest() { ... }

var test = new FizzBuzzKataTest();
test.testNormalNumberIsReturned();

If we run this code with node:

node test/FizzBuzzKataTest.js

We will get the expected error:

/path/to/project/test/FizzBuzzKataTest.js:3
        this.assertTrue("1" === fizzBuzz(1));
                               ^

ReferenceError: fizzBuzz is not defined

So, let’s define this function:

// test/FizzBuzzKataTest.js

function FizzBuzzKataTest() { ... }

function fizzBuzz() {}

If we run our test again, we will get the following error:

/path/to/project/test/FizzBuzzKataTest.js:3
        this.assertTrue("1" === fizzBuzz(1));
             ^

TypeError: this.assertTrue is not a function

Clearly, to fix it we need to define assertTrue on FizzBuzzKataTest object. Obviously, we do not want our user to define all their assertion for every test suite. This means, that we want to define it on FizzBuzzKataTest object outside of the definition of FizzBuzzKataTest.

There are two ways to go about it:

inheritance: make FizzBuzzKataTest inherit from some other object function assertTrue, or
composition: make FizzBuzzKataTest accept a special object with function assertTrue defined on it.

I would like to go with composition method since it gives us more flexibility in the long run:

function FizzBuzzKataTest(t) { ... }

and the usage of assertTrue has to change appropriately:

t.assertTrue("1" === fizzBuzz(1));

and t has to be created and passed in correctly:

function FizzBuzzKataTest(t) { ... }

function fizzBuzz(number) {}

var assertions = {
    assertTrue: function(condition) {}
};

var test = new FizzBuzzKataTest(assertions);

If we run the test suite again, we will not get any failure anymore. But we were expecting assertTrue to fail, so let’s make it fail:

assertTrue: function(condition) {
    throw new Error("Expected to be true, but was false");
}

When we run the test suite, we get:

/path/to/project/test/FizzBuzzKataTest.js:11
        throw new Error("Expected to be true, but got false");
        ^

Error: Expected to be true, but got false

Now, let’s customize the error message a bit:

this.testNormalNumberIsReturned = function() {
    t.assertTrue("1" === fizzBuzz(1), "Expected to equal " + "1" + ", but got: " + fizzBuzz(1));
}

// ...

assertTrue: function(condition, message) {
    throw new Error(message || "Expected to be true, but got false");
}

When running this, we are getting the expected error:

/path/to/project/test/FizzBuzzKataTest.js:11
        throw new Error(message || "Expected to be true, but got false");
        ^

Error: Expected to equal 1, but got: undefined

This looks better now. Let’s fix the error now by implementing the simplest thing that could work:

function fizzBuzz(number) {
    return "1";
}

And as we run our test suite we get:

/path/to/project/test/FizzBuzzKataTest.js:13
        throw new Error(message || "Expected to be true, but got false");
        ^

Error: Expected to equal 1, but got: 1

Oh, it should have passed the test. I know why it didn’t: we throw this error unconditionally, let’s add an appropriate if statement to assertTrue function:

assertTrue: function(condition, message) {
    if (!condition) {
        throw new Error(...);
    }
}

If we run this code, it does not fail. That was our first green state - it took as awhile to get here. The reason for this is that we are not only test-driving FizzBuzzKata, additionally, we are writing a feature test for a non-existing testing framework. Now that we are green, we should think about refactoring, i.e.: making the structure of our code right.

Obviously, we should move our testing framework code outside of the test suite file. Probably, somewhere in src/TestingFramework.js. For that, we need to first parametrize FizzBuzzKataTest and extract the function to run the test suite.

Parametrize:

var testSuiteConstructor = FizzBuzzKataTest;
var testSuite = new testSuiteConstructor(assertions);
testSuite.testNormalNumberIsReturned();

and extract method:

function runTestSuite(testSuiteConstructor) {
    var testSuite = new testSuiteConstructor(assertions);
    testSuite.testNormalNumberIsReturned();
}

var testSuiteConstructor = FizzBuzzKataTest;
runTestSuite(testSuiteConstructor);

and inline the variable testSuiteConstructor:

runTestSuite(FizzBuzzKataTest);

Now it is time to move testing code to src/TestingFramework.js:

// src/TestingFramework.js

var assertions = {
    assertTrue: function (condition, message) {
        if (!condition) {
            throw new Error(message || "Expected to be true, but got false");
        }
    }
};

function runTestSuite(testSuiteConstructor) {
    var testSuite = new testSuiteConstructor(assertions);
    testSuite.testNormalNumberIsReturned();
}

And to be able to require runTestSuite function:

// src/TestingFramework.js

var assertions = { ... };

function runTestSuite(testSuiteConstructor) { ... }

module.exports = runTestSuite;

And, finally, let’s use that from our test suite:

// test/FizzBuzzKataTest.js

var runTestSuite = require("../src/TestingFramework");

function FizzBuzzKataTest(t) { ... }

function fizzBuzz(number) { ... }

runTestSuite(FizzBuzzKataTest);

If we run the test suite again, everything should pass. Somehow, I don’t feel comfortable now, let’s try to break the test suite and see if it will fail as expected:

function fizzBuzz(number) {
    return "2";  // <-- "1" was changed to "2" here
}

And run tests:

/path/to/project/src/TestingFramework.js:4
            throw new Error(message || "Expected to be true, but got false");
            ^

Error: Expected to equal 1, but got: 2

Yes, it still works as expected. We have just introduced a Mutation to our code, to see if it is still tested properly. Let’s undo the Mutation and see the test still pass. And it does.

If you look closely now, it should be possible to inline FizzBuzzKataTest definition as an argument of runTestSuite call:

var runTestSuite = require(...);

function fizzBuzz(number) { ... }

runTestSuite(function (t) { ... });

And if we run our test suite, it still works. Just to check, that we are still good, let’s repeat our Mutation from the previous step. It should still fail as expected. And it does. Undo the mutation and the test is still passing. Great.

I think we are done with Refactoring step, for now, let’s get back to writing another failing test.

Writing the second test

this.testAnotherNormalNumberIsReturned = function() {
    t.assertTrue("2" === fizzBuzz(2), "Expected to equal " + "2" + ", but got: " + fizzBuzz(2));
};

If we run these tests, they do not fail. This is strange, let’s look at runTestSuite function again:

function runTestSuite(testSuiteConstructor) {
    var testSuite = new testSuiteConstructor(assertions);
    testSuite.testNormalNumberIsReturned();
}

Great, it just runs one specific function, we should probably run all functions starting from test instead:

function runTestSuite(testSuiteConstructor) {
    var testSuite = new testSuiteConstructor(assertions);

    for (var testName in testSuite) {
        if (testName.match(/^test/)) {
            testSuite[testName]();
        }
    }
}

REMARK: this code is Javascript specific. Other programming languages will have their own way of iterating over the function/method list and calling a function by its name. Usually, it is some sort of reflection for compiled languages and meta-programming features for interpreted languages.

If we run tests now, we get the expected failure:

/path/to/project/src/TestingFramework.js:4
            throw new Error(message || "Expected to be true, but got false");
            ^

Error: Expected to equal 2, but got: 1

If we try to change return "1" to return "2", of course this test will pass, but the other will fail:

/path/to/project/src/TestingFramework.js:4
            throw new Error(message || "Expected to be true, but got false");
            ^

Error: Expected to equal 1, but got: 2

This is great for couple of reasons:

It validates, that our change to how test* functions are discovered is correct, and
We have to have a bit smarter implementation to pass both tests now:

function fizzBuzz(number) {
    return number.toString();
}

And if we run the tests, they pass. Now, that we are in Green state, we should start refactoring. Have you noticed the duplication already?

t.assertTrue("1" === fizzBuzz(1), "Expected to equal " + "1" + ", but got: " + fizzBuzz(1));

// and:

t.assertTrue("2" === fizzBuzz(2), "Expected to equal " + "2" + ", but got: " + fizzBuzz(2));

Extracting "1" and "2" as variable expected, and fizzBuzz(1) and fizzBuzz(2) as variable actual, makes these 2 lines identical:

this.testNormalNumberIsReturned = function () {
    var expected = "1";
    var actual = fizzBuzz(1);
    t.assertTrue(expected === actual, "Expected to equal " + expected + ", but got: " + actual);
};

this.testAnotherNormalNumberIsReturned = function() {
    var expected = "2";
    var actual = fizzBuzz(2);
    t.assertTrue(expected === actual, "Expected to equal " + expected + ", but got: " + actual);
};

Specifically, this is identical:

t.assertTrue(expected === actual, "Expected to equal " + expected + ", but got: " + actual);

This sounds like t.assertEqual(expected, actual) to me. So let’s extract it:

// src/TestingFramework.js

var assertions = {
    assertTrue: function(condition, message) { ... },

    assertEqual: function(expected, actual) {
        this.assertTrue(
          expected === actual,
          "Expected to equal " + expected + ", but got: " + actual
        );
    }
}

Now, let’s use it and inline our expected and actual variables:

this.testNormalNumberIsReturned = function () {
    t.assertEqual("1", fizzBuzz(1));
};

this.testAnotherNormalNumberIsReturned = function() {
    t.assertEqual("2", fizzBuzz(2));
};

This looks much more readable. If we run tests, they still pass. If we try to break our code by using some Mutation, the tests fail as expected. Great, our refactoring was a success!

Let’s finish test-driving our fizzBuzz function.

Test-Driving Fizz Buzz Kata

First test for Fizz case:

// test:
this.testFizzIsReturned = function () {
    t.assertEqual("Fizz", fizzBuzz(3));
};

// Error: Expected to equal Fizz, but got: 3

// implementation:
function fizzBuzz(number) {
    if (number === 3) return "Fizz";
    return number.toString();
}

This is pretty stupid implementation, but it works for that one tests, so let’s write the test, that will break this implementation and force us to write real if condition:

// test:
this.testFizzIsReturnedForDifferentNumber = function () {
    t.assertEqual("Fizz", fizzBuzz(6));
};

// Error: Expected to equal Fizz, but got: 6

// implementation:
function fizzBuzz(number) {
    if (number % 3 === 0) return "Fizz";
    return number.toString();
}

This technique is called Triangulation:

the first test is to force us to write some if statement with a correct body,
second is to force us to make the condition right.
If we had an else clause, we would have had another test to make that part right.

OK, that looks like a right implementation for Fizz, let’s write the test for Buzz now:

// test:
this.testBuzzIsReturned = function () {
    t.assertEqual("Buzz", fizzBuzz(5));
};

// Error: Expected to equal Buzz, but got: 5

// stupid implementation:
function fizzBuzz(number) {
    if (number === 5) return "Buzz";
    if (number % 3 === 0) return "Fizz";
    return number.toString();
}

// Triangulation:
this.testBuzzIsReturnedForDifferentNumber = function () {
    t.assertEqual("Buzz", fizzBuzz(10));
};

// Error: Expected to equal Buzz, but got: 10

// correct implementation:
function fizzBuzz(number) {
    if (number % 5 === 0) return "Buzz";
    if (number % 3 === 0) return "Fizz";
    return number.toString();
}

And finally let’s implement final requirement FizzBuzz:

// test:
this.testFizzBuzzIsReturned = function () {
    t.assertEqual("FizzBuzz", fizzBuzz(15));
};

// Error: Expected to equal FizzBuzz, but got: Buzz

// stupid implementation:
function fizzBuzz(number) {
    if (number === 15) return "FizzBuzz";
    if (number % 5 === 0) return "Buzz";
    if (number % 3 === 0) return "Fizz";
    return number.toString();
}

// Triangulation:
this.testFizzBuzzIsReturnedForDifferentNumber = function () {
    t.assertEqual("FizzBuzz", fizzBuzz(30));
};

// Error: Expected to equal FizzBuzz, but got: Buzz

// correct implementation:
function fizzBuzz(number) {
    if (number % 15 === 0) return "FizzBuzz";
    if (number % 5 === 0) return "Buzz";
    if (number % 3 === 0) return "Fizz";
    return number.toString();
}

I think we are done with the implementation. FizzBuzzKata has an extended set of requirements, but they are out of the scope of this article. These requirements force us to introduce Strategy pattern and stop using this unmaintainable chain of if statements.

Refactoring this code to Strategy pattern is left as an exercise for the reader.

Bottom Line

Congratulations! Using FizzBuzzKata we have test-driven bare-bones testing framework to the point, that we can do Test-Driven Development for a simple Kata. And all that without having any testing framework in place.

The code is available on Github: https://github.com/waterlink/BuildYourOwnTestingFrameworkPart1

Now, with this minimal framework in place, it should be possible to unit-test the framework itself, so that we can support more use cases. This will be covered in next series of “Build Your Own Testing Framework”. Stay tuned!

Thanks!

Thank you for reading, my dear reader. If you liked it, please share this article on social networks and follow me on twitter: @tdd_fellow.

If you have any questions or feedback for me, don’t hesitate to reach me out on Twitter: @tdd_fellow.

TL;DR on Sustainable Software Development Paper

Alex Fedorov Jun 28th, 2016 2:22 am code-ownership, extreme-programming, grounded-theory, sustainable-software-development, tldr, white-paper

This is a TL;DR of a white paper Sustainable Software Development through Overlapping Pair Rotation that is created by Todd Sedano, Paul Ralph, and Cécile Péraire.

NOTE: this TL;DR includes a lot of compressed awesome information.

Theory of Sustainable Software Development

The paper introduces us to principles, policies, and practices, that are the results of applying of Constructivist Grounded Theory research method to the five projects at Pivotal Labs.

Principles

By Engendering Positive Attitudes Toward Disruption teams can transform a challenge of team disruption into a business opportunity:
- team members rolling off from the project are replaced with new members,
- new members are viewed as a source of fresh perspective and as an opportunity,
- new members often questioned team’s assumptions (challenging “cargo culting”).
Policies & practices that Encourage Knowledge Sharing and Continuity mitigate the risk of significant knowledge loss for the project when the team is disrupted. Policies are:
- Team Code Ownership, and Shared Team Schedule. While practices are:
- Continuous Pair Programming, Overlapping Pair Rotation and Knowledge Pollination.
Sustainable development is not possible if the team is incurring the technical debt. Caring about Code Quality is a way to mitigate this risk. Policy here is:
- Avoid Technical Debt, and practices are:
- Test-Driven Development / Behavior-Driven Development and Continuous Refactoring.

Policies

Team Code Ownership: every developer is empowered to work on and to refactor any part of the team’s code.
Shared Schedule: every team member has agreed to a fixed schedule to achieve the benefits of Sustainable Software Development. Teams are, preferably, co-located.
Avoid Technical Debt: a pair creates well-crafted code without shortcuts and short-term fixes. Solutions are best and simplest for current present without over-engineering. When inherited a legacy code base, a team is actively paying down technical debt while delivering new features.

Removing Knowledge Silos Practices

Continuous Pair Programming: two developers are collaborating to write software together as their normal mode of software development. It enables knowledge transfer, reduces knowledge silos and improves code quality. Developers always work in pairs, unless exceptional circumstances arise.
Overlapping Pair Rotation: one developer rolls off the track of work and another developer rolls on, keeping the continuity of one developer at each rotation. It results in knowledge continuity. There are three strategies for spreading the knowledge via pair rotation when fighting emerging knowledge silos:
- Optimizing for people rotation: developers try to pair with the person they “least recently paired with”;
- Optimizing for personal preferences: developer pick with whom they will pair at each rotation, based on personal preferences;
- Optimizing for context sharing: a developer who has not been working on the track for the longest track is rotating onto the track. The goal each day is for the developer leaving the track next to empower the developer who will remain on the track.
Knowledge Pollination:
- Daily stand-ups create awareness of who is working on what.
- Using backlog to communicate information and status of stories.
- Osmotic communication: a developer overhears a discussion in other pair and offers their help.
- The pair can interrupt another pair, product manager or designer to gain the needed information.
- Calling out updates to the entire team.
- Interruptions are encouraged - it makes the team more efficient as knowledge pollinates across the team.

Caretaking the Code Practices

Test-Driven Development/Behavior-Driven Development: developers use BDD (to describe interactions between the user and the system) and TDD (at unit test level). Design emerges from the creation and exploration of the test cases. Teams use a variety of TDD strategies, including:
- testing the responsibilities and interactions, or
- contract testing using mocks.
Continuous Refactoring: developers do refactoring as part of implementing feature stories. Developers are encouraged to improve the code’s design, make the code easier to read, understand, and increase the discoverability of a component based on its responsibility. The team prefers pre-refactoring, where the developer makes a complicated work to make the current story as simple and as easy as possible.
Live on Master: every pair merges their code to master many times a day. Developers may use branches to save “spikes” and to move work-in-progress between pairing stations when rotating.

Thanks!

This is where this TL;DR ends. If you are interested in the topic, go ahead and read the paper, it has the following:

Detailed description of the research method, data collection, and analysis;
Research context and one project case study;
Detailed description of Principles, Policies, and Practices of Sustainable Software Development; also anti-patterns for each policy and practice;
Theory evaluation and conclusions.

Great thanks to the original authors of the paper, researchers, and all participants!

Why I Don’t Use Mocking Frameworks Anymore

Alex Fedorov Jun 21st, 2016 8:04 am architecture, clean-code, mocking, pseudo-code, testing

Some time ago, I have discovered, that using your own custom test double classes, instead of a test framework makes my test code more readable and maintainable. Here is an example (pseudo-code):

function test_password_revealer_when_toggled_reveals_password() {
  passwordController = MockPasswordController.new()
  passwordRevealer = PasswordRevealer.new(passwordController)

  passwordRevealer.toggle()

  expect(passwordController.isRevealed()).toBeTrue()
}

The same test with mocking framework would look this way:

function test_password_revealer_when_toggled_reveals_password() {
  passwordController = MockFramework.Mock.new("PasswordController")
  passwordRevealer = PasswordRevealer.new(passwordController)

  spy = spyOn(passwordController.reveal())

  passwordRevealer.toggle()

  expect(spy.haveBeenCalled()).toBeTrue()
}

If you take a closer look at the last example, and, specifically at these 2 lines:

spy = spyOn(passwordController.reveal())

expect(spy.haveBeenCalled()).toBeTrue()

They use a language, that is not relevant to the domain, therefore, they make the test less readable.

Additionally, they have knowledge of which exactly method PasswordRevealer#toggle() should call on passwordController. If we were to rename reveal method, all tests for PasswordRevealer would fail. The same thing would happen if we were to extract methods/functions/objects out of the PasswordRevealer.

Of course, creating such test doubles yourself will involve some boilerplate code - this is a trade-off. Example:

class MockPasswordController() {
  this.state = "hidden"

  this.reveal() {
    this.state = "revealed"
  }

  this.hide() {
    this.state = "hidden"
  }

  this.isRevealed() {
    return this.state == "revealed"
  }
}

Making this trade-off, will simplify the case when we were to change the name: we would change the name at 3 places:

in test double class,
in caller class,
in “real” implementation class.

Whereas with a mocking framework, we would have to hunt for all failing tests, and usually, it means hundreds of failing tests.

Thank you!

If you, my dear reader, have any thoughts, questions or arguments about the topic, feel free to reach out to me on twitter: @tdd_fellow.

If you liked my ideas, follow me on twitter, and, even better, provide me with your honest feedback, so that I can improve.

TDD #5: English Numbers Kata

Alex Fedorov Jun 19th, 2016 6:41 pm baby-steps, divide-and-conquer, iterative, kata, problem-solving, screencast, tdd, tdd-as-if-you-meant-it, tdd-screencasts, test-driven-development

Test-Driven Development Screencast #5: English Numbers Kata

(Recommended to watch full-screen and HD: 720p or more)

List of all TDD Screencasts can be found here.

This screencast was born from my previous blog post: Iterative Divide & Conquer or How to solve problems.

TDD as an Enabling Practice or How to Be Faster With TDD

Alex Fedorov Jun 13th, 2016 7:52 am agile, continuous-delivery, continuous-integration, dreyfus-model, enabling-practices, exploitative-practices, pair-programming, practices, professionalism, skills, tdd, xp

Recently I had a lot of conversations with so many different programmers, with different backgrounds, contexts of work and opinions. What stroke me the most, is that majority feel that TDD is slower than simple automated testing, i.e., TestFirst is slower than TestAfter.

While digging deeper in their context of work, I could only agree with them: “True, in that context it will be about 50% slower”.

Most of the time, though, the context is somewhat looking like this:

We have some company ACME, that does some sort of Agile Software Development (probably Scrum);
The company ACME focuses only on the business parts of Agile Software Development;
The company ACME tried to introduce TDD as a development practice;
Everything took much longer to be done.

Enabling practice

Now, let us dive into the development practices of Agile Software Development: development practices are usually coming from Extreme Programming (XP). In XP there are 2 terms: EnablingPractice and ExploitativePractice.

Exploitative practice gives direct benefit to the team, e.g.: speed boost and quicker feedback from users and stakeholders.

Enabling practice is required for certain other exploitative practice(s) to work.

A good example of exploitative practice is Continuous Delivery. It requires Continuous Integration, Pair-Programming, and Testing to be in place. These 3 are enabling practices.

Removing slow practices

Additionally to allowing usage of practices, that make a team go faster and deliver at the higher quality level, enabling practices allow removal of practices, that make a team go slower. For example, Pair-Programming together with TDD allows removal of code review. On most of the teams (especially, of bigger size), this makes for an instant productivity boost.

Pair-Programming, TDD and Continuous Integration, additionally to enabling the team to do Continuous Delivery, also allows replacing feature-branch VCS flow with a trunk-based flow. This allows for smaller iterations and faster user feedback.

Pair-Programming Done Right

It is worth noting, that removal of code review and introducing of Continuous Delivery is only possible, if Pair-Programming is done right:

in no case, two beginners should be working in the pair;
beginners should work together with advanced beginners/competents and proficients/experts;
advanced beginners/competents should split their time in half between working with beginners and working with proficient/experts.

Terminology Beginner, AdvancedBeginner, Competent, Proficient and Expert is from Dreyfus Skill Acquisition Model.

That allows for a good trust and mentorship models in your team(s). It enables quick growth and knowledge sharing for every member of the team.

There is another enabling practice which speeds up the knowledge sharing, it is Pair Rotation, that should be done from 1 to 2 times per day, so that for small and middle-sized teams, the bigger the feature is, the higher chance, that everyone on the team have participated in its development, and therefore have enough knowledge about it.

Additionally, Pair Rotation allows for Code Detachment and removal of Code Silos. This in turn, together with TDD, enables Ruthless Refactoring, because you are not afraid:

to break the code, thanks to TDD,
to upset an owner of the Code Silo, because there is no owner, thanks to Pair Rotation.

Bottomline

I think I can go on like this forever, but I believe the idea should be clear. I will be following up with more articles in details on each technique in the future. Stay tuned!

Thank you for reading!

If you, my dear reader, have any thoughts, questions or arguments about the topic, feel free to reach out to me on twitter: @tdd_fellow.

If you liked my ideas, follow me on twitter, and, even better, provide me with your honest feedback, so that I can improve.

Automating Manual Checks or How to Save Time and Get Rapid Feedback

Alex Fedorov Jun 5th, 2016 12:37 pm automation, programming, pseudo-code, tests

It is tedious, slow and error-prone to test correctness of program manually. This article is an introduction, how to make this process as simple as a hit of one button, very fast and precise.

In the last issue about Iterative Divide & Conquer we have found out, how it is tedious to manually check that our program works correctly.

We have concluded, that it is tedious and slow to manually check that the output of our print statements is correct. Additionally, it is really easy to make a mistake and to oversee incorrect lines of the output, especially, when we have tons of them. As well, this applies to any other form of manual testing, including testing the application as an End User (via clicking/tapping for UI applications, via executing tons of commands for CLI applications).

Mitigating human error-proneness via automation

Let’s get back to our example from the previous issue:

print english_integer_in_tens(20)    # twenty
print english_integer_in_tens(27)    # twenty-seven
print english_integer_in_tens(42)    # forty-two
print english_integer_in_tens(70)    # seventy
print english_integer_in_tens(99)    # ninety-nine

What if we could ask our computer to run english_integer_in_tens function with provided arguments and check that the output is exactly what we provided in a comment?

Let’s try to put it in pseudo-code:

equal(english_integer_in_tens(20), "twenty")
equal(english_integer_in_tens(42), "forty-two")

If we run that code, of course, we will get some sort of compilation error, or runtime error (depending on the programming language), because function equal is not defined yet. Let’s define it! Presumably, it should be comparing left argument with right argument and printing something useful on the screen.

function equal(left, right) {
  if (left != right) {
    print("Failure: " + left + " should be equal to " + right)
  } else {
    print("OK")
  }
}

If we run the program after defining this function we will get:

OK
OK

If we were to break the function english_integer_in_tens’s implementation, we might get something like:

OK
Failure: twenty should be equal to forty-two

Making automation nicer

Having all this output every time we import our small library in any application going to the standard output would be annoying to say the least. How about separating the test automation from the library?

Let’s extract all our testing print-s into the separate file, import original file from it and replace all print-s with equal-s:

import english_integer

function equal(left, right) { ... }

equal(english_number(0), "zero")
equal(english_number(1), "one")
...
equal(english_integer(43), "forty-three")
equal(english_integer(-97), "minus ninety-seven")

Running only this separate file will produce expected output:

OK
OK
...
OK
OK

Importing or running directly our original library will not produce any output. This is much nicer.

Thank you for reading!

Today we have made our testing:

easier: hit of one button, or one command on the terminal;
faster: from minutes of manual verification to milliseconds of automated checks;
preciser: automated checks can’t make a mistake, if we have something except OK - we have an error, if everything is OK - the program is working correctly;
better feedback cycle: we can see if our change is correct in the matter of 1-2 seconds after making this change.

Next time we will build a fully-functional mini testing framework. Stay tuned!

← Older Blog Archives Newer →

Summary

Testing assertTrue(condition, message)

Refactoring AssertTrueTest

Testing assertThrow(expectedMessage, action)

Testing assertEqual(expected, actual)

Final assertions object after all the testing

Bottom Line

Thanks!

FizzBuzzKata

Writing your first test

Writing the second test

Test-Driving Fizz Buzz Kata

Bottom Line

Thanks!

Theory of Sustainable Software Development

Principles

Policies

Removing Knowledge Silos Practices

Caretaking the Code Practices

Thanks!

Thank you!

Enabling practice

Removing slow practices

Pair-Programming Done Right

Bottomline

Thank you for reading!

Mitigating human error-proneness via automation

Making automation nicer

Thank you for reading!

Testing `assertTrue(condition, message)`

Refactoring `AssertTrueTest`

Testing `assertThrow(expectedMessage, action)`

Testing `assertEqual(expected, actual)`

Final `assertions` object after all the testing