Parallel Change Refactoring

clean-code, pseudo-code, refactoring, software-craftsmanship

Parallel Change is the refactoring technique, that allows implementing backward-incompatible changes to an API in a safe manner. It consists of 3 steps:

  • Expand - add new functionality as an extension of the interface (e.g.: add new methods), instead of editing signatures of existing interfaces
  • Migrate - mark (or log a warning) existing interface as deprecated and give time to the clients of the interface to migrate to the new interface. This might be as simple as changing your own code-base one client of this interface at a time in case when you do not have 3rd party clients of these interfaces
  • Contract - once all clients have migrated remove old interfaces

This technique is tremendously useful when you have 3rd party clients for your API (open-source library, SaaS with REST API, etc). Also, it is as useful for normal application development because it allows such breaking changes to never break your test suite. It allows deploying your code in the middle of the refactoring, in fact, every few minutes (Continuous Integration + Continuous Deployment).

This article contains code examples in a pseudo-code.

Build Your Own Testing Framework. Part 2

build-your-own-testing-framework, javascript, tdd, test-driven-development, testing, xunit

Today we are going to unit-test existing functionality of our own testing framework, that we have test-driven with FizzBuzzKata in the previous part.

This needs to be done, since currently only happy paths are implicitly tested via FizzBuzzKata test suite, i.e.: when all the tests pass. The following unhappy paths are not tested at the moment:

  • when assertTrue fails, it throws an error,
  • when assertEqual fails, it throws an error,
  • when the test fails, it renders the error and fails the whole test suite run, i.e.: non-zero exit code.

This article is the second one of the series “Build Your Own Testing Framework”, so make sure to stick around for next parts! All articles of these series can be found here: http://www.tddfellow.com/blog/categories/build-your-own-testing-framework/.

Shall we get the ball rolling?

Testing assertTrue(condition, message)

Let’s create a test suite for assertTrue with the first test for the case, when assertTrue succeeds:

1
2
3
4
5
6
7
8
// test/AssertTrueTest.js
var runTestSuite = require("../src/TestingFramework");

runTestSuite(function (t) {
    this.testSuccess = function () {
        t.assertTrue(true);
    };
});

If we run this test suite, it should not fail:

1
2
3
/usr/local/bin/node AssertTrueTest.js

Process finished with exit code 0

Let’s check if that test actually is testing anything by trying to break the implementation of assertTrue:

1
2
3
4
5
6
7
// src/TestingFramework.js

assertTrue: function (condition, message) {
    if (condition) {    // <- this condition was inverted
        throw new Error(...);
    }
},

And if we run it, we get the expected error:

1
2
3
4
5
6
7
8
9
/usr/local/bin/node AssertTrueTest.js
/path/to/project/src/TestingFramework.js:4
            throw new Error(message || "Expected to be true, but got false");
            ^

Error: Expected to be true, but got false
    .. stack trace ..

Process finished with exit code 1

OK, it fails as expected, now we should undo our breaking change in the implementation and run the test suite again and it should pass:

1
2
3
4
5
6
7
// src/TestingFramework.js

assertTrue: function (condition, message) {
    if (!condition) {    // <- the breaking change here have been undone
        throw new Error(...);
    }
},
1
2
3
/usr/local/bin/node AssertTrueTest.js

Process finished with exit code 0

Now, let’s write a new test for the case, when assertTrue fails:

1
2
3
4
5
6
7
8
9
// test/AssertTrueTest.js

this.testFailure = function () {
    try {
        t.assertTrue(false);
    } catch (error) {
        t.assertEqual("Expected to be true, but got false", error.message);
    }
};

And if we run the test suite, it passes. If I was confident in the previous test, that I know why it didn’t fail, here I feel a bit uncomfortable about writing these 5 lines of test code and never seeing them fail. So let’s break the code once again!

1
2
3
4
5
6
7
// src/TestingFramework.js

assertTrue: function (condition, message) {
    if (!condition) {
        throw new Error(message || "oops");   // <- we have changed error message here
    }
},

And if we run tests, we get an expected failure:

1
2
3
4
5
6
7
8
9
/usr/local/bin/node AssertTrueTest.js
/path/to/project/src/TestingFramework.js:4
            throw new Error(message || "oops");
            ^

Error: Expected to equal Expected to be true, but got false, but got: oops
    .. stack trace ..

Process finished with exit code 1

And if we undo our breaking change in the implementation the test should pass:

1
2
3
4
5
6
7
8
// src/TestingFramework.js

assertTrue: function (condition, message) {
    if (!condition) {
        throw new Error(message || "Expected to be true, but got false");
        //                         ^ we have restored correct message ^
    }
},
1
2
3
/usr/local/bin/node AssertTrueTest.js

Process finished with exit code 0

And the last test for assertTrue to test the custom failure message:

1
2
3
4
5
6
7
8
9
// test/AssertTrueTest.js

this.testCustomFailureMessage = function () {
    try {
        t.assertTrue(false, "it is not true!");
    } catch (error) {
        t.assertEqual("it is not true!", error.message);
    }
};

If we run the test suite, it passes:

1
2
3
/usr/local/bin/node AssertTrueTest.js

Process finished with exit code 0

Even that I am confident enough, that this try { ... } catch (..) { ... } construction does the right thing, let’s be diligent about it and break the implementation of that functionality and see this test fail:

1
2
3
4
5
6
7
8
// src/TestingFramework.js

assertTrue: function (condition, message) {
    if (!condition) {
        throw new Error("Expected to be true, but got false");
        //              ^ 'message || ' was removed here   ^
    }
},

If we run the test suite:

1
2
3
4
5
6
7
8
9
/usr/local/bin/node AssertTrueTest.js
/path/to/project/src/TestingFramework.js:4
            throw new Error("Expected to be true, but got false");
            ^

Error: Expected to be true, but got false
    .. stack trace ..

Process finished with exit code 1

It fails, but this change breaks assertEqual too so that we don’t see any meaningful error message now. We can figure out if that is an expected failure or not by inspecting the stack trace:

1
2
3
4
5
6
7
8
9
10
11
Error: Expected to be true, but got false
    at Object.assertions.assertTrue (/path/to/project/src/TestingFramework.js:4:19)
    at Object.assertions.assertEqual (/path/to/project/src/TestingFramework.js:9:14)
    at testCustomFailureMessage (/path/to/project/test/AssertTrueTest.js:20:15)
    at runTestSuite (/path/to/project/src/TestingFramework.js:21:32)
    at Object.<anonymous> (/path/to/project/test/AssertTrueTest.js:3:1)
    at Module._compile (module.js:409:26)
    at Object.Module._extensions..js (module.js:416:10)
    at Module.load (module.js:343:32)
    at Function.Module._load (module.js:300:12)
    at Function.Module.runMain (module.js:441:10)

and precisely this line is the most interesting one:

1
at testCustomFailureMessage (/path/to/project/test/AssertTrueTest.js:20:15)

If we open the code at this stack trace frame, we will see:

1
t.assertEqual("it is not true!", error.message);

Great, this is exactly what we expected. Undoing the breaking change and running the test suite again:

1
2
3
4
5
6
7
8
// src/TestingFramework.js

assertTrue: function (condition, message) {
    if (!condition) {
        throw new Error(message || "Expected to be true, but got false");
        //              ^ 'message || ' was inserted here again       ^
    }
},
1
2
3
/usr/local/bin/node AssertTrueTest.js

Process finished with exit code 0

Interestingly enough, I know how to break the code now and all the tests will pass:

First, let’s extract local variable in assertTrue:

1
2
3
4
5
6
7
8
9
// src/TestingFramework.js

assertTrue: function (condition, message) {
    var errorMessage = message || "Expected to be true, but got false";

    if (!condition) {
        throw new Error(errorMessage);
    }
},

And that still passes its tests:

1
2
3
/usr/local/bin/node AssertTrueTest.js

Process finished with exit code 0

Now let’s expand message || into the proper if statement:

1
2
3
4
5
var errorMessage = "Expected to be true, but got false";

if (message) {
    errorMessage = message;
}

And that still passes its tests:

1
2
3
/usr/local/bin/node AssertTrueTest.js

Process finished with exit code 0

Now, what if I were to replace errorMessage = message; with errorMessage = "it is not true!"; inside of the if statement:

1
2
3
if (message) {
    errorMessage = "it is not true!";
}

The test still passes!

1
2
3
/usr/local/bin/node AssertTrueTest.js

Process finished with exit code 0

This is rather interesting. It seems, that we will have to make sure, that message parameter actually gets passed to new Error(...). We can do that by applying the Triangulation technique focusing on message parameter:

1
2
3
4
5
6
7
8
9
// test/AssertTrueTest.js

this.testCustomFailureMessage_withOtherMessage = function () {
    try {
        t.assertTrue(false, "should be true");
    } catch (error) {
        t.assertEqual("should be true", error.message);
    }
};

And if we run the test suite, we get our expected failure:

1
2
3
4
5
6
7
8
9
10
11
/usr/local/bin/node AssertTrueTest.js
/path/to/project/src/TestingFramework.js:10
            throw new Error(errorMessage);
            ^

Error: it is not true!
    ...
    at testCustomFailureMessage_withOtherMessage (/path/to/project/test/AssertTrueTest.js:28:15)
    ...

Process finished with exit code 1

And if we look at the stack trace frame, it points us to the correct line of code:

1
t.assertEqual("should be true", error.message);

And we can now fix the test by fixing the implementation:

1
2
3
4
5
6
// src/TestingFramework.js

// inside of `assertTrue` function:
if (message) {
    errorMessage = message;   // <- here we actually have to use 'message' now
}

And when we run the test suite, it passes:

1
2
3
/usr/local/bin/node AssertTrueTest.js

Process finished with exit code 0

I think we have all required tests for assertTrue now. This is great! And have you spotted the duplication already?

Refactoring AssertTrueTest

The duplication that is present here is our try { ... } catch (..) { ... } construct. Let’s make it a completely duplicate code by extracting couple of local variables:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
// first, let's extract the Action Under the Test variable:
var action = function () { t.assertTrue(false); };

try {
    action();
} catch (error) {
    t.assertEqual("Expected to be true, but got false", error.message);
}

// next, let's extract the Expected Error Message:
var expectedMessage = "Expected to be true, but got false";

try {
    action();
} catch (error) {
    t.assertEqual(expectedMessage, error.message);
}

If we apply the same refactoring for all tests in the AssertTrueTest suite, we will see the duplication:

1
2
3
4
5
try {
    action();
} catch (error) {
    t.assertEqual(expectedMessage, error.message);
}

Let’s give that operation a name and extract is as a function: assertThrow(expectedMessage, action):

1
2
3
4
5
6
7
function assertThrow(expectedMessage, action) {
    try {
        action();
    } catch (error) {
        t.assertEqual(expectedMessage, error.message);
    }
}

Use this function in all tests and inline all the extracted local variables:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
this.testFailure = function () {
    assertThrow("Expected to be true, but got false", function () {
        t.assertTrue(false);
    });
};

this.testCustomFailureMessage = function () {
    assertThrow("it is not true!", function () {
        t.assertTrue(false, "it is not true!");
    });
};

this.testCustomFailureMessage_withOtherMessage = function () {
    assertThrow("should be true", function () {
        t.assertTrue(false, "should be true");
    });
};

Function assertThrow seems to be useful for any test, not just AssertTrueTest suite. Let’s move it to the assertions object. We will be doing that by using Parallel Change technique:

  1. Create new functionality first;
  2. Step by step migrate all calls to use new functionality instead of old one;
  3. Once old functionality is not used, remove it.

The advantage of that method is that it consists of very small steps, that can be executed with confidence and each such small step never leaves the user in a red state (failing tests or compile/parsing errors).

Let’s see how this one can be applied here:

1. Create new functionality first - we can do it by copying the assertThrow function to assertions object:

1
2
3
4
5
6
7
8
9
10
11
12
13
// src/TestingFramework.js

var assertions = {
    // ...
    assertThrow: function (expectedMessage, action) {
        try {
            action();
        } catch (error) {
            // `t` needs to be changed to `this` here
            this.assertEqual(expectedMessage, error.message);
        }
    }
};

The tests should still pass:

1
2
3
/usr/local/bin/node AssertTrueTest.js

Process finished with exit code 0

2. Step by step migrate all calls to use new functionality instead of old one - we do it by calling assertThrow on t (our assertions object) in the test suite. Since we still haven’t removed the old assertThrow function, we can do that one function call at the time and the tests will always be green:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
this.testFailure = function () {
    t.assertThrow("Expected to be true, but got false", function () {
// ^ 't.' added here ^
        t.assertTrue(false);
    });
};

// run tests and they still pass.

this.testCustomFailureMessage = function () {
    t.assertThrow("it is not true!", function () {
// ^ 't.' added here ^
        t.assertTrue(false, "it is not true!");
    });
};

// run tests and they still pass.

this.testCustomFailureMessage_withOtherMessage = function () {
    t.assertThrow("should be true", function () {
// ^ 't.' added here ^
        t.assertTrue(false, "should be true");
    });
};

3. Once old functionality is not used, remove it. - now we can remove our assertThrow function defined inside of the AssertTrueTest suite and run tests:

1
2
3
/usr/local/bin/node AssertTrueTest.js

Process finished with exit code 0

And they pass. Let’s see the complete AssertTrueTest suite again:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
// test/AssertTrueTest.js
var runTestSuite = require("../src/TestingFramework");

runTestSuite(function (t) {
    this.testSuccess = function () {
        t.assertTrue(true);
    };

    this.testFailure = function () {
        t.assertThrow("Expected to be true, but got false", function () {
            t.assertTrue(false);
        });
    };

    this.testCustomFailureMessage = function () {
        t.assertThrow("it is not true!", function () {
            t.assertTrue(false, "it is not true!");
        });
    };

    this.testCustomFailureMessage_withOtherMessage = function () {
        t.assertThrow("should be true", function () {
            t.assertTrue(false, "should be true");
        });
    };
});

The only problem here is that we are relying on the untested assertThrow assertion here. Let’s unit-test it.

Testing assertThrow(expectedMessage, action)

Let’s create a new test suite with first test when assertThrow succeeds:

1
2
3
4
5
6
7
8
9
10
// test/AssertThrowTest.js
var runTestSuite = require("../src/TestingFramework");

runTestSuite(function (t) {
    this.testSuccess = function () {
        assertThrow("an error message", function () {
            throw new Error("an error message");
        });
    };
});

And the test should pass:

1
2
3
/usr/local/bin/node AssertThrowTest.js

Process finished with exit code 0

The code can be broken without test failure by not using expectedMessage parameter:

1
2
3
4
5
6
7
8
9
10
// src/TestingFramework.js

assertThrow: function (expectedMessage, action) {
    try {
        action();
    } catch (error) {
        this.assertEqual("an error message", error.message);
        // ^ here 'expectedMessage' was changed to constant ^
    }
}

Let’s triangulate the code to make sure expectedMessage is used correctly by adding a new test:

1
2
3
4
5
6
7
// test/AssertThrowTest.js

this.testSuccess_withDifferentExpectedMessage = function () {
    t.assertThrow("a different error message", function () {
        throw new Error("a different error message");
    });
};

And that test fails as expected:

1
2
3
4
5
6
7
8
/usr/local/bin/node AssertThrowTest.js
/path/to/project/src/TestingFramework.js:10
            throw new Error(errorMessage);
            ^

Error: Expected to equal an error message, but got: a different error message

Process finished with exit code 1

Undoing the breaking change (Mutation) will make the test pass:

1
2
3
/usr/local/bin/node AssertThrowTest.js

Process finished with exit code 0

Next test is to make sure, that assertThrow is actually comparing actual and expected error messages correctly:

1
2
3
4
5
6
7
8
9
// test/AssertThrowTest.js

this.testFailure = function () {
    t.assertThrow("Expected to equal an error message, but got: a different error message", function () {
        t.assertThrow("an error message", function () {
            throw new Error("a different error message");
        });
    });
};

And it passes. The last test, that assertThrow needs is the case, when action() is not throwing any error. In that case assertThrow should fail:

1
2
3
4
5
6
7
8
9
// test/AssertThrowTest.js

this.testFailure_whenActionDoesNotThrow = function () {
    t.assertThrow("Expected to throw an error, but nothing was thrown", function () {
        t.assertThrow("an error message", function () {
            // does nothing
        });
    });
};

Oh, and that test is passing! We clearly don’t have that functionality yet. We have to add another test without usage of outer t.assertThrow to make sure that we get a test failure:

1
2
3
4
5
6
7
8
9
10
11
12
13
this.testThrows_whenActionDoesNotThrow = function () {
    var hasThrown = false;

    try {
        t.assertThrow("an error message", function () {
            // does nothing
        });
    } catch (error) {
        hasThrown = true;
    }

    t.assertTrue(hasThrown, "it should have thrown");
};

And if we run our tests we get the expected failure:

1
2
3
4
5
6
7
8
/usr/local/bin/node AssertThrowTest.js
/path/to/project/src/TestingFramework.js:10
            throw new Error(errorMessage);
            ^

Error: it should have thrown

Process finished with exit code 1

We can fix that by verifying, that catch block is executed in the assertThrow function:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
// src/TestingFramework.js

assertThrow: function (expectedMessage, action) {
    var hasThrown = false;  // <- we initialize hasThrown here

    try {
        action();
    } catch (error) {
        hasThrown = true;   // <- and we set it to true in the catch block
        this.assertEqual(expectedMessage, error.message);
    }

    this.assertTrue(hasThrown);  // <- and we check that it is true
}

And now if we run the test suite, the original test that we were trying to write fails as expected:

1
2
3
4
5
6
7
8
/usr/local/bin/node AssertThrowTest.js
/path/to/project/src/TestingFramework.js:10
            throw new Error(errorMessage);
            ^

Error: Expected to equal Expected to throw an error, but nothing was thrown, but got: Expected to be true, but got false

Process finished with exit code 1

Now we need to fix the custom message provided to assertTrue call inside of assertThrow:

1
2
3
4
// src/TestingFramework.js

// inside of assertThrow:
this.assertTrue(hasThrown, "Expected to throw an error, but nothing was thrown");

And the test pass:

1
2
3
/usr/local/bin/node AssertThrowTest.js

Process finished with exit code 0

Great, I think we are done with testing the assertThrow function. Let’s see the whole AssertThrowTest suite:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
// test/AssertThrowTest.js
var runTestSuite = require("../src/TestingFramework");

runTestSuite(function (t) {
    this.testSuccess = function () {
        t.assertThrow("an error message", function () {
            throw new Error("an error message");
        });
    };

    this.testSuccess_withDifferentExpectedMessage = function () {
        t.assertThrow("a different error message", function () {
            throw new Error("a different error message");
        });
    };

    this.testFailure = function () {
        t.assertThrow("Expected to equal an error message, but got: a different error message", function () {
            t.assertThrow("an error message", function () {
                throw new Error("a different error message");
            });
        });
    };

    this.testFailure_whenActionDoesNotThrow = function () {
        t.assertThrow("Expected to throw an error, but nothing was thrown", function () {
            t.assertThrow("an error message", function () {
                // does nothing
            });
        });
    };

    this.testThrows_whenActionDoesNotThrow = function () {
        var hasThrown = false;

        try {
            t.assertThrow("an error message", function () {
                // does nothing
            });
        } catch (error) {
            hasThrown = true;
        }

        t.assertTrue(hasThrown, "it should have thrown");
    };
});

Last one for today is assertEqual:

Testing assertEqual(expected, actual)

Let’s create a test suite with the first test, when assertEqual succeeds:

1
2
3
4
5
6
7
8
// test/AssertEqualTest.js
var runTestSuite = require("../src/TestingFramework");

runTestSuite(function (t) {
    this.testSuccess = function () {
        t.assertEqual(42, 42);
    };
});

This test passes, and it can be broken without test failure by always comparing to 42:

1
2
3
4
5
6
7
8
// src/TestingFramework.js

assertEqual: function (expected, actual) {
    this.assertTrue(
        42 == actual,   // <- here 'expected' is replaced with constant
        "Expected to equal " + expected + ", but got: " + actual
    );
},

Triangulation to fix that:

1
2
3
4
5
6
7
8
9
10
11
// test
this.testSuccess_whenExpectedIsDifferent = function () {
    t.assertEqual(29, 29);
};

// Error: Expected to equal 29, but got: 29

// fix implementation:
expected == actual,   // <- here 'expected' is restored

// and the test passes

Next mutation that does not break any tests looks this way:

1
2
3
4
5
6
7
8
9
// src/TestingFramework.js

assertEqual: function (expected, actual) {
    this.assertTrue(
        expected == actual,
        // "Expected to equal " + expected + ", but got: " + actual
        "oops"    // <- replace error message
    );
},

Let’s add the test to protect from this kind of mutation:

1
2
3
4
5
this.testFailure = function () {
    t.assertThrow("Expected to equal 42, but got: 29", function () {
        t.assertEqual(42, 29);
    });
};

This fails with Error: oops, because assertThrow uses assertEqual. Stack trace shows, that the failure is happening here:

1
2
// src/TestingFramework.js in assertThrow
this.assertEqual(expectedMessage, error.message);

So that is the expected failure. We can fix it by always providing the message "Expected to equal 42, but got: 29":

1
2
3
4
5
6
7
8
// src/TestingFramework.js in assertions
assertEqual: function (expected, actual) {
    this.assertTrue(
        expected == actual,
        "Expected to equal 42, but got: 29"
    // ^ here the exact constant is used ^
    );
},

This needs some more triangulation:

1
2
3
4
5
6
7
8
9
10
11
12
13
this.testFailure_withDifferentExpectedAndActual = function () {
    t.assertThrow("Expected to equal 94, but got: 1027", function () {
        t.assertEqual(94, 1027);
    });
};

// Error: Expected to equal 42, but got: 29

// fix:
this.assertTrue(
    expected == actual,
    "Expected to equal " + expected + ", but got: " + actual
);

I think we are done now with testing the assertEqual function. The AssertEqualTest suite is looking like that now:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
// test/AssertEqualTest.js
var runTestSuite = require("../src/TestingFramework");

runTestSuite(function (t) {
    this.testSuccess = function () {
        t.assertEqual(42, 42);
    };

    this.testSuccess_whenExpectedIsDifferent = function () {
        t.assertEqual(29, 29);
    };

    this.testFailure = function () {
        t.assertThrow("Expected to equal 42, but got: 29", function () {
            t.assertEqual(42, 29);
        });
    };

    this.testFailure_withDifferentExpectedAndActual = function () {
        t.assertThrow("Expected to equal 94, but got: 1027", function () {
            t.assertEqual(94, 1027);
        });
    };
});

Final assertions object after all the testing

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
// src/TestingFramework.js
var assertions = {
    assertTrue: function (condition, message) {
        var errorMessage = "Expected to be true, but got false";

        if (message) {
            errorMessage = message;
        }

        if (!condition) {
            throw new Error(errorMessage);
        }
    },

    assertEqual: function (expected, actual) {
        this.assertTrue(
            expected == actual,
            "Expected to equal " + expected + ", but got: " + actual
        );
    },

    assertThrow: function (expectedMessage, action) {
        var hasThrown = false;

        try {
            action();
        } catch (error) {
            hasThrown = true;
            this.assertEqual(expectedMessage, error.message);
        }

        this.assertTrue(
            hasThrown,
            "Expected to throw an error, but nothing was thrown"
        );
    }
};

Bottom Line

Congratulations! We have completely unit-tested our assertions assertTrue and assertEqual. This resulted in natural emergence of the new assertion - assertThrow. We have unit-tested it too!

Additionally, we have practiced usage of Mutational Testing and Triangulation Technique to detect missing test cases and derive them from the code. Also, we have slightly touched the Parallel Change refactoring technique - we will see more of that in the future in these series.

The code is available on Github: https://github.com/waterlink/BuildYourOwnTestingFrameworkPart2

Now that we have unit-tested some basic assertions, we should unit-test our testing framework test runner: runTestSuite function. This will be covered in next series of “Build Your Own Testing Framework”. Stay tuned!

Thanks!

Thank you for reading, my dear reader. If you liked it, please share this article on social networks and follow me on twitter: @waterlink000.

If you have any questions or feedback for me, don’t hesitate to reach me out on Twitter: @waterlink000.

Build Your Own Testing Framework

build-your-own-testing-framework, javascript, tdd, test-driven-development, testing, xunit

Today we are going to test-drive the testing framework without any external testing framework. This will be done through test-driving a simple kata (FizzBuzzKata). For example:

  • every time we expect a test to fail and it doesn’t, this is a failing test for our testing framework, that we will be fixing,
  • every time we expect a test to pass and it doesn’t, this is another failing test for our testing framework, that we will be fixing.

For practical reasons, today we are going to use concrete programming language instead of pseudo-code - javascript. Except for small details, that we will point out, the techniques shown here are language-agnostic.

This article is only first one of the series “Build Your Own Testing Framework”, so make sure to stick around for next parts! All articles of these series can be found here: http://www.tddfellow.com/blog/categories/build-your-own-testing-framework/.

Shall we begin?

FizzBuzzKata

Given the number,

  • return Fizz when the number is divisible by 3,
  • return Buzz when the number is divisible by 5,
  • return FizzBuzz when the number is divisible by 3 and 5,
  • return string representation of number otherwise.

Writing your first test

How do we write our first test, when we don’t have a testing framework and we want to create one? - It seems, that we have to design how the test should like in our brand new testing framework.

I personally, would go with the xUnit-like design, since it is relatively simple. Given this, we might write our first test and it will look something like that:

1
2
3
4
5
6
7
// test/FizzBuzzKataTest.js

function FizzBuzzKataTest() {
    this.testNormalNumberIsReturned = function() {
        this.assertTrue("1" === fizzBuzz(1));
    };
}

This test should fail, because function fizzBuzz is not defined, but it doesn’t fail, since function testNormalNumberIsReturned is never called. In fact, the object with FizzBuzzKataTest is never being created.

Easiest way to solve that:

1
2
3
4
5
6
// test/FizzBuzzKataTest.js

function FizzBuzzKataTest() { ... }

var test = new FizzBuzzKataTest();
test.testNormalNumberIsReturned();

If we run this code with node:

1
node test/FizzBuzzKataTest.js

We will get the expected error:

1
2
3
4
5
/path/to/project/test/FizzBuzzKataTest.js:3
        this.assertTrue("1" === fizzBuzz(1));
                               ^

ReferenceError: fizzBuzz is not defined

So, let’s define this function:

1
2
3
4
5
// test/FizzBuzzKataTest.js

function FizzBuzzKataTest() { ... }

function fizzBuzz() {}

If we run our test again, we will get the following error:

1
2
3
4
5
/path/to/project/test/FizzBuzzKataTest.js:3
        this.assertTrue("1" === fizzBuzz(1));
             ^

TypeError: this.assertTrue is not a function

Clearly, to fix it we need to define assertTrue on FizzBuzzKataTest object. Obviously, we do not want our user to define all their assertion for every test suite. This means, that we want to define it on FizzBuzzKataTest object outside of the definition of FizzBuzzKataTest.

There are two ways to go about it:

  • inheritance: make FizzBuzzKataTest inherit from some other object function assertTrue, or
  • composition: make FizzBuzzKataTest accept a special object with function assertTrue defined on it.

I would like to go with composition method since it gives us more flexibility in the long run:

1
function FizzBuzzKataTest(t) { ... }

and the usage of assertTrue has to change appropriately:

1
t.assertTrue("1" === fizzBuzz(1));

and t has to be created and passed in correctly:

1
2
3
4
5
6
7
8
9
function FizzBuzzKataTest(t) { ... }

function fizzBuzz(number) {}

var assertions = {
    assertTrue: function(condition) {}
};

var test = new FizzBuzzKataTest(assertions);

If we run the test suite again, we will not get any failure anymore. But we were expecting assertTrue to fail, so let’s make it fail:

1
2
3
assertTrue: function(condition) {
    throw new Error("Expected to be true, but was false");
}

When we run the test suite, we get:

1
2
3
4
5
/path/to/project/test/FizzBuzzKataTest.js:11
        throw new Error("Expected to be true, but got false");
        ^

Error: Expected to be true, but got false

Now, let’s customize the error message a bit:

1
2
3
4
5
6
7
8
9
this.testNormalNumberIsReturned = function() {
    t.assertTrue("1" === fizzBuzz(1), "Expected to equal " + "1" + ", but got: " + fizzBuzz(1));
}

// ...

assertTrue: function(condition, message) {
    throw new Error(message || "Expected to be true, but got false");
}

When running this, we are getting the expected error:

1
2
3
4
5
/path/to/project/test/FizzBuzzKataTest.js:11
        throw new Error(message || "Expected to be true, but got false");
        ^

Error: Expected to equal 1, but got: undefined

This looks better now. Let’s fix the error now by implementing the simplest thing that could work:

1
2
3
function fizzBuzz(number) {
    return "1";
}

And as we run our test suite we get:

1
2
3
4
5
/path/to/project/test/FizzBuzzKataTest.js:13
        throw new Error(message || "Expected to be true, but got false");
        ^

Error: Expected to equal 1, but got: 1

Oh, it should have passed the test. I know why it didn’t: we throw this error unconditionally, let’s add an appropriate if statement to assertTrue function:

1
2
3
4
5
assertTrue: function(condition, message) {
    if (!condition) {
        throw new Error(...);
    }
}

If we run this code, it does not fail. That was our first green state - it took as awhile to get here. The reason for this is that we are not only test-driving FizzBuzzKata, additionally, we are writing a feature test for a non-existing testing framework. Now that we are green, we should think about refactoring, i.e.: making the structure of our code right.

Obviously, we should move our testing framework code outside of the test suite file. Probably, somewhere in src/TestingFramework.js. For that, we need to first parametrize FizzBuzzKataTest and extract the function to run the test suite.

Parametrize:

1
2
3
var testSuiteConstructor = FizzBuzzKataTest;
var testSuite = new testSuiteConstructor(assertions);
testSuite.testNormalNumberIsReturned();

and extract method:

1
2
3
4
5
6
7
function runTestSuite(testSuiteConstructor) {
    var testSuite = new testSuiteConstructor(assertions);
    testSuite.testNormalNumberIsReturned();
}

var testSuiteConstructor = FizzBuzzKataTest;
runTestSuite(testSuiteConstructor);

and inline the variable testSuiteConstructor:

1
runTestSuite(FizzBuzzKataTest);

Now it is time to move testing code to src/TestingFramework.js:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
// src/TestingFramework.js

var assertions = {
    assertTrue: function (condition, message) {
        if (!condition) {
            throw new Error(message || "Expected to be true, but got false");
        }
    }
};

function runTestSuite(testSuiteConstructor) {
    var testSuite = new testSuiteConstructor(assertions);
    testSuite.testNormalNumberIsReturned();
}

And to be able to require runTestSuite function:

1
2
3
4
5
6
7
// src/TestingFramework.js

var assertions = { ... };

function runTestSuite(testSuiteConstructor) { ... }

module.exports = runTestSuite;

And, finally, let’s use that from our test suite:

1
2
3
4
5
6
7
8
9
// test/FizzBuzzKataTest.js

var runTestSuite = require("../src/TestingFramework");

function FizzBuzzKataTest(t) { ... }

function fizzBuzz(number) { ... }

runTestSuite(FizzBuzzKataTest);

If we run the test suite again, everything should pass. Somehow, I don’t feel comfortable now, let’s try to break the test suite and see if it will fail as expected:

1
2
3
function fizzBuzz(number) {
    return "2";  // <-- "1" was changed to "2" here
}

And run tests:

1
2
3
4
5
/path/to/project/src/TestingFramework.js:4
            throw new Error(message || "Expected to be true, but got false");
            ^

Error: Expected to equal 1, but got: 2

Yes, it still works as expected. We have just introduced a Mutation to our code, to see if it is still tested properly. Let’s undo the Mutation and see the test still pass. And it does.

If you look closely now, it should be possible to inline FizzBuzzKataTest definition as an argument of runTestSuite call:

1
2
3
4
5
var runTestSuite = require(...);

function fizzBuzz(number) { ... }

runTestSuite(function (t) { ... });

And if we run our test suite, it still works. Just to check, that we are still good, let’s repeat our Mutation from the previous step. It should still fail as expected. And it does. Undo the mutation and the test is still passing. Great.

I think we are done with Refactoring step, for now, let’s get back to writing another failing test.

Writing the second test

1
2
3
this.testAnotherNormalNumberIsReturned = function() {
    t.assertTrue("2" === fizzBuzz(2), "Expected to equal " + "2" + ", but got: " + fizzBuzz(2));
};

If we run these tests, they do not fail. This is strange, let’s look at runTestSuite function again:

1
2
3
4
function runTestSuite(testSuiteConstructor) {
    var testSuite = new testSuiteConstructor(assertions);
    testSuite.testNormalNumberIsReturned();
}

Great, it just runs one specific function, we should probably run all functions starting from test instead:

1
2
3
4
5
6
7
8
9
function runTestSuite(testSuiteConstructor) {
    var testSuite = new testSuiteConstructor(assertions);

    for (var testName in testSuite) {
        if (testName.match(/^test/)) {
            testSuite[testName]();
        }
    }
}

REMARK: this code is Javascript specific. Other programming languages will have their own way of iterating over the function/method list and calling a function by its name. Usually, it is some sort of reflection for compiled languages and meta-programming features for interpreted languages.

If we run tests now, we get the expected failure:

1
2
3
4
5
/path/to/project/src/TestingFramework.js:4
            throw new Error(message || "Expected to be true, but got false");
            ^

Error: Expected to equal 2, but got: 1

If we try to change return "1" to return "2", of course this test will pass, but the other will fail:

1
2
3
4
5
/path/to/project/src/TestingFramework.js:4
            throw new Error(message || "Expected to be true, but got false");
            ^

Error: Expected to equal 1, but got: 2

This is great for couple of reasons:

  • It validates, that our change to how test* functions are discovered is correct, and
  • We have to have a bit smarter implementation to pass both tests now:
1
2
3
function fizzBuzz(number) {
    return number.toString();
}

And if we run the tests, they pass. Now, that we are in Green state, we should start refactoring. Have you noticed the duplication already?

1
2
3
4
5
t.assertTrue("1" === fizzBuzz(1), "Expected to equal " + "1" + ", but got: " + fizzBuzz(1));

// and:

t.assertTrue("2" === fizzBuzz(2), "Expected to equal " + "2" + ", but got: " + fizzBuzz(2));

Extracting "1" and "2" as variable expected, and fizzBuzz(1) and fizzBuzz(2) as variable actual, makes these 2 lines identical:

1
2
3
4
5
6
7
8
9
10
11
this.testNormalNumberIsReturned = function () {
    var expected = "1";
    var actual = fizzBuzz(1);
    t.assertTrue(expected === actual, "Expected to equal " + expected + ", but got: " + actual);
};

this.testAnotherNormalNumberIsReturned = function() {
    var expected = "2";
    var actual = fizzBuzz(2);
    t.assertTrue(expected === actual, "Expected to equal " + expected + ", but got: " + actual);
};

Specifically, this is identical:

1
t.assertTrue(expected === actual, "Expected to equal " + expected + ", but got: " + actual);

This sounds like t.assertEqual(expected, actual) to me. So let’s extract it:

1
2
3
4
5
6
7
8
9
10
11
12
// src/TestingFramework.js

var assertions = {
    assertTrue: function(condition, message) { ... },

    assertEqual: function(expected, actual) {
        this.assertTrue(
          expected === actual,
          "Expected to equal " + expected + ", but got: " + actual
        );
    }
}

Now, let’s use it and inline our expected and actual variables:

1
2
3
4
5
6
7
this.testNormalNumberIsReturned = function () {
    t.assertEqual("1", fizzBuzz(1));
};

this.testAnotherNormalNumberIsReturned = function() {
    t.assertEqual("2", fizzBuzz(2));
};

This looks much more readable. If we run tests, they still pass. If we try to break our code by using some Mutation, the tests fail as expected. Great, our refactoring was a success!

Let’s finish test-driving our fizzBuzz function.

Test-Driving Fizz Buzz Kata

First test for Fizz case:

1
2
3
4
5
6
7
8
9
10
11
12
// test:
this.testFizzIsReturned = function () {
    t.assertEqual("Fizz", fizzBuzz(3));
};

// Error: Expected to equal Fizz, but got: 3

// implementation:
function fizzBuzz(number) {
    if (number === 3) return "Fizz";
    return number.toString();
}

This is pretty stupid implementation, but it works for that one tests, so let’s write the test, that will break this implementation and force us to write real if condition:

1
2
3
4
5
6
7
8
9
10
11
12
// test:
this.testFizzIsReturnedForDifferentNumber = function () {
    t.assertEqual("Fizz", fizzBuzz(6));
};

// Error: Expected to equal Fizz, but got: 6

// implementation:
function fizzBuzz(number) {
    if (number % 3 === 0) return "Fizz";
    return number.toString();
}

This technique is called Triangulation:

  • the first test is to force us to write some if statement with a correct body,
  • second is to force us to make the condition right.
  • If we had an else clause, we would have had another test to make that part right.

OK, that looks like a right implementation for Fizz, let’s write the test for Buzz now:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
// test:
this.testBuzzIsReturned = function () {
    t.assertEqual("Buzz", fizzBuzz(5));
};

// Error: Expected to equal Buzz, but got: 5

// stupid implementation:
function fizzBuzz(number) {
    if (number === 5) return "Buzz";
    if (number % 3 === 0) return "Fizz";
    return number.toString();
}

// Triangulation:
this.testBuzzIsReturnedForDifferentNumber = function () {
    t.assertEqual("Buzz", fizzBuzz(10));
};

// Error: Expected to equal Buzz, but got: 10

// correct implementation:
function fizzBuzz(number) {
    if (number % 5 === 0) return "Buzz";
    if (number % 3 === 0) return "Fizz";
    return number.toString();
}

And finally let’s implement final requirement FizzBuzz:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
// test:
this.testFizzBuzzIsReturned = function () {
    t.assertEqual("FizzBuzz", fizzBuzz(15));
};

// Error: Expected to equal FizzBuzz, but got: Buzz

// stupid implementation:
function fizzBuzz(number) {
    if (number === 15) return "FizzBuzz";
    if (number % 5 === 0) return "Buzz";
    if (number % 3 === 0) return "Fizz";
    return number.toString();
}

// Triangulation:
this.testFizzBuzzIsReturnedForDifferentNumber = function () {
    t.assertEqual("FizzBuzz", fizzBuzz(30));
};

// Error: Expected to equal FizzBuzz, but got: Buzz

// correct implementation:
function fizzBuzz(number) {
    if (number % 15 === 0) return "FizzBuzz";
    if (number % 5 === 0) return "Buzz";
    if (number % 3 === 0) return "Fizz";
    return number.toString();
}

I think we are done with the implementation. FizzBuzzKata has an extended set of requirements, but they are out of the scope of this article. These requirements force us to introduce Strategy pattern and stop using this unmaintainable chain of if statements.

Refactoring this code to Strategy pattern is left as an exercise for the reader.

Bottom Line

Congratulations! Using FizzBuzzKata we have test-driven bare-bones testing framework to the point, that we can do Test-Driven Development for a simple Kata. And all that without having any testing framework in place.

The code is available on Github: https://github.com/waterlink/BuildYourOwnTestingFrameworkPart1

Now, with this minimal framework in place, it should be possible to unit-test the framework itself, so that we can support more use cases. This will be covered in next series of “Build Your Own Testing Framework”. Stay tuned!

Thanks!

Thank you for reading, my dear reader. If you liked it, please share this article on social networks and follow me on twitter: @waterlink000.

If you have any questions or feedback for me, don’t hesitate to reach me out on Twitter: @waterlink000.

TL;DR on Sustainable Software Development Paper

code-ownership, extreme-programming, grounded-theory, sustainable-software-development, tldr, white-paper

This is a TL;DR of a white paper Sustainable Software Development through Overlapping Pair Rotation that is created by Todd Sedano, Paul Ralph, and Cécile Péraire.

NOTE: this TL;DR includes a lot of compressed awesome information.

Theory of Sustainable Software Development

The paper introduces us to principles, policies, and practices, that are the results of applying of Constructivist Grounded Theory research method to the five projects at Pivotal Labs.

Principles

  1. By Engendering Positive Attitudes Toward Disruption teams can transform a challenge of team disruption into a business opportunity:
    • team members rolling off from the project are replaced with new members,
    • new members are viewed as a source of fresh perspective and as an opportunity,
    • new members often questioned team’s assumptions (challenging “cargo culting”).
  2. Policies & practices that Encourage Knowledge Sharing and Continuity mitigate the risk of significant knowledge loss for the project when the team is disrupted. Policies are:
    • Team Code Ownership, and Shared Team Schedule. While practices are:
    • Continuous Pair Programming, Overlapping Pair Rotation and Knowledge Pollination.
  3. Sustainable development is not possible if the team is incurring the technical debt. Caring about Code Quality is a way to mitigate this risk. Policy here is:
    • Avoid Technical Debt, and practices are:
    • Test-Driven Development / Behavior-Driven Development and Continuous Refactoring.

Policies

  1. Team Code Ownership: every developer is empowered to work on and to refactor any part of the team’s code.
  2. Shared Schedule: every team member has agreed to a fixed schedule to achieve the benefits of Sustainable Software Development. Teams are, preferably, co-located.
  3. Avoid Technical Debt: a pair creates well-crafted code without shortcuts and short-term fixes. Solutions are best and simplest for current present without over-engineering. When inherited a legacy code base, a team is actively paying down technical debt while delivering new features.

Removing Knowledge Silos Practices

  1. Continuous Pair Programming: two developers are collaborating to write software together as their normal mode of software development. It enables knowledge transfer, reduces knowledge silos and improves code quality. Developers always work in pairs, unless exceptional circumstances arise.
  2. Overlapping Pair Rotation: one developer rolls off the track of work and another developer rolls on, keeping the continuity of one developer at each rotation. It results in knowledge continuity. There are three strategies for spreading the knowledge via pair rotation when fighting emerging knowledge silos:
    • Optimizing for people rotation: developers try to pair with the person they “least recently paired with”;
    • Optimizing for personal preferences: developer pick with whom they will pair at each rotation, based on personal preferences;
    • Optimizing for context sharing: a developer who has not been working on the track for the longest track is rotating onto the track. The goal each day is for the developer leaving the track next to empower the developer who will remain on the track.
  3. Knowledge Pollination:
    • Daily stand-ups create awareness of who is working on what.
    • Using backlog to communicate information and status of stories.
    • Osmotic communication: a developer overhears a discussion in other pair and offers their help.
    • The pair can interrupt another pair, product manager or designer to gain the needed information.
    • Calling out updates to the entire team.
    • Interruptions are encouraged - it makes the team more efficient as knowledge pollinates across the team.

Caretaking the Code Practices

  1. Test-Driven Development/Behavior-Driven Development: developers use BDD (to describe interactions between the user and the system) and TDD (at unit test level). Design emerges from the creation and exploration of the test cases. Teams use a variety of TDD strategies, including:
    • testing the responsibilities and interactions, or
    • contract testing using mocks.
  2. Continuous Refactoring: developers do refactoring as part of implementing feature stories. Developers are encouraged to improve the code’s design, make the code easier to read, understand, and increase the discoverability of a component based on its responsibility. The team prefers pre-refactoring, where the developer makes a complicated work to make the current story as simple and as easy as possible.
  3. Live on Master: every pair merges their code to master many times a day. Developers may use branches to save “spikes” and to move work-in-progress between pairing stations when rotating.

Thanks!

This is where this TL;DR ends. If you are interested in the topic, go ahead and read the paper, it has the following:

  • Detailed description of the research method, data collection, and analysis;
  • Research context and one project case study;
  • Detailed description of Principles, Policies, and Practices of Sustainable Software Development; also anti-patterns for each policy and practice;
  • Theory evaluation and conclusions.

Great thanks to the original authors of the paper, researchers, and all participants!

Why I Don’t Use Mocking Frameworks Anymore

architecture, clean-code, mocking, pseudo-code, testing

Some time ago, I have discovered, that using your own custom test double classes, instead of a test framework makes my test code more readable and maintainable. Here is an example (pseudo-code):

1
2
3
4
5
6
7
8
function test_password_revealer_when_toggled_reveals_password() {
  passwordController = MockPasswordController.new()
  passwordRevealer = PasswordRevealer.new(passwordController)

  passwordRevealer.toggle()

  expect(passwordController.isRevealed()).toBeTrue()
}

The same test with mocking framework would look this way:

1
2
3
4
5
6
7
8
9
10
function test_password_revealer_when_toggled_reveals_password() {
  passwordController = MockFramework.Mock.new("PasswordController")
  passwordRevealer = PasswordRevealer.new(passwordController)

  spy = spyOn(passwordController.reveal())

  passwordRevealer.toggle()

  expect(spy.haveBeenCalled()).toBeTrue()
}

If you take a closer look at the last example, and, specifically at these 2 lines:

1
2
3
spy = spyOn(passwordController.reveal())

expect(spy.haveBeenCalled()).toBeTrue()

They use a language, that is not relevant to the domain, therefore, they make the test less readable.

Additionally, they have knowledge of which exactly method PasswordRevealer#toggle() should call on passwordController. If we were to rename reveal method, all tests for PasswordRevealer would fail. The same thing would happen if we were to extract methods/functions/objects out of the PasswordRevealer.

Of course, creating such test doubles yourself will involve some boilerplate code - this is a trade-off. Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
class MockPasswordController() {
  this.state = "hidden"

  this.reveal() {
    this.state = "revealed"
  }

  this.hide() {
    this.state = "hidden"
  }

  this.isRevealed() {
    return this.state == "revealed"
  }
}

Making this trade-off, will simplify the case when we were to change the name: we would change the name at 3 places:

  • in test double class,
  • in caller class,
  • in “real” implementation class.

Whereas with a mocking framework, we would have to hunt for all failing tests, and usually, it means hundreds of failing tests.

Thank you!

If you, my dear reader, have any thoughts, questions or arguments about the topic, feel free to reach out to me on twitter: @waterlink000.

If you liked my ideas, follow me on twitter, and, even better, provide me with your honest feedback, so that I can improve.

TDD as an Enabling Practice or How to Be Faster With TDD

agile, continuous-delivery, continuous-integration, dreyfus-model, enabling-practices, exploitative-practices, pair-programming, practices, professionalism, skills, tdd, xp

Recently I had a lot of conversations with so many different programmers, with different backgrounds, contexts of work and opinions. What stroke me the most, is that majority feel that TDD is slower than simple automated testing, i.e., TestFirst is slower than TestAfter.

While digging deeper in their context of work, I could only agree with them: “True, in that context it will be about 50% slower”.

Most of the time, though, the context is somewhat looking like this:

  • We have some company ACME, that does some sort of Agile Software Development (probably Scrum);
  • The company ACME focuses only on the business parts of Agile Software Development;
  • The company ACME tried to introduce TDD as a development practice;
  • Everything took much longer to be done.

Enabling practice

Now, let us dive into the development practices of Agile Software Development: development practices are usually coming from Extreme Programming (XP). In XP there are 2 terms: EnablingPractice and ExploitativePractice.

Exploitative practice gives direct benefit to the team, e.g.: speed boost and quicker feedback from users and stakeholders.

Enabling practice is required for certain other exploitative practice(s) to work.

A good example of exploitative practice is Continuous Delivery. It requires Continuous Integration, Pair-Programming, and Testing to be in place. These 3 are enabling practices.

Removing slow practices

Additionally to allowing usage of practices, that make a team go faster and deliver at the higher quality level, enabling practices allow removal of practices, that make a team go slower. For example, Pair-Programming together with TDD allows removal of code review. On most of the teams (especially, of bigger size), this makes for an instant productivity boost.

Pair-Programming, TDD and Continuous Integration, additionally to enabling the team to do Continuous Delivery, also allows replacing feature-branch VCS flow with a trunk-based flow. This allows for smaller iterations and faster user feedback.

Pair-Programming Done Right

It is worth noting, that removal of code review and introducing of Continuous Delivery is only possible, if Pair-Programming is done right:

  • in no case, two beginners should be working in the pair;
  • beginners should work together with advanced beginners/competents and proficients/experts;
  • advanced beginners/competents should split their time in half between working with beginners and working with proficient/experts.

Terminology Beginner, AdvancedBeginner, Competent, Proficient and Expert is from Dreyfus Skill Acquisition Model.

That allows for a good trust and mentorship models in your team(s). It enables quick growth and knowledge sharing for every member of the team.

There is another enabling practice which speeds up the knowledge sharing, it is Pair Rotation, that should be done from 1 to 2 times per day, so that for small and middle-sized teams, the bigger the feature is, the higher chance, that everyone on the team have participated in its development, and therefore have enough knowledge about it.

Additionally, Pair Rotation allows for Code Detachment and removal of Code Silos. This in turn, together with TDD, enables Ruthless Refactoring, because you are not afraid:

  • to break the code, thanks to TDD,
  • to upset an owner of the Code Silo, because there is no owner, thanks to Pair Rotation.

Bottomline

I think I can go on like this forever, but I believe the idea should be clear. I will be following up with more articles in details on each technique in the future. Stay tuned!

Thank you for reading!

If you, my dear reader, have any thoughts, questions or arguments about the topic, feel free to reach out to me on twitter: @waterlink000.

If you liked my ideas, follow me on twitter, and, even better, provide me with your honest feedback, so that I can improve.

Automating Manual Checks or How to Save Time and Get Rapid Feedback

automation, programming, pseudo-code, tests

It is tedious, slow and error-prone to test correctness of program manually. This article is an introduction, how to make this process as simple as a hit of one button, very fast and precise.

In the last issue about Iterative Divide & Conquer we have found out, how it is tedious to manually check that our program works correctly.

We have concluded, that it is tedious and slow to manually check that the output of our print statements is correct. Additionally, it is really easy to make a mistake and to oversee incorrect lines of the output, especially, when we have tons of them. As well, this applies to any other form of manual testing, including testing the application as an End User (via clicking/tapping for UI applications, via executing tons of commands for CLI applications).

Mitigating human error-proneness via automation

Let’s get back to our example from the previous issue:

1
2
3
4
5
print english_integer_in_tens(20)    # twenty
print english_integer_in_tens(27)    # twenty-seven
print english_integer_in_tens(42)    # forty-two
print english_integer_in_tens(70)    # seventy
print english_integer_in_tens(99)    # ninety-nine

What if we could ask our computer to run english_integer_in_tens function with provided arguments and check that the output is exactly what we provided in a comment?

Let’s try to put it in pseudo-code:

1
2
equal(english_integer_in_tens(20), "twenty")
equal(english_integer_in_tens(42), "forty-two")

If we run that code, of course, we will get some sort of compilation error, or runtime error (depending on the programming language), because function equal is not defined yet. Let’s define it! Presumably, it should be comparing left argument with right argument and printing something useful on the screen.

1
2
3
4
5
6
7
function equal(left, right) {
  if (left != right) {
    print("Failure: " + left + " should be equal to " + right)
  } else {
    print("OK")
  }
}

If we run the program after defining this function we will get:

1
2
OK
OK

If we were to break the function english_integer_in_tens’s implementation, we might get something like:

1
2
OK
Failure: twenty should be equal to forty-two

Making automation nicer

Having all this output every time we import our small library in any application going to the standard output would be annoying to say the least. How about separating the test automation from the library?

Let’s extract all our testing print-s into the separate file, import original file from it and replace all print-s with equal-s:

1
2
3
4
5
6
7
8
9
import english_integer

function equal(left, right) { ... }

equal(english_number(0), "zero")
equal(english_number(1), "one")
...
equal(english_integer(43), "forty-three")
equal(english_integer(-97), "minus ninety-seven")

Running only this separate file will produce expected output:

1
2
3
4
5
OK
OK
...
OK
OK

Importing or running directly our original library will not produce any output. This is much nicer.

Thank you for reading!

Today we have made our testing:

  • easier: hit of one button, or one command on the terminal;
  • faster: from minutes of manual verification to milliseconds of automated checks;
  • preciser: automated checks can’t make a mistake, if we have something except OK - we have an error, if everything is OK - the program is working correctly;
  • better feedback cycle: we can see if our change is correct in the matter of 1-2 seconds after making this change.

Next time we will build a fully-functional mini testing framework. Stay tuned!

Iterative Divide & Conquer or How to Solve Problems

coaching, kata, problem-solving, programming, pseudo-code, teaching

Let’s imagine a beginner programmer, who have learned a programming language basics, can write programs and can read someone else’s programs. Now, for any somewhat non-trivial problem, they have trouble coming up with a solution. And if they read one of the possible solutions, they will understand it and conclude: “I understand how this program works, but I don’t know how to get there.”. If that sounds like you, my dear reader, or someone you would like to coach or teach, then come and learn Iterative Divide & Conquer problem-solving technique.

English Numbers Kata

Given an integer number in the range of -999 999 999 to 999 999 999, write a program, that will output that number in English words.

Example:

Given number 37545, the program outputs thirty-seven thousands five hundred forty-five.

I can solve that!

If you are confident, that you are able to solve that problem, my congratulations, you have problem-solving skill at a necessary level!

If you still feel, that you can solve this Kata easily, although there are more complex problems that give you troubles, then, probably, your problem-solving strategy is not scalable. The technique described in this article is scalable.

Let’s dive in

NOTE: this article uses pseudo-code, that doesn’t belong to any programming language, so it makes sense for you, my reader, to implement this step-by-step in the programming language you know.

OK. Let’s imagine, that you don’t know how to solve the whole English Numbers Kata and you don’t even know where to start.

Now, let’s try to solve much simpler problem:

Given an integer number in the range of 0 to 9, write a program that will output that number in English words.

Can you solve that? I am pretty sure you can. And easily at that.

Let’s write down some of our possible inputs and corresponding outputs:

1
2
3
4
print english_number(0)   # zero
print english_number(1)   # one
print english_number(7)   # seven
print english_number(9)   # nine

If we run that program, we will get an error, that english_number function is not defined yet. Let’s define it:

1
2
3
function english_number(number) {
  return ""
}

Now if we run our program, we will get four empty lines on the screen as expected. Easiest implementation of english_number function would probably look like that:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
if (number == 0) {
  return "zero"
} else if (number == 1) {
  return "one"
} else if (number == 2) {
  return "two"
} else if (number == 3) {
  return "three"
} else if (number == 4) {
  return "four"
} else if (number == 5) {
  return "five"
} else if (number == 6) {
  return "six"
} else if (number == 7) {
  return "seven"
} else if (number == 8) {
  return "eight"
} else if (number == 9) {
  return "nine"
}

If we run our program, then we will get our expected output:

1
2
3
4
zero
one
seven
nine

If you already know arrays and access array by index, english_number can be simplified greatly:

1
2
3
4
5
6
NUMBERS = ["zero", "one", "two", "three", "four",
           "five", "six", "seven", "eight", "nine"]

function english_number(number) {
  return NUMBERS[number]
}

After making this change, let’s run the program, we should see the same output:

1
2
3
4
zero
one
seven
nine

OK. Seems like we have solved our current problem at hand. What about the initial problem? How do we get there?

Increase size of the problem slightly

Or as you would say in real-world programming: Add a new requirement.

Our initial problem has only two axes, where we can add requirements to our current solution to move it towards final solution:

  • increase the supported range
  • allow negative integers

Let’s go with latter, it seems easier. First we write down our possible inputs and outputs:

1
2
3
4
5
print english_integer(0)     # zero
print english_integer(4)     # four
print english_integer(-3)    # minus three
print english_integer(9)     # nine
print english_integer(-9)    # minus nine

Now, implement english_integer as a simple call to english_number (that will make half of our inputs produce correct output):

1
2
3
function english_integer(number) {
  return english_number(number)
}

Depending on what your programming language this can:

  • output partly incorrect data (for negative values)
  • fail at run time
  • fail at compile time

We can fix that by handling the case of negative numbers:

1
2
3
4
5
if (number < 0) {
  return "minus " + english_integer(-number)
} else {
  return english_number(number)
}

That might be confusing at first. Especially, part, where we call english_integer from the body of the same function. Let’s draw a diagram on how that function works:

Diagram for english_integer after adding negative integers

If we were to unwind this diagram into possible paths, we would end up with two possible cases:

  1. When number < 0:
    • english_integer(number)
    • if (number < 0) - YES
    • english_integer(-number)
    • if (number < 0) - NO
    • english_number(number)
    • prepend result of last call with minus
    • and return it
  2. When number >= 0:
    • english_integer(number)
    • if (number < 0) - NO
    • english_number(number)
    • return result of last call

Running this program results in expected output:

1
2
3
4
5
zero
four
minus three
nine
minus nine

Implementation of english_integer(number) function can be simplified by eliminating else clause and treating number < 0 as an edge case and using “guard clause” to handle it:

1
2
3
4
5
function english_integer(number) {
  if (number < 0) return "minus " + english_integer(-number)

  return english_number(number)
}

If we run our program, the output should be the same as before.

Add more requirements!

Now, that we are done with handling a negative case, we have only one requirement axis left: range of integers. Currently, we support integers from -9 to 9. Now let’s extend support for integers from 10 to 19. This problem should be as trivial as the problem for the range of 0...9.

Our inputs with corresponding outputs:

1
2
3
4
5
print english_teen_number(10)    # ten
print english_teen_number(11)    # eleven
print english_teen_number(13)    # thirteen
print english_teen_number(18)    # eighteen
print english_teen_number(19)    # nineteen

And the implementation to make the output of the program correct (very similar to english_number function):

1
2
3
4
5
6
7
8
9
TEEN_NUMBERS = ["ten", "eleven", "twelve", "thirteen", "fourteen",
                "fifteen", "sixteen", "seventeen", "eighteen", "nineteen"]

function english_teen_number(number) {
  // "number - 10" comes from the fact, that range 10..19
  // maps to a range of 0..9 of our array, so we need to
  // shift 10..19 by -10 to get 0..9
  return TEEN_NUMBERS[number - 10]
}

Now we want to glue our current solution with english_teen_number to add support for range 10..19. This sounds like another if .. else case handling:

  • when number < 10, use english_number
  • when number >= 10, use english_teen_number

Let’s change our input/output pairs for english_integer function:

1
2
3
4
5
6
7
8
print english_integer(0)     # zero
print english_integer(4)     # four
print english_integer(-3)    # minus three
print english_integer(9)     # nine
print english_integer(-9)    # minus nine
print english_integer(10)    # ten
print english_integer(15)    # fifteen
print english_integer(19)    # nineteen

That should fail in some way (compile error, runtime error or incorrect output). Now it is time to implement missing functionality for last 3 examples in our english_integer function. Replace the last call to english_number with:

1
2
3
4
5
if (number < 10) {
  return english_number(number)
} else {
  return english_teen_number(number)
}

If we run the program, we should see correct output:

1
2
3
4
5
6
7
8
zero
four
minus three
nine
minus nine
ten
fifteen
nineteen

What about negative numbers in the range of -19...-10? Let’s add input/output examples and see what happens if we run the program:

1
2
3
print english_integer(-12)    # minus twelve
print english_integer(-16)    # minus sixteen
print english_integer(-19)    # minus nineteen

And the output if we run the program:

1
2
3
minus twelve
minus sixteen
minus nineteen

Yes, it works already thanks to the guard statement that we have at the top of english_integer function. And the whole function body:

1
2
3
4
5
6
7
8
9
function english_integer(number) {
  if (number < 0) return "minus " + english_integer(-number)

  if (number < 10) {
    return english_number(number)
  } else {
    return english_teen_number(number)
  }
}

Going further!

Next smallest requirement seems to be an addition of the range 20...99. Let’s write down our example inputs and outputs:

1
2
3
4
5
print english_integer_in_tens(20)    # twenty
print english_integer_in_tens(27)    # twenty-seven
print english_integer_in_tens(42)    # forty-two
print english_integer_in_tens(70)    # seventy
print english_integer_in_tens(99)    # ninety-nine

So how do we solve this problem? Can we follow the same pattern as before, for ranges 0...9 and 10...19?

We certainly do. We create a constant INTEGERS_IN_TENS = [...], where [...] will contain 79 (mostly two-word) strings. It seems like an excess effort to me. So can we do better?

Yes! We can apply the same problem-solving technique here. What is the smallest problem, that we can solve here easily and independently from other problems?

What about solving the problem only for numbers 20, 30, 40, …, 90? Sounds simple enough! Let’s write our example input/outputs:

1
2
3
4
print english_ten(20)    # twenty
print english_ten(30)    # thirty
print english_ten(60)    # sixty
print english_ten(90)    # ninety

This is no longer different from ranges 0...9 and 10...19. The only detail is that we need to find a way to convert range 20...90 to 0...7 to be able to access the array by that index. This can be done in 2 steps:

  1. obtain the first digit of the number: number / 10, which results in the range 2...9,
  2. and shift resulting digit by -2: number / 10 - 2, which results in the range 0...7,

exactly, what we expect. Now, we can implement english_ten similarly to english_number and english_teen_number:

1
2
3
4
5
6
TENS = ["twenty", "thirty", "forty", "fifty",
        "sixty", "seventy", "eighty", "ninety"]

function english_ten(number) {
  return TENS[number / 10 - 2]
}

Now it is time to return to the original requirement: support range 20...99. Looking at our example input/outputs for a function english_integer_in_tens, we can conclude, that we have 2 different cases:

  • When second digit is 0 (number % 10 == 0):
    • we output only twenty, thirty, …, ninety, depending on the first digit of the number;
    • by the way, this is exactly, what english_ten function is doing. Great!
  • When the second digit is not 0:
    • we output twenty, thirty, …, ninety, depending on the first digit of the number;
    • and we output a hyphen character: -;
    • and we output one, two, …, nine, depending on the second digit;
    • by the way, latter is exactly, what english_number function is doing. Great!

So, let’s implement that in english_integer_in_tens:

1
2
3
4
5
6
7
8
9
function english_integer_in_tens(number) {
  second_digit = number % 10

  if (second_digit == 0) {
    return english_ten(number)
  } else {
    return english_ten(number) + " " + english_number(second_digit)
  }
}

Running the program will produce the expected output:

1
2
3
4
5
twenty
twenty-seven
forty-two
seventy
ninety-nine

Now we should glue the solution for range 20...99 with our main solution, that currently supports only -19...19. As always, start with example input/outputs:

1
2
3
print english_integer(60)   # sixty
print english_integer(43)   # forty-three
print english_integer(-97)  # minus ninety-seven

This results in incorrect output or failure. How can we merge two solutions together now? Let’s remember our current main solution’s different cases:

  • Guard number < 0, that prepends “minus ” and makes number non-negative.
  • When number < 10, use english_number.
  • Otherwise, use english_teen_number.

Seems, like first two cases should not be touched and we are interested in the last one. We should split it in two:

  • Otherwise:
    • When number < 20, use english_teen_number.
    • Otherwise, use english_integer_in_tens.

I believe, we are done and can implement merged version:

1
2
3
4
5
6
7
8
9
10
11
function english_integer(number) {
  if (number < 0) return "minus " + english_integer(-number)

  if (number < 10) {
    return english_number(number)
  } else if (number < 20) {
    return english_teen_number(number)
  } else {
    return english_integer_in_tens(number)
  }
}

Running the program confirms that our new solution now supports integers in the range -99...99.

Final solution

I’m leaving the final solution as an exercise for you, my dear reader. There is nothing more to this technique. Careful application of this technique and careful choice of smallest baby-steps as new requirements to your current solution will get you there - to the final solution, that supports the range of -999 999 999...999 999 999.

If you have any troubles, don’t hesitate to share your questions, ideas and code with me. I will try my best to help you, my dear reader. I am reachable via Twitter by handle @waterlink000.

Challenge

Extend solution to support floating-point numbers to a precision level of 6 digits after the dot.

Thank you for reading!

Final note: did you notice, how tedious it is to check, that output of all these print-s is correct, each time you run the program? This can and should be automated! Stay tuned, I am going to publish an article on how to easily automate these checks really soon!

Next issue is out: Automating Manual Checks or How to Save Time and Get Rapid Feedback!

TDD #4: Path Finding

graph-theory, screencast, tdd, tdd-screencasts, test-driven-development

Recommended to watch in a full-screen and on 720p or higher quality.

List of all TDD Screencasts can be found here.

Script

Hello. I am Oleksii Fedorov and I am wearing a hat of That TDD Fellow. That means you are watching TDD Screencast episode #4.

As mentioned in the previous episode, today we are going to implement a path finding algorithm.

Specifically, our first problem in a path finding theme will be the following:

  • We are given a directional graph, that consists of nodes and edges between them.
  • We are given two nodes: start and finish.
  • We need to answer such questions:
    • Is there a path from start to finish?
    • If there is, What this path is?

I need to make a remark, that we clearly dont have to return the shortest path, just any path, that we can find. We will tackle shortest path problem in later episodes.

Now, I think we can start.

1
2
./watch.sh
vim         # coding session

This was pretty easy to make a DFS algorithm to emerge on its own, by driving it with the specification. Lets see if it is possible to do the same with shortest path problem.. Next time, on the TDD Screencast. Have a nice day.