I’ve been writing software since I was 12 years old and I’m at a healthy age now lol. I’ve been developing software professionally for 37 years (meaning I’ve been getting paid for it). I love it and hope to be coding till I die Unit testing was introduced to me some 15 years ago, but I never really grasped its significance till about 2010 when I was introduced to JUnit. I soon realised I’d got a new way of controlling my little hacks when I was trying to test something that I’d written. So rather than having several static pubic void main() method, each with its own series of print statements and an ad-hoc collection of if statements to control what I was trying to test and print out, I could organise things into tidy little functions each focused on testing one thing. And so the journey into unit testing and TDD began.

With this wonderful toolset and the philosophy of TDD, there is an approach that is missing. What do you do when you are faced with a problem that is foreign to you and have very little understanding of the API required to solve the problem? Pre “unit testing and TDD” I would begin by hacking some code together and using static public void main() as the testing ground, with the code growing ever more complex with each piece of new understanding. But during the COVID lockdown across the world, I decided to enter a new phase of learning (every day is a school day lol). I wanted to follow a TDD approach to learning how to use an API and at the same time develop the solution.

In this talk, I will be using Java and make a number of assumptions - you know what a test double is, and how to write a JUnit test case. What I say here is equally applicable if you are a C++, C# or another language developer.

Test doubles are an integral part of your testing toolkit. There are five types of test doubles that we commonly speak about, Dummy, Fake, Mock, Spy, Stub. Martin Fowler summarises them beautifully here

In this section, I will focus on Stubs, Mocks, and Spies.

It is a commonly held view that a Dummy, Fake, Spy, and Stub are Mocks. They are not. Each one is a type-of Test Double with a very specific purpose.

A Mock and Spy can be categorized as interaction Test Doubles - they focus on peering on the inside of CUT to ensure that it behaves as expected and that the CUT interacts with the Mock and Spy as expected. However, Mocks are very useful for stateful testing using expectations.

There are lots of Mocking frameworks out there, I am using Mockito in my discussion.

So what’s the problem we’re trying to solve?

To answer this, we need to ask another question. What are the basic tenets of a unit test?

Test one method in a class
Not be an integration test
Execute quickly
Be independent of other unit tests, in other words, isolated
Does not interact with a non-in-memory database
Does not perform any file IO
Does not perform any network IO
The scope of the test should be between the Unit Test and the CUT, and NOT any periphery objects to the CUT, including its grand children

Points 5, 6, and 7 are of important interest here. Unit tests are ultimately deployed to a build server. In doing so there are no guarantees that tests that exhibit these capabilities will pass because the database will not be accessible, the file system will more likely be blocked, files may have moved or changed, and networking will be restricted if not totally blocked.

Let’s write some code

So here is the problem we will walk through and discuss - an application that reads a text file and stores the contents of that file in a database. Each line in the text file is stored as a row in a database table. The class model for the database will be

The testing environment will look like this

So my initial thinking was to have three classes

FileToDbLoader - performs the transfer of data from a file object of some kind
FileLoader - loads the data from the file
DbConnector - opens a connection to the DB, and performs the writes to the DB

You might be wondering how I was able to come up with these three classes so quickly. I’ve been working in an OO world since 1988 (some of the first implementations of C++ and modeling software with OMT, Booch, then UML). I can read a requirement and very quickly perform a noun analysis and determine the required classes. Using TDD helped refine that thinking.

We are going to be using already tested APIs such java.nio.Files and java.sql.DriverManager. So we are not testing these calls. We are using these APIs in our code and want to be sure that our code performs as expected.

I began my design on the FileLoader by writing an initial test

package com.celestial.mockito.filetodb;

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.List;
import org.junit.Test;
import static org.junit.Assert.*;

/**
 *
 * @author selvyn
 */
public class FileLoaderTest 
{
    // The inital design is described in this test
    /*
     The weakness should be obvious?  The file to be loaded and it's location
     I use a shared network drive to run this code from different machines, normally
     dev-ing from a PC.  When I ran the code on the laptop from a cafe it 
     immediately failed because the C: on the laptop was completely different
     to the PC, so the original file C:/tmp/KeyboardHandler.txt did not exist
    
     THIS IS A GREAT EXAMPLE OF WHY THE UNIT TEST AND CUT SHOULD NOT BE STRONGLY
     LINKED TO ANY IO - NETWORK, DB, AND FILE SYSTEM
    */
     @Test
     public void load_all_of_file_using_inbuilt_Files_type() 
     {
         // arrange
         String fileToLoad = "c:/tmp/KeyboardHandler.txt";
         FileLoader cut = new FileLoader(fileToLoad);
         int expectedBytesRead = 301;
         
         // act
         int bytesRead = cut.loadFile(fileToLoad);
         
         // assert
         assertEquals(expectedBytesRead, bytesRead);
     }
}

It soon became apparent that this test would fail.

And the CUT - the issue is at line 28, tight coupling to the static method

package com.celestial.mockito.filetodb;

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.*;


/**
 *
 * @author selvyn
 */
public class FileLoader 
{
    String fileToLoad;
    List<String> lines = Collections.emptyList();

    public FileLoader(String fileToLoad) 
    {
        this.fileToLoad = fileToLoad;
    }

    int loadFile(String fname) 
    {
        try
        {
            lines = Files.readAllLines(Paths.get(fname), StandardCharsets.UTF_8);
        }
        catch (IOException e){}

        return calculateFileSize();
    }    
    
    private int calculateFileSize()
    {
        IntWrapper result = new IntWrapper();
        
        lines.forEach(line -> {
                result.value += line.length();
        });
        
        return result.value;
    }
}

The failure comes from this simple fact. The code is stored on a network drive so I can access it and develop it from a number of machines. Each of these machines has a C: drive but their contents are very different. So initially the test worked, but when I moved to work on one of the laptops it failed because the file in the test did not exist. So yes I could have stored the test text file within the project, but if I had done so, the problem would never have been revealed, So the test failed one of the tenets of good unit tests; no File IO.

For what I was trying to do I liked the java.nio.Files class. The coupling between FileLoader and Files is high, but at some point, there is going to be some level of dependency in your code. You need to decide what is an acceptable level. Unit tests are great for revealing if you got that level of dependency correct - the testability test.

My initial thought was to use constructor injection. Inject the Files class (not the object since we need the static methods on the class) into the FileLoader, but this was wrought with complications that I don’t want to discuss here and would lead us away from the main discussion. So the approach I have chosen is to pass an anonymous object into the FileLoader via the loadFile() method. The loadFile() method requires the field member “lines” to have been set before calculateFileSize() is called.

The implementor of the anonymous object can then add whatever functionality they want but it must return an instance of a List<string>. In other words, they can stub the required Files behaviour. This is a very old technique from the world of C++ programming called a functor (function object).

So I have a new test because I maintain the story of the evolution in my thinking

     // Redesign the FileLoader so that the machenism to load files up can be 
     // passed in as a lambda - still titghtly coupled the file system
     @Test
     public void load_all_of_file_using_inbuilt_Files_type_as_lambda() 
     {
         // arrange
         String fileToLoad = "c:/tmp/kipstor-settings.js";
         FileLoader cut = new FileLoader(fileToLoad);
         int expectedBytesRead = 301;
         
         // act
         int bytesRead = cut.loadFile((fname) ->
         {
            List<String> result = null;
            try
            {
                result = Files.readAllLines(Paths.get(fname), StandardCharsets.UTF_8);
            }
            catch (IOException e){}
            return result;
         });
         
         // assert
         assertEquals(expectedBytesRead, bytesRead);
     }

To support this, we will create a functional interface

And add a new method to FileLoader that can take this anonymous object

So I’m using the functor (lambda) to decouple the FileLoader from java.nio.Files.

This test works but it’s still constrained because it is trying to access the local file system. So, let’s stub out the functor with some dummy data so that loadFile() has something to work with other than the local file system

A new test

To support our overall design aim, we need one more method, a getter to retrieve the List<string> of lines from the FileLoader.

Using the functor approach and Mockito to create a stub for the Files.readAllLines(Paths.get(fname), StandardCharsets.UTF_8); call we can rewrite the load_all_of_file_using_inbuilt_Files_type_as_lambda() test to

Now we have code that can be deployed onto the build server and describes how to use the FileLoader

We are now ready to work on the DBConnector

As I began to work on the test for this class it soon became apparent that was going to have a similar problem as with the FileLoader. The java.sql.DriverManager.getConnection(…) is a static method. So again I set about creating a functional interface so that I could create a functor as and when I need one. Again the functor will be used to decouple java.sql.DriverManager from our implementation code. This was my initial test

What I needed was this (yes I know bad security around the user name and password - not the point of the discussion). Notice the use of the functor again (lines 7-10)?

The above allowed me to create the functional interface

Originally openConnection(…) was a void operation, but then I realised I couldn’t test if the connection had succeeded. Also, without the Connection, the DBConnector would not be able to perform any DB requests.

So the implementation class currently looks like this

Now we are ready to redesign the test so that it’s deployable on a build server

This test works just as expected. The CUT is working as designed. Changing the port number as shown in line 5 does not cause the test to fail because the test is isolated from the network and physical DB. And the cut is not dependent on that value

Writing a line to the database

My thinking here is to have a method that takes the string to be written to the DB and returns its line number/row in the DB.

So here is the test

And here is the initial implementation in DBConnector{ … }

This does result in an interesting design decision that must be made - running a SQL query after each insert would give us the row id/line #, but be very slow. A more optimised approach would be to query the last row # when we connect to the DB and store this in the DBConnector. So we are going to have to modify the openConnection() method so that it queries the DB for the row count.

So here is the redesign of the writeLine()

The DBConnector class fields and constructor look like this

In keeping with not breaking the tests, openConnection() has been updated to

You will notice two new methods updateLineCount() and queryLinesInDB()

The tests haven’t greatly changed and they still pass, which is a great and good demonstration of following the TDD rules.

Now we are ready to translate the write_line_to_live_connection() test method so that it can be deployed on a build server. Before we do this, I want to re-examine the open_connection() test method. Now that we have bought line number into the equation, and it is retrieved when the DB connection opened, we can add this to the test result - so that a valid DB connection would be that we connect to the database and are able to retrieve the current line number

Notice on line 12 I’m expecting a call the queryLinesInDB() if the connection was successful and no exception is thrown. We can now test the DBConnector’s state by reading this field value @line 16.

We are now able to mock out the write_line_to_mocked_connection() test method

DIS - TDD (Java, C#, VB)

Discovery through unit testing

So what’s the problem we’re trying to solve?

Let’s write some code

I began my design on the FileLoader by writing an initial test

We are now ready to work on the DBConnector

Writing a line to the database

Related articles