So what’s the problem we are trying to solve

I want to load up from the disk or even off the network files containing varying types of content, straight, XML, JSON etc.

Once the source has been loaded into the program, I wan to process it, this can vary from as simple as counting the number chars (bytes) in the file, the looking for complex patterns in the data source.

Initial Model

TextFileSource

Responsible for loading text files and returning an Iterable class of String objects. Each String corresponds to a single line (a string of characters terminated with a CR/LF). It does not process the raw data but handles the use of JDK API classes File, FileReader, and BufferedReader. It also supports a custom method that allows you to customise the way in which data is loaded into the Iterable class, the current algorithm is

        ICollectionLoader<ArrayList<String>> functor = (c, l) -> {
            c.add(l);
            return c;
        };

Which by default gets passed into the loadData method.

public class TextFileSource implements IDataSource
{
    @Override
    public  Iterable<String>  loadData(  String fname ) {
        // We create a lambda expression to do the work in the TextFileLoader
        ICollectionLoader<ArrayList<String>> functor = (c, l) -> {
            c.add(l);
            return c;
        };
        return loadData(fname, functor);
    }

    @Override
    public  Iterable<String>  loadData(  String fname, ICollectionLoader func )
    {
        FileReader fr = null;
        ArrayList<String> lines = new ArrayList<>();
        try
        {
            String fileName = fname;
            File file = new File(fileName);
            fr = new FileReader(file);
            BufferedReader br = new BufferedReader(fr);
            String line;
            while((line = br.readLine()) != null)
            {
                // process the line
                // we use a lambda here to do the heavy lifting
                func.addElement( lines, line);
            }       
        } catch (IOException ex)
        {
            Logger.getLogger(App.class.getName()).log(Level.SEVERE, null, ex);
        } 
        finally
        {
            try
            {
                fr.close();
            } catch (IOException ex)
            {
                Logger.getLogger(App.class.getName()).log(Level.SEVERE, null, ex);
            }
        }
        return lines;
    }
}

Processing the data

BasicDataProcessor

Once the data has been loaded into the collection we are ready to process it. This is the job BasicDataProcessor (I’m going to assume that once we get the design on this right, we can build out an interface from which other types of DataProcessors can be implemented)

public class BasicDataProcessor
{
    private final IDataSource dataSource;
    
    public BasicDataProcessor(IDataSource ds )
    {
        dataSource = ds;
    }
    
    public  long    loadData(String fname)
    {
        var data = dataSource.loadData(fname);
        long count = 0;
        for( var datum : data )
        {
            count += datum.length();
        }
        
        return count;
    }
}

We’ve used constructor injection to supply a DataSource to the BasicDataProcessor. As can be seen from the code, we simply total up the number of characters in the data source.

Testing

We are responsible for testing all the code we write. In this article, we are only focusing on the BasicDataProcessor. It’s ideal for illustrating the power and use of mocking.

Test environments must be stable. An engineer must understand the chaotic and unpredictable and turn them into the predictable.

I’ve scrawled over our initial model (not the best handwriting)

We are not testing File and BufferedReader. They’re part of the JDK and have already been tested by Sun and the millions of developers who have been using them for years. Both BasicDataProcessor and TextFileSource are deterministic pieces of code (we wrote them).

TextFileSource is deterministic code but it’s using API calls that could lead to unpredictable results

The file might not be there (an exception would be thrown)
The file content might change
The file might become none readable (an exception would be thrown)

Our first breaks one of the tenets of writing good tests - NO IO

Line 6 points to a file on my computer at the time of writing this article. Yes you can copy this code, and repoint the file location to something on your machine, but it will fail if try this on a build server (yes we can repoint the path to a file within the project, but it’s still a bad test.

Let’s look at the issue and how mocks can help us. In this first part of our discussion I’ve introduced two classes A and B is nested local classes

Remember testing environments must be predictable

If you execute this code it works as expected, the result and the expected are both 18. But it’s highly coupled code (line 10). Tight coupling here isn’t about the number of methods on a required class that consuming classes uses, it’s about ownership and substitution. The instance of B can NOT be substituted with another instance with the interface. If B were making IO calls, then A becomes unpredictable.

In this next version, we go part way to decoupling A and B

We’ve removed some of the tight coupling between the instances but not the types. We create an of B and pass it into A at lines 25-26. But we could never create a different type of B that pretends to be B, in other words, a test double of some kind. So we still have the same problem if B were making IO calls, then A becomes unpredictable.

Now we have completely decoupled A from B through the use of an interface called IB. We then created a different implementation of IB called X and passed an instance of X to A, lines 39-40. Class X can do whatever we want it to do as long as it meets the requirements of the contract of the interface IB.

Finally, we introduce a mock

We are using mockito here at lines 28 and 36.

The mocking library will define a mock (proxy) implementation of the interface. For each method on the interface, it will simply put in place placeholders for the methods. Any method that returns a value will by default return the default value for the return type. So an int return type will by default return 0, a boolean will return false, etc.

Line 36 is extremely important, and is known as setting up your expectations. You are basically overriding a method on the interface but without providing a method implementation. You are indicating that for a given set of inputs expect a specific value to be returned. For example in the code above change the value of 3 or 6 to some other value, and run the test. You will find the result will be 9. You might be perplexed by this but your shouldn’t. When you set an expectation, and when the code executes it does not have input values that match those in the expectation, the method will return the default value for its return type.

Let’s dig into this a little deeper, play around with the following code

Line 3 should give you the result you expect, the assert passes.

Line 21 also passes even though we have not added any values to the array.

Line 26 fails because we told the mock to always return 4 (line 15).

Important note here, you control the mock

Now in this example, we don’t set up an expectation for the isEmpty() method on the ArrayList.

We’ve set up an expectation for ArrayList.size() but not for ArrayList.isEmpty().

It doesn’t matter how many times you add or remove elements isEmpty() will always return false.

We are now ready to apply learning and understanding of mocks to our initial problem

Line 5 we mock the TextFileSource.

Lines 6 - 9 we set up the data that needs to be returned from TextFileSource.loadData()

Line 11 we set up an expectation for TextFileSource.loadData(). We specify any() as the input and items as the return. Because we don’t really care about loading a file from the disk we pass any() indicating that mockito can match the input against any value passed to loadData().

Step through the code using a debugger to see how this all fits together.

DIS - TDD (Java, C#, VB)

A Worked Example