Fundamentals of Unit Testing

In the past we already dealt with various aspects of the topic “Testing”. Now, it’s time to dig deeper – it’s time to get our hands dirty. So – let’s code!

I will share with you how I approach unit testing without claiming absolute truth or perfection. What follows are neither rules nor instructions. It’s a description of what works well for me.

Spoiler: this is not (always) test-driven development (TDD) and I don't (always) follow the "rules" or recommendations of Robert "Uncle Bob" Martin or the "one assertion per test" paradigm. I think I'm following a pragmatic middle path and I can sympathize with Rob Martin's experience, which he describes in "Just 10 minutes without a test". It’s always an unsettling feeling to state in the daily scrum: "The feature is finished. I just have to finish writing the unit tests today".

Don't get me wrong; I'm sure everyone has a way that works well for them personally. It's just not mine. And do we all always have to follow the same path? I don't think so. It's far more important that we all agree on one thing: tests are important! Unit tests are particularly important!

My basic assumption with unit tests is practically Murphy's Law: anything that can go wrong will go wrong – somewhere, sometime, under some circumstance. We all know the "this should never happen" cases that we provide for in our code, for example when catching exceptions. And then, these cases still occur and we are surprised.

Therefore, my conclusion from Murphy's Law for unit tests is: everything that can be tested meaningfully will be tested. But what does meaningful mean here? Are we aiming for 100 percent test coverage? I strive for a high test coverage of relevant code parts and also pay attention to this in reviews. There were also a few thoughts on this in an earlier article.

But let’s end the prologue. How do I approach this in detail?

Example: A Simple Calculator

As a test object, we take a simple Java class Calculator, which can process simple basic arithmetic operations using the calculate() method. A string is used as input, which is split into the operands and the operator using regular expression before the actual calculation is performed:

1package de.pentacor.blog.unittesting;
2
3import java.util.regex.Pattern;
4
5public class Calculator {
6
7  private static final String CALCULATION_REGEX = "^(?<a>\\d+)\\s*(?<operator>[+\\-\\*/])\\s*(?<b>\\d+)$";
8  private static final Pattern CALCULATION_PATTERN = Pattern.compile(CALCULATION_REGEX);
9
10  public Double calculate(String calculation) {
11    var matcher = CALCULATION_PATTERN.matcher(calculation);
12
13    if (matcher.find()) {
14      var operator = matcher.group("operator");
15      var a = Double.valueOf(matcher.group("a"));
16      var b = Double.valueOf(matcher.group("b"));
17
18      return switch (operator) {
19        case "+" -> a + b;
20        case "-" -> a - b;
21        case "*" -> a * b;
22        case "/" -> a / b;
23      };
24    }
25
26    throw new IllegalArgumentException("Invalid calculation: " + calculation);
27  }
28}

This is not a complicated example and yet there is already a lot going on here. And where there's something going on, something can go wrong. So what should be tested?

• that the operands are correctly extracted from the input

• that the operator is correctly extracted from the input

• that the basic arithmetic operations are performed correctly

• that everything doesn't blow up in our faces if the input is somehow "unexpected"

The way the example is currently written, our ability to test these different things independently is limited because we only have one method that does everything. So the unit tests for this one method would also have to cover everything. That doesn't sound very fun, especially since a regular expression is involved – and that's what makes it interesting. And let's just imagine what would happen if we also had external dependencies that are also used in this method…

Making Code (Meaningfully) Testable in the First Place

The functionality provided and used externally can be as complex as you like. This speaks for equally complex tests. Meh. I want it to be simple...

So let's rework the Calculator class to make our lives easier and to be able to test different things separately.

Division into fine-grained Methods

Just because I want to be able to test individual parts separately, I don't want them to be usable separately at the same time. I therefore restrict the visibility of the methods accordingly:

• Only the calculate() method remains public - and can therefore be called from anywhere.

• All other methods become private and can therefore only be used by the class itself.

The result of this refactoring then looks as follows:

1package de.pentacor.blog.unittesting;
2
3import java.util.regex.Matcher;
4import java.util.regex.Pattern;
5
6public class Calculator {
7
8  private static final String CALCULATION_REGEX = "^(?<a>\\d+)\\s*(?<operator>[+\\-\\*/])\\s*(?<b>\\d+)$";
9  private static final Pattern CALCULATION_PATTERN = Pattern.compile(CALCULATION_REGEX);
10
11  public Double calculate(String calculation) {
12    var operator = extractOperator(calculation);
13    var a = extractOperand(calculation, "a");
14    var b = extractOperand(calculation, "b");
15
16    return switch (operator) {
17      case "+" -> add(a, b);
18      case "-" -> subtract(a, b);
19      case "*" -> multiply(a, b);
20      case "/" -> divide(a, b);
21      default -> throw new IllegalArgumentException("Invalid calculation: " + calculation);
22    };
23  }
24
25  private Double extractOperand(String calculation, String operand) {
26    return Double.valueOf(extractNamedGroupFrom(calculation, operand));
27  }
28
29  private String extractOperator(String calculation) {
30    return extractNamedGroupFrom(calculation, "operator");
31  }
32
33  private String extractNamedGroupFrom(String calculation, String groupName) {
34    Matcher matcher = CALCULATION_PATTERN.matcher(calculation);
35    if (matcher.find()) {
36      return matcher.group(groupName);
37    }
38    return null;
39  }
40
41  private double add(Double a, Double b) {
42    return a + b;
43  }
44
45  private double subtract(Double a, Double b) {
46    return a - b;
47  }
48
49  private double multiply(Double a, Double b) {
50    return a * b;
51  }
52
53  private double divide(Double a, Double b) {
54    return a / b;
55  }
56}<code data-language="java"></code>

The public method is primarily the one that provides the essential functionality of my class. It's obvious that it has to be tested. As it is publicly visible and can be used from anywhere, this is not a problem. But the devil lies in the details.

The private methods encapsulate the individual parts of the functionality in small, easy-to-understand units. And what is easy to understand is also easy to test. However, this should only help us internally and not be used externally – after all, we want to keep our dependencies under control. Accordingly, the methods are not only encapsulated but also hidden.

Problem solved! Yes, nobody can now really get at it. Not even our test, which is implemented in a separate class in the src/test/java directory in the same package. Well, that’s annoying.

For a test of the publicly visible calculate() method, this is initially not a problem. An example unit test could look like this:

1@Test
2void calculate_withAddition_returnsExpectedResult() {
3  // arrange
4  Calculator calculator = new Calculator();
5  String input = "2 + 3";
6  Double expectedResult = 5d;
7
8  // act
9  Double result = calculator.calculate(input);
10
11  // assert
12  assertNotNull(result);
13  assertEquals(expectedResult, result);
14}

Making Methods Accessible for the Test

For a private method, however, the situation is completely different. We first have to find a way to access the method.

Reflections to the rescue! Java offers the option of accessing classes and methods via reflections, changing their visibility and then executing them to make them accessible in our unit tests… problem solved again!

1@Test
2void add_withValidOperands_returnsExpectedResult()
3    throws NoSuchMethodException, InvocationTargetException, IllegalAccessException {
4  // arrange
5  Double a = 2d;
6  Double b = 3d;
7  Double expectedResult = 5d; 
8
9  Method add = Calculator.class.getDeclaredMethod("add", Double.class, Double.class);
10  add.setAccessible(true); 
11
12  // act
13  Double result = (Double) add.invoke(calculator, a, b); 
14
15  // assert
16  assertNotNull(result);
17  assertEquals(expectedResult, result);
18}

Before we can test the add() method, we must first gain access and make the method executable before we also run it via reflections on the previously used calculator instance. Unfortunately, the reflections are not generic, so we first have to cast the result to the desired type. By using reflections alone, this fairly simple test could (theoretically) result in three different exceptions, which need to be declared; not to mention possible runtime exceptions, such as an exception when casting from Object to Double.

Of course, these are comparatively artificial considerations that should never happen. "This should never happen". Wait, that sounds familiar... Murphy's Law. What can go wrong will go wrong. I would feel better if I could eliminate the complexity of reflections from my tests.

My solution: I use the default visibility of the methods, so I remove the access modifier private. This makes them accessible from the same package as the implementing class and our exemplary test can be significantly simplified:

1@Test
2void add_withValidOperands_returnsExpectedResult() {
3  // arrange
4  Double a = 2d;
5  Double b = 3d;
6  Double expectedResult = 5d; 
7
8  // act
9  Double result = calculator.add(a, b); 
10
11  // assert
12  assertNotNull(result);
13  assertEquals(expectedResult, result);
14}<code></code>

If exceptions are still thrown, they come from the method that I want to test, i.e. from the business logic instead of the test implementation.

Anatomy of a Unit Test

The test cases show some of the conventions I follow when writing my unit tests:

Name of the Test Method

The names of my test cases, i.e. test methods, have three separate parts – separated by _:

1. the name of the tested method: add

2. the test condition, i.e. a brief description of the test case: withValidOperands

3. the expected result: returnsExpectedResult

This results in a reasonably readable and meaningful description of the test case. Depending on the method, condition and expected result, however, this can lead to extremely long method names (not untypical for Java) – but we’ll come back to this later on.

Arrange-Act-Assert Pattern

I am a fan of the Arrange-Act-Assert pattern, which makes it possible to clearly structure test cases with simple comments and divide them into sections:

Arrange – preparing the test

Input values are prepared and expected results are defined. For external dependencies that are mocked, I also use this area to define the behavior of the mocks. Does the test data need to be inserted into the database? That's where it takes place (but then, we are actually already talking about an integration test).

Act – the test itself

The actual test is performed. The method to be tested is executed or the REST call relevant to the test is made. Nothing else happens here. However, in every case, some kind of result falls out of the Act section.

Assert – evaluating the test

The assertions are applied to the result. There should not be too many but at least one. In some cases, for example with more complex results with elements in lists or maps, it can also make sense to get individual elements and apply further assertions to them.

‍

Inevitably, the Arrange and Assert sections are longer than Act. In many cases, Arrange is probably the longest, especially when a lot of preparation is required because the inputs or expected results are very complex, a lot of mocking is required, etc. In these cases, the question arises as to whether the same procedure can and should be used for test code as for productive code: can parts be outsourced to separate (support)methods?

If, on the other hand, I have a lot of assertions in my test case, I should ask myself whether I am maybe trying to test too many things at once. Perhaps it would be better to split my test case into several smaller test cases, each containing the assertions for a partial aspect? Something I am not a fan of in this context: writing support methods for assertions à la checkResult(), which then in return contain all assertions and are potentially used again and again in different tests.

Unit tests normally have the great advantage that they "document" methods by using exemplary execution – this is made considerably more difficult by outsourcing assertions to (intransparent) separate methods. The same applies, by the way, if the inputs used under Arrange are largely taken from variables and constants defined elsewhere.

Finding the right balance is therefore a balancing act in many cases. Personally, however, I prefer to accept redundancy and have a test, in which I can see the input and output at a glance, without having to look at six other files and scroll to three other methods.

Speaking of clarity and capturing tests at a glance: in some cases, I also combine the Act and Assert sections, especially if the assertion requires the tested method to be called within the assertion:

1@Test
2void calculate_withInvalidInput_throwsException() {
3  // arrange
4  Calculator calculator = new Calculator();
5  String input = "foo bar";
6
7  // act & assert
8  assertThrows(IllegalArgumentException.class, () -> calculator.calculate(input));
9}<code data-language="java"></code>

If I wanted to stick to the previous three-part structure by all means, I would have to write the test case like this:

1@Test
2void calculate_withInvalidInput_throwsException() {
3  // arrange
4  Calculator calculator = new Calculator();
5  String input = "foo bar";
6
7  // act
8  Executable methodCall = () -> calculator.calculate(input);
9
10  // assert
11  assertThrows(IllegalArgumentException.class, methodCall);
12}

Although this is possible, I personally think it's pointless. There's one possible exception: the method-call to be tested has a lot of arguments, is somehow confusing or complex for other reasons. Even in this case, the problem and solution most likely lie elsewhere.

What I also don't do in most cases is reducing tests to one line. As much as I like to keep my productive code concise and reduce it to just a few lines where possible, I like to stick to the Arrange-Act-Assert pattern for the sake of consistency and readability. But judge for yourself whether you like this test better than the previous examples:

1@Test
2void calculate_withAddition_returnsResult() {
3  assertEquals(5, calculator.calculate("2 + 3"));
4}<code data-language="java"></code>

Test-Class-Organization

Depending on how much functionality and complexity, there is in a class and how many different test cases need to be covered, a test class can quickly become quite long and confusing.

You could, for example, start writing several test classes for a single "production class". However, I am not convinced by this idea. CalculatorTest as a test for Calculator is logical and obvious. But what should I do if I want to write individual classes for testing individual methods? Will I end up with the following constructs?

• AbstractCalculatorTest as an abstract parent class for all individual classes for the initialization and preparation of the tests as a whole

• CalculateCalculatorTest for tests of the calculate() method

• AddCalculatorTest for tests of the add() method

Or should I rather name the individual classes such that Calculator is at the front and the tests of individual methods of a class are sorted alphabetically?

• CalculatorAbstractTest

• CalculatorAddTest

• CalculatorCalculateTest

• …

Naming - one of the most difficult problems. We all know the drill.

But what I do instead is to use an IDE feature: IntellIJ IDEA allows you to use comments in the code to define regions that can then be collapsed. This would result in a test class looking something like this:

1package de.pentacor.blog.unittesting;
2
3import static org.junit.jupiter.api.Assertions.*;
4
5import java.lang.reflect.InvocationTargetException;
6import java.lang.reflect.Method;
7import org.junit.jupiter.api.Test;
8import org.junit.jupiter.api.function.Executable;
9
10class CalculatorTest {
11
12  Calculator calculator = new Calculator();
13
14  // region calculate()
15
16  @Test
17  void calculate_withAddition_returnsResult() {
18    // arrange
19    String input = "2 + 3";
20    Double expectedResult = 5d;
21
22    // act
23    Double result = calculator.calculate(input);
24
25    // assert
26    assertNotNull(result);
27    assertEquals(expectedResult, result);
28  }
29
30  @Test
31  void calculate_withInvalidInput_throwsException() {
32    // arrange
33    String input = "foo bar";
34
35    // act
36    Executable methodCall = () -> calculator.calculate(input);
37
38    // assert
39    assertThrows(IllegalArgumentException.class, methodCall);
40  }
41  
42  // endregion
43  
44  // region add()
45
46  @Test
47  void add_withValidOperands_returnsResult() {
48    // arrange
49    Double a = 2d;
50    Double b = 3d;
51    Double expectedResult = 5d;
52
53    // act
54    Double result = calculator.add(a, b);
55
56    // assert
57    assertNotNull(result);
58    assertEquals(expectedResult, result);
59  }
60  
61  // endregion
62}<code data-language="java"></code>

The regions or sections can also be nested within each other as needed, offering a good way of structuring and organising long source files. Incidentally, this also works in any other file format, such as YAML or SQL, and is therefore an absolute recommendation.

Selection of Test Cases

We now have seen how I generally write my tests and I also indicated at the beginning that I wanted to cover all conceivable cases. This means in particular:

• Input value null for all arguments of a method

• Empty input values for all arguments of a method, i.e. in particular "", Collections.emptyList() etc.

• Invalid input values

• Valid input values for the happy path (of course)

At first glance, this does not appear to be necessary, even when looking at the example of the calculator class because the string used as input must pass the regular expression for the calculation. And the expression ^(?<a>\d+)\s*(?<operator>[+\-\*/])\s*(?<b>\d+)$ provides for this:

• The string begins with a number consisting of at least one digit.

• The first operand a is followed by any number of spaces.

• This is followed by the operator, which is either +, -, * or /.

• The operator is again followed by any number of spaces.

• Finally, the string ends with a number consisting of at least one digit.

So what could go wrong?

Admittedly, in the very first version of the calculator, there was actually not too much that could go wrong. We only introduced some problems by splitting the functionality into separate methods (ironically): due to the fact that the extraction of the operator from the string and its subsequent evaluation now take place in different contexts - in particular if the switch-statement is executed independently of a match of the input to the regular expression - we suddenly added the very real risk of a NullpointerException to the code. This occurs when we call the calculate() method with an invalid input.

There are various options for solving the problem, which in return can be tested at various points. For example, the new method extractOperator() could result in an exception being thrown directly, if no valid operator can be extracted from the input. Or the calculate() method is extended by an additional null check. But let's not go into further detail here.

It’s much more important to keep in mind that methods should technically always be tested with (currently) "not possible" conditions in order to avoid surprises later on. As we have seen, these can arise quickly and easily through simple refactorings.

And let's think a little further: we have also extracted the methods for executing the actual calculations as private methods. At this point, one could certainly discuss whether they could or should not also be public, as this makes perfect sense for a computer. There isn't necessarily a comprehensible reason to hide them, except that the methods that were previously only used internally could now be used by completely different, potentially unknown callers, in an unclear way, with possibly invalid input values. These invalid and unexpected inputs then lead to unexpected errors, if we have not prepared for all eventualities beforehand when selecting our test cases.

If we fail to do this at the beginning, it becomes much more difficult later on. It’s therefore essential to write a test that reproduces the bug before it is fixed when bugs occur. This definitely increases the resilience and robustness of the application in the long term, also with regard to future refactorings.

Parameterized Tests

Frequently, many tests only differ in their inputs, but expect the same result or the execution of the test is basically identical and the test is - so to say - a function f (input, output). In these cases, JUnit 5 makes life much easier for us with the test annotation @ParameterizedTest because we only need to define the basic test once and can then execute it as often as we like with different parameters.

Which parameters are used depends on other annotations. The simplest is @NullSource - this simply passes null as a parameter to the test. Admittedly, this is not super useful on its own, but in combination with @ValueSource it makes a lot more sense. Let's take a look at an example:

1@ParameterizedTest
2@NullSource
3@ValueSource(doubles = {0d, 1d, 2d, 3d, 4d})
4void add_withParameters_returnsExpectedResult(Double a) {
5  // arrange
6  Double b = 1d;
7  Double expectedResult = a != null ? a + b : b;
8
9  // act
10  Double result = calculator.add(a, b);
11
12  // assert
13  assertEquals(expectedResult, result);
14}<code data-language="java"></code>

To ensure that this test is actually successful, I "secretly" adapted the method so that null is treated as 0d for the input. 🤓

In my opinion, this type of parameterized test is particularly suitable for testing error cases in which a wide variety of invalid inputs should all lead to the same result, meaning handling the errors in alignment with the expectations.

I prefer to use the annotation @MethodSource, which is much more flexible. This specifies a method that provides the arguments for the parameterized test. Unlike @ValueSource, this can be any number of arguments of any type, so that a test of the add() method could look like this:

1@ParameterizedTest
2@MethodSource("add_params")
3void add_withParameters_returnsExpectedResult(Double a, Double b, Double expectedResult) {
4  // act
5  Double result = calculator.add(a, b);
6
7  // assert
8  assertEquals(expectedResult, result);
9}
10
11static Stream<Arguments> add_params() {
12  return Stream.of(
13    Arguments.of(null, null, 0d),
14    Arguments.of(1d, null, 1d),
15    Arguments.of(null, 1d, 1d),
16    Arguments.of(2d, 3d, 5d),
17    Arguments.of(23d, 6d, 29d),
18    Arguments.of(1200d, 34d, 1234d)
19  );
20}

Fine Tuning the Output

Previously, I described the convention for naming test methods and also mentioned that the names are not always easy to read. With JUnit 5 there is also a solution for this, which really comes in handy - especially in connection with parameterized tests: we can use the annotation @DisplayName to assign any name for test classes and methods. Using the name attribute, this is also possible for the individual parameterized test executions, whereby parameter values can also be used in the name – for example:

1@DisplayName("Calculator.add() - Happy Cases")
2@ParameterizedTest(name = "Case {index}: {0} + {1} = {2}")
3@MethodSource("add_params")
4void add_withParameters_returnsExpectedResult(Double a, Double b, Double expectedResult) {
5  // act
6  Double result = calculator.add(a, b);
7
8  // assert
9  assertEquals(expectedResult, result);
10}<code data-language="java"></code>

The result of the test execution is then perfectly readable and "human-friendly".

Conclusion

With this article, I gave an insight into my personal view of the basics of unit testing. We have seen how to structure the code so that it can be tested at a fine-granular level. You also got to know my preferences for the design or organization of test classes and I mentioned some essential cases that, in my opinion, should not be missing in any unit test - and why or which pitfalls can lead to the "impossible" cases becoming quite important. To make testing really fun and nice to look at, you also got to know my favorite annotations from JUnit 5.

There is only one thing missing: the required dependencies. The examples in this article require the junit-jupiter-engine and junit-jupiter-params libraries. Then you can get started!

Happy testing! And don't forget: what can go wrong will go wrong - what can be tested should be tested!