Methods to Obtain JUnit Test Results

2018-05-03

During past few days, I have been working on JUnit test and tried to find out an easy way to get JUnit test results in real time from outer application (and finally post to my front end webpage). Indeed there are several ways to do that but some of which could be quite roundabout.

In this blog, I am going to discuss some methods I found to do that.

Possible Solutions

IDEA Test Runner

Eclipse is not concerned here since I don’t use it, though its usage should be similar to that of IDEA.

IDEA can launch tests very well, result of tests are listed out and updated in real time. It can even generate graceful html report for the last compiled test suite. The problem is how to get those runtime output. IDEA JUnit launcher should be implemented by Jetbrains and users don’t have privileges to access those implementations.

Console Launcher

JUnit provides a command line Java application that lets users run JUnit via console. See JUnit Console Launcher guide here.

However, it seems that this tool doesn’t provide methods to obtain tests detail during runtime from outer console (actually I failed to use it for some configuration problems that I have not solved yet). If I want to get test details during runtime, a console listener would be necessary, which could be quite awkward.

Maven Surefire Plugin

Maven probides surefire to launch JUnit tests(actually JUnit group mentioned this in their user guide). Besides Maven, Gradle can also do similar jobs, though I have never used Gradle. :P

Maven handles lots of job for users and will generate reports for JUnit tests. In those report files, details of every tests will be presented. However, the problem is similar to that of console launcher.

See susrefire docs here.

TestExecutionListener

JUnit 5 provides a TestExecutionListener interface so that users can implement their own listener and register it. By doing so, users can do something more complex. This method will stuck developers not know much about JUnit test, though it is the most powerful way to do that so.

See TestExecutionListener docs here.

AfterTestExecutionCallback

This is another interface of JUnit 5. It is a functional interface. AfterTestExecutionCallback provides API for behaviours to be done immediately after each test case is done. And it would be called before any user defined teardown methods (e.g., @AfterEach methods). An ExtensionContext parameter could be used to get some brief information about the test. However, ExtensionContext can’t provide users the test result directly, though we can figure it out via .getExecutionException().isPresent() method because assertion fails will throw an AssertionFailedError. But do remenber that other exceptions will also throw exceptions and make the method return value true. Then we can use if statement to trigger some behaviours (like notify the front end webpage).

See ExtensionContext docs here)
See AfterTestExecutionCallback docs here.

Implementation

The implementation could be quite simple as the following codes show.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
import org.junit.jupiter.api.extension.AfterTestExecutionCallback;
import org.junit.jupiter.api.extension.ExtensionContext;

public class NotifierExtension implements AfterTestExecutionCallback {
@Override
public void afterTestExecution(ExtensionContext extensionContext) throws Exception {
Boolean testResult = extensionContext.getExecutionException().isPresent();
if (testResult) {
//do something
} else {
//do another thing
}
}
}

And before every class or methods to use custom AfterTestExecutionCallback add @ExtendWith(NotifierExtension.class) annotation.

If we run command mvn test to launch JUnit tests, some report files will be generated under project/target/surefire-reports directory. We can make the backend application notify the front end by some ways that the test results are ready, then the front end can load those report files.

Maven Dependency

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
<project>
[...]
<build>
<pluginManagement>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.21.0</version>
<dependencies>
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-surefire-provider</artifactId>
<version>1.2.0</version>
</dependency>
</dependencies>
<!-- sepecify test cases -->
<configuration>
<includes>
<include>**Test*.java</include>
</includes>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>2.21.0</version>
</plugin>
</plugins>
</pluginManagement>
</build>
[...]
</project>

RWBY 3.0 : Payback

..-. . -.