API Testing with Java and REST Assured - Allure reporting

In my previous posts I've walked through how to get started with REST Assured and API testing. We've setup our test endpoints and created a variety of test methods that validate those endpoints. You can find the complete code for this series of blog posts in my Github project.

Up until now we've been using the IDE to run the tests and viewing our test results in the terminal window. In this post I want to show you how to generate an HTML report using the Allure reporting framework.

Here we go!

Allure Reporting

So what is Allure? Allure is an open sourced framework that leverages junit or testng test results to create an HTML report with both high level and detailed views. You can experiment with an Allure sample report to get a general idea of what the report looks like.

Adding reporting to our project

Add the Allure dependency

Let's update the project we've already been working on in previews posts. To get started with Allure we'll need to update our pom.xml file by adding an Allure dependency:

1        <dependency>
2            <groupId>io.qameta.allure</groupId>
3            <artifactId>allure-testng</artifactId>
4            <version>2.13.2</version>
5            <scope>test</scope>
6        </dependency>

Add testng.xml file

In my sample project I now have 2 test classes, and I want to run the tests from both classes. To do this I need to create a testng.xml file and put it in the root of my project:

1<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
2<suite name="REST Assured test examples">
3    <test name="All">
4        <classes>
5            <class name="SchemaValidation"/>
6            <class name="MyTestClass"/>
7        </classes>
8    </test>
9</suite>

The file lists the test classes I want to run. Time to run the test!

Run the test suite

After making sure you have started the json server endpoint with json-server --watch music-db.json, let's just run our test from the command line using maven:

1mvn test site

Hopefully you'll see a successful test run:

View results

Now take a look at the files in target/allure-results. These are testng result files that Allure uses to create a report:

In addition, you should have an index.html file in your target/site/allure-maven-plugin folder. Right-click that file and open in browser. The report should open in your browser:

The left nav of the page will give you different views into your tests. Drilling into the suite will give you insights on each individual test:

View test history

Each time you run the report from the command line with the mvn test site command, new testng json results file will get added in the allure-results folder. Allure can use those files to include a historical view of your tests. Let's give that a try.

To get started run mvn test site a few time and watch how the number of file in the allure-reports folder grows.

Now go back a view your report. Select Suites from the left nav, select one of your tests, and click Retries in the right pane. You should see history of test runs for that test:

Of course no one cares about tests that are green. This historical view can give you insights on patterns with failed tests. For example, this test looks pretty flaky!

It's a wrap!

Hopefully this blog post gives you some insights on using Allure to generate reports for your REST Assured tests. I should mention, Allure is not limited to REST Assured reports, and is not even limited to Java-based test frameworks. In addition to Java, Allure supports frameworks for Python, JavaScript, Ruby, Groovy, PHP, .Net, and Scala.

I've barely scratched the surface with this introductory post. Check out the Allure getting started page to learn more.

As I wrap up this Allure post, I am also wrapping up this 4-part series on REST Assured. I've learned a lot working through these posts, and hopefully you've learned something as well.

You can find the complete code for this series of blog posts in my Github project.

Thanks and happy testing!