Background Information

In the world of software engineering we frequently face change. The arrival of the concept of “Microservices” is one of those recent events, which changes not only the architecture of software, but also the way teams are organized and how they work together.
Here is one of the definitions of Microservices, introduced by M. Fowler and J. Lewis: “…is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies.” In addition to that, the concept of microservices gives us the flexibility to deploy services independently of each other.

In this blog post we would like to give an overview of how testing changes in the new world of microservices. We will also introduce the details of Consumer-Driven Contract Testing and the frameworks that support it. Looking back to the well known concept of the Agile Testing Pyramid, we want to do a re-check if the idea in it is still applicable and valid when we speak about microservices testing.

For better understanding, we will use the following example model to describe the concept behind microservices testing:

On Figure 1 above, we can see that we have two microservices, communicating between each other via REST. The first service is in the role of a Consumer, while the second is in the role of a Provider. The Provider or Product service in our case provides all product relevant information such as name, type and description, by a given product identification number. The Consumer, on the other hand, fetches the data and consumes the relevant information. There can be of course as many consumers as you need. Later on, we are going to show an example with two consumers, but those should be enough for building up an understanding.

Microservices Testing Approaches

Unit Testing

Should we still have unit tests when we speak about microservices? – Yes, of course unit tests have proven to be a reliable, fast and also not so costly approach for testing your business logic. But here it can guarantee only the functionality of your own service. If you work on the consumer microservice, unit tests cannot guarantee that the Provider service works properly.

End-to-End (System) Testing

Should we still have end-to-end tests when we speak about microservices? –  Yes, having end-to-end tests is important, but when we speak about microservices the level of complexity could be extremely high. Imagine having more than five microservices that have to be deployed up and running in order for the end-to-end tests to be executed.

Integration Testing

Moving up to the next level in the pyramid (we have intentionally left integration testing last)- should we still have integration tests when we speak about microservices? – Yes, definitely. There are a couple of approaches such as service virtualization that can help you simulate the microservices you are communicating to. Some of the known frameworks that provide such functionality are WireMockHoverfly. One of the downsides of doing integration tests here is their complexity and also the number of combinations you need to test and simulate, if you have more than two microservices (which we could assume is the case in a real-world project). And the integration tests can not still guarantee that the service you use, does the job it is supposed to do. WHY? Let’s look at a simple example:
You have an integration test, where the Provider service is mocked. Before deploying your consumer service, you would like to prove that it works properly. On the first run all tests are green and then you can deploy your service (see Figure 2).

However, if you run your service against the production Provider, not the mocked version, it fails. In this example the Provider has changed the data format, und updated the name of a field from “name” to “names“. The integration tests could not catch this problem, because they are being run against an outdated version of the Provider (see Figure 3).

HOW can you fill this gap in your test definitions? Here comes the time to introduce the concept of Consumer Driven Contract Testing.

Consumer Driven Contract Testing (CDC Testing)

Consumer Driven Contract approach is nothing more than an agreement between the Consumer and Provider about the format of data that they transfer between each other. Normally, the format of the contract is defined by the Consumer and shared with the corresponding Provider. Afterwards, tests are being implemented in order to verify that the contract is being kept. One of the prerequisites of CDC Testing is the possibility to have a good, at best case close communication with the Provider service team (for example when you are the owner of the Consumer and Provider). Sharing those contracts and communication on test results is important part of implementing proper CDC tests.

PACT

PACT is an open source CDC testing framework. It also provides a wide range of language supports like Ruby, Java, Scala, .NET, Javascript, Swift/Objective-C. We will go through the two major steps with code examples.

Getting started
Demo Project on Github: Pact Demo Github Repo
For following the guide, you first need to have at least two microservices implemented. Our demo project is based on the product case, introduced earlier. In addition, we use Spring boot for the microservices implementation and Gradle as dependency and configuration management. On the Provider, the PACT Gradle plugin is being used (see https://github.com/DiUS/pact-jvm/tree/master/pact-jvm-provider-gradle for detailed information). The guide consists of two major steps that contain smaller steps. The first step focuses on what should be done on the Consumer side and the second step groups the activities on the Provider side.

Step 1 Actions Overview

1.1. Define the expected result on the Consumer side service
1.2. Generate the PACT file
1.3. Share the generated PACT file with Provider service

 

1.1. Define the expected result on the Consumer side service

We start by implementing the ConsumerDemoTest, where we specify the expected result from the Provider. There are two major approaches, either to extend the base class ConsumerPactTest or to use annotations. The second approach gives us the flexibility to have more than one test per test class and removes the necessity to inherit, thus it can be recommended as the better one. In the source code, you can see both of the options presented.

Firstly, we have to build the request and the expected response for it. In the expectations part, we say that given a GET request on a specific URL path is being performed, a response with status OK, headers and json body is being expected. Afterwards, in the runTest() implementation, we execute the test against a mocked service to set up the Provider expectations and expect the particular values for product name, type and description properties.

1.2. Generate the PACT file

Now, let’s run the test. The result is positive as expected:

If we open the /target folder, we will be able to see that a new subfolder /pacts with a JSON file have been created. This is actually our contract with the Provider:

1.3. Share the generated PACT file with Provider service

Lastly, the Consumer should share the contract with the Provider. You can use either a simple file sharing, cloud service or the PACT Broker. In our demo project, we have used file sharing, but as a more advanced technique it is worth looking at the Pact broker.

Step 2 Actions Overview

2.1. Start the Provider service
2.2. Execute requests against the Provider
2.3. Check verification result

 

2.1. Start the Provider service

Moving on to the Provider service. Let us assume that we have received the Consumer contract and we want to verify it against the real provider implementation. Firstly, the Provider service should be up and running. There are a couple of possibilities in there. You can start it from your preferred IDE (IntelliJ or Eclipse), or use the already implemented Gradle task startProviderService.

2.2. Execute requests against the Provider

Afterwards, you can run the verification test by either using JUnit (see ProviderJunitContractTest implementation) or the Pact Gradle plugin: gradle pactVerify. Resulting output:

 

2.3. Check verification results

In this case, everything seems to be working as expected. The results should be shared of course with the Consumer and then given a green light for the Consumer to deploy its service in production.

However, we also want to monitor what would have happened in the case of a problem. Therefore, for the demo purposes, we are going to inject a failure on the Provider Service and add a Second Consumer service. The new Consumer uses only the product name and type, whereas the first Consumer uses product name, type and description. The idea is presented on Figure 6 below.

We change the Provider property description by removing it:

And then we run our pact verification step (note that now the configured consumers are two). Resulting output:

As a result, we can see that we broke the contract, but only with the first consumer. This is because it uses the description property, while the second one does not use it, thus its contract passes.

Summary

The introduction of microservices opens new challenges for testing. How one can actually test this combination of distributed microservices, that to the user look as one service. The minimum that we can do on top of the well known approaches in the Agile Testing Pyramid is to start defining our contracts. Those contracts can be surely written before the provider service is implemented based on the consumer API requirements. The Consumer Driven Contract Testing creates the need for frameworks supporting it. In this blogpost we have made an introduction to PACT, in the following one we are going to discuss Spring Cloud Contract.

 

 

Leave a Comment

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.

Close