Spring Cloud Contracts and Spring Cloud Services on PCF


We had a customer recently who were quite interested in the idea of using Spring Cloud Contract (SCC) in order to prevent API ‘drift’ between microservices teams where individual development teams look after the individual API’s that form part of an enterprise application.

Spring Cloud Contract is an implementation of the ‘Consumer Driven Contracts‘ concept for the Spring platform. From the docs…

Spring Cloud Contract provides support for Consumer Driven Contracts and service schemas in Spring applications. [It provides] a range of options for writing tests, publishing assets, and asserting that a contract is kept by both producers and consumers. It works with both HTTP and message-based interactions.

To help the customer get started with SCC, I created a demonstration app for them that used the 1.0 GA version of the Ssoftware. During this process, I learned that SCC is undergoing some rapid development at the moment and this meant that SCC v1.0 was occasionally a little bit ‘temperamental’ when things like filenames or folder locations change within your project. I found first few days with SCC were a learning curve but I did come to love it as the results of my effort were paid off.

I found that Spring Cloud Contract publishes very clear and helpful information about your services, improves the clarity of your testing, adds fantastic wiremock stubbing capabilities, and alerts you early to any API drift which may have occurred between projects (which is essential in multi-team microservice development environments). I’ll definitely be recommending SCC to clients in the future.

To try and help other newbies out, I used the original SCC samples but added plenty of comments into the code and the README’s to make it easier for people to just pick it up and run with it.

The code for the demo is here: https://github.com/benwilcock/spring-cloud-contracts

Extra Credit – Spring Cloud Services on PCF

The same customer also wanted a demo of the Spring Cloud Services (SCS) components for Pivotal Cloud Foundry so I built one and added additional Zipkin tracing (not part of SCS) into the mix. This demo should make it super easy for anyone giving PCF and SCS a trial run. It should even work on PCF Dev (if started with the SCS services) so any Spring developer, even those without PCF access at work can still give it a try.

https://github.com/benwilcock/pcf-spring-cloud-services-demo 

I enjoyed building them, and I hope that these are useful to you.

About the Author

Ben Wilcock works for Pivotal as a Senior Solutions Architect. Ben has a passion for microservices, cloud and mobile applications and helps Pivotal’s Cloud Foundry customers to become more responsive, innovate faster and gain greater returns from their software investments. Ben is a respected technology blogger who’s articles have featured in DZone, Java Code Geeks, InfoQ, Spring Blog and more.

Microservices with Docker, Spring Boot and Axon CQRS/ES


The pace of change in software architecture has rapidly advanced in the last few years. New approaches like DevOps, Microservices and Containerisation have become hot topics with adoption growing rapidly. In this post, I want to introduce you to a microservice project that I’ve been working on which combines two of the stand out architectural advances of the last few years: command and query responsibility separation (CQRS) and containerisation.

In this first installment, I’m going to show you just how easy it it to distribute and run a  multi-server microservice application using containers.

In order to do this I’ve used Docker to create a suite of containers containing all the microservices required to run the demo. At the time of writing there are seven microservices  in this suite; they are:-

 

The source code for this demo is available on Github and demonstrates how to implement and integrate several of the features required for ‘cloud native’ Java including:-

  • Microservices with Java and Spring Boot;
  • Build, Ship and Run anywhere using Docker containers;
  • Command and Query Responsibility Separation (CQRS) and Event Sourcing (ES) using the Axon Framework v2, MongoDB and RabbitMQ;
  • Centralised configuration, service registration and API Gateway using Spring Cloud;

How it works

The microservice sample project introduced here revolves around a fictitious `Product` master data application similar to that which you’d find in most retail or manufacturing companies. Products can be added, stored, searched and retrieved from this master  data using a simple RESTful service API. As changes happen, notifications are sent to interested parties using messaging.

The Product Data application is built using the CQRS architectural style. In CQRS commands like `ADD` are physically separated from queries like `VIEW (where id=1)`. Indeed in this particular example the Product domain’s codebase has been quite literally split into two separate components – a command-side microservice and a query-side microservice.

Like most 12 factor apps, each microservices has a single responsibility; features its own datastore; and can be deployed and scaled independently of the other. This is CQRS and microservices in their most literal interpretation. Neither CQRS or microservices have to be implemented in this way, but for the purpose of this demonstration I’ve chosen to create a very clear separation of the read and write concerns.

The logical architecture looks like this:-

CQRS Architecture Overview

Both the command-side and the query-side microservices have been developed using the Spring Boot framework. All communication between the command and query microservices is purely `event-driven`. The events are passed between the microservice components using RabbitMQ messaging. Messaging provides a scalable means of passing events between processes, microservices, legacy systems and other parties in a loosely coupled fashion.

Notice how neither of the services shares it’s database with the other. This is important because of the high degree of autonomy it affords each service, which in turn helps the individual services to scale independently of the others in the system. For more on CQRS architecture, check out my Slideshare on CQRS Microservices which the slide above is taken from.

The high level of autonomy and isolation present in the CQRS architectural patterns presents us with an interesting problem – how should we distribute and run components that are so loosely coupled? In my view, containerisation provides the best mechanism and with Docker being so widely used, it’s format has become the defacto standard for container images, with most popular cloud platforms offering direct support. It’s also very easy to use, which definitely helps.

The Command-side Microservice

Commands are “actions which change state“. The command-side microservice contains all the domain logic and business rules. Commands are used to add new Products, or to change their state. The execution of these commands on a particular Product results in `Events` being generated which are persisted by the Axon framework into MongoDB and propagated out to other processes (as many processes as you like) via RabbitMQ messaging.

In event-sourcing, events are the sole record of state for the system. They are used by the system to describe and re-build the current state of any entity on demand (by replaying it’s past events one at a time until all previous events have been re-applied). This sounds slow, but actually because events are simple, it’s really fast and can be tuned further using rollups called ‘snapshots’.

In Domain Driven Design (DDD) the entity is often referred to as an `Aggregate` or an `AggregateRoot.`

The Query-side Microservice

The query-side microservice acts as an event-listener and a view. It listens for the `Events` being emitted by the command-side and processes them into whatever shape makes the most sense (for example a tabular view).

In this particular example, the query-side simply builds and maintains a ‘materialised view’ or ‘projection’ which holds the latest state of the individual Products (in terms of their id and their description and whether they are saleable or not). The query-side can be replicated many times for scalability and the messages held by the RabbitMQ queues can be made to be durable, so they can even temporarily store messages on behalf of the query-side if it goes down.

The command-side and the query-side both have REST API’s which can be used to access their capabilities.

For more information, see the Axon documentation which describes how Axon brings CQRS and Event Sourcing to your Java apps as well as lots of detail on how it’s configured and used.


Running the Demo

Running the demo code is easy, but you’ll need to have the following software installed on your machine first. For reference I’m using Ubuntu 16.04 as my OS, but I have also tested the app on the new Docker for Windows Beta successfully.

  • Docker (I’m using v1.8.2)
  • Docker-compose (I’m using v1.7.1)

If you have both of these, you can run the demo by following the process outlined below.

If you have either MongoDB or RabbitMQ already, please shut down those services before continuing in order to avoid port clashes.

Step 1: Get the Docker-compose configuration file

In a new empty folder, at the terminal execute the following command to download the latest docker-compose configuration file for this demo.

$ wget https://raw.githubusercontent.com/benwilcock/microservice-sampler/master/docker-compose.yml

Try not to change the file’s name – Docker defaults to looking for a file called ‘docker-compose.yml’. If you do change the name, use the -f switch in the following step.

Step 2: Start the Microservices

Because we’re using docker-compose, starting the microservices is now simply a case of executing the following command.

$ docker-compose up

You’ll see lots of downloading and logging output in the terminal window as the docker images are downloaded and run.

There are seven docker images in total, they are mongodb, rabbitmq, config-service, discovery-service, gateway-service, product-cmd-side, & product-qry-side.

If you want to see which docker instances are running (and also get their local IP address), open a separate terminal window and execute the following command:-

$ docker ps

Once the instances are up and running (this can take some time at first) you can have a look around immediately using your browser. You should be able to access:-

  1. The Rabbit Management Console on port `15672`
  2. The Eureka Discovery Server Console on port `8761`
  3. The Configuration Server mappings on port `8888`
  4. The API Gateway Routes on port ‘8080’

Step 3: Working with Products

So far so good. Now we want to test the addition of products.

In this manual system test we’ll issue an `add` command to the command-side REST API.

When the command-side has processed the command a ‘ProductAddedEvent‘ is raised, stored in MongoDB, and forwarded to the query-side via RabbitMQ. The query-side then processes this event and adds a record for the product to it’s materialised-view (actually a H2 in-memory database for this simple demo). Once the event has been processed we can use the query-side microservice to lookup information regarding the new product that’s been added. As you perform these tasks, you should observe some logging output in the docker-compose terminal window.

Step 3.1: Add A New Product

To perform test this we first need to open a second terminal window from where we can issue some CURL commands without stopping the docker composed instances we have running in the first window.

For the purposes of this test, we’ll add an MP3 product to our product catalogue with the name ‘Everything is Awesome’. To do this we can use the command-side REST API and issue it with a POST request as follows…

$ curl -X POST -v --header "Content-Type: application/json" --header "Accept: */*" "http://localhost:8080/commands/products/add/01?name=Everything%20Is%20Awesome"

If you don’t have ‘CURL’ available to you, you can use your favourite REST API testing tool (e.g. Postman, SoapUI, RESTeasy, etc).

If you’re using the public beta of Docker for Mac or Windows (highly recommended), you will need to swap ‘localhost’ for the IP address shown when you ran docker ps at the terminal window.

You should see something similar to the following response.

* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080(#0)
> POST /commands/products/add/01?name=Everything%20Is%20Awesome HTTP/1.1
> Host: localhost:9000
> User-Agent: curl/7.47.0
> Content-Type: application/json
> Accept: */*$ http://localhost:8080/commands/products/01
< HTTP/1.1 201 Created
< Date: Thu, 02 Jun 2016 13:37:07 GMTThis
< X-Application-Context: product-command-side:9000
< Content-Length: 0
< Server: Jetty(9.2.16.v20160414)

The response code should be `HTTP/1.1 201 Created.` This means that the MP3 product “Everything is Awesome” has been added to the command-side event-sourced repository successfully.

Step 3.2: Query for the new Product

Now lets check that we can view the product that we just added. To do this we issue a simple ‘GET’ request.

$ curl http://localhost:8080/queries/products/1

You should see the following output. This shows that the query-side microservice has a record for our newly added MP3 product. The product is listed as non-saleable (saleable = false).

{
  name: "Everything Is Awesome",
  saleable: false,
  _links: {
    self: {
    href: "http://localhost:8080/queries/products/1"
    },
  product: {
    href: "http://localhost:8080/queries/products/1"
    }
  }
}

That’s it! Go ahead and repeat the test to add some more products if you like, just be careful not to try to reuse the same product ID when you POST or you’ll see an error.

If you’re familiar with MongoDB you can inspect the database to see all the events that you’ve created. Similarly if you know your way around the RabbitMQ Management Console you can see the messages as they flow between the command-side and query-side microservices.

About the Author

Ben Wilcock is a freelance Software Architect and Tech Lead with a passion for microservices, cloud and mobile applications. Ben has helped several FTSE 100 companies become more responsive, innovate, and agile. Ben is also a respected technology blogger who’s articles have featured in Java Code Geeks, InfoQ, Android Weekly and more. You can contact him on LinkedIn, Twitter and Github.