#functional-testing

I finally got some time to do some more work / hacking / fun with the HTTP Spock extension I started working on before Christmas. The project is still far, far away from being stable. The latest changes are all about trying to come up with a nice API for testing REST / HTTP services.

Unstable as the project and its API might be, the ideas and assumptions behind it remains the same. This post is all about writing down the ideas and perhaps getting some feedback.

Service contracts - The basic idea

Assumption: Most restful web services can be expressed as a Java interface.

Having an interface is useful because it makes it very easy to set up expectations and return data using Spock's awesome interaction API.

Have a look at the example below. It describes the contract for a tiny part of GitHub's API. It takes a username as input and returns a list of usernames following that user. Writing the contract shouldn't require any knowledge of how the input parameters are transmitted or how the response should be formatted. Whether the input parameters are given as a part of the URI path or query, or the response is formatted as JSON or XML is completely irrelevant for the interface.

interface FollowerService { List<String> followers(String username); }

Request to Contract

Assumption: Http requests to well defined and behaving REST APIs can easily be translated into method invocations on the service interface / contract.

In contrast to the contract / interface, this part knows and cares about how input parameters are transmitted and what return data should look like. They serve a similar role as controllers in typical MVC web frameworks. Ultimately they will be invoked as part of a Jetty request handler, but this happens behind the scenes.

class GitHubFollowersRequestToContract { FollowerService contract // a mock @EndpointRoute("/api/v2/xml/user/show/@username/followers") def findUserFollowers = { List followers = contract.followers(routeParams.username) xmlResponse { users(type: "array") { followers.each { follower -> user(follower) } } } } }

The API is inspired by Grails and Ratpack. The challenge is to provide a nice and effective API without re-inventing a web framework. After all, it's all about verifying how our code interacts with external web services, not about re-implementing them.

Finally - the test

Assumption: It's possible to describe this in a very declarative way by leveraging Spock.

The last part of the puzzle is the actual test. You should be familiar with Spock and especially the interaction API in order to wrap your head around this part.

class GitHubExampleSpec extends Specification { // Test server configuration is injected (port / host) @HttpServerCfg HttpTestServer server // The Spock extension will find this field and wire up // the end-point as a Jetty handler in the test server. @HttpServiceEndpoint(GitHubFollowersRequestToContract) FollowerService githubFollowerService = Mock() def "Should be able to look up a users followers"() { given: "a http client capable of parsing xml responses" HTTPBuilder http = new HTTPBuilder(server.baseUri) when: "we invoke the http service using the http client" NodeChild xmlResponse = http.get(path: "/api/v2/xml/user/show/superman/followers") then: "the http request is translated into a method call on our Spock mock" 1 * githubFollowerService.followers("superman") >> [ "lex.luthor", "lana.lang" ] and: "the xml response contains the expected data" xmlResponse.user*.text() == [ "lex.luthor", "lana.lang" ] } }

There are a few things going on here. All of them are (hopefully) explained below:

@HttpServerCfg: The Spock extension will inject a reference to the test server into the field annotated with this annotation. The main purpose of this is to be able to find out on what port the server is running. As a convenience it's possible to get a base URI from this instance.

@HttpServiceEndpoint(GitHubFollowersRequestToContract): Notice the call to the Mock() method. This will construct a Spock mock which in turn will be passed to the request-to-contract implementation configured by the annotation.

given: A http client (HTTP Builder) wired to use the base URI of the test server that is started up in the background. The host name and port number might change depending on environment so hard coding it in the test is not such a good idea.

when: Send a http request to the endpoint our request-to-contract implementation will transform into a method invocation on the injected mock. This is the trick that allows us to make assertions on how our code interacts with the remote service. Wrong parameters, too many or too few requests will cause the test to fail.

then: This is where we set up mock expectations in Spock. We're expecting a single request to /api/v2/xml/user/show/superman/followers. Being a mock, we're also able to specify the data the mock should reply with, and this is the interesting part! The method call invoked by the request-to-contract implementation will return with this data (if the mock expectations are satisfied).

and: The request-to-contract implementation returns XML data with the expected format. The http client automatically converts the XML response to a Groovy data structure we can traverse using a GPath expression.

Why?

End-to-end: The most obvious answer to this question is the ability to unit test a larger part of the stack. This will not only let you verify that http requests are actually being made, but also the how many requests as well as their headers and parameter.

Remote side effects: Another advantage is the ability to run tests without worrying about potential side effects on remote servers. Not all web services come with a sandbox version to play around with or the owner of the service might charge you per request. Using this approach you can write tests to verify that your code interacts with the remote service as intended. Just don't throw out your acceptance / integration tests! You might do a mistake when setting up the service interface or it might even change! Good REST APIs should be versioned and is not suppose to change in a way that breaks existing clients, but when they do you won't notice until it's too late if you run all your tests against a mock.

Parallel development: You could in theory swap mocks for stub implementations and do all your front end programming against the stub services while another team is programming the actual backend. Just agree on contracts up front! I think this is a very intriguing concept that might manifest itself as a Grails plugin... The code

The code is hosted at GitHub as usual :-)

Spock is an awesome testing framework. It's written almost entirely in Java, while Spock tests them self are written in Groovy.

Some days ago I wanted to unit test some http client code for a Grails plugin I'm working on. This inspired me to have a look at Spock's extension API. The API is very good and easy to understand! All you have to do in order to write an annotation driven extension is to implement an annotation, method interceptor and an extension class connection everything together.

Example from the Grails sitemap plugin

Most search engines exposes a service where you can send a "ping" when your sitemap has been updated. These are usually invoked by sending a simple http request. Below is test code from the sitemap plugin.

@WithHttpServer(port=23456) def "pinging dummy search engine works"() { given: "mocked http server" TestHttpServer.mock = Mock(HttpServer) and: "register url for a dummy service to ping" String base = "http://localhost:23456" String pingUri = base + "/searchEngine/ping?sitemap=%s" searchEnginePinger.addPingUrl "dummy", pingUri when: "ping all registered services" boolean allSuccess = searchEnginePinger.pingAll() then: "one ping request for with the expected sitemap url" 1 * TestHttpServer.mock.request("get", "/searchEngine/ping", { it["sitemap"] == "http://localhost:8080/sitemap.xml" }, _) >> "ok" and: "it should return true if every ping returned http 200" allSuccess == true }

Pros and cons

Pros: What I really like about it is that setup and tear down of the http server is handled outside the actual test code. Another really neat feature is that it allows use of Spock's interaction API.

Cons: Error messages could be a lot more helpful. The API for specifying constraints on parameters and headers could be more readable.

La siguiente prueba, la hice para automatizar una prueba funcional en una aplicación Grails. Para la realización de esta prueba se han instalado los plugins webdriver, fixtures y build-test-data. Para instalarlos, simplemente introducir los siguientes comandos:

$ grails install-plugin webdriver $ grails install-plugin fixtures $ grails install-plugin build-test-data

Cada uno de estos plugins, nos da una funcionalidad distinta:

Webdriver nos permitirá automatizar las pruebas de la UI.

Fixtures nos permitirá cargar datos de prueba.

Build-test-data nos permitirá generar estos datos de prueba de forma más sencilla.

A continuación editaremos el fichero application.properties de nuestra aplicación, y en caso de no tener la siguiente línea, se la añadiremos.

plugins.build-test-data=1.1.1

Seguiremos un poco el ejemplo del otro día donde teníamos un controlador (HomeController) para procesar la página principal de nuestro sitio. En primer lugar trazaremos una prueba funcional que valide la UI, para la prueba unitaria del ejemplo del otro día. Si estuviesemos haciendo BDD esta prueba funcional la tendría que haber escrito antes que la unitaria. Definición de la página en test/functional/pages/HomePage.groovy

El siguiente código hará la llamada a la página y comprobará que se hayan recuperado 3 posts.

A continuación, cargaremos los datos de pruebas ayudados por el los plugins fixtures y build-test-data. La ubicación del código de carga de estos datos de pruebas será el fichero grails-app/conf/BootStrap.groovy.

Después escribiremos la vista, para ello crearemos el fichero grails-app/views/home/index.gsp con el siguiente contenido:

Ejecutaremos todas las pruebas de nuestra aplicación, incluidas las funcionales, usando este comando:

$ grails test-app

Si hemos realizado todos los pasos correctamente, veremos como se ejecuta la prueba y esta pasa:

O en caso de dar error, tendremos que ir a buscar los resultados a la carpeta target/test-reports, donde veremos dos carpetas:

html: Donde podremos ver los reportes en formato html