#play-framework

iHeart/play-swagger is a Swagger spec generator for REST APIs built with Play Framework. It generates swagger specs from play route files and case class reflections without the need for any code annotation.

The main principles we follow when designing this library are:

No code pollution (e.g. annotation)

DRY (extract as much information from the code as possible)

When documenting an endpoint, it should be just swagger specification that you need to write. You shall not need to learn another API or spec format.

  Which translate to following features we implemented:

Write your swagger specification in your routes files as comments (json or yml)

Reference your case classes in your swagger spec and play-swagger will generate definitions

Override anything in either the swagger spec in comment or the base swagger spec file.

Here is a simple example that demonstrates how iheart/play-swagger works.

In a cards.routes which is referenced in routes as

-> /api/cards cards.Routes

You write the following swagger spec in comment.

### # summary: create a card # responses: # 200: # description: success # schema: # $ref:'#/definitions/com.iheart.api.Protocol.CardCreated' ### POST /users/:profileId/contexts/:contextName/cards controllers.api.Cards.createCard(profileId: Int, contextName: Option[String])

Note that everything in the comment is just standard swagger definition, and it $refs to a case class CardCreated, which is defined in a Protocol object, which then references another case class Card. Here is the source code:

package com.iheart.api object Protocol { case class CardCreated(card: Card) case class Card(id: Int, name: String) }

With the minimum swagger def written in the routes file as comments, the result swagger specs will look like:

For more detail, please go to the project page: http://iheartradio.github.io/play-swagger/

We recently decided to port code over from a Play Framework app written in Java over to Scala for many reasons like Scala’s expressiveness and asynchronous capabilities. Our code was messy, excessively verbose and highly blocking so the plan was to introduce Scala, clean up as much as possible and to write non-blocking code for new pages.

We started with the Corporate site pages like the homepage, about us, contact, etc. These pages were backed by controller actions that looked like:

This was not incredibly bad, but it was not as fluid as we wanted. The code was also making blocking service calls, which was hurting performance so we decided to first make the action declaration read better. This is when we started looking into Play’s Action Composition. We wanted to use verbs (since these were actions) for action function names. For example, we wanted an action that required authentication and notifications to look like:

Our first thought was to validate the request and inject data into it for use in the action. So, in pseudo code this read like:

We explored ActionBuilder[R], which extends from the ActionFunction[R] and is composable. The main functionality is placed in the invokeBlock[A] method. We easily ported the @Authenticated annotation logic here:

Note the use of a custom request case class AuthRequest, which wraps the original request, layering it with additional data. Basically, this builder accepts a request and a function (that accepts our custom AuthRequest and returns a Future[Result]). Since invokeBlock returns a Future[Result] it was very easy to compose async builders like this. Using a case class for AuthRequest was not the only option. A trait would’ve worked too, but case classes offer pattern matching, which could be useful down the line.

Next, we had to create a similar ActionFunction to inject notifications. This time we played with ActionTransformer[R, P], which transforms a request from type R to type P. Here’s what we did:

Again, using the NotificationRequest case class we were able to layer an optional list of the users notifications to the request, except this time the request was an AuthRequest. Here we used an ActionTransformer that transforms the AuthRequest to a Future[NotificationRequest].

So as you can see we can keep adding more ActionBuilders, ActionTransformers, ActionRefiners and ActionFunctions to layer more functionality onto a request before it reaches the core action code to keep it lean and focused on doing one thing: rendering a view, redirecting, etc. Here’s what our action looked like at the end of all this manipulation:

Maintaining a complex web page full of different components with their own requirements can be challenging. Often you will start with a simple web page but slowly you start adding new components and integrating with other services.

When you have a dynamic, feature-rich page caching becomes more difficult and testing an entire page becomes impractical. Your tests can become difficult to maintain and can become slow.

Micro components

By breaking your page up into separate components you get the following advantages:

Easier to maintain distinct pieces of functionality

Reusability

More flexible caching

Ability to independently deploy components so you can enhance and fix bugs without a "big bang" release.

Inspired by the LinkedIn talk by Yevgeniy Brikman, Rob and I wanted to deliver a proof of concept of this approach for SpringerLink’s abstract/fulltext page. (e.g http://link.springer.com/article/10.1007/s10817-012-9258-1).

We wanted to break up our page into smaller components, pull them in via HTTP asynchronously.

Our proof of concept

The components

We developed 4 simple web pages in the Play framework which represent the main features on an article page.

http://localhost:9000/abstract/{ID} - Displays the HTML abstract for a given ID

http://localhost:9000/fulltext/{ID} - Displays the HTML for the entire document for a given ID

http://localhost:9000/downloads/{ID} - Displays total number of downloads and downloads by country for a given ID

http://localhost:9000/time - Returns the server’s local time

Putting it together

Now that these components are created you can imagine developers being able to work on them without worrying about the rest of the website. They can be tested and deployed completely independently from the big web app.

Now we have the problem that we want to bring all these components to construct a web page

Scala is really great at asynchronous programming, so this is as easy as just making 4 HTTP calls and wrapping them in Future. You can then compose them together, plug it into a template and then you have the page fully rendered.

Play is a great choice for this because it allows you as a developer to have your controllers return Future's and it will handle the rest.

Here is the code for the aggregating controller

https://gist.github.com/quii/a8165f87656c031cf9d6

As you can see, there is no real complexity in breaking up the page into pieces, fetching them asynchronously and then composing it together into a view.

Edge side includes

Edge side includes are server side tags which contain a URL. Proxies like Varnish and Akamai can read and then fetch the HTML from its cache and then output it to the page seamlessly for the user.

High quality diagram

Given our components are now pulled in via HTTP, we wanted it so if you were hitting the website via Varnish the web server would create ESI tags rather than making potentially slow HTTP calls itself.

The simplest way to accomplish this is to edit the getBodyOf function:

https://gist.github.com/quii/e9407898d701d2b0e623

The net result for this is the server can output ESI tags and delegate the hard work to the cache which can store these potentially heavy requests, rather than doing the heavy work itself.

You'll notice that we check for the presence of a Varnish request, the reason for this is we wanted to make sure that development could be done without the need of Varnish running on a local dev box.

By doing this the page remains dynamic, yet highly cacheable. We can serve a cached version of the page depending on your access rights.

What about client side?

You can find plenty of tutorials on bringing in data asynchronously via the client. This approach works fantastically on a number of successful websites, like Facebook and LinkedIn.

However you can face a number of tricky issues around this approach, Twitter famously reverted from a client side approach to moving this complexity back to the server

"To improve the twitter.com experience for everyone, we’ve been working to take back control of our front-end performance by moving the rendering to the server"

I feel client side rendering and pulling of data is great for auxiliary content but it does add complexity to the client which I feel is a bit easier to handle on the server. This post by Karl Seguin goes over some of these problems.

In addition, managing things like SEO and testing are a bit easier on the server side and for the main content on our site we need to make sure these things are bullet-proof.

That said, the micro-component approach doesn’t restrict you using the client side, it actually allows you to be flexible in how you want content to be delivered to your users.

You may prefer in some cases to output a bespoke HTML tag which your client side code knows to look for and then subsequently retrieve the data.

Next steps

The code above is just a proof of concept and as such is a bit messy so it would be nice to encapsulate the behaviour of "Retrieve the contents of this URL by this means" in some kind of re-usable way.

I would like to write a nice declarative way of writing things like

“Fetch the contents of URL-A”

“Output ESI tags for URL-B”

“Output client side include tags for URL-C”

and then be able to compose these calls into a view for rendering.

The code I didn’t show was of the components, which you need to make sure set the appropriate cache headers.

Conclusions

In spite of our page being very dynamic the main bulk of the response can be brought in from a cache, which brings in the benefits of decreased load and faster performance.

This architecture has the happy side-effect that you have to make your components small and independent, which in the long run makes a more maintainable, re-usable platform.

With modern languages the old objections of this approach being too complicated are largely irrelevant. Scala's monadic composition makes this very easy but would be just as doable in other languages such as Javascript with Node.js

You can accomplish all of these things client side too and there is no real right answer as to where this logic should live; but I hope this post gives you a heads up on a different approach which might be more suitable for your needs.

I've recently wanted to set up integration tests that do end-to-end testing of a play-framework application written in Scala, using MongoDB for persistence. The idea was to have an instance of MongoDB spun-up by the test harness, do some stuff to that instance, check data, shutdown instance.

Getting an Embedded MongoDB running

To Embed mongodb into your play project, you can simply use the specs2-embedmongo project. In the background, it uses the very nice Flapdoodle Embed Mongo library.

Add the following to your ApplicationBuild object (./project/Build.scala):

"com.github.athieriot" %% "specs2-embedmongo" % "0.5.1" % "test"

The Embed Mongo library simply creates a command to launch a specified version of mongo on a specific port, and then allows you to shut it down. The play plugin mentioned above wraps the embedded mongo application in a Specs2 BeforeAfterExample trait, which allows you to set a before and after action after each Example (in Specs2 terminology, an example is a test, more here).

Setting up Tests

In your integration tests, you just need to mix-in the trait EmbedConnection with your test spec:

class PatientsSpec extends Specification with EmbedConnection

This should now be set-up to launch an instance of MongoDB for each of your examples. There is an issues, however, with the way the plugin is written. In short, there is some sort of conflict with ReactiveMongo plugin and the Embedded mongo plugin, where the manifestation of the conflict is that the first test runs fine, but subsequent tests fail. ReactiveMongo can't seem to get a connection open to the server. To fix this, add the following to the top of your Specification:

class PatientsSpec extends Specification with EmbedConnection { isolated }

The isolated argument makes each example run in its own "cloned" Specification, so that there is no overlap

Fixing problems, making new ones

This isn't the end, however. The way that the plugin is written, it will attempt to launch multiple instances of Mongo with the same port information, which will obviously fail. To accommodate this shortcoming, I created the following trait, which takes care of launching a new instance of MongoDB on an unique port:

package util import de.flapdoodle.embed.mongo.distribution.Version import play.api.test.Helpers._ import play.api.test.FakeApplication import reactivemongo.bson.BSONObjectID import org.specs2.Specification import com.github.athieriot.EmbedConnection import org.specs2.matcher.ThrownMessages trait MongoSetup extends EmbedConnection { val rnd = new scala.util.Random val range = 12000 to 12999 val portNum = range(rnd.nextInt(range length)) override def embedConnectionPort() = { portNum } override def embedMongoDBVersion() = { Version.V2_4_3} def testMongo[T](code: => T) = { running( FakeApplication( additionalConfiguration = Map( ("mongodb.uri", "mongodb://localhost:" + embedConnectionPort().toString + "/" + BSONObjectID.generate.stringify), ("mongodb.channels", "1") ) ))(code) } }

A little more detail

Here, we generate a random port between 12000 and 12999 (random) and set that port as the port used by our embedded MongoDB instance (notice the mixed-in EmbedConnection)

val rnd = new scala.util.Random val range = 12000 to 12999 val portNum = range(rnd.nextInt(range length)) override def embedConnectionPort() = { portNum }

Next, we use that port + a dynamically generated DB name to set as the mongo URI for our project. For me, I didn't want any interference with any other tests, I wanted the tests to be completely stand-alone. This way, it's assured. You don't have to do this, instead, set it as the name of the DB that you normally use.

def testMongo[T](code: => T) = { running( FakeApplication( additionalConfiguration = Map( ("mongodb.uri", "mongodb://localhost:" + embedConnectionPort().toString + "/" + BSONObjectID.generate.stringify), ("mongodb.channels", "1") ) ))(code) }

Now, I can just simply use this new mixin instead of the EmbedConnection and get all of that boilerplate out of my Specs:

Just a quick mental note for others; When you compile your Play Framework application to an uncompressed war folder:

play war my-play-app -o my-compiled-app-war-output-dir

You can still change the Play Framework Id after compile but before in the WEB-INF\web.xml (play.id).

By doing this you can use an alternative configuration (prefixing properties in configuration.conf) for production/staging/development when deploying to Google App Engine via a DTAP procedure.

Note: When your GAE application boots it will show the following log:

play.Logger info: Forcing PROD mode because deploying as a war file.

play 1.2.4

conf/application.confにて

jvm.memory= -Xloggc:gc.log

jvm.memoryは本来メモリサイズなどを指定する時に使用するオプションのようだけどjvmに渡せるオプションは全部ここで渡せそうなのでgcログの指定もできる。