#serversidelove

Over the past couple of days we've rolled out some big changes to make iNewsletter faster and more reliable. These changes take effect immediately so you can enjoy the improvements right away.

We've completely overhauled the publishing process to make it more reliable and faster. Previously when the Publish button was clicked an API call was made to the server which would run through a list of tasks: cache the newsletter, send out email notifications, post a Facebook update etc. If any of these tasks failed there would be no record of it failing. Also, if one of these tasks took a while to complete it would hold up the user, who was waiting for the newsletter to finish publishing.

We've implemented a task queue using Celery and Redis. This allows us to execute tasks asynchronously. Instead of waiting for one task to finish before executing the next task, they can all run at the same time. You'll notice when you publish a newsletter now it takes less than two seconds. Caching the newsletter, sending the emails, posting a Facebook update and posting a Twitter tweet are all done at the same time allowing the user to continue using iNewsletter. Also, if any of these tasks throw an error, we will be emailed of the details of the error straight away, so we can look into what the issue was.

Another improvement we've made is caching and minifying our Javascript. Previously, all the javascript for a particular page was concatenated into one file, and then served to the client. This required significant server time to generate the javascript (sometimes up to 1 second) before it could be sent to the client. And this happened on every single page requested!

We're now caching all the javascript in the database so that when a client requests javascript for a particular page, the app can just retrieve it from the database without having to go to disk. We're also minifying our javascript with Google's Closure Compiler so it will download even quicker.

Another change we've made is to the way a newsletter is cached. When you publish a newsletter, it is compiled into one HTML file and stored on Amazon S3. Then, when the newsletter is requested our server retrieves the file off S3 and serves it. The biggest problem with this method is that Amazon S3 is not designed for low latency. It is great for storing static files such as images and CSS, but it is not great a serving them fast - not as fast as a database. We now store the compiled newsletter in the database, so instead of our server retrieving it off Amazon S3, it retrieves it from our database.

One final change we've made is we've mirrored our Amazon S3 buckets with Amazon Cloudfront, so downloading images from any location in the world should be even faster.