The Beanstalk

From EC2 Classic to EC2 VPC with minimum downtime

Recently we moved our whole production infrastructure from EC2 Classic to EC2 VPC. We are very happy with this move and this is an attempt to outline the motivation for this move and our experience doing it.

Cloud 66 Production Infrastructure - before the move We have servers in all the other cloud providers we support but the main bulk of our servers run on EC2 and Rackspace. EC2 runs our production servers and Rackspace is used for our supporting systems: admin systems, monitoring tools, dashboards and more.

Our EC2 servers run Ubuntu 12.04 backed by EBS volumes and in case of some servers (like our Redis, file servers and metrics servers) we use high IOPS volumes.

We use Elastic Load Balancer to handle our frontend traffic and Multi AZ AWS RDS for our DB instances with read replicas for our reporting services.

As well as that we have Redis, Collectd, ElasticSearch and multi-terabytes of S3 storage. I'm going to write more in future about each one of these components.

EC2 Classic We use a group of gateway servers to connect to deploying (client) servers. Those servers require to have fixed IP addresses to the client's firewalls can be configured to allow only Cloud 66 IP addresses in. We achieve this by using Elastic IP addresses for our gateway servers. However this doesn't work when deploying to servers running in the same data centre as they will be called with the internal IP address of our gateway servers. With EC2 Classic you cannot fix the internal IP address of the servers. So every time a server is restarted you are going to get a new internal IP address.

This made our maintenance jobs and upgrades very tricky.

EC2 VPC Amazon VPC or Virtual Private Cloud works by allocating a private network subnet to your servers. You can separate your servers into different subnets inside your own VPC as you would do that in your own data centre. This give you a lot of flexibility and control over security and traffic routing.

Around a month ago, we started our move from EC2 Classic to EC2 VPC. This was a major move as we built our whole production site from scratch and changed the way we arrange our servers and route our internal and external traffic.

While we use Cloud 66 production to deploy Cloud 66 Staging, Development and Test stacks as well as all of our support systems, there is one stack we can't deploy with Cloud 66 production: Cloud 66 production itself. It's like pulling oneself over a fence with one's bootstraps!

The move from EC2 Classic to EC2 VPC The most important thing about the move is to know exactly what you want to achieve by the end of it. Once you have a clear picture of the endgame, it becomes a relatively easy task.

Currently you can't move an EC2 instance into a VPC: you have to build a new server from an image in VPC. You also cannot use Elastic IPs (EIP) from EC2 Classic in EC2 VPC. Those two where the biggest challenges we had during our move.

Step 1: Design your VPC Our new VPC has 5 different subnets: a public subnet, two private subnets and two RDS MultiAZ subnets.

The public subnet includes our Internet Gateway and our load balancers.

Each private subnet is protected with a different set of firewalls and access keys and houses all of our servers. A NAT server connects the private subnet 1 to the public subnet. This is an unusual setup to have on VPC: usually web servers and other public facing servers live in the public subnet while the backend server a located in the private subnet. However our requirement for all of our servers to reach the outside world meant we needed to have a single EIP to act as a one way gateway from our servers to the outside world. This is achieved with a NAT node.

Step 2: Rebuild the servers We started the move by taking the latest images from our EC2 instances and building them again within our new VPC. We also built a new RDS cluster in the VPC.

The easiest way to rebuild the servers is to shut them down, take an AMI image (if you don't have one already) and use that image to fire up the new EC2 VPC instance.

Step 3: Warm up whatever you can We use Redis quite heavily at Cloud 66. From Sidekiq to storing temporal logs, Redis has been a great asset to have. After building our VPC Redis instances, we setup the new VPC servers to be readonly slaves to our Classic Redis master. This would keep new instances warm and ready to switch over.

Currently it is not possible to setup RDS replication between EC2 Classic and EC2 VPC, so we didn't do that.

Step 4: Reduce your TTL on DNS This is always the case when switching over traffic between two servers. DNS records need to be updated and any router on the way can cache the records which can cause delays in the switch over for some visitors. The most common way to deal with this is to reduce your DNS record TTLs to a low number like 5 minutes at least 24 hours before the move. This way all the caches will have to check for new values every 5 minutes. This does not guarantee a smooth switchover as some routers might not honour the TTL.

Also, since we use Amazon Route 53 for our DNS, we use ALIAS records which make switching over much quicker for load balancers.

Step 5: Tunneling With everything in place we started the switch over. To minimise downtime, we used SSH tunnels between EC2 Classic and EC2 VPC instances to switch over different services gradually: a quick shutdown of the service allowed us to use our latest DB backup to build our VPC database. Once database was in place we used an SSH tunnel from the Classic services to connect to the new VPC databases (remember, the Classic instances are considered public internet as far as VPC is concerned so you can't just get the old servers to connect to the new DB servers).

We did the same thing for Redis: switching VPC redid servers to master and tunnelling Redis traffic from Classic servers to VPC Redis allowed us to replace old Redis servers as well.

Step 6: The final move With all data being served from the new VPC instances, we shut the site down again shortly to move our data volumes. This involved shutting down our file servers, unmounting EBS volumes from those and mounting them back on the new VPC file servers.

With the last piece of backend service being moved, we then switched our DNS records to the new load balancers and restored the production traffic fully on the new VPC stack.

Conclusion The move to VPC has given us great flexibility and control over our security, traffic routing and availability. It wasn't easy and we had to plan for it. The most important thing was making sure we understand the limits of VPC and designing the new stack around that. Things like NAT servers, routing tables and internet gateways had to be configured to allow our unusual traffic requirements through.

Cloud 66 and Honeybadger integration

Today we are excited to announce our partnership with Honeybadger: A great exception monitoring service used by many great companies from Pivotal Labs and Thoughtbot to DigitalOcean and eBay.

This partnership allows us to bring the goodness of Honeybadger to Cloud 66 users everywhere. All you need to do is to add the Honeybadger add-on to your stack which notifies Honeybadger about the deployments you do on Cloud 66 and resets the exception counters, complete with the name of the person who triggered the deployment and the git reference of the release.

We have been using Honeybadger here at Cloud 66 and really love the complete picture they present about the errors and exceptions in the apps as they happen.

Speeding up deployments.

Step 1: Parallel Deploys

You asked and we listened! We're working hard on making your stack builds and deployments faster, and are happy to announce a piece of that puzzle: parallel deployments.

You can now deploy against each server in a stack simultaneously, which has some very big speed implications in stacks with more than one app server.

We used a well-known open source project to test this new feature, and the results are promising. For example, deploying our app to two app servers in parallel took 53% of the time it would take doing so in serial! And the deploy time only gets better from there:

This is the first step in our effort to make deployments faster. We are not going to stop here and will introduce more improvements continuously within the course of coming months.

Deploy Shopify Dashing with Cloud 66

Here at Cloud 66 there is a lot to measure: from site visitors to signups and temperature outside to the tweets about us. But more importantly we have a ton of metrics that are about our production systems: number of servers running, error rates, cloud provider metrics and server heartbeats. On top of that we display KPIs like MRR as well as London Underground's realtime line updates for stations near the office.

All this needs a good, flexible and configurable dashboard tool. We looked at many strong contenders and finally settled on Dashing an elegant Sinatra based solution by the awesome guys of Shopify.

Obviously we used Cloud 66 to deploy Dashing to Rackspace London with little effort. Here is how we did it:

Build a Dashing dashboard. It is really simple.

Add gem 'sinatra' to your Gemfile. This will tell Cloud 66 that Dashing is a Sinatra app, which it really is.

Add a file called Procfile to your project's root with this line: 

custom_web: bundle exec dashing start --socket /tmp/web_server.sock --pid /tmp/web_server.pid -e $RACK_ENV -d

This will instruct Cloud 66 to use a custom web server (in this case dashing instead of Passenger)

Now you can check in your project into your favourite git repository and build a stack with it with Cloud 66. 10 minutes later you have your own awesome dashboard up and running.

These are the widgets we use here:

Time

London Underground Tube Lines

Weather

MRR

Assembla Milestone Open Tickets

Number of test servers running

Google Analytics Metrics

Number of Signups

Tweets about us

Unassigned Helpscout Tickets

We're very happy to announce our new partnership with Flatiron School, a New York based program that offers full-time, intensive classes (plus pre-work) "designed to turn you into a web developer". We share their belief that programming can be learnt by anyone, and they empower their students with the equivalent skill-set and experience of an entry-level Ruby developer in just 12 weeks. Students don't need any programming experience to attend, all they ask for is a "strong desire to create and a whole lot of grit".

Following the program, they help you get a job placement at one of their partner companies, which range from companies like Artsy to Statuspage.io. One of the aspects that sets Flatiron School apart from similar programs is their focus - they don't have fancy offices and millions of dollars in venture capital. They're also not trying to scale geographically or across curriculums, which allows them to focus on what they do best.

In a similar vein, they intentionally keep a low acceptance rate into their program, which helps them maintain a diverse and passionate group of students. For example, former students have backgrounds as diverse as professional poker players and Major League Baseball scouts to Wall Street traders and lawyers.

Our partnership

Cloud 66 is delighted to offer all Flatiron students free services for 6 months, to help them get their applications up and running in the cloud of their choice as smoothly as possible.

At Cloud 66 we pride ourselves in helping our customers deploy and manage their code on their cloud of choice and in any part of the world. As part of this effort, we're happy to announce that we now support deploying to DigitalOcean's Singapore datacenter. This will be greatly beneficial to users throughout Southeast Asia and nearby regions such as Australia and India.

Our customers in Asia and Australia now have the option of deploying in Hong Kong (Rackspace), Singapore (AWS, DigitalOcean), Sydney (AWS, Rackspace) and Tokyo (AWS, Linode). Aside from Asia and Australia, we also support Europe, North and South America.

Replicate your database between different stacks, data centers and clouds.

We now make it easy for you to replicate your databases between stacks. There are many use cases for this, but here are the main ones:

Preparing a failover for an existing stack

This allows you to have a replica of your production database somewhere else (eg. a different region) in case of disaster. Should the master database fail, you can easily switch over to use the slave.

Moving your stack to a different cloud vendor/region

You find out that most of your users are based in Asia, while your servers are in Europe. You can migrate your stack from one data center to another, setting up a slave database in the new region that will constantly read the master database. Once you're happy with your migration, you can simply switch the slave to a master. 

Using your database in applications like reporting tools

It's often not desirable to use your production database for non-critical functions like reporting tools. For example, if a user queries the database incorrectly, this could bring down your production database! In this case, it's better to have a replicated database that simply reads from your production database.

Setting up replication between stacks involves just a few clicks for our users, but this is what happens in the background:

We take a full backup of the database in your source stack and restore it on the target server

The target server is configured to be a slave of the source database, and vice-versa

The relevant environment variables are updated for use in your code and scripts

Realtime notifications: Hosted or Self-hosted?

As the web moves towards being more and more realtime across platforms and applications, developers are increasingly confronted with the decision of using hosted versus self-hosted realtime services. Before the introduction of WebSockets, there were plenty of workarounds for achieving this functionality - polling for new messages by the client versus pushing messages straight from the server.

However, these two solutions resulted in unnecessary load on the server as well as timing problems, whereby a message on the server may reach the client before the client asks for it for example. This was not realtime communication in any sense of the word.

WebSockets

With WebSockets, we have a full-duplex single socket connection over which messages can be sent between client and server. This creates truly realtime communication, allowing browsers to communicate in realtime between server and client by pushing messages directly to the browser that needs the information.

There are countless libraries, startups and open source projects that make use of WebSockets, which makes it easy for developers to quickly use this technology in their projects. For example, Faye is an open source publish-subscribe messaging system that works in all major web browsers.

Faye (self-hosted) example

There are lots of great resources out there on using Faye, but for illustrative purposes we'll create a very simple  app:

1. Install the Faye and Thin gems:

gem install faye gem install thin

2. Create a Rackup file in your application root (faye.ru):

require 'faye' faye = Faye::RackAdapter.new(:mount => '/faye', :timeout => 45) run faye

3. You can now start the Faye server in your command line (it'll run on port 9292):

rackup faye.ru -s thin -E production

4. This server will have a Javascript file that needs to be included in our application layout. The name of the file depends on the :mount option provided above:

<%= javascript_include_tag :defaults, "http://localhost:9292/faye.js" %>

5. Now we'll create a new client - add this code to your application.js

$(function() {     var faye = new Faye.Client('http://localhost:9292/faye'); });

6. Create a channel and subscribe to it in the client, also in your application.js

$(function() {     var faye = new Faye.Client('http://localhost:9292/faye');     faye.subscribe('/test', function (data) {     alert(data);     }); });

7. Check if it works by opening your Rails application view in your browser. Now cURL it from the command line:

curl http://localhost:9292/faye -d 'message={"channel":"/test", "data":"hello"}'

This should result in a notification in your browser!

Although open source solutions like Faye are relatively easy to set up, scaling real time systems can quickly become a nightmare. And there's not very much documentation about this out there.

Pusher (hosted) example

Luckily, this is where hosted solutions like Pusher come into the picture - a hosted API that makes it incredibly easy to implement scalable realtime systems. Whether you're developing for iOS, Android or web, Pusher and similar services have plug-and-play libraries that can get you up and running in no time.

1. Stick this Javascript in your client.

<script src="http://js.pusher.com/2.1/pusher.min.js" type="text/javascript"></script> <script type="text/javascript"> var pusher = new Pusher('7c8dcbc303f793f4b4b1'); var channel = pusher.subscribe('test_channel'); channel.bind('my_event', function(data) { alert(data.message); }); </script>

2. Use cURL to check if it works!

curl -H "Content-Type: application/json" -d "{\"data\":\"{\\\"message\\\":\\\"hello world\\\"}\",\"name\":\"my_event\",\"channel\":\"test_channel\"}" \ "http://api.pusherapp.com/apps/66571/events?"\ "body_md5=8a3501faef6636ca9a5ebbe6f31b5409&"\ "auth_version=1.0&"\ "auth_key=AUTH_KEY&"\ "auth_timestamp=1392908459&"\ "auth_signature=c27b0df7e42efcfa2af35db0db7f7f7c97bbd3a63f19f684e1e7547ccc5f4cf6&"

Fix nogroup and nobody when using NFS on Linux

You can use NFS to mount a drive on a different server. This is useful when you need to share a volume between different servers however in ext4, Linux group and user access rights to directories and files can cause issues like showing up a nogroup or nobody on the mounted drive.

To make sure both servers use the same group and user, first make sure the desired group and user exists on the destination server (the one mounting the remote NFS volume).

Now edit the /etc/idmapd.conf file:

sudo vi /etc/idmapd.conf

These are the important parts to change:

Domain under [General]: It needs to be the same on both servers.

Also add the following to the end of the file:

[Translation] Method=nsswitch

The end result will look something like this:

[General] Verbosity=7 Pipefs-Directory=/var/lib/nfs/rpc_pipefs Domain=gw.example.com [Translation]  Method=nsswitch

Once done, you need to restart the idmap service on both client and server:

service idmapd restart

And the NFS server on the server:

Put your free startup cloud credits to work

tl;dr;

Startups receive tons of free cloud computing credit from infrastructure providers, but hardly use them because managing servers is not their specialty. Put your credits to use now!

Startup Cloud Credits Everywhere

As the cloud computing space grows by leaps and bounds, so does the competition amongst the providers of the crucial computing horsepower that underpins the industry. These players are “poaching each other’s employees, slashing prices” and “upending long-held strategies” to fight it out over one of the fastest-growing areas of technology. 

For consumers of cloud computing, this competition translates into great value, and startups are perhaps the biggest beneficiaries of this. Startups around the world are often given tens of thousands of dollars in credit from major infrastructure as a service (IaaS) providers, who hope to get a foothold in up and coming companies before they make it big.

Amazon for example offers $12,000 of credit for its AWS offering, while Rackspace offers $36,000 and Microsoft Azure offers $60,000. These are incredible offers for cash-strapped startups, and brings the lean startup mentality to a whole new level.

Despite these and many other benefits, there is one big problem - namely that many startup founders have no expertise in managing servers in these complex cloud infrastructures. We have personally witnessed a handful of startups who simply don’t know what to do with all these credits and ultimately end up not using them!

Other startup founders might decide to learn about configuring their server infrastructure to leverage these credits, but this ends up being detrimental to their main value-add. Still some startups choose to use PaaS providers to make things easier, but end up having a blackbox infrastructure with unsustainable pricing.

Our Offer

With all this in mind, Cloud 66 would like to announce a startup offer of their own, which finally allows startups to concentrate on building their products while managing their cloud infrastructure transparently on any cloud. As we support seven different cloud providers with data centers all over the world, we’re confident that we can help startups everywhere leverage their free credits and stay cost-effective at a time when they need it most.

Get $10 free hosting with the OpenStack Canadian Cloud Provider Vexxhost

Today we are happy to announce a deal with our partners Vexxhost, the awesome Canadian OpenStack cloud providers:

All Cloud 66 customers can get $10 free credit from Vexxhost by just linking their Cloud 66 account to Vexxhost.

Vexxhost prices start from $5 per month for a 2 CPU Cores, 512MB Memory and 2TB of transfer.

Claim your $10 free credit by selecting Vexxhost from the list and clicking on the "Sing up for Vexxhost - get $10 credit". You will have to then sign up with Vexxhost.

Your new Vexxhost account will be added to your account automatically. No need do anything else. As simple as that!

With the London Underground strike in full swing, we wanted to keep an eye on the state of the lines around the office at Cloud 66!

So we wrote a simple widget for the awesome Shopify Dashing.

If you are in London and want to have a realtime dashboard of London Underground Lines, why not use the London Underground Tube Line Status Widget with your Dashing dashboard.

How much does it cost your business to setup and run a Rails production app?

Many want to know the cost comparison of building and managing their infrastructure with Cloud 66 vs the alternatives. We thought we should see if we can put a number on it!

What is our sample app?

For this we chose a very small production Rails app with very common requirements: scalable (load balanced, backed by a single database and protected by SSL). There is nothing special in this stack that requires specific components to be installed.

Framework: Rails 3

DB: PostgreSQL

Environment: Production

Other requirements: SSL

Heroku

Let's see how much it is going to cost us to run this on Heroku:

1x Dyno (512MB, 1x Shared CPU) = $0.05 x 730 = $36.50

PostgreSQL: Standard Yanari (400MB RAM, 64GB HDD) = $50

SSL certificate = $20

Total: $106.50 per month + $0 setup

DIY

We all know the value of automation when it comes to infrastructure and deployment. We automate things not to speed them up, but to make them repeatable, testable and less prone to human errors.

For our app, we are going to assume an experienced DevOp is going to put the needed scripts together. They can be Chef, Puppet, Ansible, Saltstack, Capistrano or anything our dev is comfortable to start and maintain within the business.

1x VPS (512MB, 2x CPU) = $5

1x Load Balancer = $5

SSL certificate = Free

PostgreSQL (1GB RAM, 30GB HDD) = $10

Maintenance and updates = $60

Monitoring service 3x server: $45

Total: $125.00 per month + $300 Setup

Cloud 66

Here is how much it is going to cost to do the same thing on Cloud 66.

1x VPS (512MB, 2x CPU) = $5

1x Load Balancer = $5

SSL certificate = Free

PostgreSQL (1GB RAM, 30GB HDD) = $10

Cloud 66 Charge: $19 (first server of the account) + $9 (second server of the stack)

Maintenance = 1x Backup = $12

Monitoring = Free

Total: $60 per month + $0 setup

Note: this will drop to $50 per month for the second stack as only the first server of the account is $19.

What's the setup fee?

Assuming a salary of $60k for a web developer (you can add 50% on top of that to calculate the cost to the business). $60k is around $40 per hour for the dev ($60 per hour to the business).

We are assuming the developer involved knows everything needed and is going to write the scripts (Bash, Chef, Puppet, Ansible, Capistrano,...) following best practices.

The scripts will have to perform the following:

Setup the servers, web server, database server, access rights, load balancer, SSL certificate

Deploy the app

Manage the firewalls

Backup DBs, send them offsite and rotate them

Setup server protection and updates

Setup monitoring for new servers.

We are going to assume around 5 hours of total time spent on setting all this up for the stack: $300

What's the maintenance and updates fee?

Doing DIY also will require around 1 hour per stack per month maintenance on average: patching servers and the framework with the latest updates, updating scripts with app requirements, managing firewalls and access rights etc.

Assumptions

Cloud provider: DigitalOcean or Vexxhost.

Monitoring provider: Server Density

Developer salary: $60k per year.

How consumerization of the enterprise internet fueled by the Lean movement and venture money can lead to simplistic products.

This is part of our Software Economy Series.

Remember the days of the VHS video recorder? How difficult it was to set a recording time for your favourite program?

Two technologies (Digital Video Broadcasting and hard disk technology) made it possible to enjoy products like Tivo. Technology however is not always the reason things can get easier: Compare the latest Creative Nomad MP3 players and the earliest iPods. Even if Apple picked up the micro hard disk technology from Toshiba earlier than others, it was the clever design and user interface that made a lot of the difference.

Consumer internet products are getting better and easier to use. Mobile apps that check for train routes and times are the best examples because most of them are still much more difficult to use (UK National Rail iOS app) while the new generation of them use the same technologies (mobile, SaaS and touch) to deliver a much better user experience (Citymapper).

Most of this has been achieved by simplicity through minimalism: stripping away clutter and non-core features. What is left is a simple, minimalist and easy to use user experience. User interfaces that are optimised for 80% of the use cases.

On the other hand in the enterprise world the user experience movement has been making inroads for a while. The new generation who has grown up using Facebook and Twitter's simple design cannot understand why enterprise software should be so difficult to use.

This generation of users have champion companies like Huddle against Microsoft SharePoint and Box against EMC storage systems.

Many developers who have seen the opportunity in the enterprise internet world come from the consumer internet space, bringing their product design skills with them.

This is where problem starts.

In the consumer word, simple is minimal, reduced of clutter and focusing on core features. "Do less, do it better" is the mantra. I am typing this post on iA Writer, a prime example of this trend.

However a literal translation of this into the enterprise products doesn't produce simple products. It produces simplistic products.

A new generation CRM app starts with a correct assumption, a vision that sees enterprise CRM software as complex. It sets out to change that by brining simplicity to this system: instead of spending days integrating your new CRM tool into the rest of your systems, importing current customer records and training staff on how to use the system, just drop a Javascript on your website and you're done.

The concept is intoxicatingly simple and it works for many customers when they start. But then it runs into problems: new workflows and requirements from customers who are growing starts to push the company to make a choice: focus on smaller customers and simpler workflows and keep the product simple or to add new features to the product making it more complex as it matures.

Many enterprise internet companies come to making this choice sometime during their life. Companies either end up with simplistic solutions to complex problems and companies that grow to produce solutions as complex as the ones they set out to disrupt.

I think this is caused by a combination of different factors:

Team

Many of these products are started by teams that don't have any experience of actually working for a large company with complex requirements that is their dream customer. A CRM company driven by devs who have never worked in the sales team of a large enterprise is a typical example.

This is a curse and blessing at the same time.

In short term the fresh eyes they bring to the play is extremely valuable. They see things in the light of the latest product design trends of the market. They bring experience from the consumer internet space. And when it comes to the business the naïvety of being able to solve the complex enterprise problems works great in breaking into new markets and disrupting them. This is arguably the most potent value of startups and works very well in the short term.

But in the long term it can also be their undoing. The natural resistance towards adding features their customers want - achieving simplicity purely through minimalism - and seeing them as clutter means they usually confuse lack of features with simplicity.

MVPs and The Lean Movement

Another reason for this is the Lean Movement. As great as it is for many use cases, the Minimum Viable Product and the Lean Movement can entrench this confusion between power and simplicity in favour of a simplistic product that is good as a prototype but fails to grow deeper into the enterprise after that, resulting in large churn that is visible only after a while.

Money

The third reason is the capital markets backing those startups. VCs mostly backing consumer internet companies who are seeing a bubble in that market with expensive deals and silly valuations move to the enterprise market with little knowledge of the space and applying the same economic mathematics of the consumer internet to the enterprise SaaS companies: build a simple product, forget unit economics by subsiding the customers with VC money, grow very fast and very wide, raise a ton of money not as a business requirement but as a stake in the ground that works like a benchmark for a high valuation when it comes to a quick exit, usually to a market incumbent that kills the product and keeps the team.

We have all seen many of those businesses where we loved the simplicity of the product, the fresh approach they took just to witness their product being destroyed by the acquirer: think Sparrow, Lala, Delicious and pretty much anything Yahoo! touched until recently.

A good business for a few, a bad one for many

On average building an enterprise business takes longer than a consumer oriented one. The winners of this game are the founders and the early investors who usually are bought out by the funders of the next rounds.

The losers of the game are two groups: the customers who usually get screwed by the new owner of the company and the other enterprise SaaS vendors who want to build a sustainable business that is fuelled by customer generated revenue and a great product. Those businesses have to ensure their customers that their more expensive - no subsidised - product is better for them in the long run because they are not going to sell out to a big incumbent just because they have taken so much debt.

Apply the rules accordingly

I think the best way to avoid building simplistic products is to understand that simple doesn't mean rudimentary. Simple does not mean simplistic. Minimal does not mean featureless.

Instead of blindly applying the consumer internet product design rules to enterprise software, we can be more clever about building products that are flexible and powerful but are still simple through conventions, user education, strong product culture, clever use of user interface elements and most importantly widening our arsenal of tools to solve complex problems.

Customize your Nginx configuration

Cloud 66 is built around the main concept of simplicity. Our mission is to make building servers and running applications simple.

The main flow of an application deployment at Cloud 66 is to take the source code and in a short period of time have it running on our own servers. We also make running and managing your applications simple. Once an app is deployed and running, Cloud 66 takes care of backups, scaling, portability and many other aspects of keeping your app up and running as well.

This is all very simple with Cloud 66. However, we don't want to be simple by being simplistic. There is a fine line there and we would like to stay on the right side of it by providing enough flexibility so you can benefit from the automation we bring to your infrastructure while having full control on all aspects of it.

This is where CustomConfig comes in.

CustomConfig allows you to see and modify the configuration templates used for the deployed components on your servers.

Take nginx for example. The default nginx configuration deployed on all Cloud 66 servers is secure and follows the best practices. It is automatically modifed as you add more servers, load balancers, SSL certificates and takes care of different application servers you use: passenger or a custom one like unicorn.

But what if you want to add some custom configuration to your nginx across your stack, like forcing an SSL redirect or banning a specific IP address?

Until now you had to break the symlink for the nginx configuration and added your custom configuration to the server. This was flexible, but you would lose the future upgrades that are constantly delivered to all Cloud 66 stacks.

Another option was to have the custom configuration done as a post deploy hook. This was also not an ideal solution as it required shell scripts and relying on the underlying configuration not changing much.

CustomConfig solves that problem by opening up the nginx configuration template so you can configure it however you like, while keeping it under Cloud 66 management.

Here is how:

Select CustomConfig from your Server Group detail page:

And edit the nginx configuration file in Liquid:

and deploy it to all of your web servers at once.

You can also see the history of your changes the way you are used to!

We have enabled CustomConfig for Nginx to all of our customers for free today. Other components like MySQL and HAProxy will follow soon!

OpenStack Canadian Cloud Servers

Today we are excited to announce our new fully supported cloud vendor at Cloud 66: Vexxhost. Vexxhost is our first OpenStack and first Canadian cloud provider.

During the course of past year, we have been receiving a lot of requests to support new cloud providers that are reliable and affordable. We received a lot of suggestions from our awesome customers and finally settled on one we think is one of the best.

Our criteria for choosing our next cloud vendor was one with great technology, attention to user experience and support and a price point that matches our growing customer base.

We think we found all of them in Vexxhost.

Vexxhost Cloud Compute runs on OpenStack, an ever growing open standard for cloud infrastructure, has a nice and tidy user interface and is affordable: the servers start at $5 per month for a 2 vCPU box with 512MB or memory, 20GB of HDD with a whopping 2TB of monthly data transfer.

We were also looking for a Canadian cloud vendor as many of our customers have requested it and Vexxhost ticks this box too: As well as being a Canadian company, their data centre is located in Canada with great connections to the US.

We are very excited about our partnership with Vexxhost and both Cloud 66 and Vexxhost support teams are going to be available to help any customers who wish to try and use Vexxhost with Cloud 66 with their awesome web apps, mobile backends and APIs!

Enjoy!

Preface:

This is a guest post by Joseph Acklin who kindly offered to help other Cloud 66 customers with the setup of their stacks on their own servers (Bring Your Own Server).

Introduction

Cloud66 is a brilliant service that helps us developers stick to the development instead of system administration.

We've all struggled through configuring a rails environment from scratch by rummaging through a millions stack overflow posts.

When you're finished, if you managed to get your app running at all, you always feel like maybe you left something out. You might be wondering if you've sealed all the security holes. Then in 2 months when you update a package and everything stops working the real panic and frustration starts.

Cloud66 does all the scary stuff for you!

Cloud66 has a few providers built in, but I decided to do things the hard way.

In this document I'm going to be going through the steps to show you how to prepare a VM purchased from CloudAtCost.com for Cloud66.

The steps in this guide are geared towards a CloudAtCost VM, but its likely that they are relevant for any similar cloud provider.

Prepare Your VM

When you first purchase your VM from CloudAtCost.com you'll be asked to select your OS. You'll need to make sure you pick Ubunutu 12.04.XX. Currently this is the only choice Cloud66 will support.

If you've already built your VM with another choice its okay. Login to your management panel for CloudAtCost and rebuild the VM with Ubuntu. Once you have the VM loaded with the correct OS you'll want to log in to the box. You can do this through the management panel but I prefer using a terminal application vs. the web terminal emulators.

ssh [email protected]

Your root password can be found by logging in to CloudAtCost.com and finding your VM under the products menu. Once you're logged in the first thing you'll need to do is create the user you'll want Cloud66 to use.

sudo useradd -d /home/joe -m joe

Now lets set joe user's password:

passwd joe

Cloud66 requires that the user you provide can connect without using a password. We'll need to do a few things to set this up securely. Lets create a new group:

groupadd sudoers

And put joe user in that group:

sudo usermod -a -G sudoers joe

Now lets make it so anyone in that group won't have to enter a password for sudo: We'll edit /etc/sudoers file by entering:

visudo

Then below this line:

%sudo ALL=(ALL:ALL) ALL

Add the line:

%sudoers ALL=(ALL) NOPASSWD:ALL

Type ctrl+x to exit, and 'Y' to save and you're all set. Now that we have joe user all ready to go lets take him for a spin:

su joe

You might notice that your default profile isn't bash. You'll want to change that by typing:

bash

That's a temp fix though. Lets make your profile always use bash. This is recommended to a smoother deploy through cloud66.

chsh

After entering joe's password enter the path:

/bin/bash

You should now have a prompt something like:

joe@ubuntu1204:/home$

Okay, now its time to generate some ssh keys. There are several guides on this subject so i'll leave out the details and just list the steps. Make sure you are still the joe user.

whoami

Then type the following substituting your email address:

ssh-keygen -t rsa -C "[email protected]"

You'll have a series of about 3 prompts after that. Just hit enter leaving them all blank. This will eventually bring you back to your prompt. Now enter the following ( being sure to enter your VM's real IP address ):

scp ~/.ssh/id_rsa.pub [email protected]:/home/joe/.ssh/uploaded_key.pub

And:

ssh [email protected] "echo `cat ~/.ssh/uploaded_key.pub` >> ~/.ssh/authorized_keys"

This is all the setup you will need to prepare your VM for Cloud66!

Building The Cloud66 Stack

Now that we have our VM ready and waiting for our rails environment to be setup its time to build our stack!

Login to Cloud66.com and from the 'Dashboard' click 'New Stack'. The UI for Cloud66 is pretty easy to use.

The first screen is pretty straightforward. You'll need to provide the github url for your app, which environment you want to use, and the name of your stack. The stack name is used for identification within Cloud66 only so enter something like:

MySweetApp

When you're ready click the big blue 'Analyze' button. It'll take a few seconds while Cloud66 figures out what kind of DB you're using along with some other items.

The first thing you'll see is a section where you can add some environment variables. Cloud66 will analyze your database.yml and determine what your database user and password should be. If you don't have your database.yml checked in to github (which you shouldn't) then Cloud66 will generate a random password for you. You can edit this information in the environment variables.

Next up is the ruby version. Pay close attention here because if you choose the wrong ruby its not easy to change later.

The "Where are you Deploying To?" section is where it gets interesting for us. Choose: "Deploy to my own server" for Deployment target. Here's where all the SSH key business comes into play. Click 'Add new Key'. Now from back in your VM (logged in as joe user) type:

vi /home/tiki/.ssh/id_rsa

Copy the resulting text (including the beginning and ending comments) into the Private SSH Key text area. You can enter anything you want for the name. I usually name mine so I have a hint to what server its from:

CloudAtCostVM1

Next enter the IP address of your VM in the “Deploy Web Server” input.

For this app i’m just going to install the DB on the same VM so for “Deploy Mysql” I’m going to leave the default as “Locally -- Same as the web server”. Give everything a quick look. Especially the ruby version, username (in our case joe), and web server IP. Then click the pretty blue ‘Deploy’ button. If you get an error right away that says something like:

Failed to download host file from server

It's likely due to a user permission problem. Make sure you have created the user and groups as recommended in the CloudAtCost section.