Screechy Threat’s Tech Blog #5: Assemble the Fleet!
One of the key principles of economics is supply and demand. The pricing of a good is directly related to the amount available and the desire to purchase the good. If there are too few of an object that is highly desired, the provider can charge a higher price. Were as, if there is too many of an item, the provider would want to lower the price in order to make the item more buyable. Amazon's spot instances apply this principle.
Amazon owns a vast amount of server farms with many different types of servers. So many that they have some instances sitting idle, doing nothing. If they remain unused, Amazon has no gain from these servers. Nor is there a public need for these servers, otherwise they would be in use. So their value, based on the supply and demand, is very low. Instead of wasting these machines, Amazon decided to make them available to the public at a different price scale.
Normally, when renting an ec2 instance from AWS, there is a set price you pay for on demand access to an instance. The better the criteria of the hardware, the more it costs. Amazon EC2 Spot Instances is a system for bidding on Amazon's spare instances. According to the EC2 Spot website, "Since Spot instances are often available at a discount compared to On-Demand pricing, you can significantly reduce the cost of running your applications." The way the pricing for spot instances works is that you set a cost you are willing to pay for a type of instance. If the price for that instance is lower than your bid you are able to get one. But you don't pay what you bid; you pay what the instances are valued at, based on the pricing of each hour.
The main reason to use spot instances is that you get the same instances for a cheaper or equal price. If you are debating between getting an on demand or a spot instance and decide to go with a spot instance, it makes most sense to bid at least the value of the on demand instance. First of all, you were willing to pay that much to begin with if you had gone with an on demand instance. Secondly, it gives you more leeway if the price goes up. From my experience, the value of the spot instances is usually around 20 percent of the on demand price. So, if the price rises a little, you don't have your instances taken down. Thirdly, you most likely won't pay the on demand price anyway. Like stated earlier, you don't pay what you bid; you pay the value of the spot instance. Finally, if the value does go over the on demand price, your instances will be taken down, because you only get the instances if they are valued below your bid. To counteract this, you can always bring up an on demand instance. Therefore, you will never be paying more than what you would have paid for an on demand instance.
One of the pains of spot versus on-demand instances is that Amazon will shutdown the spot instances when the value exceeds the bid. The processes that run on the spot instances have to be able to deal with a shut down at anytime. This is why Amazon recommends spot instances run temporary or short running processes. If the process is short lived then there is less time to be affected by a price change and less of a threat or risk of being taken down. Otherwise, AWS gives a two minute warning for when they are going to shut down an instance. Unfortunately, the best way to listen to this warning is to have a separate process querying amazon every X minutes to see if the instance is marked as going down. If the processes require zero down time, it is recommended to use dedicated instances, either instead or in addition. It can take minutes to bring up new instances, so spot is unreliable for zero down time.
Amazon provides a very helpful tool for managing your spot instances, called spot fleet. Spot fleet has a view of the prices of the different kinds of instances. It also allows for diversification and scaling of instances in groups. Diversification is very important because it makes having spot instances available more likely. There are two means of diversification.
The first kind of diversification is having multiple kinds of instance types. Choosing different types of instances lowers the likelihood that all your instances will be taken down at the same time. If only one instance type is selected and then that instance type becomes in high demand, all the spot instances may be taken down. If multiple instances types are selected, if one of the instance types becomes in high demand, those instances will be scaled down and another type can be brought up in its stead. Furthermore, there are enough instance types similar or higher in performance from different EC2 families (i.e. m, r, t) that you can swap them out, almost insignificantly.
The other type of diversification is by availability zones. It is possible that all the instances of one type in one availability zone will be taken. It is less likely that all the instances of that type in all availability zone in the region will be unavailable. Specifying multiple availability zones increases the chances of scaling up another instance in the region and maintaining the amount of instances desired. By utilizing both types of diversification, the likelihood of having no spot instances available is very low.
Spot instances can also be put in load balancers for serving requests. In a normal situation of an on demand instance, the instance would be put in an autoscale group and then into the elastic load balancer. Spot fleet can be used as the autoscale group. Each fleet is a group of instances that can be diversified and has a minimum, maximum, and desired capacity for scaling purposes. These instances can be grouped together with an EC2 group name, and not an autoscale group name because they are not actually in an autoscale group.
Spot instances allows Amazon to utilize their unused servers, while giving their clients a reduced price. It's a win win situation. Amazon makes money they wouldn't be otherwise, but still reserve the right to take back the machines if they need them. The clients potentially get a significant price cut and are still able to use the EC2 instances as if they are on demand. And, if the instances are taken back by Amazon, they can just re-spawn with on demand instances.
