#boot script

Requirements:

Intermediate knowledge of Amazon Web Services(AWS) EC2 and CLI.

An understanding of AWS security groups and networking.

Intermediate knowledge of RHEL/Linux and Bash.

Skip the rambling and go straight to the code!

Backstory

Recently I was tasked with setting up a server environment for a web application which could support a potential rise in traffic. Because "potential" was in the sentence I didn't feel the need to set up an infrastructure to support heavy amounts of traffic that doesn't exist yet. I wanted to keep it as slim as possible so that my cost would be just as slim.

So what I had in mind as far as infrastructure was...

Utilizing Amazon's AWS EC2 services I would create:

1 Micro instance for the Web Application.

1 Micro instance for the Database.

I decided to separate the two because a Micro instance has very limited resources. So I wanted the resources of each instance to be dedicated to its specific tasks. The first serving the web pages and the other providing the datastore.

In this kind of infrastructure both instances (web app & datastore) private ips need to be added to the security group in order for communication between both instances to be possible.

So what if the potential rise in traffic occurs?

We scale it! AWS has great tools to makes this easy. If we find that resources are starting to run out because of traffic we simply add another instance, which is usually a snap shot of the current live instance, add its internal ip to the security group, and attach it to a load balancer.

Damn that's awesome! But do I have to constantly monitor the servers to see if there are performance issues due to a spike in traffic in order to scale out the infrastructure? That sucks because I would rather be watching "Big Bang Theory" and occasionally catch up on a little shut eye.

The answer is NO!

Let's welcome CloudWatch! A service that monitors the instances to make sure that they are running as intended. I don't have to worry about keeping an eye on performance. CloudWatch knows that once the CPU begins to rise and reaches a user set threshold to spawn a new instance and load balance them. The same thing goes if the CPU goes below a user set threshold to shut down the attached instance.

I could go into all the details on how to create an Auto Scaling Policy and how to attach it to CloudWatch, but that's not what this post is about.

The Problem

The post is about an issue that I ran into when the application automatically scales. Every time a new instance is spawned it's assigned a new private ip. This private ip needs to be added to the security group to allow for communication between the database server and the new instance. It defeats the purpose of using CloudWatch If I have to do this manually every time the app scales.

My Solution

I say my solution because it's by no way "THE" solution. There may be better ways to do this but mine was to create a boot script which adds the new instances internal ip to the security group and removes it when the instance is shut down.

This solution requires the setup of AWS-tools API CLI on the instances.

Step 1. The startup/shutdown script that does the meat of the work is created and placed in /etc/init.d

mng_security_group

#! /bin/sh # Author: David Salazar , 2011 # # /etc/init.d/mng_security_grp # and its symbolic link # /usr/sbin/mng_security_group ### BEGIN INIT INFO # Provides: mng_security_grp # Required-Start: $network # Required-Stop: # Default-Start: 3 5 # Default-Stop: 0 1 2 4 6 # Short-Description: aws call to manage security group for instance # Description: Initial boot adds new instance to security # group if not already added. ### END INIT INFO . /etc/init.d/functions . /root/.bashrc BIN_FILE=/usr/sbin/mng_security_grp EC2_BASE=/usr/sbin/ec2-api-tools/bin EC2_SEC_BIN=/usr/sbin/ec2-api-tools/bin/ec2-describe-group LOCKFILE=/var/lock/subsys/mng_security_grp GROUP='security_group' test -x $BIN_FILE || { echo "$BIN_FILE not installed"; exit 5;} test -x $EC2_SEC_BIN || { echo "$EC2_SEC_BIN not installed"; exit 5;} INTERNAL_IP=`curl http\://169.254.169.254/latest/meta-data/local-ipv4` EC2_PERM_EXISTS=`$EC2_BASE/ec2-describe-group $GROUP -F ip-permission.cidr=$INTERNAL_IP/32` case "$1" in start) if [ "'$EC2_PERM_EXISTS'" != "''" ]; then echo "ALREADY EXISTS IN SECURITY GROUP"; exit 0; fi touch $LOCKFILE $EC2_BASE/ec2-authorize $GROUP -p 0-65535 -s $INTERNAL_IP/32 echo "ADDED $INTERNAL_IP TO SECURITY GROUP $GROUP" ;; stop) if [ "'$EC2_PERM_EXISTS'" == "''" ]; then echo "ALREADY REMOVED FROM SECURITY GROUP"; exit 0; fi rm -f $LOCKFILE $EC2_BASE/ec2-revoke $GROUP -p 0-65535 -s $INTERNAL_IP/32 echo "REMOVED $INTERNAL_IP FROM SECURITY GROUP $GROUP" ;; esac

Step 2.

# Make sure that the file is executable. chmod +x /etc/init.d/mng_security_grp #Copy mng_security_grp over to /etc/rc.d/init.d cp /etc/init.d/mng_security_grp /etc/rc.d/init.d #Create a symlink in /usr/sbin cd /usr/sbin ln -s /etc/init.d/mng_security_grp mng_security_grp

Step 3.

Add the boot script to the appropriate runlevels.

Definition of runlevels: The runlevel field designates system state. For example, a runlevel of 0 corresponds to a halted system while a runlevel of 6 corresponds to a system reboot. Unfortunately, all Linux distributions do not follow the same definition for runlevels.

For example, mng_security_grp would be executed(add) when entering the runlevels 3 and 5 and execute(remove) when entering the runlevels 0, 1, 2, 4, and 6.

chkconfig --level 35 --add mng_security_grp chkconfig --list | grep mng_security_grp # mng_security_grp 0:off 1:off 2:off 3:on 4:off 5:on 6:off

For details on managing init scripts with chkconfig click here.