#deployment technologies

Post 332 – by Gautam Shah . Mile stones of Human developments are marked by distinctive technological innovations that have given very competitive advantage to several fields. These through wide range of applications changed the course of developments.   Mile stones of Human developments are marked by distinctive technological innovations that have given very competitive advantage to several…

What exactly is a ‘Deployment Technology?’

Who doesn’t love technical jargon?  One thing that can be intimidating to new developers approaching web development is the number of terms dropped casually when discussing the development process.  For example, a part of this process which is an absolute necessity in order to take code from a fun demo that ‘runs on my machine’ to an actual application is deployment.  When we talk about modern deployment technologies, we are just talking about the technologies that support and describe the ways developers take applications and make them available to consumers.

Of course, there are many ways, physical and digital, to accomplish the deployment of software.  Trying to cover all of these solutions in a single blog post would be a silly, somewhat Sisyphean task.  But one common, recently developed solution to the deployment of applications, and particularly webapps, involves the use of containers.  But what is a container?  How does it work?  And how are containers managed?  All of these are important questions, essential to understanding and deploying containerized software.  In order to address these points, we need to cover three topics essential to the process: the platform which runs the container, the container itself, and the many options for the infrastructure used to orchestrate multiple containers.  

Platform

Docker, the most popular container framework, still needs to run inside of a host OS.  Though containers, as will be described later, allow for the configuration of many contextual details, there still must be an environment to configure: this is where the platform comes in.

CoreOS

Docker itself, as well as Rocket, run on top of an OS called CoreOS.  Put simply, CoreOS is a barebones operating system, built with the idea of running scalable containerized applications in mind.  Actually a fork of ChromeOS, CoreOS focuses on providing the bare minimum for running containerized applications, in order to conserve computational resources and aid in the scalability, reliability, and security of those applications.  It provides certain things, like service discovery and an interface for communication, that are useful specifically for running clustered cloud applications, while omitting many traditional OS features, like package managers, for the sake of compute- and space-efficiency.

Containers

Containers are a simple way of encapsulating an application and its dependencies, intended to be run in an isolated namespace ensuring resource isolation, while sharing system-critical components like the kernel with other containers.  This is not actually a new idea. Starting with FreeBSD Jails and Solaris Zones, the idea of tunable but small application specific environments has appealed to developers for quite some time.  Docker itself is built on top of LXC, or Linux Containers.  Unlike more heavyweight solutions such as hypervisors, running a group of containers generally involves a stack of lean virtual machines running atop a shared kernel in order to preserve system resources.  This allows for a number of benefits over heftier solutions, like improved startup times, flexibility, and portability, at the cost of a certain amount of security: a shared kernel can become a liability.  There are currently two main technologies competing as container solutions: Docker and Rocket.

Docker

Docker was created to ease the process of distributing applications for deployment by offering a way to configure the local context.  Isolation features of the Linux Kernel allow multiple Docker containers to run in unique contexts without requiring the overhead of spinning up an entire virtual machine for each one.  This is particularly important when considering the logistics of distributing and supporting applications to the cloud.  By running concurrent application containers instead of full VMs, cloud resources can be used more efficiently, allowing providers of web services to get more bang for their buck.

Rocket

Following up on claims that Docker has gone from being the lightweight and portable container technology that was promised at its inception to ‘bloated’ solution for containerization, coming packaged with features that many users may not need or want, the team at CoreOS has designed a competitor called Rocket.  Though Rocket provides many of the same things as Docker, it goes about providing these things in a different way.  For example, while Docker runs a central daemon process which executes within a Docker namespace, Rocket runs under the system in which it is initialized.  Ultimately, where Docker is tending closer to becoming a platform in-and-of-itself, with all the commiserate tooling, Rocket intends to provide composable, secure container units, true to the manifesto written in the early days of Docker, while keeping development an open process.

Container Orchestration

Though containers offer many benefits when compared to traditional PaaS solutions, they does not cover the operational aspects of code deployment .  There are a number of operational considerations, like balancing workloads and dealing robustly with failure, that still need to be managed.  This is where container orchestration technologies like Kubernetes, Swarm and Panamax come in.  Further, in addition to technologies intended specifically for use with containers, there are other, more general-purpose technologies, like Ansible and Mesos, which are intended for managing and configuring distributed computational resources.

Swarm

Perhaps the most simple of the solutions listed here, Docker’s own Swarm service is a very simple way of grouping hosts into a single virtual host.  This ‘swarm’ of hosts still communicates through the same Docker API, making this clustering of containers invisible to the outside user, and providing one simplified interface for communicating across multiple hosts.  Ultimately, this is a very limited solution, and is intended for simple use cases, and to be easily swapped out with more robust solutions like Kubernetes and Mesos.

Kubernetes

Kubernetes is an orchestration system for Docker, developed by Google, intended for distributing containers across clusters of compute resources (like physical computers or VMs) in a way that meets a declarative set of user constraints.  Basically, Kubernetes provides a framework for supporting management of and interaction with containers.  

One concept central to the architecture of Kubernetes is the ‘pod,’ which is a group of tightly coupled containers meant to be served together.  This ensures that, depending on the requirements of the application, separate docker containers can be run consistently on the same machine, sharing resources and allowing for performant communication.  Pods are be further grouped into ‘Services,’ a useful abstraction of functionality performed by pods allowing for a simplified API.  One could imagine a service intended for operating a web server - while Kubernetes may internally run multiple pods, each consisting of multiple containers, in order to ensure reliability and redundancy of a service, an outside user simply interacts with an abstracted interface for that service, without needing to understand the details of how that service is internally provided.

Kubernetes also provides an interface for things like container-to-container communication, without requiring overly complex workarounds to account for things like dynamic port allocation, and optimistic distributed concurrency, obviating the need for acquiring distributed locks.  Further, a declarative interface for stating constraints ensures that developer-specifications are consistently met, doing things like keeping a certain number of server instances up and running, and balancing containers to minimize vulnerability to any particular point of failure.  Fundamentally though, Kubernetes tries to provide a ‘unified computation substrate’, abstracting away the details of the multiple resources across which containers must be distributed and providing a simpler and more functional interface.

Ansible

Much like Kubernetes, Ansible is a deployment management system providing a declarative interface, intended for managing and configuring multiple computing nodes.  Unlike Kubernetes, Ansible is not intended to work specifically with containers (though it does include support for Docker containers) , but more broadly to ensure the general consistency applications running on computing resources with constraints declared by the developer.  Rather than containers, Ansible depends on ‘Ansible Modules’, allowing the specification of certain resources and their desirable states (like requiring a web service to always be up).  Machines are divided into ‘playbooks’ based on operations intended to be executed, so that computing resources can be divided among the variety of tasks necessary to run a particular application.  This allows for a clearly understandable and configurable separation of concerns, and the modularization of both development and resource management.

Mesos

Apache Mesos has a similarly minded, but somewhat more abstract goal than the platforms listed above.  Fundamentally, Mesos sets out to abstract the resources of multiple computers or clusters (like RAM, CPU, etc) away from the physical machines they are resident of.  By so doing, it provides an abstracted interface to manage the general execution of a number of services, including containers but also more traditional cluster-oriented applications such as Hadoop, Spark, etc. across a scalable number of machines.  Essentially Mesos is a distributed kernel, providing many of the same features for a cluster that a traditional kernel would for a single machine.  It also provides features for things like resource isolation and rack locality, and has a common interface for accessing cluster resources.  But it is less opinionated about the way these distributed applications should be structured and run than the other platforms in this post, for better or for worse:  Mesos is a bit less optimized for running containers and is a heavier weight solution than something more targeted, like Kubernetes.  The technologies can actually be used together - with Kubernetes handling the configuration and management of container pods and Mesos handling the distribution of computational resources to those pods.

Panamax

Panamax takes abstraction one step back from a framework like Kubernetes, and is intended to handle the orchestration of not just containers, but also existing Linux resource orchestration systems.  This includes frameworks like Kubernetes and Apache’s Mesos.  Marketed as ‘Docker Management for Humans’, Panamax offers a much friendlier UI for management of Docker containers, giving developers who might not be interested in learning the minutiae of command-line container management access to a simpler set of tools.  Panamax is a free and open-source project, intended to help developers understand and deploy multi-container apps without requiring the mastery of the multiple technologies typically involved. 

Conclusions

Last week’s blog posts discussed Brendan Burns’ talk on deployment technology.

Strengths

Most groups discussed the advantages of Kubernetes’ declarative approach: rather specifying how to keep a deployed system in a production-ready state, one only specifies what the production-ready state is, and Kubernetes automatically keeps the system in that state.

Some groups mentioned the power of Kubernetes’ networking layer: since each pod is assigned a unique IP, there’s no need to deal with weird bugs resulting from dynamic port allocation.

Almost everyone touted Kubernetes’  integration with Docker as a big plus.

Weaknesses

One group mentioned that Kubernetes’ documentation didn’t really make clear why it’s useful.

Another group observed that the declarative event loop might be less efficient than an imperative system, in part because the declarative approach permits less control over low-level optimizations.

Comparisons

As mentioned above, many groups also discussed Docker, which is one of the foundational elements of Kubernetes: a Docker VM is self-contained, stripped-down virtual machine, that allows users to modularize an app and all of its dependencies and configurations for easy deployment.

Other groups mentioned CoreOS, an operating system (based off of ChromeOS, apparently) designed to incorporate Docker’s design goals into an actual OS, rather than just a VM.

One group compared and contrasted Kubernetes and Ansible. They noted that while the two projects have different goals­—the former focuses on cluster management and the latter focuses on host configuration­—they share a number of design principles. For example, they both use a declarative description of state rather than an imperative one, and they both focus heavily on modularization.

Last week, the founder of the Kubernetes project Brendan Burns gave an overview of the power of the Kubernetes cluster management system.

What is Docker? Before understanding the implications of Kubernetes, one must understand what Docker is and why it’s so prevalent.

Traditionally, deployment of applications were done through virtual machines that sits on top of a hypervisor. These virtual machines were essentially blank slates that needed to be configured with a guest operating system, external libraries, and the application binaries.

Containers are similar to VMs in that they are for deploying web applications. However, containers are different in that they serve only the application binaries and external libraries need to be installed and configured, whereas VMs must virtualize hardware and other system requirements. This “leaner” model allows system administrators to fully utilize their metal by running more contained instances on a machine.

This containerization not only helps in deploying, but also with developing. The docker layer can also help developers onboard their development environments to emulate the production environment in a faster manner.

Cool. So what’s Kubernetes? Although setting up a container has become very easy to do, scaling and deploying these containers for large complex application is no trivial task. Kubernetes is open source architecture that handles scheduling, allocating, and deploying a container cluster with some cool features like self-healing to ensure uptime, minimizing same-server usage for maximizing resource utilization, and minimizing same-server deployments of containers to maximize uptime.

These features are baked into Kubernetes with a unique paradigm. Rather than handling errors and events, Kubernetes is architected such that the container deployment requirements are part of an evaluation feedback loop that tries to match the current state of deployments to the desired state of deployments. Brendan said that this was inspired by his time in robotics path planning. It turns out that this programming paradigm is extremely helpful in the deployment context as well. When the cluster administrator specifies that a web service needs 5 containers to scale it out, Kubernetes spins up 5 containers to match the desired state. When a container suddenly goes down, Kubernetes will spin up another container to preserve the desired deployment state.

What problem/s do they try to address?

Developing applications in our modern day cloud infrastructure has various moving parts. There are parts that have to deal with deployment, maintenance, scaling, load balancing, data storage…etc. However, Brendan Burns, a staff engineer at Google, talks about his distributed system Kubernetes that is designed to make web development easy. However to talk about Kubernetes we also talk about Docker integrates well with Kubernetes philosophy of making web development easy. Docker is an open-sourced project that establishes a container format for applications. Each module can be put into a container and containers can be put together into a pod. This fundamentally changes the way we think about cloud based applications because instead of abstracting the VM as a whole where very little information is given to the user, if the abstraction is at the level of the application (through the containers) then then much more control and information is available to the engineer. 

What are some strengths?

As hinted in above, we start off our discussion with the strengths of Docker as it forms the foundation from which we discuss Kubernetes. One advantage is that the abstraction at the level of the application (often represented as a pod, a collection of working containers) allows for greater control and information available to the engineer. Currently we have access to VM’s in the cloud and we just tell the cloud application services how many and what kinds of machines we wish to spin and consequently we only have access to information at the level of asking the system as a whole: is it on? how much CPU is being used? With Docker container based applications we have a standardized container API for applications and can peek into it and can tweak and debug each component of it. Now onto Kubernetes. Kubernetes manages large applications in a contained environment, making it easy to manage the lifecycle of cloud based applications. Servers can be wound up and down as necessary because everything is in easily isolated containers.  Furthermore, Kubernetes is managed in a declarative way so that if we say that we want 3 of x server running, Kubernetes takes care of making sure that that state is maintained and we as the software engineer don’t have to worry about that. Another strength is Kubernetes its networking. For a system orchestrating a complex interaction between various containers, it is key that the system is robust and communication is consistent. Its networking is perhaps one its biggest advantages. The goals is to assign each pod its own IP in a simplified and flat namespace. That way, it can avoid all the problems of dynamic port allocation.

Other notable features?

One of the related deployment technologies listed alongside Kubernetes this week is Ansible. Ansible (and other related tools such as Puppet and Chef) are IT automation engines to manage server provisioning, configuration, management across many servers. Where as Kubernetes focuses on cluster management, Ansible focuses on host configuration. Despite their different focus, when exploring more about Ansible we were able to see many of the same design principles that went Brendan described about Kubernetes.

The first similarity we noted is the importance of a declarative description of the state. That is, the input to the systems should be *what* the state of the system should be, not necessarily *how* to do it. This allows for a great deal of flexibility in how the system operates underneath the covers, and for server systems, how often human intervention (which is expensive) is necessary. For example, if a cluster of three computers has one failure, a declarative system can reconcile the observed state to reach the desired state, where as an imperative system may be left waiting for instructions on how to proceed. We see declarative descriptions exemplified in Ansible in the form of Ansible Modules, which allow the administrator to simply specify a resource and its desired state. For example, in Ansible instead of saying on reboot run “/etc/init.d/apache start”, you would indicate in the configuration that the web server service should always be up.

The second common principle we observed between Kubernetes and Ansible is modularization. Kubernetes is all about self-containment and modular design for launching applications. This abstraction frees the developer from worrying about interactions of their application with the underlying OS/physical machine, etc. In Ansible, machines are divided into playbooks based on similar operations to be performed on them. For example, you may have a playbook full of data-crunching machines, and a playbook for database servers. Furthermore, the Module system described earlier isolates independent operations from each other. For example, a playbook’s ssh module is independent from those machines’ user permissions module. These ideas seem fairly simple, but we’d imagine that when dealing with an enormous number of heterogenous machines, being able to reason about the configuration of machines in this clear way is extremely valuable.

This week we had Brendan Burns, staff software engineer at Google, talk about distributed system management and his project at Google - Kubernetes.

Kubernetes is an open source cluster management system that manages containerized applications across multiple hosts. It also has basic mechanisms for deployment, maintenance, and scaling of applications.

Kubernetes Architecture

Here, I will talk about two of the main architectural components of Kubernetes - the API server and pods.

Before we get into the nitty gritty, it is important to note is that one of the key methodologies of Kubernetes is the use of declarative primitives. This allows the system to easily maintain the state requested by a user. For example, let’s say a user requests that 3 containers be running at any given time. If Kubernetes detects that the number of containers running falls below 3, it will automatically spawn new running containers. Likewise, if the number is more than 3, it will kill any extra containers. This declarative nature allows Kubernetes to have self-healing mechanisms that makes it much easier on the cluster maintainer. No one likes being woken up in the middle night because of some server error.

API server The REST API server serves up the main Kubernetes API and allows the outside world to communicate with Kubernetes. Some examples of what you can do are scheduling pods (explained below) to worker nodes and retrieving pod information (where they are, what ports are they using, etc). This is the main hub for orchestration of the distributed system.

Pods Another main unit of the Kubernetes architecture is the pod. Pods are basically a group of tightly coupled containers that are scheduled onto the same host. They serve as distinct, containerized environments that serve as the basic units of scheduling and deployment. The containers within a pod also share the same fate and resources such as storage volumes and IP addresses.

Kubernetes take the complication out of managing cloud infrastructure. What it allows the developer to do is worry about the design and operation of the server and not worry about the physical machine that their code is running on at all. Application oriented architecture gives the developer freedom to create their code as well as gives the manager freedom to contract services out rather than doing them in house. The Kubernetes approach is to have the user express a desire, while the system expresses a state, then the system reconciles the two states to be equal without interaction or involvement by the user. The approach firmly enforces the bounds of abstraction in order to provide a unified, API like view for the user to see of the cloud.

Kubernetes isn’t alone though. It is one of an increasing number of tools designed to simplify the life of a dev ops engineer and increase the deployment capabilities of a project. The most well known of these in recent times is Docker. Docker’s approach is to containerize projects, essentially creating an upgraded version of a VM, thereby allowing for rapid and easy deployment and management. It is perhaps a testament to Docker’s own success and effectiveness that Kubernetes interfaces with Docker for managing individual machines.