Watch "Asking Interns How Much Money They Make" on YouTube
seen from Sri Lanka
seen from Germany
seen from Brazil
seen from Estonia
seen from Germany
seen from United States

seen from United States

seen from T1

seen from Singapore
seen from United States

seen from Finland

seen from T1
seen from United States

seen from United Kingdom

seen from Australia
seen from United States
seen from China
seen from Brazil
seen from Malaysia

seen from T1
Watch "Asking Interns How Much Money They Make" on YouTube

Anya is live and ready to show you everything. Watch her strip, dance, and perform exclusive shows just for you. Interact in real-time and make your fantasies come true.
Free to watch ⢠No registration required ⢠HD streaming
Lol all i want from this hackathon is the huge teddy bear
Scaling Summary
This past weekâs blog posts were very centered on Finagle, and the idea of scaling. Marius Eriksen made this the focus as he talked about how twitter handled their scaling problem.
A common theme among the blog posts were the idea of data centers and horizontal scaling. Unlike vertical scaling, horizontal scaling is most focused on the idea of adding new computers to the data center. One thing Marius made clear was that data center computers really are not meant to handle modern day applications, as very little has been done to improve them in years. The problems with data centers were a clear focus, with a common theme being addressing the fact that data centers often involve partial failures and painful abstractions.
Another common theme was the idea of RPCs, and using Finagle as a RPC as a means of avoiding the headaches commonly seen in data centers. This idea communicated that data centers could use RPC as a means of creating a more natural way of communication between the user and the data center. Using finagleâs model of fixtures, services, and filters, the use of an RPC becomes much clearer.
A lot of focus was also placed on alternatives to Finagle. As scaling becomes a larger problem, more alternatives will come out. For example, Nutanix, Thrift, and other solutions are quickly entering this market, and it will certainly be interesting to see where the products move in the future.
Hadoop
This week we had Eric Baldeschwieler from Apache Hadoop. Hadoop is an open source framework for data computation and distributed data storage that can be run on commodity hardware. Hadoop is very modular, allowing the components to be developed by different teams while still working towards a common goal. The most popular parts of Hadoop are MapReduce and HDFS, which is Hadoopâs distributed file system. Hadoop follows an interesting principle for warehouse scale computing, which is that all of this software should be able to run on commodity, off the shelf hardware rather than purpose built machines that optimize for one certain task. This allows data center operators to cost optimize and scale more quickly with off the shelf hardware rather than purchasing a large software/hardware package from a company such as Oracle. Itâs 100 times cheaper to store your database in Hadoop as well.
Hadoop does huge data processing well. Thatâs what it was built for. A frequent problem in web scale companies today is that they have no methods to take enormous datasets, condense them down and parse useful metrics out of them. Thatâs where MapReduce comes in. For anyone thatâs taken 61C at Berkeley, theyâve used MapReduce or its newer cousin Spark to parse a large dataset and get useful information out of it. MapReduce works by distributing the computational load across many different computers, allowing the work to be completed quickly and in parallel. As with any task that is completed in parallel, the task must be well suited for being completed in this manner, a task that was extremely computation intensive but not very data heavy would not be well suited for example.
Hadoop also has a scalable database system, called HDFS, that it uses to keep track of these large sets of data across many disks, redundantly. It differs from a typical RAID setup in that it can be run on off the shelf equipment, rather than to many of the same model and size disks in a RAID 5 for example. This allows data center operators to have a much easier time of managing a storage group using HDFS rather than many RAID setups running in parallel. This also allows other Hadoop applications to pull from many disks at the same time, which could maximize bandwidth for an application if it needed it. Many cloud based storage providers offer HDFS as a service for data storage.
Scaling with Finagle
This week, we had Marius Eriksen from Twitter come in to tell us about Finagle and handling scale by talking about the paradigm of data centers. One thing that Marius focused on was understanding what a datacenter is, and the problems that are inherent in working with a data center. Even though these machines are powerful and useful, they are inherently terrible, wrought with problems. Such problems include partial failures, being hard to program, and abstraction that is an absolute pain to handle. In addition to this, most modern software is not optimized for data centers, instead it is optimized for local computation and not the models that a data center is meant to support. Because of this, we need a new paradigm to understand the best way to tackle these issues, and that comes with a modular model to these data centers.
Finagle helps different processes interact with each other in a datacenter scheme for a web scale big data application. A seemingly harmless request may have to traverse many different databases or pull data from different servers, itâs easy to introduce errors if you spend a lot of time parsing data between sources so Finagle provides a consistent interface across the different sets. Because Finagle was conceived for the datacenter, most of the code is asynchronous, allowing your processor to do other important things while it waits for a request to finish. This allows for cleaner code and easier optimizations because of the asynchronous nature of Finagle.
Other companies have been trying to tackle the issue of data center scaling as well. For example, on the hardware side technologies like Nutanix are allowing the seamless integration of storage into these large scale solutions. Nutanix specifically solves the issue traditionally presented by NAS systems: Theyâre non-local, slow, and really expensive. Their software allows each system to have its own local memory, but for the data center applications to treat that local storage as if it was one unified storage pool for the entire data center. These kinds of hardware-centric advancements synergize with improved frameworks and web technologies to leverage the full power of the data center.

Anya is live and ready to show you everything. Watch her strip, dance, and perform exclusive shows just for you. Interact in real-time and make your fantasies come true.
Free to watch ⢠No registration required ⢠HD streaming
HTTP2 for Web 2.0
This week we had Brad Fitzpatrick, a software engineer at Google and the maintainer of the Go standard library come in to speak to us about the improvements made in HTTP2, and how they addressed the flaws that came up within the original version of http.
One fundamental point about HTTP that led to the creation of HTTP2 was that the protocol had remained relatively untouched since 1999, and had not evolved as the web, and the content being served over the protocol had. Thus, for new use cases, such as mobile, the old HTTP was just not optimized to work well. Minor improvements that had been made from 1999 to 2013, such as the idea of pipelining, simply did not work, or had such little adoption that they found themselves effectively useless. HTTP was originally built to handle one thing at a time, with the slow startup time provided through the TCP handshake and huge requests being passed back and forth. In addition, shutdowns were not quick, and, in turn, these problems just did not support the modern web. Because of this, HTTP2 was born.
HTTP2, as most standards are, was created out of a slew of committees and as such took into account the design concerns of all of the parties with a vested interest in how the standard was implemented. Of particular note is the fact that HTTP2 operates entirely over the secure socket layer, which provides end to end encryption. This allows HTTP2 to enact new interactions without the interference of a middleman. The new standard greatly improves latency on a connection with latency, allowing for multiple streams of data to be delivered at the same time. This allows something like a tiled image to be streamed to the user rather than holding two or three connections to different web servers to download each tile as in HTTP1. The new standard also allows for push promise messages, which are resources that the server sends, knowing or thinking that the client will need them before the client asks, again improving round trip time. HTTP2 improves on most of the major issues with previous versions of the protocol.
This is part of a wider industry trend to update old web standards. Weâve already seen this take place multiple times over the last couple of years, with the biggest overhaul by far being the move towards HTML5 and the use of the web server as a delivery mechanism for rich applications, not just static web pages. This same updating has also applied to CSS, Javascript, and nearly every other framework or technique used in the creation of web content. In many ways, HTTP2 represents one of the final steps in this evolution, since it sits close to the bottom of the stack as one of the fundamental technologies that make all of these other techniques and standards possible. Evolution at this layer will lead to new possibilities and options for these other technologies as well.
Modern Day Deployment
Kubernetes take the complication out of managing cloud infrastructure. What it allows the developer to do is worry about the design and operation of the server and not worry about the physical machine that their code is running on at all. Application oriented architecture gives the developer freedom to create their code as well as gives the manager freedom to contract services out rather than doing them in house. The Kubernetes approach is to have the user express a desire, while the system expresses a state, then the system reconciles the two states to be equal without interaction or involvement by the user. The approach firmly enforces the bounds of abstraction in order to provide a unified, API like view for the user to see of the cloud.
Kubernetes isnât alone though. It is one of an increasing number of tools designed to simplify the life of a dev ops engineer and increase the deployment capabilities of a project. The most well known of these in recent times is Docker. Dockerâs approach is to containerize projects, essentially creating an upgraded version of a VM, thereby allowing for rapid and easy deployment and management. It is perhaps a testament to Dockerâs own success and effectiveness that Kubernetes interfaces with Docker for managing individual machines.
Another, more fascinating, of these tools is CoreOS. CoreOS is a linux distribution built entirely around large scale server deployments. As such, it doesnât even have a single âinstanceâ of itself. It is built entirely for hosting Docker containers. These Docker containers then act as the applications and nodes that a cloud service is built on. CoreOS also comes with services like etcd and fleet which allow it to pass around information and manage its services, respectively. The advent of tools like CoreOS shows weâve reached a maturation stage in cloud infrastructure. We now have tools designed for use in massive systems built all the way from the web services we are so familiar with, down to the most fundamental components of the operating system.
Summary : PaaS
This past weekâs blog posts focused on the newly invented paradigm of Platforms as a Service. Platform as a service focuses on one core tenet: that as an engineer on a project, the goal is for the project itself to be as polished and usable as possible. For this reason, Noah encouraged the idea of using platforms throughout a product as much as possible because, as an engineer, it allows you to solve the problems your customers want solved, and leaves the coordination to the experts. The ideal way to develop an application, Noah argued, is to focus on what you know best, your product and your customers, and to focus on your business logic without wasting precious time on boilerplate and architecture. Instead of having to make monolithic applications, the platforms would be able to keep your code scalable and maintainable.
In order to allow for this PaaS philosophy to work, platforms must meet several criteria to be truly useful. As a PaaS-based platform, it is vital that users can navigate and utilize the platform easier that they could create that platform for themselves. Thus, APIâs should be machine and human readable, and executable examples must be given. In addition, platforms should run on themselves, in order to ensure that they are able to empathize with the usersâ concerns and know where problem points are within the application.
One clear problem with this philosophy is that you are unable to be responsible for all downtime your product experiences. By using a PaaS like DigitalOcean, you are willingly allowing someone else to control your application. In addition, PaaS platforms often donât cater to specific needs, so as a business scales, it may make sense to move off of PaaS and handle everything your app needs on itâs own. Because of this, when using PaaS it is essential that you use platforms that are tried and true and have a history of running well. In addition, using services for their intended tasks allows your application to work as fast as possible, so it is important to understand how a service meets your needs.