the tourist

Last week (5/13-5/16) was the first week of my pairing tour, a two-week period where I pair with a different 8th Light craftsman each day. Last week I worked with Dave, Patrick, Dariusz, and Craig. I worked on Rails apps, wrote some Ruby code, and paired on a Node app. I learned a lot, both about code and about how to effectively pair.

There are several things about pairing that I noticed as I moved from project to project, craftsman to craftsman, each day trying to gain enough context quickly enough to contribute.

Verbally communicate at all times.

The best work happens when you and the developer you're pairing with are constantly talking. You should explain your reasoning when necessary, describe what you're doing, and ask for feedback. There should be a constant flow of communication.

Alternate driving frequently.

It can be tiresome to drive for long periods of time and similarly, you can find yourself feeling fatigued and almost absent when the other person is driving for a long time. It's really great when you can get in the rhythm of alternating frequently. One strategy for achieving this is ping-pong pairing.

Be engaged when you're not driving.

Sometimes it's strange to be the "other" person in the pair, the navigator, the one without the keyboard and mouse. It's important to be engaged in the process by asking questions, providing suggestions, and most importantly, letting the driver know when they're moving too fast or going in a confusion direction.

Pairing has many advantages. As you're pairing, you're sharing knowledge. You're learning about the codebase, the design and architecture, the domain, the technologies being utilized (language, framework, etc.). Sharing knowledge helps you avoid a very common situation where all domain knowledge belongs to one developer. If that person leaves the company or takes a week off, everything stalls.

Pairing invites debate and forces developers to explain their reasoning. It provides accountability and leads to cleaner, smarter, more well-written code. That is the most fascinating part about pairing. It's true that over the course of the day, you might get slightly less done. But in the long run - because you've written better, more maintainable code - you ultimately save time.

OSX's bash is terribly out of date, as was pointed out to me today by 8th Light Craftsman Eric Meyer.

My machine was using bash 3.2.48 and the most recent version of bash is 4.2. In this post I'll walk you through the steps needed to update your bash.

But first, an unexpected and exciting (and brief) look into tar files! TAR stands for Tape Archive. Why? Well, originally it was designed for tape backups. What the heck is a tape? There's a device called a tape drive that reads and writes data on a magnetic tape. Check it out!

Wow! That thing looks like it's a hundred years old! In contrast to a disk drive, which can move to any position on the disk in milliseconds to retrieve data, a tape drive must physically wind tape between reels to read data. But what you lose in speed you make up in cost. Tape drives are much less expensive.

So we go grab this tar file of the newest version of bash. (Note: a tar does not compress the files).

First, click here to grab the tar of bash 4.2.

Enter the Downloads folder.

cd ~/Downloads

Let's learn about some tar commands now that we have a tar file to deal with.

z uncompress before extracting

x is to extract files ending in .tar.gz or .tgz

v is to be verbose as it's extracting

f is filename to follow

tar zxvf bash-4.2.tar.gz

Wait! What is this .tar.gz extension? Remember when I told you tar files aren't compressed? .tar.gz files are compressed tar files, which is why we had to supply the z flag to uncompress before extracting it.

Look at Wikipedia's pretty picture of it all!

Alright, now hop into your brand new, freshly created bash-4.2 folder.

cd bash-4.2

Now we'll just run three commands in a row all on one line. These three steps are very standard for obtaining software directly from the source code.

We'll execute the configure script, the make command uses the makefile to determine what needs to be recompiled (and recompiles them). It does not recompile the entire program, only what's needed.

make install performs the install, also specified in the makefile (which is essentially a file of instructions). It's common for this command to install executables after the compilation process is complete.

The binary is installed at /usr/local/bin/bash.

./configure && make && sudo make install

Change your login shell to the bash shell you just downloaded.

chsh -s /usr/local/bin/bash your_user_name

chsh only allows you to switch to the shells listed in /etc/shells, so we need to register the new binary as a valid shell. The bash manual tells us that if the -c option is present commands are read from string.

sudo bash -c "echo /usr/local/bin/bash >> /private/etc/shells"

Enter the bin folder.

cd /bin

Re-name bash folder to bash-old, thereby creating a "backup" copy.

sudo mv bash bash-old

Create a symlink to point /bin/bash to /usr/local/bin/bash.

sudo ln -s /usr/local/bin/bash bash

Now if you run this command you can check the version of bash you're using.

bash --version

And finally, read about the new stuff in bash 4.X.

Some Helpful Stuff

The Ultimate Tar Command Tutorial with 10 Practical Examples

BashGuide

Today Paul asked Nhu (a fellow apprentice) and I to get a PHP app running locally so he could begin the process of determining the best way to re-write it as a Rails app.

We weren't very familiar with PHP or Apache, the web server we decided to use, so it took a while to figure everything out. We have the app running locally now and I wanted to explain how we got there.

Apache and PHP come with OSX, but they aren't enabled by default. The Apache binary running under Unix is called HTTPd (for HTTP daemon). The core of Apache is fairly basic and the server integrates easily with PHP and many databases.

cd /etc/apache2

This is where your machine's Apache server is located. You'll see a number of files and folders here, including one called httpd.conf. Apache is going to look at these *.conf files (configuration files) to determine a number of things, such as what port to listen on.

This file needs to be edited, but it's read-only so you'll have to change permissions on the file.

sudo chmod 755 httpd.conf

755 is the same as u=rwx,go=rx, meaning that you're giving the user read, write, and execute privileges, and the group and other privileges to read and execute only (but not to write). Read permission is assigned a value of 4, write is assigned 2, and execute 1. So giving read, write, and execute privileges is 4+2+1 (7), hence the reason for 755.

Next, we needed to uncomment a line in this config file that contains this command:

# LoadModule php5_module libexec/apache2/libphp5.so

Uncommenting this line is effectively enabling PHP.

At this stage we are ready to start the Apache server. We can start and stop (and restart) with these commands:

sudo apachectl start sudo apachectl stop sudo apachectl restart

But what files are we serving? If we visit localhost what will we see? Well, if we go back to the config file we edited earlier and take a look around line 165 we'll see a commented out chunk of text explaining something called DocumentRoot.

# DocumentRoot: The directory out of which you will serve your # documents. By default, all requests are taken from this directory, but # symbolic links and aliases may be used to point to other locations. DocumentRoot "/Library/WebServer/Documents"

So by default it will take files from this location to serve. There is an index file there that's served when we hit the root route. It gives us the reassuring message "It works!".

We wanted to serve our PHP app so we copied it to this directory and we were all set.

Some Helpful Stuff

An Introduction to Apache

Introduction to the Apache Web Server

Running PHP on the Mac OS X Leopard Apache Web Server

PHP: Built-in web server

A closure is a function plus a pointer to that function's scope. Peter Landin, a British computer scientist, defined the term in 1964.

I found what I consider to be a simple, understandable example that I want to share.

outer = function() { var a = 1; var inner = function() { alert(a); } return inner; } fnc = outer(); fnc();

Essentially, a closure is a function that has access to variables or values outside its scope. In the example, we define a variable a, then define a function inner within another function outer. inner has access to variable a. Because outer returns function inner, function fnc has access to a. a is local to the scope where inner was defined, so the pointer to a exists.

At run-time, when outer is invoked (outer()), a closure is formed, which consists of inner's code and references to outer's variables (a in this case) required by the closure.

It's common to read and write functions that only make use of the variables defined within their own scope. Closures, as you saw above, provide access to variables outside of a function's scope. The closure is not only a function, but also a pointer (as mentioned in the opening sentence). That pointer is a referencing environment, something Wikipedia defines as:

a table storing a reference to each of the non-local variables (also called free variables) of that function

A function pointer is a way to access executable code in memory. Instead of pointing to data, it points to a function that can be invoked after being dereferenced (the process is referred to as an indirect function call). It's different than a closure because it does not allow a function to access non-local variables.

Closures are most commonly seen in functional languages (those in which functions are first-class citizens), but are possible in OO languages as well.

So why use closures? Well they enable you as a programmer to write cleaner, more readable code, for one thing. They are a very convenient, idiomatic way to give a function access to local state without creating a new class/object, etc. A function's closure is essentially the same as a class' member variables. With closures you gain the power to access a block of code and the context in which that code was defined. You therefore can make use of that context (the variables defined within it, for example) later on if needed.

I'll leave you with this additional explanation:

A closure is a function that has one or more variables bound to it. The two are basically identical, at an implementation level at least. The real difference is in where they come from. In object-oriented programming, you declare an object class by defining its member variables and its methods (member functions) up-front, and then you create instances of that class. Each instance comes with a copy of the member data, initialized by the constructor. You then have a variable of an object type, and pass it around as a piece of data, because the focus is on its nature as data. In a closure, on the other hand, the object is not defined up-front like an object class, or instantiated through a constructor call in your code. Instead, you write the closure as a function inside of another function. The closure can refer to any of the outer function's local variables, and the compiler detects that and moves these variables from the outer function's stack space to the closure's hidden object declaration. You then have a variable of a closure type, and even though it's basically an object under the hood, you pass it around as a function reference, because the focus is on its nature as a function.

Some Helpful Stuff

Stack Overflow: What is a 'Closure'?

Cornell University Department of Computer Science: Capsules and Closures

Stack Exchange: What is a closure?

If you work in Clojure enough you'll see both require and use near the top of files, indicating the external files and functions that are needed in that particular namespace.

Let's say I want to use Ring's wrap-params function.

(ns project.core (:require [project.player.player-controller :refer [routes-handler]])) (def app-handler (-> routes-handler (wrap-params))) (defn -main [] (run-jetty app-handler {:port 3000}))

Let's say I have a test suite and I run my tests. I get an error.

java.lang.RuntimeException: Unable to resolve symbol: wrap-params in this context

I get this error because I'm attempting to invoke a function defined in a third-party library, but this namespace has no idea where to find this function. It doesn't know what wrap-params is.

Let's add a line to our code.

(ns project.core (:require [project.player.player-controller :refer [routes-handler]]) (:use ring.middleware.params))

Now it works. But what's happening? use is pulling in every single function from the ring.middleware.params library. It's as if I had all those functions defined in this namespace. That's far from ideal.

We could specify the functions we want to include with only.

(ns project.core (:require [project.player.player-controller :refer [routes-handler]]) (:use [ring.middleware.params :only [wrap-params]]))

And it works!

But let's take a couple steps back and adjust our original code a bit.

(ns project.core (:require [project.player.player-controller :refer [routes-handler]] [ring.middleware.params]))

Same error as the first time! Why? require is slightly different than use. By requiring ring.middleware.params I'm giving myself access to that namespace. Hop down further in the code and make another change and we can have this working.

(def app-handler (-> routes-handler (ring.middleware.params/wrap-params)))

Now it works! We specified the namespace where that function is defined. What if we did that from the beginning, but didn't require that namespace?

java.lang.ClassNotFoundException: ring.middleware.params

It doesn't know where to find ring.middleware.params. We can also alias the namespace to make this less verbose, yet potentially more clear.

(ns project.core (:require [project.player.player-controller :refer [routes-handler]] [ring.middleware.params :as ring])) (def app-handler (-> routes-handler (ring/wrap-params)))

That works! But for the most idiomatic solution, let's use refer, which takes a vector of functions, those you wish to pull in.

(ns project.core (:require [project.player.player-controller :refer [routes-handler]] [ring.middleware.params :refer [wrap-params])) (def app-handler (-> routes-handler (wrap-params)))

Near the top of the file we specify the function (wrap-params) and location of the function (ring.middleware.params). This allows us to invoke the function just as we would any other function that we defined in this namespace.

So overall, what's the difference between require - refer and use - only? Well, nothing really. require is more idiomatic, so it's considered best practice to include external functions by providing the library to require and specifying the individual functions to include with refer. By themselves, require loads libraries and use loads libraries and refers to their namespaces (giving you everything, which is usually/always overkill). use can also easily lead to namespace conflicts (for the exact reason I just mentioned).

Lastly, and quickly, if you hop in the REPL, you have a few options for how you can make use of all of this.

(require 'hyperion.api.set-ds! 'hyperion.api.new-datastore) (require '(hyperion.api set-ds! new-datastore)) (require '[hyperion.api.set-ds! :as ds]) (require ['hyperion.api.set-ds! :as 'ds]) (use '[hyperion.api :only [new-datastore]])

All valid syntax, all behaves similarly.

Some Helpful Stuff

Clojure Libs and Namespaces: require, use, import, and ns

Stack Overflow: Difference in Clojure between use and require

I'll borrow from someone and tell you that middleware is just plumbing. In reality, the term is used quite a bit, but tends to be defined in a horribly vague way. Want proof? Here's how Wikipedia explains it:

computer software that provides services to software applications beyond those available from the operating system

So that's a place to start, but it's ambiguous.

The developer who referred to middleware as plumbing explained his comparison by listing their similarities.

mostly invisible

provides standard way of doing things

ties together parts of complex systems

lets you worry about other things

The middleware "layer" can be considered a conversion, translation, consolidator, and integrator. It provides an abstraction (like an interface) between an application and the communication over the network. This abstraction allows us to hide the details of the process.

Diagram source

In cases where middleware isn't used, it's very likely that there will be duplication. Middleware generally connects two applications and passes data between them. These two applications exist independently and use middleware to communicate despite the fact that they may run on different platforms or come from different vendors, etc.

As I mentioned before, the term "middleware" is used a lot and the first definition I provided isn't quite appropriate for other types of middleware (although there are lots of similarities).

Rack middleware, for example, can assist with the process of accepting a request by breaking it down into different stages. It's a standardized interface in Ruby for interacting with a web server. It defines how to accept a request from the web server and return a response. As an example, when the request arrives, the user might need to be authenticated (or authorized). Later in the process, the response will need to be provided. Rack's README explains that its purpose is to provide an "adaptable interface for developing web applications in Ruby." It achieves this by wrapping "HTTP requests and responses in the simplest way possibly". Essentially, it's a filter. It passes a request along and augments the request and response as necessary.

Middleware, much like a framework, helps with things people need across many applications, but don't want (or have the time to) custom build each time.

Some Helpful Stuff

ObjectWeb: What Is Middleware

What Is Middleware?

What is Middleware? - Operating Systems and Middleware Group

Rails Casts: Rack Middleware

A "go to" statement performs a one-way transfer of control, unlike a typical function call that returns control to the caller. Go to statements are harshly criticized by many well-respected software engineers, such as Uncle Bob and Edsger Dijkstra. Interestingly, other noted engineers, namely Donald Knuth and Linus Torvalds, point out a number of instances where go to statements are useful.

In this post I mainly want to focus on Dijkstra's paper. He introduces the topic by making two observations.

First, Dijkstra explains that the programmer's job ends when he finishes a correct program (a very subjective term), but that he must keep in mind that he (his program) controls the action of the code that the machine executes. The execution of the program is then completed entirely by the machine. Essentially he's just pointing out that once the program is run, the programmer has absolutely no control over the execution.

Second, Dijkstra describes - in a very intriguing way - the reason go to statements make a program more difficult to understand. He explains that humans master static relationships better than dynamic ones. For this reason, the dynamic aspects of the program (those that are non-constant) should be made as obvious as possible. The difference between the static and the dynamic should be minimized. It's a noble goal, but one that cannot always be achieved. Programs can be complex and non-constant attributes will always exist.

He begins denouncing the use of go to statements by stating:

The unbridled use of the go to statement has an immediate consequence that it becomes terribly hard to find a meaningful set of coordinates in which to describe the process progress.

By "coordinates" he means the contents of the call stack and loop iteration stack (at any given point during the execution). By "process progress" he means program execution. So these statements make it so that we can't use the call stack - the data structure that stores information about the active subroutines of a program - to determine the value of the program variables, or invariants.

A program invariant refers to information about the data a program manipulates that remains unchanged when the program is executed. Such invariants can be invalidated when go to statements are introduced. A go to could transfer control out of a loop before it's completed or into an existing loop.

This discussion really exposes two issues. While designing the flow of the program, we can run into problems with go to statements (as we've seen) and with multiple exit points. Multiple exit points exist when a function introduces a number of return statements at different points, rather than one return at the end of the function. Often times, this problem can be solved by breaking apart the function into smaller pieces.

After a long break from working on my Java server I came back to it last week when Dave asked me to add the ability to serve a video. I hopped into my code and things came back to me fairly quickly. I noticed right away, however, that some refactoring was needed. I made a few minor changes, then began looking into how I could read a video file and serve it through the browser.

I began with a small video file, one just under 1MB. While small compared to many videos, it was quite a bit larger than the images (or text files) I had served before. I looked into reading files in "chunks" and decided to start by writing code that would read the file in smaller segments.

byte[] chunk = new byte[1024];

Instead of creating a byte array with a size equal to the number of bytes contained in the file (as I did when I served images and text files), I'm going to start by creating a byte array that can hold 1,024 bytes.

At this point I ran into a problem. Let's step back and look at how I constructed my original server. I had a class with a method in charge of getting the appropriate headers and body to generate the response based on the request that was made. So for example, when I read an image, I read the entire thing, returned a byte array, and began the process of writing the image (which involved writing a byte array to a FilterOutputStream).

With chunking, however, I needed to write the byte chunks to the output stream as I read them in. The writing portion needed to be part of the read loop, which forced me to refactor my code quite a bit.

Having refactored my code, I could see how this approach would work. The body of the response would be written in chunks and the headers would need to include several essential pieces of information:

content-type: video/[type of file, such as webm or mp4] Transfer-Encoding: chunked

By simply specifying a content type of video you'll see a video player in the browser. The chunked transfer-encoding requires you do structure your response in a specific way.

You will write to the output stream in a series of chunks. Each chunk will first contain a line with the size of the chunk data in hex. To do that, the number of bytes that were read will get converted to hex, then appended with a CRLF.

Integer.toHexString(bytesRead) + "\r\n";

The next line will contain the chunk data itself. This pattern (size followed by data) continues until the entire file has been read at which point the response will contain a 0, followed by a blank line.

As I went through this process, most of it seemed pretty straightforward, but I had to refactor again to make it easier to test.

I removed the hard-coded input stream and output stream from the method, instead choosing to set these values outside the method and using a ByteArrayInputStream and ByteArrayOutputStream, respectively, in my tests.

@Test public void writesToOutputStreamInChunks() throws IOException, InterruptedException { ByteArrayInputStream testInputStream = new ByteArrayInputStream("my string".getBytes()); ByteArrayOutputStream testOutputStream = new ByteArrayOutputStream(); chunkedResponse.setInputStream(testInputStream); chunkedResponse.setOutputStream(testOutputStream); chunkedResponse.readFile(5); assertEquals(testOutputStream.toString(), "5\r\nmy st\r\n4\r\nring\r\n0\r\n\r\n"); }

To test the readFile method, we pass a byte array to our ByteArrayInputStream and set that as the input stream. We call readFile and pass it the chunk size, in this case 5 bytes. If our code is written correctly, what's written to the output stream should be:

5 my st 4 ring 0

Since we're reading the file in chunks of 5 bytes, we respond with the size (5 bytes), the data ("my st"), the size (4 bytes), the data ("ring"), a 0, and a blank line. If this test passes, we can be confident that readFile correctly reads a file in chunks of size n and within the read loop, writes them to the output stream in the correct format.

Some Helpful Stuff

Chunked transfer encoding

Content Types - Web Developer Boot Camp

HTTP Protocol Parameters

Amending my post from the other day after gaining a better understanding of the (sometimes) absoluteness and destructiveness of Git's checkout command, I hope to provide a helpful explanation of the problem git checkout can introduce and how you can avoid this problem.

In the referenced post I explained why checkout has two seemingly different behaviors based on the options you pass (either a branch or a file). I explained Git's logic, which makes sense. That being said, there's a very dangerous situation that you can run into if you use this command with the expectation that it can be undone.

mkdir delete_me cd delete_me touch test_file.rb git init git add . git commit -m 'created test file' git branch other_branch

Now we open that file and make some changes.

git status # On branch master # Changes not staged for commit: # (use "git add ..." to update what will be committed) # (use "git checkout -- ..." to discard changes in working directory) # # modified: test_file.rb # no changes added to commit (use "git add" and/or "git commit -a") git checkout test_file.rb

Aaaaaand I'm screwed. The changes I made to test_file.rb are gone forever and there's nothing I can do. Let's pretend I didn't issue that dangerous checkout command. Instead, I did this:

git checkout other_branch

What happens? It switches to the branch called other_branch, bringing along the changes I made to test_file.rb with it. It's completely harmless. But if I gave it the file path instead of the branch name, I would've been in trouble.

As I said, Git's reasoning for this makes some sense, but I think there should be a new, separate command that permanently discards changes made to a file.

As an alternative, I've taken the wonderful advice found in this Stack Overflow post and aliased git checkout like this:

alias "gco=git stash; git stash apply; git checkout"

I still let it issue the git checkout command, but before I do that I stash my changes, apply the stash, and keep it in the stash cache.

Let's look at how this would work by returning to the example above. We'll re-do all the steps up until the git checkout command. At that point, I'll first stash my changes, then apply the stash, and then checkout the file.

If I run git status, I'll see that my working directory is clean. Let's try and recover my seemingly lost changes.

git log

Output:

11cc53a created test file

That won't work, but reflog will.

git reflog --all

Output:

d83e780 refs/stash@{0}: WIP on master: 3b117e2 created test file 11cc53a refs/heads/master@{0}: commit (initial): created test file

I see the stash I just made with a SHA referencing the state at that point. Instead of my changes being lost forever, I can quickly and easily retrieve them with a variety of commands.

git reset --hard d83e780

And I'm back to where I was before I ran git checkout. Running git stash and then git stash apply before issuing the checkout command saved the day.

Some Helpful Stuff

Git Basics - Undoing Things

Stack Overflow: Undo part of unstaged changes in git

Stack Overflow: git stash apply version

JavaScript’s popularity is almost completely independent of its qualities as a programming language.

Douglas Crockford, the JavaScript guy, says a lot in that sentence. It might not necessarily invite you to learn JavaScript, but it's an incredibly important tool, albeit one that is oft-bemoaned.

Programming is difficult business. It should never be undertaken in ignorance.

And in this sentence, Crockford - at least to me - summarizes why you need to learn JavaScript (and any other language/framework that you are likely to encounter on a client project).

JavaScript is the language of the browser, giving you the ability to add an interactive component to a web page. Crockford lists his "favorite" aspects of the language, touching on functions, loose typing, dynamic objects, and object literal notation.

Object literal notation means that you can very simply create objects by listing their properties. Properties are variables created inside an object. Below I use object literal notation (this is where we got JSON) to create an object called animal.

var animal = { first_name: "George" };

We have one property name, first_name, which has a value of "George".

animal.first_name // "George"

An object is a container of properties, where a property has a name and a value. Objects in JavaScript are not part of classes. They can be used to collect and organize data, including other objects.

var animal = { first_name: "George" }; function formName(last_name) { var another_animal = { first_name: "Herman" } return another_animal.first_name + " " + last_name; }; formName("Hermanson") // "Herman Hermanson"

When used inside of a function, the var statement defines the function’s private variables. One aspect of JavaScript that is frequently criticized is the language's dependence on global variables, something Crockford (and basically everyone else) call "evil".

var animal = { first_name: "George" }; function formName(last_name) { var another_animal = { first_name: "Herman" } return animal.first_name + " " + last_name; // even though animal isn't defined within this function, it exists within the global scope }; formName("Hermanson") // "Herman Hermanson"

In regard to JavaScript types, numbers, strings, and booleans are object-like in that they have methods, but they are immutable. In the example about, we aren't changing the string "Herman", we're creating a new string using concatenation.

var functionVariable = function() { alert("FUNCTION WAS INVOKED"); }; functionVariable();

A function can also be set to a variable and invoked with that variable name followed by open and closed parentheses. Why? Well, here's what's going on:

function returnWord(word) { return word; } returnWord("monster") // "monster" returnWord.call(undefined, "monster"); // "monster" returnWord.call(window, "monster"); // "monster"

Or functions can be defined like this:

var addingFunction = new Function("a", "b", "return a + b"); addingFunction(15, 3); // 18

Going a bit further, this example comes from John Resig's JS tutorial.

var ninja = { yell: function yell(n){ return n > 0 ? yell(n-1) + "a" : "hiy"; } };

ninja is a variable and the function yell is assigned as a property on ninja. It's the equivalent of this:

var ninja = {}; ninja.yell = function(n) { ... }; ninja.yell(4);

The yell function takes one argument, n. A ternary statement is used to determine what the function will return. If n is greater than 0, the function will return the return value of calling itself and passing itself (n-1) + "a". If not, it will return "hiy".

Lastly, a few simple debugging options include console.log and debugger, both of which allow you to hop into the code at the point where you called them, jump into the browser, inspect the element, open the in-browser console, and see what you have.

Some Helpful Stuff

JavaScript: The Good Parts

Douglas Crockford on JavaScript — Volume 1: The Early Years

Git is an incredibly powerful version control system, but for those of us with less experience, there's always an element of the unknown when using certain commands. One command that I always found odd was checkout. The reason I found it odd has to do with the two (very different) ways I most commonly use it.

In the first situation, I'm working on a branch - let's call it google_oauth - and I want to switch back master. I simply run git checkout master and I'm now working on the master branch.

In the second situation, I've made several minor changes to a file - let's call it view_controller.rb - and I realize that those changes were useless. I was going in the wrong direction and want to scrap the changes. I run git checkout view_controller.rb and my changes are wiped away.

But what if I issue that second command by accident? Why, when given a branch name, does Git switch me over to that branch, but when given a file name, seemingly destroys the changes I made to the final with no way to recover them?

Obviously Git's commands are well-thought-out, so I looked into the reasoning behind these decisions.

When a file path is not provided as an option - and a branch is - git checkout will switch branches (as we said). The key is understanding why. When this command is issued, Git updates the index, the working tree, and HEAD to reflect the branch you specified.

When a file path is provided instead of a branch, it might seem like an odd thing happens (as I mentioned before), but it really makes a lot of sense.

Remember, the command is basically a way for you to say "Hey, I'd like to update [whatever] to the most recent commit." When given a file path, it updates the file in the working tree to the index. So if I've made a few changes to my view_controller.rb file, do not stage them for commit, then run git checkout view_controller.rb, it updated this file from the index file, meaning that it takes that file back to its state at the previous commit.

HEAD is the current branch (or last committed state on current branch), index is the staging area, and working tree refers to the state of files in checkout.

Here's a quick example of some of the actions you can perform using checkout.

git checkout master ---> switch to the master branch git checkout soft_deletes -- user_controller.rb ---> does not switch branches; gets the most recent version of the user_controller.rb file from the soft_deletes branch rm -f user.rb ---> deletes user.rb file git checkout user.rb ---> restores user.rb file from the index

As is (almost) always the case with Git, even if you accidentally lose all your changes by issuing the checkout command and providing a file, you can revert back to the state you were in before you issued that command (reflog can be a life saver and I'll go into that more in another post).

Some Helpful Stuff

git-checkout

I've spent much of the last couple weeks working in Clojure and as I've gotten more comfortable with the language I've taken some notes on aspects of the language that were new to me.

-> macro

The arrow, sometimes called the "thread-first" macro because it "threads the expression through the forms", indicates the flow of data from one function to another.

(-> (Math/sqrt 64) int list) ; the result of the first function is the square root of 64, 8 ; that result is passed as an argument to the next function, int ; that result is passed as an argument to the next function, list (list (int (Math/sqrt 64))) ; these are equivalent

To me, this macro can make your code quite a bit more readable because it allows you to process the information from left to right. There's also a function signature convention where the threading macro is used to be more explicit about the purpose of the function.

(defn ->int [s] (Integer/parseInt (str s)))

This function is named ->int because the argument it's passed is converted to an integer (essentially passed in as the first argument to the integer-parsing function).

(defn another-destructuring [attributes] (-> attributes (dissoc :key_1) (dissoc :key_2)))

Again, attributes is passed as an argument to another-destructuring and it becomes the first argument passed to the dissoc function, implying that attributes is a map. The map returned by the dissoc function is then passed as the first argument to the next dissoc function.

(defn double-arrow [sequence] (->> sequence (remove #(= 3 %)) (first)))

The double-arrow threading macro is very similar to the single-arrow except that it passes the argument in as the last argument, rather than the first. In this example, sequence is passed as the last argument to the remove function and the return value of that function is passed as the last argument to the first function.

destructuring

(defn destruct [{:keys [id name]}] (println "id is " id) (println "name is " name)) (destruct {:id 14 :name "Bazooka Joe"}) ; => id is 14 name is Bazooka Joe (let [my-map {:sport "baseball" :animal "giraffe"} {:keys [sport animal] :as binding-entire-expression-to-this} my-map] (println "my map is bound to " binding-entire-expression-to-this) (println "sport is " sport) (println "animal is " animal)) ; => my map is {:sport "baseball" :animal "giraffe"} sport is baseball animal is giraffe

Destructuring is often referred to as a "mini-language" within Clojure. Formally, destructuring is defined as "abstract structural binding", meaning that a data structure, like a map, can be passed as an argument and bound to the respective parts of the expression.

overloading functions

Clojure supports arity overloading in a function.

(defn arity-overloading ([] (arity-overloading {})) ([this-map] (merge {:name "John"} this-map)))

If you invoke the arity-overloading function and do not pass it any arguments, it will call itself and pass itself an empty map and bind it to this-map. If you invoke the function and pass it a map, it will bind that map to this-map.

Some Helpful Stuff

Understanding the Clojure -> macro

Clojure: Functions

Thread Macro From Clojure

Clojure Guide Chapter #8: Arity & Overloading

In this post we're going to look at Josh Cheek's gist.

Let's start with the initialize method on line 10. Its arity is variable, as indicated by the * that precedes the variable name args. This method is defined on the anonymous class instantiated on line 5.

def initialize(*args) @attributes = Hash[members.map(&:to_s).zip args] end

What's members? On line 46, the members method is dynamically defined:

define_method :members do method_names end

Let's look at Ruby's define_method. It's simply another way to define a method. As opposed to using the builtin def, you can dynamically define methods with define_method.

def user @data[:user] end define_method(user) { @data[:user] }

members returns an array of method names. The map method is invoked on this array.

["one", "two"].map { |word| word + "!" } # => ["one!", "two!"]

As explained here, &, when preceding the last variable in a method call, allows us to call a method where the object is the parameter (instead of calling the method on the object).

[1, 2].map(&:to_s) # => ["1", "2"]

Josh invokes the map method on the array, then calls the to_s method and pass in each element (an object) from the array. This returns an array and he calls the zip method on that returned array.

[1, 2].map(&:to_s).zip # => [["1"]. ["2"]]

zip converts any arguments to arrays. The result of the method chaining we're using here is used to build a hash using Hash.

Hash["a", 100, "b", 200] # => { "a" => 100, "b" => 200 } Hash[[:method_1, :method_2].map(&:to_s).zip] # => { "method_1" => nil, "method_2" => nil }

But in Josh's example, he passes an array of arguments to the zip method.

Hash[[:method_1, :method_2].map(&:to_s).zip([8, "baseball"])] # => { "method_1" => 8, "method_2" => "baseball" }

And we set this return value to the instance variable attributes.

On line 40 remember that the equals sign in a Ruby method is just convention. It merely indicates that the method is a setter.

Also, on line 34 and line 40 Josh defines getter and setter array methods.

def [](key) key = key.to_s raise NameError, "no member '#{key}' in struct" unless @attributes.has_key? key @attributes[key] end

Let's say this getter method is defined on class Monkey and @attributes (set in the initialize method) is a hash: { "method_1" => 8, "method_2" => "baseball" }.

monkey = Monkey.new monkey["method_2"] # => "baseball"

As I mentioned, a setter method is defined just after this and works as any setter method does.

Let's go back to where we started and talk more about lines 46 and 50 where Josh dynamically defines methods.

method_names is an array containing a variable number of symbols. This array is passed as an argument to the new method and referenced on line 46 when it's returned by the members method (which we just discussed).

It's referenced again on line 50 as the array is iterated over and a method of the same name is defined for each symbol. Line 53 is a getter and 57. We saw how @attributes was set and by calling the getter method (in the same way we did before) we can retrieve the value associated with that key (the method name).

Lastly, there are several places in the code where we see &block. Very quickly, a block is very powerful and can be explained (in one way) like this (an explanation I found helpful when I was first learning about blocks):

class Array def iterate!(&block) self.each_with_index do |n, i| self[i] = block.call(n) end end end array = [1, 2, 3, 4] array.iterate! { |n| n ** 2 }

We invoke the iterate! method on array and we pass it a block (literally just a "block" of code). Within iterate! we call each_with_index which allows us to have both the index position of the array element and the array element itself. self[i] will give us access to the value associated with the index position, i, of self, the original array. When we invoke the call method on the block we passed in we pass it n (the element in the array). The block is executed (where we perform some simple multiplication) and when the iteration is complete we return an array: [1, 4, 9, 16].

A few weeks ago I presented a kata before 8th Light University. I chose the familiar coin change kata and I wrote it in Clojure. Presenting this on the projector screen required me to make both my tests, my code, and the result of running my tests visible without switching between windows. It seemed easiest to use Vim so I searched for a Clojure plugin in Vim.

I found the aptly named VimClojure. First I'll point you to a few resources that can help you with the process of installing VimClojure. This Github repo, vimclojure-easy, might be helpful and this screencast, albeit quite out of date, also might be of use.

When I began writing my tests (using Speclj) I found that the indentation was off. I dug into the VimClojure plugin by navigating to ~/.vim/bundle/VimClojure/indent/clojure.vim. At the bottom of the file you'll see this section:

" Defintions: setlocal lispwords=def,def-,defn,defn-,defmacro,defmacro-,defmethod,defmulti setlocal lispwords+=defonce,defvar,defvar-,defunbound,let,fn,letfn,binding,proxy setlocal lispwords+=defnk,definterface,defprotocol,deftype,defrecord,reify setlocal lispwords+=extend,extend-protocol,extend-type,bound-fn " Conditionals and Loops: setlocal lispwords+=if,if-not,if-let,when,when-not,when-let,when-first setlocal lispwords+=condp,case,loop,dotimes,for,while " Blocks: setlocal lispwords+=do,doto,try,catch,locking,with-in-str,with-out-str,with-open setlocal lispwords+=dosync,with-local-vars,doseq,dorun,doall,->,->>,future setlocal lispwords+=with-bindings " Namespaces: setlocal lispwords+=ns,clojure.core/ns " Java Classes: setlocal lispwords+=gen-class,gen-interface

Here they are specifying keywords that require indentation different from the default behavior. I added this section to account for Speclj.

" Speclj: setlocal lispwords+=describe,it,context,around

I opened my coin change program again, started writing tests, and the formatting was correct. I decided to submit a pull request and hopefully it gets accepted because one of the other apprentices already ran into the same issue, prompting me to write this blog.

"The price of reliability is the pursuit of the utmost simplicity. It is a price which the very rich find most hard to pay." - Sir Antony Hoare

Complexity is intrinsic to the task of delivering software. It is unavoidable. It's one of the reasons why our job as software developers is a difficult one. With that in mind, it makes sense to make a concerted effort to minimize incidental complexity, those things that serve no purpose but to obfuscate the true meaning, intended purpose, and functionality of the code. In layman's terms, don't make it harder than it needs to be.

I read this article and it led me to this paper, this paper, and this paper.

The common thread was the idea of complexity. In the second paper, Dr. Edsger W. Dijkstra offers a very interesting observation that's startlingly relevant despite the fact that it was made 25 years ago.

The educational dogma seems to be that everything is fine as long as the student does not notice that he is learning something really new.

He claims that the steep level of complexity involved in writing software is made worse by the way we introduce "radically novel" concepts. I've always maintained that in order to access "new" information we must build connections to previous knowledge. Dijkstra disagrees and admittedly, I disagree with my previous belief (at least in the sense that my suggested way of learning isn't always appropriate). Perhaps part of the struggle we endure as we learn (all of us, regardless of experience, every single day) is due to our tendency to force a square peg in a round hole. Dijkstra continues:

Coming to grips with a radical novelty amounts to creating and learning a new foreign language that can not be translated into one's mother tongue.

In my experience, I've always attempted to achieve this "translation", often times with much difficulty. Maybe that's our way (or at least my way) of coming to grips with the intrinsic complexity of software. And if that approach is flawed, we may write code without fully accepting its intrinsic complexity, leaving us prone to introducing additional complexity.

Rich Hickey, creator of Clojure, said that "overcoming complexity isn't work, it's waste". If we appropriately and accurately acknowledge the inherent complexity of code, then we can recognize unnecessary complexity and remove it, rather than wasting time overcoming it.

So we accept that it's a difficult pursuit, we approach it as a blank slate, and we tread carefully. We minimize incidental complexity and rebuke the notion that there's more value in what an application does than in how it is put together.

Our software won't be "easy" to understand. Dijkstra went so far as to say that software "is the most complex product ever produced by human effort." But we don't need to add to that intrinsic complexity. To avoid unnecessary complexity, we need to carefully think through our decisions and push back when given unrealistic deadlines. As you might expect, it is mentioned in the third paper that part of the problem lies with "the lack of education of the average engineer, the short-sightedness of the managers and the malice of the entrepreneurs." The first paper introduces ten "rules" that serve as coding guidelines. Their purpose is to prevent programmers from writing code that is so tremendously opaque that the system can't be maintained and new users can't make sense of it.

Most of our systems are much more complicated than can be considered healthy, and are too messy and chaotic to be used in comfort and confidence.

It must be important to us to change that.

Some Helpful Stuff

The Mythical Man-Month: Essays on Software Engineering by Frederick Brooks Jr.

The Art of UNIX Programming by Eric Raymond

The app I worked on this week required me to install Postgres. Postgres - originally called Postgres, but now "officially" PostgreSQL - is an object-relational database.

I ran into a few issues getting it up and running and I wanted to walk through some of those here. OS X ships with Postgres, so I already had it on my machine, but at some point I had also run brew install postgres. Because of this, the server was running, but I wasn't sure why or how.

Note: one of the first steps in the process includes initializing the Postgres database cluster with the command initdb -D /Library/PostgreSQL/9.2/data

I ran this command to try and find where my Postgres process was running, thus preventing me from properly firing up the app.

ps -e | grep post

Part of the output included this process:

/Library/PostgreSQL/9.2/bin/postmaster -D/Library/PostgreSQL/9.2/data

But upon trying to kill it, I was told that the operation was not permitted. I decided to try and solve the problem without the use of the (potentially) dangerous sudo command.

Next I ran:

launchctl stop -D /Library/PostgreSQL/9.2/bin/postmaster

Again, this didn't work. launchd is Apple's open-source replacement for cron, xinetd, mach_init, and init. It manages named jobs that are specified by having launchctl load plist files with the launchd configuration for a task. With launchctl, you can manage launchd services. launchctl can take commands from the command line.

The error returned from running the above command read:

usage: launchctl stop

I found out that by default this process runs at load, so I opened the .plist file in /Library/LaunchDaemons/ and changed the default to false.

I then ran:

sudo -u postgres /Library/PostgreSQL/9.2/bin/pg_ctl stop -D/Library/PostgreSQL/9.2/data

This command correctly stopped the process and with the run-at-load flag set to false, it wouldn't re-start. I double-checked the processes running and saw that the Postgres server had stopped.

Before I corrected this problem, when I tried to fire up the server I received an error that informed me "the server requested password-based authentication, but no password was provided". I had to edit a Postgres configuration file, called pg_hba.conf located in my /usr/local/var/postgres/ directory.

I edited this file to allow local users to access databases without a password. Having corrected the first problem and now allowing users to access the database without a password, I was finally able to get started.

psql -d postgres

Running that command allows you to enter into Postgres at the command line. You can list all the databases with \l. I successfully created my development database, fired up the app, and I was all set!

Oh, by the way, you can't use hyphens in object names in a Postgres database. As this Stack Overflow member so eloquently phrase it, "Don't do that."

Some Helpful Stuff

Creating Postgres Database Cluster

Postgres FAQ

Intro to using launchctl with MacPorts

About Daemons and Services

Postgresql: show tables, show databases, show columns, describe table

15 Practical PostgreSQL Database Administration Commands

Can not connect to local PostgreSQL