Taryn's blog

Today was the first day of my pairing tour, and I had a lot fun fun writing plain JavaScript (not just jQuery) again for the first time in months. It always takes a little while to jump back into the syntax of a language, but that's to be expected.

When pairing, though, it sometimes makes the adjustment a little more frustrating. You know what you want to write, but you're a little uncertain. You don't want make stupid mistakes, and it's tempting is to simply ask your pair if you're doing it right.

Here at 8th Light, pretty much everyone learns Clojure at one time or another. I first learned in Ruby and JavaScript, then later came to appreciate Java's static typing. Clojure was the most recent language that I've learned, and the first functional one.

It took me days to get the hang of Clojure's funky syntax. Everything's a function and functions are evaluated from left to right with the parameters following the function form. Eventually, though, I became adept enough to complete a few katas and re-build some of my old Ruby applications in Clojure.

However, I had the feeling that I was fighting the language' style. Brian Marick describes this style difference between object-oriented and dataflow style very well in his book, Functional Programming for the Object-Oriented Programmer. According to Marick, in OO languages, we model a problem using objects that may have relationships/dependencies to other object. For functional programming, on the other hand, we use functions that depend only on their inputs.

For example, I recently created a small "Duty Assigner" application designed to assign apprentices to weekly duties, taking into account their relative experience, availability, and previous assignments.

If I had written this particular program in Ruby, I'd probably have created a Duty class, an Apprentice class, and perhaps subclasses of each (for AvDuty and Foreman) to handle the special requirements of AV assignments and the apprentice who must serve as foreman. Based on variations in inputs (e.g. - are 2 or 0 people absent?), the application might call different methods within any or all of the classes. It's execution path would vary.

However, when writing the program in Clojure, I created a number of primary methods that take in apprentices and duties (both lists of maps) as arguments. Each of these methods is called in a the same order every time the application executes, each modifying the apprentices list of maps and passing it along to the subsequent method. Helper methods provide logic evaluations (e.g., all-duties-assigned?) as necessary, and all the methods would live in a file with a duties.assigner namespace.

Of course, to an object-oriented programmer, this might at first seem like dataflow style lacks structure and organization. The classes in an object-oriented program use categories and hierarchy that are easy for humans to understand.

However, I think that classes in object-oriented programs sometimes create unnecessary complexity. In the Duties Assigner example, the object-oriented Ruby version would likely include Duty subclasses for each of the 5-6 types of duties required. While each duty has different but similar attributes (name, max_people_required, min_people_required), it seems like overkill to create a separate class for each.

Last week I created a server in Java to handle various client requests, and, as a result, I'm becoming very well acquainted with HTTP request headers. However, I found it a little tricky to figure out how to read in and parse the the request line and headers.

It was easy enough to get the method and URI from the request line; I simply used regular expressions. However, the headers were less straightforward because they vary among requests. To make the header data easily accessible for any given request, I had to parse the headers and save each as keys and values in a hashmap.

However, even after parsing the headers, I was still missing data.  In HTTP requests, there are several line breaks between the headers and any body content. I was missing the body content when I stopped reading after line == null. For example, the following post request contains params--with key "data" and value "cosby."

POST /form HTTP/1.1--break--Connection: close--break--Host: localhost:5000--break--Content-Length: 10--break--Content-Type: application/x-www-form-urlencoded--break----break----break--data=cosby

To get this data, I needed to determine the length of the request by getting the Content-Length bytes, and reading up to that number of bytes. However, since the Content-Length is often missing, a better idea is to simply read in bytes until you reach the end of the file. To do this, I casted the request to bytes as I read it in from the buffered reader.

This worked great, however it had a big drawback. Since I was no longer reading in the request as "lines," I wasn't able to break apart the headers by interpolating line breaks "\r\n" into the string.

After completing my minimax tictactoe game in Clojure last week, I decided to refactor it to remove the atoms, which I had used to store state of user types (human, computer, or AI) and the maximizing player piece (used in the minimax-related functions).

Before refactoring out the atoms, I first learned a little bit about a couple of key features in Clojure: Maps and multimethods.

Maps

Similar to hashes in Ruby, maps are lists that associate keys with values. Clojure provides two different types of maps, hash-maps and sorted-maps. Hash maps are more flexible because the keys can be a mix of strings, keywords, numbers, or symbols. In sorted maps, keys must all be the same data type so that the collection can be sorted.

For performance reasons, it makes sense to only use sorted-maps when you actually need the map to be sorted by key.

Maps are a great way to store metadata about an object or entity without actually persisting the data. You can simply pass along the map to functions that know how to access its values through its keys. Functions can even change the map or add new elements to it before passing it along.

In my Clojure tictactoe application, I removed the player type atoms and update-player-types function from the game package. In its place, I added a get-player function, which returns a hash with piece and player-type key/value pairs.

I call this method from the main, which calls the player function with a board vector and two player hash map arguments. In addition to adding the two player types as arguments to play, I also needed to add them as arguments to the next-move function. Doing so ensures that the player hash maps persist throughout gameplay.

Multimethods

Next, I used multimethods to apply different get-move logic based on the value of the current player hash-map's player-type value. In Clojure, you can use multimethods to implement polymorphism, dispatching different functions based on the presence of certain attributes or metadata within the arguments.

The defmulti specifies the name of the method as well as its dispatching function, which defines the argument(s) that the method takes as well as the logic by which it dispatches to one of the various implementations of the method.

In my application, I used defmulti to define a get-move method, which takes current-player and board arguments and dispatches to a defmethod according to the :player-type value of the current-player hash map. For this implementation to work, I needed to first refactor the current-player method to return player hash-map rather than piece. This was a pretty minor change. However, I found it necessary to add a new current-piece method, which returns the current player piece based on the current board vector.

If I hadn't added the current-piece method, I would have had to pass both player hashes to the AI package, which is responsible for getting the AI player's move. This seemed like overkill to me, especially since the minimax functions already take a number of arguments.

I think that this basic multimethod elegantly handles the differing move logic for each player type. An alternative would have been to use case statements. While would have worked, it would have been less easily extendable for additional player types. It also would have violated the open-close SOLID principle.

After working with Clojure for a few days, recurring through loops to build up local variable values, I realized that I really needed to know how to handle state.

Of course, Clojure provides several ways to mutate the state of non-local variables. However, coming to Clojure from an object-oriented background, I was completely unfamiliar with the vocabulary of these reference types (atoms, refs, and agents) and, therefore, steered clear of them at first.

But, for my Clojure tic-tac-toe game, I really needed to know how to access user settings (e.g, difficulty level and player type), and this required setting and changing non-local variables. 

Atoms

I first explored atoms. Atoms are a reference type that allow you to assign an immutable Clojure data type to a non-local variable.

After defining the atom variable, you can modify its value any number of times within the namespace. You can also access the atom variable's value from anywhere in the namespace.

To create an atom, simply define a variable as usual, but call the atom function to assign the value to an atom. For example, my player-types variable might look like this:

(def player-types (atom []))

To access the value of player-types, you can now call @player-types:

=> @player-types

[]

Next, define a method to update the atom's value. To update the value of an atom, you can use either the swap! or the reset! functions. Both swap! and reset! set the atom's value to the new value and return the new value, but swap! modifies the existing value while reset! replaces it entirely.

For my player-types example, I created a  method called update-player-types to add player type values to the @player-types atom.

(defn update-player-types [new-type]

    (swap! player-types conj new-type))

Now, in my tic-tac-toe application, I can call this method with new values to add them to the @player-types atom.

(update-player-types "human")

(update-player-types "computer")

Now, within other methods, I can easily access these values:

=> @player-types

["human" "computer"]

=> (first @player-types)

"human"

=> (second @player-types)

"computer"

Refs

Next, I looked into implementing the same variables using refs. Refs are variables that store values in a single location. They allow for the values to change through "transactions," which are triggered by using certain functions, including ref-set, alter, and commute. 

To create a ref, define a variable and call the ref method on the value. For my player-types example, I call ref on an empty vector.

(def player-types (ref []))

Exactly like atoms, you can now access the ref value throughout the namespace by calling @player-types.

=> @player-types

[]

Similar to atoms, you'll now want to create a method for modifying the ref value. You can use ref-set, alter, or commute. Ref-set simply sets the new value regardless of the current value, alter changes the current value, and commute is similar to alter but it better handles concurrency issues resulting from order of operation dependencies. 

Whichever ref function you choose, you must wrap it in a dosync function. The dosync function starts the transaction and runs the nested calls.

(defn update-player-type [new-type]     (dosync (alter player-types conj new-type)))

After defining the update method, you can call it with the new values that you want to assign to the ref.

(update-player-type "human") (update-player-type "computer")

=> @player-types

["human" "computer"]

=> (first @player-types)

"human"

=> (second @player-types)

"computer"

As you can see, the implementation of the refs player-types example is almost exactly like the corresponding atom player-types. So, which one should I use in my tic-tac-toe application?

According to the documentation, refs should be used for synchronous, coordinated and shared changes, and atoms should be used for synchronous, independent and shared changes. In other words, refs are intended to be used when you need to update multiple refs within a single dosync transaction.

In the case of setting a tic-tac-toe game's player-types values, all changes to the values will be simple one-off updates. So, in this case, it makes more sense to use an atom rather than a ref.

Also, after implementing the player-types atom variable in my application, I realized that using a vector to hold the types wasn't the best approach because the atom vector still hangs on to all previously selected types.

The iterator patter is when you provide a way to access the elements of an aggregate class without exposing the innards of the objects itself. According to Olsen, you can accomplish this by using either an internal iterator or by using an external iterator.

External iterators are objects that are separate from the aggregated elements--meaning they are defined in a separate class. An external iterator accepts the aggregate object as an argument, and it defines methods that move sequentially through it, accessing its data along the way. An internal iterator approach is where the aggregate's class defines (or imports) its own methods for iterating through the elements.

For example, for an applicant-tracking app, we might want to get and sort all applicants by the date at which they submitted their application. To create an internal iterator, I would simply include the Enumerable module in the ApplicantPool class and define methods for <=> the element (applicant object) and each for the applicant pool aggregate.

After re-building our Sinatra app in Rails this week and implementing the repository pattern for our tables, we spent a fair amount of time rewriting and refactoring our RSpec tests. In doing so, I learned about some helpful RSpec methods and conventions that I'd never before used.

Testing controllers with assigns

Before we began writing our controller tests, we had to extract some database queries and other params handling conditions to our repository classes and helper modules. After we couldn't extract out anything else, we began writing controller tests.

To test controllers, you use RSpec to simulate an HTTP request (with an optional params hash) and then write an assertion on what you expect the instance variables or cookies to be. This seems pretty straightforward, and if you've written lean controller methods, then it probably will be.

Before writing our controller tests, I had never before used the assigns method. This method accepts a symbol, and it returns the value of the instance variable of the same name in the specified controller method. This method allows you to thoroughly test even those controller methods that are make multiple instance variable assignments based on params values.

For controller methods that are simply displaying a page, you can assert that they render a template or successfully display the page with a 200 response status. You can also test that they redirect to the correct path when appropriate.

When some routes in the application require user authentication, you can stub the method used to check authentication and simply set the method to return true/authenticated, as shown in the example below. 

Setting up conditions for testing

Test setup can easily get very complicated when dealing with complex objects. However, there are a few strategies you can use to make your RSpec tests cleaner.

First, don't create and destroy objects unnecessarily. In other words, don't use a before :each block when a let will do. Before each blocks create and/or destroy objects before each individual test is run. A let block simply allows you to create and objects and assign it to a variable that each test can access.

In addition, let blocks are also a nice way to give meaningful names to the values that are passed in to create objects. For example, since our applicant objects take in a number of date values that must be assigned in a certain sequence, we found it helpful to create Date objects called, yesterday, today, tomorrow, etc., to more clearly communicate the intention of the date values.

Finally, when you have two different classes that share functionality (such as our MemoryRepository and ActiveRecord Repository), you can write tests that are used to test both classes. Simply define the tests as "shared examples," shared_examples/<testname>_example.rb. Within a shared_examples block in the file, the described_class keyword serves as a variable for the class name of all of the classes that use the tests.

In the spec file for the class that uses the shared example, simply assert it_behaves_like and provide the shared example's name. This runs all of the tests in the shared examples file with for the given class. 

Testing for raised exceptions

This week we migrated our Sinatra app to Rails 4. This presented a few unexpected challenges. The first challenge was how to normalize our date inputs before saving and updating applicants.

In our Sinatra app, we had simply called the helper method modify_date_params in the controller to modify the params before using them to create or update and applicant in the database.

However, this approach wasn't possible in Rails 4 due to its "strong parameters" feature. This features requires you to specify which fields are allowed to be updated through mass assignment (e.g, form submission) to prevent the user from updating field values, such as password, that they shouldn't be allowed to.

When using strong parameters, you must define a private controller method to specify which params fields are required and which are permitted. Our applicant_params method simply states that the :applicant params hash is required, and it lists the fields in the applicants table that are permitted to be updated through mass assignment. In the create and update actions, we call the applicant_params method as an argument in the create and update method.

Now, if somebody tries to add a form field to the page in the browser and pass params to update a field that is not listed, such as :state, it identifies the unpermitted parameter and does not update that field in the database. I tried to do this by changing the ttt_repo_link field to state and adding a value; it updated all other permitted fields in the action but did not change the state value.

Rails 4's strong params is a nice feature because it adds security to your forms.

However, we struggled with how to modify the date values in the form params since we were passing the params through the applicant_params method to the create and update methods. We initially tried to add modify_date_params as a before_validation callback method, however this caused errors because the date values were still hitting the database as strings instead of Date objects. We needed to modify the information before it was sent to the database.

The Rails filters methods turned out to be an excellent solution. Filters are similar to callbacks, except they trigger methods around controller actions rather than database actions. By using the before_filter method on the :create and :update actions, we could call the modify_date_params method on the params hash before it hits the create and update actions.

The Composite design is a pattern of organizing a system in which functionality can be broken up into functional sub-components. These sub-components, in turn, may also be broken up into two or more subcomponents.

This branching can continue until you reach the lowest possible component and it doesn't make logical sense to break it up further. For example, in a dinner party planning app, you could break an event into food, meal options, and side dishes; but it wouldn't make sense to create sub-components of individual peas or lettuce leaves.

This design pattern works when there can be base class (the interface of the component) of attributes that all sub-components share. In Design Patterns in Ruby, Olsen mentions GUIs as a case where the composite pattern makes sense. Different interface objects can be made to include different combinations of widgets and UI features. My composite pattern example is the dinner party planning application. 

The higher-level "composite" objects don't have any parent objects, meaning they are not sub-components of any other object. For the dinner party planning app, these composite objects might include food, venue, and guest list. 

Each higher-level composite class inherits from an abstract class that defines how to add subcomponents, remove subcomponents, and (optionally) get data about the object's subcomponents. In the dinner party planning app example, each composite object has a price, and the price is dependent upon its sub-components.

The Observer is a design pattern that allows classes in an application to share information about state without adding undesirable dependencies between the different classes.

For a class to share information about its state with other classes, you can add these other classes as "observers" of the given class (e.g., an array of objects assigned to the instance variable @observers).

For each registered observer, you can call a notify_observers action after certain actions or state changes occur in the given class. This method iterates through the list of observers and can pass along information to each one, perhaps triggering them to take other actions or change state.

In the Design Patterns in Ruby, Olsen recommends using the Observable module provided in the Ruby standard library. By including this module in a given class, the class receives access to several utility methods for initializing @observers, adding observers, removing observers, etc.

After adding custom date validation to our Sinatra app, I wanted to add flash messages for when users update and create new applicants. These messages should appear after a user submits a form, indicating whether or not the create/update was successful and, if not, what the errors were.

Since Sinatra does not come with flash messages like Rails does, I needed to install a gem that supports session-based flash messages. I chose to use the sinatra-flash gem.

After installing the gem and enabling sessions in the app, I added conditions to the controller's post routes for creating new applicants and updating existing applicant profiles. If an applicant successfully saves, a success message is added to the :notice key in the flash session. If the applicant doesn't save, then its error object messages are added to the :error and :errors_list keys.

For the :error key, the full_messages method returns the error messages as an array of strings. For the :errors_list key, the messages method returns a hash of messages where the key is the field symbol.

I captured the errors in both ways because I use the two lists differently in the form view. I iterate through the flash[:error] array above the form to list each of the errors (if there are any). I use the flash[:errors_list] hash to conditionally add the "required" class to any input field that has an error. As a result, each input field with an error displays with a red border, making it easier for users to identify the problematic fields.

To apply the conditional "required" class, I needed to create a helper method called error?, which accepts a field symbol and returns a boolean indicating whether or not the field has an error, based on the contents of the flash[:errors_list] hash.

Our Footprints Sinatra app tracks applicants through the application process, and it depends greatly on the accurate input of "status" date information for each applicant. For each of the six dates in the "applicant profile", none of the dates can be in the future, each date must be ahead of the prior status dates, and each date must be earlier than all subsequent status dates.

Because the validation of each date value depends on the other status date values, I had to create a custom validator to handle each of the six status dates. 

To do so, I created a DateValidator class that inherits from the ActiveModel::Validator class. This class provides the validate method, which takes a record object. In the DateValidator class, I needed to redefine the validator method to validate the six status dates.

I used two private helper methods for these validations. The first one, future_date_validation checks that the value in each date field is not in the future. It adds an error message to the applicant's error object for each field that does have a date in the future.

The second helper method, date_sequence_validation, checks that the subsequent field dates are always later than prior ones. When a violation occurs, an error with a message referencing both dates (violation date and the date it conflicts with) is added to the error object. Both of the validation helpers are called in the validate method.

This week we added even more applicant queries to our Sinatra Footprints application. This time, we wanted to add an integer value for each applicant on the applicants index page indicating the number of days since the last action has been taken.

The logic to implement this was pretty straightforward, however, continuing to add applicant queries as helper methods to state_helpers.rb was becoming confusing, cumbersome, and worst of all -- slow to render in the browser.

We obviously needed a state field in the Applicant database so that we could easily query for all applicants at a given "state" in the applicant process. I wasn't sure of the best way to implement this, though. 

 A finite-state machine is a device that tracks the initial and current state of an object. There are a known number (finite) possible states and defined events that can trigger an object's transition from one state to another. Since I knew that tracking the "state" of an object seemed similar to the use case of finite-state machines, I decided to first investigate gems support this type of functionality.

I chose to implement the state_machine gem in my Applicant model. I added a state field to the applicants table and wrote a state machine block defining the initial applicant state as well as the events triggering the transition to other states. 

It was easy enough to define the state machine events. The tricky part was determining how to trigger these events in the application. For Footprints, applicant state only changed when creating or updating an applicant. As a result, I decided to add a callback to trigger the state machine events.

Callbacks are methods that you can use to trigger logic (i.e., call another method) when the state of an object changes in a certain way. The ActiveRecord documentation thoroughly describes all of the callback options. I chose to use the after_validation callback so that the update_state method would only be called after valid object creates/updates. To add the callback, simply "register" it in the model with the desired actions and options.

However, after adding the callback to the state machine implementation, I realized that I had unknowingly created an infinite loop. Every time I created or updated an applicant, the callback triggered the update_state method, which updated the applicant state, again triggering the callback and calling update_state, etc.

This was no good. It seemed like I was adding too much complexity with the state machine. Besides the state machine interfering with my callback logic, the state_machine gem also included a ton of functionality that I wasn't using in my very simple implementation. It seemed silly to add a dependency to a third-party library when it wasn't really necessary.

So, I decided to get rid of the gem and state machine and instead just use the callback with a refactored update_state method. In my revised update_state method, I sort of created my own state machine. On the after_validation event, the update_state method is called, which transitions the applicant object to a new state based on its field values. It defaults to the initial state, "applied_on." 

Creating long user input forms with many predefined options can quickly lead to ugly, bloated view files in Sinatra apps. We made a lot of effort this past week to keep our applicant form under control using helpers, iterators, and partials.

First, once we realized that we needed to use basically the same form in both the new applicant view and the applicant profile view, we created a partial view called applicant_form.erb that we could pull into both views.

Next, since the form includes a number of date fields that are identical except for their names and labels, we used an array to store the field names. We then iterated through the array to create the input tags and other markup for each field.

For each field, the applicant instance's property is called in the input's value attribute so that any existing value for the applicant instance populates the field in the profile page. Since the field name is being passed as a variable in the block, you have to use the send method to call the getter on the instance.

We used this same approach for each of the text fields in the form (:assigned_craftsman, :email, :ttt_repo_link).

For the select menus in the form, we created an array of the menu options to generate the necessary option tags. Since we wanted to show values in the database as "selected" values on the applicant profile page, we had to use conditional logic as well. This conditional logic checks to see if the given option matches the instance's property value in the database, and it adds the selected attribute when it does.

Finally, we extracted the logic for formatting labels and date values for display on the page into helper methods.

One of the challenges with building the Footprints Sinatra application for tracking applicants is that it requires a number of pretty complicated database queries for to get all of the applicants with a given status.

To prevent our controller from becoming a massive mess of DataMapper queries, we added a new StateHelpers module. This module defines methods that return the correct collection of applicants for all of the possible applicant state queries. 

Using these methods, we can easily get collections to pass into other analytics methods in the analytics view. These analytics methods return summary values, such as averages, percentages, and total applicants.

We also used the StateHelpers methods to simplify the params helper method used to get the correct collection of objects during filtering. 

Up until now, we had been using a CSV file to seed dummy data into our Sinatra Footprints app. As we continued to update our database model, though, it became more and more cumbersome to keep generating new CSV files of dummy data.

Even worse, the online tool we were using didn't provide enough configuration options to generate valid date values (in the application process, dates for earlier steps were sometimes later than the dates in subsequent steps). Unsurprisingly, this produced ridiculous analytics values.

So, I decided to replace our CSV solution with a script using custom methods and randomized, realistic values courtesy the ffaker gem. Because I figure that we'll continue to modify the database throughout the next few iterations, I wanted to make the script as easy to update as possible.

To seed appropriate date values for each status step in the application process, I created a method called random_datetime to generate a DateTime object with a date value that would be random but not overlap with the subsequent step's value. To do this, the random_datetime accepts two arguments, the earliest possible date and the latest possible date.

To make the main generate_applicants! method easy to configure, I wanted to pass in the date range values in the familiar "MM/DD/YYYY" format. I then used our helper method convert_to_datetime to convert each string date into a DateTime object. I was then able to call rand(early_datetime_object..late_datetime_object) to get a random datetime object within the range.

I also used a counter to conditionally fill in certain properties in each applicant instance. The conditions were set so that only certain applicants had dates for certain statuses in the application process. For example, all applicants greater than 40 had a completed_ttt_on value. However, those below 40 are have received a reply (initial_reply_on date set) but haven't completed the TTT challenge (completed_ttt_on value is nil).

I used helper methods to generate randomized values for the other string Applicant properties. This allowed me to modify the Faker data where necessary and simply call the helper methods in the generate_applicants! method.

Our Footprints Sinatra app has an "All Applicants" page that uses a form to filter a list of all applicants. The user selects the radio button of a stage in the application process, clicks the apply filter button, and a GET request is made.

This GET request passes the status value param as part of the query string, and the applicants/ page is reloaded with the URL, a "?" separator, and this query string. For example, applying a filter for applicants at the "currently refactoring" status loads the /applicants?status=refactoring page. This visual cue helps give context to the user.

We were mostly happy with this solution, but there was one remaining UX issue. The selected radio button would become deselected when the new URL with the query string would load. This disrupts the continuity of the user experience, and might erroneously suggest that the filtering didn't work correctly.

As a result, we decided to add a little JavaScript to the page to re-add the selected attribute to the appropriate select menu option on page reload. This seemed simple enough. We knew that the params always contained the selected filtering status string, but we weren't sure of the best way to access this param.

After struggling for a bit, we eventually figured out that we could easily access the GET request query string using the JavaScript window object.  The Window object represents the current window, and Window's location property gets the Location object, which contains information about the window URL.

Using the Location property search, we could access the search string. For example, after filtering on the Reviewed status, you can run window.location.search in the console, which returns "?status=reviewed".