Code Mason

Recently I had a requirement in an application I was working on to add an internal Settings model.  By my own standards, I wanted it to be very lightweight and flexible. There are several gems that provide such drop-in functionality, such as https://github.com/Squeegy/rails-settings or https://github.com/railsjedi/rails_config.  However, I felt this added an unnecessary dependency to our project for a seemingly simple feature.

Instead, I ventured to create our own ActiveRecord Based Settings Model.  I went in knowing that it needed to be lightweight and flexible.  The following is a brief walkthrough of how I went about implementing a Settings model that ultimately has one record in the Settings table and leverages Ruby's dynamic programming to create attributes when required.

The Migration

class CreateSettings < ActiveRecord::Migration def change create_table :settings do |t| t.text :preferences t.timestamps end end end

Pretty straightforward here. Only one column in addition to the timestamps, :preferences. This will be a serialized field allowing me to enter in key value pairs in a hash.

The Pseudo-Singleton

I wanted to ensure that there would never be more than one record in the Settings table, but didn't feel the single Settings object would be accessed often enough to warrant a proper Singleton instantiation.  Instead, I added the following to the Settings Model:

class Settings < ActiveRecord::Base ... before_create :confirm_singularity ... def confirm_singularity raise Exception.new("A Setting record already exists") if Settings.count > 0 end end

This simple bit of code ensures that an exception will be raised if an attempt to create a Settings beyond the first is made.

Leveraging Method Missing

I wanted to allow that single Settings preferences Hash to not rely on predefined keys. Rather, it would be far more flexible for any key to be set throughout the lifetime of the application.  To achieve this I relied upon method_missing:

def self.method_missing(method, *args) method_name = method.to_s super(method, *args) rescue NoMethodError settings = self.first_or_create if method_name =~ /=$/ name = method_name.gsub('=', '') value = args.first settings.preferences[name.to_sym] = value settings.save else settings.preferences[method_name.to_sym] end end

In this method, when a method that is not found is caught by the 'NoMethodError'.  Once caught, I first retrieve or create the settings record.  This allows for anyone to assign settings without needing to worry about whether a Setting record has been created or not.

From there I employ a regex to determine whether the method sent to Settings is a setter and thus contains a '='.  If it is then I parse the name of the method from the method, and assign the value of the key as the first of args. Then I access the settings.preferences  Hash and add the key value pair and subsequently save the settings record.

In the case that the method sent to Settings is a getter, then the else statement simple retrieves and returns the value of the key from settings.preferences.

Conclusion

After completing this task of implementing a flexible Settings model. I felt far more comfortable with Ruby's dynamic/meta programming.  Of course there is an overhead cost to dynamic programming, but I feel that for something as innocuous as application settings it was worth it.

I recently finished reading Russ Olsen's Eloquent Ruby. I found it to be an accessible and informative read as both a introduction to Ruby, but also a great supplement to one's existing knowledge-base.

My introduction into Ruby came via Rails and that inherently brought a number of downsides, mainly a limited understanding of the full depth of Ruby itself. I feel as if I am only now separating my Ruby knowledge from Rails, and Russ' book goes a long way in helping that effort.

Eloquent Ruby is a high level explanation of all the quirks and particulars that make Ruby a unique and powerful programming language. Each chapter follows a standard structure of explaining a concept, discussing how it can be used on a sample 'Document' project, then displaying the concept in practice in the Ruby world. This is an excellent way to structure a book that ranges on so many topics and it allows the reader to come away from the book with a 'Document' project that utilizes so many core Ruby concepts.

The portrayal of Ruby concepts in public gems and tools was eye-opening. Before reading this book, especially the final few chapters, I really had never thought about how gems like Rspec or Rake work but now I have a rather concrete understanding how both of those popular tools are essentially just DSL's. 

Overall, Eloquent Ruby isn't a reference book but a primer on the cool things about Ruby.  It sparks intrigue in a wide range of topics from dynamic_methods to passing around blocks. However, because so many topics are discussed I didn't come away with an expertise on any of them. 

As a self-taught programmer, I often feel as if I don't know enough and that one day I'll be confounded by some gap in my knowledge.  Looking back, I was extremely fortunate to find a job a few months after deciding to code, have had the ability to found two startups of my own, and luckily again find employment.  It's only recently that I have come to terms with the fact that I may not be particularly accomplished at algorithm analysis, but I have established a good reputation wherever I have gone and I am able to get the job done.  That being said, as any other programmer I am compelled by the complexity of the challenge. Eventually, the level of work I am doing now will become stale and there is no guarantee that at that point I will be any more prepared to face greater challenges.  Thus, I have made it my main focus to advance my programming skills and knowledge to the next level wherever that may lie.

My main goals in this undertaking will be:

Mastering Ruby

Mastering Rails

Solid understanding of Data Structures and Algorithms

Releasing a 'complex' gem

Over the past few months I have been working extensively with sports related data coming from various API's such as SeatGeek, ESPN, Bleacher Report and XmlSoccer.com.  The latter proved somewhat of a pain to interact with as it is a SOAP web service with a 5min or 60min rate limit on most calls.  However, it is by far the cheapest and most extensive Soccer API I've found. Providing both historic and current data for nearly every league, XmlSoccer.com is consistent and reliable in all its responses.

To help out other ruby coders I've developed a gem wrapper around the API that can be found @ https://github.com/kavinderd/xmlsoccer. It doesn't yet cover all the methods provided by the API, but I'll get to them when I get the chance.

Working in a startup has been a great experience. I would recommend any young programmer to at least work for a short while at a startup or start your own.  For one, the ability to feel close ownership of a product is something that slowly vanishes as a company becomes larger and larger.  The anxiety of releasing and the accomplishment afterwards is a thrill and sharing that in a small office or coffee shop with a co-founder or small team is an experience that all programmers should experience.

Obviously, there are downsides to working in a startup that are mitigated at larger perhaps slower companies.  In between prototyping and early releases there is a constant pressure to code, release, iterate.  As iterations require product pivots, code continually changes. In this fast-paced environment, I have found that code quality is often the first thing sacrificed.

Non-technical People Don't Understand Refactoring:

I think the biggest obstacle to keeping an emphasis on refactoring in a startup is due to the fact that anyone that doesn't code, doesn't quite understand the purpose of refactoring.  It can be a challenge to say to a VP or Co-founder, 'Hey, we need X amount of time to clean up the code. It may not lead to any changes in performance or features, but afterwards it will be a big boost to our efficiency'.  To the iteration obsessed individual, code quality is simply measured by whether something works or not, not how the code is structured. Secondly, its perhaps equally as difficult to explain how poor code quality isn't simply a product of poor coders or code; that instead it comes about almost organically like dust on a window sill.

Code Quality Can Always Be Pushed Back

Another reason I feel code quality is never made a priority of any kind is simply because of all technical tasks it is the easiest to put back.  A product needs IT infrastructure, features, tests, UI, but streamlining code in your controller, or refactoring a bloated method isn't necessary.  So, a few lines here become useless, a class there becomes irrelevant, and slowly a project becomes a mess, even though the outward facing product may work wonders. 

'It's the Developers Responsibility'

Obviously, code quality can easily be pointed to the author of the code. In the perfect world, a developer would code with perfect efficiency, a biz-dev person would always seal a deal, and a startup would never need to pivot.  Therefore, just as developers understand that product requirements change or that deals can fall through, non-technical members of a startup should understand that the need to refactor is just the nature of the fast-paced environment.

Conclusion

After much more research I decided to delve into Graphity.  It was written as part of a thesis in order to create a news-feed data structure that would only be governed by the size of the news feed and not the degree of the relationships or the amount of nodes. 

At first the information seems academic and complex, but the solution is really quite simple especially for a neo4j environment.  The solution that Graphity proposes is to not just create relationships from a user to each friend but to create a seperate relationship, Ego_Network, that maintains the order of the user's network in order of the last_update time for each friend.  Additionally each user would have connections to content that they are linked to, but also a seperate relationship, Activity_Feed, that would be linked in order of creation.

In implementing this data model I opted to not use the Active Model supported Neo4j Relationship type, but rather simply user Neo4j::Relationship for both the Activity_Feed and Ego_Network relationship.  Both relationship types have a reference to the original user id.  The code for generating and updating the activity feed and ego network is virtually the same and was fairly simple.  The query to retrieve either feed is also straightforward:

user.outgoing(:ego_network).depth(20).filter do |path|       path.last_relationship[:user_id] == user.id end

I chose a depth of 20 for both the Ego Network and Actvity Feed lookup as current requirments really don't need much more.  As Pickhardt mentions in his own research, the efficiency is dependent on the amount of news-feed items needed, so this should be watched closely for performance.

Coming from using the Geocoder gem in an ActiveRecord environment, I would have liked Neo4j.rb to have a similar gem or library to quickly calculate nearby objects based on coordinates.

As I worked on this issue I found three points from which to start:

1. Either customize the Geocoder gem to support Neo4j or Active Model objects,

2. Get the neo4j-spatial.rb gem working in my current project, which hasn't been updated in over a year

3. Write my own code that fits my needs.

The problem with #1 is that Geocoder's code is very specific to Active Record objects or MongoDB ones. To add Neo4j into the existing gem for all its features was too time consuming to make it an effective solution. In the future,if I have enough time I intent to create a geocode-neo4j version so others can benefit from the gem, which is already very good.

The problem with #2 is that the neo4j-spatial.rb gem, although incredibly promising, has not been updated in a long time and would have required too much time in just troubleshooting.  That being said, Neo4j-Spatial is a very powerful library that is geared towards advanced geospatial queries on the graph database.  My project only has a small location component, so it isn't really required.

So, after much investigation I decided upon #3.  I used the mathematical equation referenced by the Geocoder gem and wrote my neo4j query with that. One issue I'm seeing with this is that I still haven't been able to write a query that will only query a subset of locations in the interest of speed.

I have also decided to piggyback on Geocoder for the IP and Coordinate lookups that it does so well. This hybrid solution is working well for me because it fits all the needs I had from the outset.

There are thousands of API's available for all sorts of data.  I thought I'd share a few tips on picking an API that will work best for you.

First of all, Most API's require an account and a key, others have adopted OAUTH as their authentication, such as Yelp. I have come across some API's, such as SeatGeek.com, that don't require any user account.  Nonetheless, all API's have T&C's about using the API.  In general, these conditions usually require that the API consumer must attribute the source. 

Secondly, it's incredibly important to thoroughly read the documentation of the API and make note of what data can be returned to you.  You can never assume that just because a company has an API it will automatically provide data x,y & z. 

Currently I'm working on a project that is using neo4j as its database. I've enjoyed implementing the neo4j.rb wrapper into my Rails project.  Initially, I was a little hesitant of switching from PostgreSQL to Neo4j.rb simply because I would have to also switch to jRuby.  However, the switch has been without many complications, though the specific devise-neo4j gem is not up to date with the current rails version, but it still works for my needs. 

I've decided to start a blog on my career as a software engineer, both for my own personal edification and to help other coders out there with any problems I'm able to solve.