#method_missing

SQL Awesomeness

Today we learned more about SQL. After clearing up some questions over the assessment and yesterday's more challenging exercises, Max split SQL into the DML and the DDL.

DDL - Data Definition Language

The Data Definition Language (DDL) is all about establishing a database's schema.

A database's schema is its blueprint. It includes such information as which tables are in the database, the names and types of each table's columns, and constraints placed on data.

Some common SQL statements that belong to the DDL are:

CREATE - to create objects in the database

ALTER - alters the structure of the database

DROP - delete objects from the database

TRUNCATE - remove all records from a table, including all spaces allocated for the records are removed

COMMENT - add comments to the data dictionary

RENAME - rename an object

Source- StackOverflow: What is DDL and DML

Yesterday we ran a shell script that setup a database and populated several tables for us, so we didn't really have any exposure to the DDL yesterday.

DML - Data Manipulation Language

The Data Manipulation Language (DML) is all about managing data within schema objects.

Some common SQL statements that belong to the DML are:

SELECT - retrieve data from the a database

INSERT - insert data into a table UPDATE - updates existing data within a table

DELETE - deletes all records from a table, the space for the records remain

MERGE - UPSERT operation (insert or update)

CALL - call a PL/SQL or Java subprogram

EXPLAIN PLAN - explain access path to data

LOCK TABLE - control concurrency

Source- StackOverflow: What is DDL and DML

All of the different SQL queries we ran yesterday were DML statements.

SQLite3

Max walked us through setting up an SQLite3 database.

SQLite is a database system that is great for situations when:

The database and application are on the same machine.

We don't need to have more than one connection to the database at any given time.

The data set is relatively small.

In a small demo, Max wrote a shell script to CREATE a database table of people in the Game of Thrones Series.

Once the table had been set up with the appropriate column names, data types, and constraints, he added some lines to the script to seed the table by using INSERT to add some of the characters.

It was nice that there is so much overlap in syntax between PostgreSQL and SQLite3!

Ruby's SQLite3 Gem

There's a gem for ruby called (fittingly enough) sqlite3. Max demonstrated how to use it to build an application to interface with our database. This sort of application employs object relational mapping (ORM).

Establishing the Connection

The code below is all we had to do to connect to our SQLite3 database.

require 'sqlite3' require 'singleton' class PeopleDatabase < SQLite3::Database include Singleton def initialize super('game_of_thrones.db') self.results_as_hash = true # Query result returns an array of rows # {'column_name' => value } instead of # ['column_name', 'value'] self.type_translation = true # Change results to correct ruby type end end

Including the Singleton module implements the singleton pattern, which prevents instantiation of more than one object of the class. This is useful to prevent concurrent connections to the database.

Including the module makes PeopleDatabase::new a private method, so now we must use PeopleDatabase#instance in order to access the instance methods inherited from the SQLite3::Database class.

Executing statements

With the database singleton ready to go, we executed SQL statements using the following syntax:

PeopleDatabase.instance.execute(<<-SQL, 'Arya', 16, 'Stark') INSERT INTO people ( name, age, house ) VALUES ( ?, ?, ? ) SQL

It turns out the Database#execute instance method is a lot more powerful than we knew today, so I'm looking forward to refactoring our code with a block to process results and hashes to make order-independent bind variables and placeholders.

Building the ORM in Pairs

Armed with the tools from lecture, my partner and I set out to build a simple ORM to manage three classes of object: users, questions, and replies. Additionally, users can like and follow questions.

Today's partner was stupidly smart (again), and we raced through the day's work together.

Setup

We wrote a simple shell script to set up the database's schema (DDL) and seeded it with some simple data (DML).

Models

From there we built classes to interface with each table and methods that handled DML tasks, like looking up the top n most-liked questions, and a user's "average karma" (average number of likes for a User's questions).

Bonus: Extract Superclass

Once we had the skeleton of our ORM working, we did some refactoring based on a great code review from Winnie. The bonus had us refactor our classes using the Extract Superclass refactoring pattern.

Each of our models had similar #find_by_id(id) instance methods, so we made a ModelBase class and moved the generalized behavior there.

Return of the ::method_missing

Finally we ended the day by building a method_missing delegator that intercepted any call to an unknown method whose name began with find_by....

We had a lot of fun getting this to work, and I'm looking forward to revisiting it armed with the discovery of Database#execute additional behavior.

Welcome dear reader!  Today we tackle metaprogramming in Ruby, what it is, why it's useful in your everyday programming life, and what the basic concepts, techniques and resources available to you are.  

In brief, metaprogramming is writing code that will generate additional code as your program runs based on what external data is provided.  Ruby provides various tools allowing you to examine and alter existing modules, classes and methods or to add brand new functionality programmatically, rather than by hand.  

So, why should you learn this stuff?  Well one of the primary uses of metaprogramming is to DRY out your code, as it is excellent for eliminating the repetition of highly similar, but not identical, functionality from your codebase.  This process typically involves some logic that is repeated throughout your program with slight variations each time it is called.  Relying on user input, a database schema (think of how ActiveRecord defines methods for you that provide access a parent model's child objets, or that allow you to search by specific properties of a model), or other external data source, you can automatically have your code define unique variations of these chunks of logic during runtime.

DRY'ing out your code is well and good, and is an important skill for any developer.  But a more compelling reason for familiarizing yourself with metaprogramming is to increase your Ruby literacy.  I believe this is one of the most effective ways of improving your understanding of other peoples' Ruby code, be it Rails or one of any number of popular Gems.  When debugging these third-party libraries you'll find that metaprogramming is commonplace, and therefore understanding it is a major part of understanding what is actually happening in the code.

Below are some examples of essential metaprogramming techniques, with images courtesy of the wonderful RubyMonk.com.  Ruby Monk has an excellent guide on this topic which I can't recommend highly enough.  Code School's "Ruby Bits 2" course also has great material on the subject that I found extremely helpful.

1) Code inspection, send and method_missing:

Oftentimes, in order to be able to generate code on-the-fly you'll need to perform some checks on the existing code you're dealing with.  For example, what methods does a certain object respond to, what self currently is (the scope or context), or what variables are currently defined within a given scope.  This is code inspection.  

These tools are very handy, and while they don't dynamically generate any code for you, they are invaluable in situations where you want to do so, because they allow your program to examine any piece of code, without knowing its contents ahead of time, and determine what course of action to take when adding or modifying functionality on-the-fly.

Fairly self-explanatory are Object#methods and Object#public_methods, which return arrays of all an object's methods or public methods, including those it inherits from parent classes.  If you pass false to public_methods it will return the object's public methods, but none of its ancestors'.  You can also call Object.method(:some_method_name_goes_here) to return an instance of class Method, which allows you to examine the properties of the method, for example by calling Method#parameters.

If you've used the Gem pry then you've already seen another very useful code inspection technique:  Binding objects.  Kernel#binding (you can simply call binding nakedly, without the reference to Kernel), creates an instance of the Binding class which captures, among other things, the scope and state of the program on whatever line you place it.  You can define getter methods around bindings and place them at various points in your code to reference multiple contexts in this way.

Ruby also gives us some quick ways to access basic info about the program itself.  The __FILE__ method returns the current file name, __LINE__ returns the current line of the program, and the global variable $0 returns the name of the program that was initially executed to arrive at that point in the code.

Get giddy, beloved reader, because you're about to add two of the most useful methods in Ruby to your toolkit: send and method_missing.  Sticking with the theme of being able to deal with code without knowing its exact nature in advance, send is your friend, it allows you to call any method, passed in as a string or symbol, along with any arguments or block it may require.  

This functionality is great for delegating methods to other objects when a particular object doesn't know how to respond or if you simply want to extend another class's functionality.  But how does this work exactly?  As you may know, when you call a method in Ruby a search is made through the method lookup chain (check out the image below) for that method.  First the class of the receiving object is searched, followed by any modules included in that class.  If the method isn't found the search is repeated in that class's parent class and its modules, with this process repeating all the way up Ruby's class hierarchy.     

The Ruby method look up chain. From the allenlsy blog <3

However, Ruby also checks within each class to see if method_missing is defined.  If it is, the logic in that method will be executed in lieu of the original method that was called.  This is amazingly helpful as you can define custom behavior for when an unknown method is called on one of your objects.  Inside of a particular class's method_missing you'll often see calls to other classes via send, effectively saying "if my class doesn't know how to deal with this particular method, send it to another class I know can handle it".  This might seem to be a strange use case, but imagine it in the context of metaprogramming.  You will have methods generated dynamically based on specific conditions in your codebase, and it will not always be clear exactly what methods are defined for a given class.  

2) eval is evil

To get the value of some arbitrary piece of code you can pass the eval method a string and a Binding object to specify the scope (eg, is this a variable from the global scope, or the one with the same name that you've defined within a method?), and it will fire up the interpreter and parse that string as code.  This means that, as with some of the code inspection techniques mentioned above, you don't need to know ahead of time what code will be passed in in order to evaluate it... pretty sweet!  

But when you encounter eval in your meta-studies more than likely the only thing you'll hear is how you should never use it and how eval is evil.  So why all the hate?  Take heed!  eval is computationally very expensive because you need to fire up the interpreter, and it's also highly insecure, as you're allowing someone to pass in potentially malicious code to be blindly executed.  

So, what's a dev to do in this situation, where you need to examine a dynamically generated bit of code, but don't want to expose yourself to the pitfalls of eval?  Best practice here is to pass a string to const_get (called on a class or module) or instance_variable_get (called on a specific instance) to return the actual Ruby object of whatever class, module or instance variable you're looking for.

3) Lifecycle callbacks, instance_eval and class_eval: 

Check out the code example above from Ruby Monk's Metaprogramming book, it uses a pattern commonly found in Gems and one that we'll break down for you here.  

Rearing its head again is the concept of context:  what is the scope for a given object?  The instance_eval, class_eval, and module_eval methods give you safer (although still expensive) alternatives to eval, and are great for modifying existing objects.  Where with eval you pass in the context you want to evaluate via a binding, these methods set their context to the object they are called on, and then define some functionality for that context via a passed in string or block of code.  

instance_eval defines methods particular to a specific instance, so class methods when called on a class, and singleton methods when called on instances of a class.  class_eval can only be called on a class or module, and defines instance methods on that class or module.  module_eval is simply an alias for class_eval, and obviously is more readable/semantic for defining instance methods within modules.

You can see that in this example, we're extending the ClassMethods module and including the InstanceMethods module into an object, using instance_eval and class_eval to add those modules within the correct context.  instance_eval, you'll recall defines methods exclusive to a single instance, so that means class methods when dealing with a class object, and conversely, class_eval defines instance methods.

But the real clever part to this code is the use of included.  included is an object life-cycle callback, or life-cycle hook.  These are methods that Ruby will call automatically after certain events occur in your code.  There are callbacks for when a module is included or extended, when a class is inherited from, for when a method is added or removed, and even for less common situations such as when a method that was defined becomes undefined.  So, when you define include as was done in the example, you're saying run this code whenever the Gym module is included in an object, then take that object as a parameter and perform some operations on it.

Putting all of that together, the above code says that when you include the Gym module in a given object, take that object (a class presumably) and  extend the ClassMethods module as a set of class methods (so, in the context of a specific instance of a class), and include the InstanceMethods module as a set of instance method (so, in the context of that instance's class).  This provides a succinct way for third-party library authors to add both class methods and instance methods into a class with only one call to include.

4)  define_method and instance_variable_set:

Finally, we have this scrumtrulescent beauty of a code snippet.  Here, define_method is called, passing in the name of the method to be defined:  initialize.  In this case the method name is a hard coded value, but as mentioned above, names are often passed in from some dynamically generated data source.  You can pass define_method a block, where you have access to any arguments that will be passed to the method and can define the method body.

In this Ruby Monk example we're creating a module that can be included in any class to provide a hash initializer (a common design pattern allowing you to supply arguments to initialize in any number and order).  This allows us to write our initializer in only one place, as opposed to in each class, DRY'ing out our code.  Let's see how they go about doing it.

First, the hash_initialize method takes a splat argument, allowing you to pass in as many arguments as needed.  That array is then compared to the array keys for the hash that will be passed in to the dynamically defined initialize method (that hash is accessed as the variable h within the block passed to define_method).  This is done to prevent initializing instance variables that have not been specified in the call to hash_initialize.  Now if any arguments are not passed to hash_initialize, but are passed to initialize, the missing array will contain those extra arguments and an error will be thrown.

If no extra arguments are passed to initialize then each of the key-value pairs of the hash passed in to initialize will be made into instance variables for the object being initialized.  This is done via instance_variable_set, which as the name implies, accepts a variable name and value as arguments.  Note the use of string interpolation here to add the "@" to our instance variable name.

The end result is very terse, but allows every class to have a hash initializer, with control over what instance variables are allowed to be defined, in only two lines of code:  including the module, and calling hash_initialize.

WHEW.  Did you make it all the way through?  Kudos to you, intrepid reader, if you did.  Hopefully this has been a good high-level intro into metaprogramming, and one that has you all set to dive deeper into this powerful part of the Ruby language.

In Scala 2.10 class Dynamic would be enabled by default, so I think that it's the right time to look at it in more detail. This class can be used for many different purposes. I think, that an integration with dynamic languages is the most obvious one, but I would like to show you something different. I will demonstrate you how to create parameterizable extractors, that can be parametrized directly within case expressions.

Class Dynamic works very similar to Ruby's method_missing or Groovy's methodMissing/propertyMissing. So you can call non-existing methods on classes that extend Dynamic. Such calls would be rewritten by compiler to following method invocations: applyDynamic, applyDynamicNamed, selectDynamic, updateDynamic. I will describe each of these methods in the next sections.

method_missing is still considered scary, but here's something more scary: forgetting to override respond_to?.

Whoa, right? Just gave you an anxiety attack there.

How about #method, though? Does that still work?

Go ahead, try it. Here, I'll try it with you:

require 'ostruct' class Order def user @_user ||= OpenStruct.new(name: 'Mike', age: 28, occupation: 'slacker') end def method_missing(method_name, *arguments, &block) if method_name.to_s =~ /user_(.*)/ user.send($1, *arguments, &block) else super end end def respond_to?(method_name, include_private = false) method_name.to_s.start_with?('user_') || super end end

OK now I'll load that into irb and play with it:

ruby-1.9.2-p290> order = Order.new => #<Order:0x00000001e40948> ruby-1.9.2-p290> order.user_name => "Mike" ruby-1.9.2-p290> order.respond_to?(:user_name) => true ruby-1.9.2-p290> order.method(:user_name) NameError: undefined method `user_name' for class `Order' from (irb):23:in `method' from (irb):23 from /home/mike/.rvm/rubies/ruby-1.9.2-p290/bin/irb:16:in `<main>'

Denied!

There is a solution! How would you like to solve your #method problem while also using a method with a better name? For the low price of Ruby 1.9 you, too, can have this:

require 'ostruct' class Order def user @_user ||= OpenStruct.new(name: 'Mike', age: 28, occupation: 'slacker') end def method_missing(method_name, *arguments, &block) if method_name.to_s =~ /user_(.*)/ user.send($1, *arguments, &block) else super end end def respond_to_missing?(method_name, include_private = false) method_name.to_s.start_with?('user_') || super end end

But you don't have to take my word for it:

ruby-1.9.2-p290> order = Order.new => #<Order:0x00000001c8d678> ruby-1.9.2-p290> order.user_name => "Mike" ruby-1.9.2-p290> order.respond_to?(:user_name) => true ruby-1.9.2-p290> order.method(:user_name) => #<Method: Order#user_name>

Today I was writing a library to access some API, in the design process I started wondering if in Ruby is possible to use some sort of namespacing on class.

All that I wanted was concatenating methods, like you do on active record objects. I show you the solution that I found:

module Test module Foo def bar @namespace = "bar" self end def bar_quack puts "Quack!" end end end module Test class Base def initialize puts "Init!" end def method_missing(method, *args) if @namespace self.send("#{@namespace}_#{method}", *args) else super end end include Test::Foo end end

In this way you can do this:

test = Test::Base.new #=> Init! test.bar #=> test.bar_quack #=> Quack! test.bar.quack #=> Quack!

The code works by implementing a method missing in the Base class, and a namespace method in the module that you want namespaced. All the method in the module must have the format: namespace_method. Now when you call the namespace method an instance variable is set with the namespace, and self is returned. Because the method cannot be found method missing is triggered, and is the instance variable is set we send the method with the namestace.

Recently, decorators have become a big part of my Ruby on Rails life.

We used them heavily in a recent client project, Harold Giménez wrote a great post about them, Avdi Grimm is writing about them in Objects on Rails, and Jeff Casimir has a great presentation about them.

Until recently, I still had some questions, however, such as:

Should I roll my own decorators?

If I roll my own, what are the tradeoffs of different implementations?

Do I care about the "transparent interface" requirements of the Gang of Four's decorator definition?

Is it good or bad that the decorated object's class is the decorator instead of the component?

I'd like to try to answer those questions here.

Intent

A decorator is a design pattern. Its intent, as described in Design Patterns by the Gang of Four is:

> Attach additional responsibilities to an object dynamically. Decorators provide a flexible alternative to subclassing for extending functionality.

Use it instead of inheritance

An example I've seen a couple of times for decorators is the "coffee with milk and sugar" example. One that I found particularly helpful was Luke Redpath's article.

Here's a Ruby implementation of that example using class inheritance (subclassing):

class CoffeeWithSugar < Coffee def cost super + 0.2 end end class CoffeeWithMilkAndSugar < Coffee def cost super + 0.4 + 0.2 end end

The problems with inheritance include:

Choices are made statically.

Clients can't control how and when to decorate a component.

Tight coupling.

Changing the internals of the superclass means all subclasses must change.

In Ruby, including a module is also inheritance:

module Milk def cost_of_milk 0.4 if milk? end end class Coffee include Milk include Sugar def cost 2 + cost_of_milk + cost_of_sugar end end

How a decorator works

Using Gang of Four terms, a decorator is an object that encloses a component object. It also:

conforms to interface of component so its presence is transparent to clients.

forwards (delegates) requests to the component.

performs additional actions before or after forwarding.

This approach is more flexible than inheritance because you can mix and match responsibilities in more combinations and because the transparency lets you nest decorators recursively, it allows for an unlimited number of responsibilities.

Alternative implementations in Ruby

I've researched and found four common implementations of decorators in Ruby:

Module + Extend + Super decorator

Plain Old Ruby Object decorator

Class + Method Missing decorator

SimpleDelegator + Super + Getobj decorator

There's probably others but these seem to be the most common.

Module + Extend + Super decorator

This implementation is described, and I think advocated for, in the Design Patterns in Ruby book. It consists of:

module

super

extend

Staying with the "coffee with milk and sugar" example for consistency, it is implemented like this:

class Coffee def cost 2 end end module Milk def cost super + 0.4 end end module Sugar def cost super + 0.2 end end coffee = Coffee.new coffee.extend(Milk) coffee.extend(Sugar) coffee.cost # 2.6

The benefits of this implementation are:

it delegates through all decorators

it has all of the original interface because it is the original object

The drawbacks of this implementation are:

can not use the same decorator more than once on the same object

difficult to tell which decorator added the functionality

The "Plain Old Ruby Object" (PORO) decorator

My recommendation is to start with this style of decorator in your Ruby programs, including Rails apps, and then move to something else only when this style fails you.

class Coffee def cost 2 end def origin "Colombia" end end class Milk def initialize(component) @component = component end def cost @component.cost + 0.4 end end coffee = Coffee.new Sugar.new(Milk.new(coffee)).cost # 2.6 Sugar.new(Sugar.new(coffee)).cost # 2.4 Sugar.new(Milk.new(coffee)).class # Sugar Milk.new(coffee).origin # NoMethodError

The benefits of this implementation are:

can be wrapped infinitely using Ruby instantiation

delegates through all decorators

can use same decorator more than once on component

The drawbacks of this implementation are:

cannot transparently use component's original interface

This drawback also means that this decorator isn't really a decorator under the Gang of Four definition. I maintain that we should still call it a decorator, however, because it otherwise looks and acts overwhelmingly like a decorator.

This is a sticky opinion, however, so let's consider the "transparent interface" requirement from Gang of Four in more detail.

Do we care about the "transparent interface" requirement?

Let's say the interface we care about decorating is cost. If so, we don't need to also support origin method. Then, the PORO decorator meets our practical needs.

By redefining the scope of "interface" to be the subset of the object's entire interface that we care about, we meet the Gang of Four definition. Is that cheating?

I say no. Consider how many methods are on Object in Ruby 1.9.3:

> Object.new.methods.size => 56

An Object in Rails has more than double the number of methods:

> Object.new.methods.size => 118

An ActiveRecord Object has even more:

> User.new.methods.size => 366

We're not using hundreds of methods on each object, especially in typical use from a Rails view.

However, if we used the PORO decorator in a Rails app to decorate an ActiveRecord object, we're probably reducing the interface by about 300 methods.

Depending on how we use the object in the app, that may or may not be a problem.

In practice, when TDD'ing a new feature, I have not found this to be an actual problem. That's why I say start with this decorator.

If it's not a problem, great. Your test suite should tell you. You might decide to add one or two more methods that do very clear delegation:

def comments @component.comments end def any? @component.any? end

However, you might feel like this is tedious or repetitive. So, let's say we do care about the "transparent interface" requirement later on.

That's usually accomplished with method_missing or something from Ruby's delegate library like Delegator, SimpleDelegator, DelegateClass, or Forwardable.

"Method Missing" decorator

Here's a method_missing implementation of a Ruby decorator:

module Decorator def initialize(component) @component = component end def method_missing(meth, *args) if @component.respond_to?(meth) @component.send(meth, *args) else super end end def respond_to?(meth) @component.respond_to?(meth) end end class Coffee def cost 2 end def origin "Colombia" end end class Milk include Decorator def cost @component.cost + 0.4 end end coffee = Coffee.new Sugar.new(Milk.new(coffee)).cost # 2.6 Sugar.new(Sugar.new(coffee)).cost # 2.4 Sugar.new(Milk.new(coffee)).origin # Colombia Sugar.new(Milk.new(coffee)).class # Sugar

The benefits of this implementation are:

can be wrapped infinitely using Ruby instantiation

delegates through all decorators

can use the same decorator more than once on the same component

transparently uses component's original interface

The drawbacks of this implementation are:

uses method_missing

the class of the decorated object is the decorator

Do we care that the class is the decorator?

We should not. This is Ruby, land of duck typing.

However, Rails inflects (object.class.name) for polymorphic relationships, form_for, and other places. When I tried to convert a few older Rails view helpers to decorators, this was an actual problem in the form of ActiveRecord errors during test runs.

In some cases, this revealed a deeper problems. By re-working the model code, I was able to use PORO decorators and the overall codebase was cleaner. Other times, I just thought it was less time-consuming to make the class appear to be the component's class and move on.

"SimpleDelegator + Super + Getobj" decorator

So, a Rails compromise is this decorator implementation. I'm trying to use it only as a last resort.

class Coffee def cost 2 end def origin "Colombia" end end require 'delegate' class Decorator < SimpleDelegator def class __getobj__.class end end class Milk < Decorator def cost super + 0.4 end end coffee = Coffee.new Sugar.new(Milk.new(coffee)).cost # 2.6 Sugar.new(Sugar.new(coffee)).cost # 2.4 Milk.new(coffee).origin # Colombia Sugar.new(Milk.new(coffee)).class # Coffee

The benefits of this implementation are:

can be wrapped infinitely using Ruby instantiation

delegates through all decorators

can use same decorator more than once on component

transparently uses component's original interface

class is the component

The drawbacks of this implementation are:

it redefines class

I honestly don't know exactly what sort of problems can result from using method_missing or redefining class but I imagine they'll be manifested in the form of a time-consuming debugging session.

Plan of action

I think I now understand the tradeoffs of different Ruby implementations and have a plan for using them:

PORO to start

SimpleDelegator if I need the "transparent interface" requirement or the decorated object's class is causing problems for Rails

I still have other questions like:

What's the difference between a decorator and a presenter?

What's the difference between a decorator and a strategy? And a composite?

Should I use a gem like Draper?

Is there any reason to use classic Rails view helpers anymore?

Should I be using a different templating system like Mustache?

I think I have answers to some, but that's another post...