#datamapper

Simple, but undocumented:

User.first(:links => [User.relationships[:company].inverse])

or

Things are starting to get exciting. I feel that we're now close to the tipping point at which we know enough of the fundamentals to take almost any idea and stand a fighting chance of building it. The smart thing about the Makers "curriculum" is that it teaches you the essentials, and then provides you with the tools to figure the rest out for yourself - and I'm only now understanding why.

As a developer in the real world, there's no practical way that you can memorise or even learn everything you'll ever need; the web changes daily and the languages and frameworks that you use today may be different tomorrow. Being a successful developer means knowing where to go to find information, and knowing what to do with it once you've found it. I've now started doing this for myself and it feels like I've stepped into a world of unfathomable possibilities, where every app idea suddenly feels like something I could potentially build for real. Best not get too confident though, I've still got a hell of a long way to go.

This week was another struggle, at least for the first few days. The teaching style didn't fit my learning style, and it resulted in me feeling overwhelmed and at times, disengaged. We were following a tutorial written by MA for the most part, and unlike in the other weeks, it felt very "hand-holdy" and lengthy. The topic was databases and security, a fundamental part of almost anything we'll build and there are so many moving parts that I didn't feel like the knowledge was sinking in at all. I much prefer to be given a brief primer and then thrown directly into the deep-end, at which point I can learn to swim for myself. Given the material, I can now sort of understand why it might be taught in that fashion, but there must be a way to improve it. It sounds like our cohort isn't the first to become frustrated by it, as it's now being reworked.

As is now becoming routine, my fears of falling behind were proven to be unfounded by the weekend challenge. A surprising amount of knowledge had made its way into my brain, and the walkthrough was a great backup reference for the few times that I had difficulty researching the problems I faced. For the most part, I acted like a well-trained developer and sought out the official documentation for tools like DataMapper (an Object Relational Model), PostgreSQL (a database manager) and our old friend Sinatra. There were also a fair few gems that were used this week for various tasks; I'm starting to appreciate the extensibility that Ruby provides as a language.

The app I finished last night and can now show off is a simple Twitter clone named 'Chitter', and it's live on the web here: http://chitterapp-tomcoakes.herokuapp.com

I'm quite proud of this one, there's a fair bit going on at the back-end. I think there are mixed feelings on the course about designing on the front-end, perhaps because it's so fiddly and isn't really 'development work' per se. For me though, I actually really enjoy playing around with HTML and CSS, especially after spending a long amount of time back-end coding - it's a welcome break from having to worry about the logic. I'm not the greatest designer in the world but I've found that just trying to keep it really simple usually produces something not horrendous looking. I look forward to working more on honing this skill because I've got ambitions to become a full-stack developer.

Datamapper won't destroy a record if an association will be left orphan. However, beside from returning false, it won't tell you which association is preventing it. To find out:

前回は SQLAlchemy で ActiveRecord パターンを実装してみた。 今回は同様にデータベースに関わるアーキテクチャパターンとして DataMapper パターンを実装してみる。 まず、あらかじめシステムに SQLAlchemy をインストールしておく。

$ pip install sqlalchemy

DataMapper パターンではデータソースとドメインの分離に主眼が置かれている。 データベースのスキーマとドメインオブジェクトの間に、両者をマッピングするレイヤを入れることで抽象度を高めるパターンと理解した。 以下のサンプルコードではドメインオブジェクトの User とデータソースの UserRecord 及び AddressRecord を UserMapper で変換して扱っている。

#!/usr/bin/env python # -*- coding: utf-8 -*- import abc from contextlib import contextmanager from sqlalchemy import create_engine from sqlalchemy import Column from sqlalchemy import ForeignKey from sqlalchemy import String from sqlalchemy import Integer from sqlalchemy.ext.declarative import declarative_base from sqlalchemy.orm import scoped_session from sqlalchemy.orm import relation from sqlalchemy.orm import sessionmaker # SQLite3 のオンメモリ DB を使う engine = create_engine('sqlite:///', echo=True) # セッションはスレッドローカルにする Session = scoped_session(sessionmaker(bind=engine)) # ドメインが継承するスーパークラス Base = declarative_base() # データベーストランザクション用のコンテキストマネージャ @contextmanager def transaction(): session = Session() try: yield session session.commit() except: session.rollback() raise finally: session.close() # データソースのスーパークラス class RecordBase(object): id = Column(Integer, primary_key=True) def __repr__(self): items = [ '{key}={value}'.format(key=key, value=value) for key, value in self.__dict__.items() if not key.startswith('_') ] return '<{cls_name}: {attrs}>'.format( cls_name=self.__class__.__name__, attrs=', '.join(items), ) # ユーザの持つメールアドレス class AddressRecord(Base, RecordBase): __tablename__ = 'addresses' user_id = Column(Integer, ForeignKey('users.id')) address = Column(String, nullable=False) # ユーザ class UserRecord(Base, RecordBase): __tablename__ = 'users' name = Column(String, nullable=False) age = Column(Integer, nullable=False) addresses = relation( AddressRecord, order_by=AddressRecord.id, backref='user', cascade='all, delete-orphan', ) # データソースとドメインを変換するマッパーのスーパークラス class MapperBase(object): __metaclass__ = abc.ABCMeta @classmethod def get_session(cls): return Session() # User と UserRecord, AddressRecord を変換するマッパー class UserMapper(MapperBase): @classmethod def insert(cls, user): session = cls.get_session() addresses = [ AddressRecord(address=address) for address in user.addresses ] user_record = UserRecord( name=user.name, age=user.age, addresses=addresses, ) session.add(user_record) return user_record.id @classmethod def find_by(cls, **kwargs): session = cls.get_session() user_record = session.query(UserRecord).filter_by(**kwargs).first() user = User( id_=user_record.id, name=user_record.name, age=user_record.age, addresses=[ address.address for address in user_record.addresses ] ) return user @classmethod def delete(cls, user): session = cls.get_session() user_record = session.query(UserRecord).filter_by(id=user.id).first() session.delete(user_record) # ドメイン class User(object): def __init__(self, name, age, addresses, id_=None): self.id = id_ self.name = name self.age = age self.addresses = addresses if __name__ == '__main__': # テーブルを作る Base.metadata.create_all(engine) # トランザクション開始 with transaction(): # ユーザを保存する user = User( name='foo', age=20, addresses=[ '[email protected]', ], ) UserMapper.insert(user) # 名前からユーザを取り出す user = UserMapper.find_by(name='foo') print(user) # ユーザを削除する UserMapper.delete(user)

実行結果は以下の通り。

...(省略)... 2014-08-31 15:20:00,536 INFO sqlalchemy.engine.base.Engine BEGIN (implicit) 2014-08-31 15:20:00,537 INFO sqlalchemy.engine.base.Engine INSERT INTO users (name, age) VALUES (?, ?) 2014-08-31 15:20:00,537 INFO sqlalchemy.engine.base.Engine ('foo', 20) 2014-08-31 15:20:00,538 INFO sqlalchemy.engine.base.Engine INSERT INTO addresses (user_id, address) VALUES (?, ?) 2014-08-31 15:20:00,538 INFO sqlalchemy.engine.base.Engine (1, '[email protected]') 2014-08-31 15:20:00,540 INFO sqlalchemy.engine.base.Engine SELECT users.id AS users_id, users.name AS users_name, users.age AS users_age FROM users WHERE users.name = ? LIMIT ? OFFSET ? 2014-08-31 15:20:00,540 INFO sqlalchemy.engine.base.Engine ('foo', 1, 0) <__main__.User object at 0x10c6aff28> 2014-08-31 15:20:00,541 INFO sqlalchemy.engine.base.Engine SELECT users.id AS users_id, users.name AS users_name, users.age AS users_age FROM users WHERE users.id = ? LIMIT ? OFFSET ? 2014-08-31 15:20:00,541 INFO sqlalchemy.engine.base.Engine (1, 1, 0) 2014-08-31 15:20:00,543 INFO sqlalchemy.engine.base.Engine DELETE FROM addresses WHERE addresses.id = ? 2014-08-31 15:20:00,544 INFO sqlalchemy.engine.base.Engine (1,) 2014-08-31 15:20:00,544 INFO sqlalchemy.engine.base.Engine DELETE FROM users WHERE users.id = ? 2014-08-31 15:20:00,544 INFO sqlalchemy.engine.base.Engine (1,) 2014-08-31 15:20:00,545 INFO sqlalchemy.engine.base.Engine COMMIT

I'm relearning for my benefit; the journey calls for retracing steps. This book is good, but running into installation issues with gems. Had to use:

sudo gem install sqlite3-ruby

Object Relational Mapping (ORM for short) refers to a framework that simplifies using databases to store information in applications. 

In Ruby, it takes the structure of classes and objects and maps them into a database where the a class is a table and each object of the class corresponds to a row in that table. The column names refer to attributes of the object, which often correspond to its instance variables.

This week, we experimented with a few different ORMs like Active Record, DataMapper and Sequel (not to be confused with mySQL). I created a basic Sinatra application with a rabbits resource, and I want to run through the differences between using DataMapper and Active Record. 

The primary difference between both is that Active Record requires migrations to inform the database, whereas Datamapper does this somewhat automatically. 

I'll walk through the different steps required to set up a database using each ORM, and you can reference my complete code here.

DATAMAPPER

First, add the 'data_mapper' and 'dm_sqlite-adapter' gems.

# Gemfile gem 'data_mapper' gem 'dm-sqlite-adapter'

Then set up the database in app.rb. If there is no database set up already, it will create one called rabbits.db.

At the bottom of the file, call the auto_upgrade! method to update the database. This line is usually added at the end (after you've described your table), and it is what DataMapper uses in place of migrations. It will update the database with any changes you make in your Rabbit class.

# app.rb # at the top of the file DataMapper.setup(:default, ENV['DATABASE_URL'] || "sqlite3://#{Dir.pwd}/rabbits.db") # RESTful routes go here # at the bottom of the file DataMapper.auto_upgrade! # updates the database

Create a Rabbit class and include 'DataMapper::Resource.' Then list out the columns and data types for the table. You must specify a primary key, since DataMapper does not do this automatically like ActiveRecord does. Also, note the way validations are described and the created_at and updated_at timestamps.

# rabbit.rb class Rabbit include DataMapper::Resource property :id, Serial property :name, String, :required => true property :description, Text property :age, Integer property :colour, String property :created_at, DateTime property :updated_at, DateTime end

Finally, add your routes/handlers to app.rb. I'm only including one route here for the sake of illustration, but the entirety of my code can be view on Github.

# app.rb # ... # show rabbit get '/rabbits/:id' do @rabbit = Rabbit.get(params[:id]) haml :show end # ...

ACTIVE RECORD

The same basic process goes for Active Record, but I'll point out the differences.

Add three gems for Active Record. Sqlite3 is the database, activerecord is the interface for the database, and sinatra-activerecord is the bridge that let's us connect use it.

# Gemfile gem 'sqlite3' gem 'activerecord' gem 'sinatra-activerecord'

This sets up the database in app.rb

set :database, "sqlite3:///rabbits.db"

The Rabbit class inherits from ActiveRecord::Base and does not include migration for the table (just validations).

# rabbit.rb class Rabbit < ActiveRecord::Base end

We need to create some rake tasks so that we can communicate with the database from the command line. The 'sinatra-activerecord' gem includes some, so we just need to require that in a Rakefile.

# Rakefile require "./app" require "sinatra/activerecord/rake"

Then, write a migration to create a migration file. You can also create the file manually, but in that case will also have to append the timestamp to the filename manually as well.

# creates a migration file w/timestamp in a db/migrations folder $ rake db:create_migration NAME=create_rabbits

After you create the migration, you still need to give it instructions before the table is actually created. Def up describes the columns and datatypes to create a table and is run when migrating, while def down describes the required action to do the reverse (delete the table) and is run when rolling back.

# [timestamp]_create_rabbits.rb class CreateRabbits < ActiveRecord::Migration def up create_table :rabbits do |t| t.string :name t.text :description t.integer :age t.string :color t.timestamps end end def down drop_table :rabbits end end

Notice that Active Record automatically generates and ID for every database entry and create_at and updated_at timestamps. 

Next, run the migration to create the table.

$ rake db:migrate

Now that the table has been created, add your routes to app.rb. It's done almost exactly like with DataMapper, except for one difference. Active Record uses the method 'find', whereas DataMapper uses the method 'get' to retrieve records. Here's an example of one route:

# app.rb # .... # show rabbit get '/rabbits/:id' do @rabbit = Rabbit.find(params[:id]) haml :show end # ...

Databases are nice and all but in order to use em we have to deal with a whole different language called SQL. Now since we're lovers of technology we'll gladly become bi-lingual and work in SQL BUT wouldn't it just be peachy if whilst programming, there was a way to talk to the database in our current native tongue? In Ruby perhaps?

Well ladies and germs make way for ::trumpets:: Object Relational Mapping (or ORM for the LOAs out there! You know who you are). Object Relational Mappers are like translators for data exchange between technologies who don't naturally communicate with each other (like me and m'lady amIright? amIright? ::nervous laugh::...no I'm wrong...I'm always wrong...) There are many ORM tools available for a variety of programming languages.

An ORM's job is to be the glue between our database and our objects. So we can use Ruby syntax to "talk" to the database (Ha! Talk? Look at me getting all cozy with the DB...yea we're on two letter terms....don't be jelly...anywayz...). The idea is that you should be able to model your class as a table and the rows in the table as objects.

This post we're going to talk about three Ruby ORMs:

DataMapper: http://datamapper.org/docs/

Sequel: http://sequel.jeremyevans.net/

ActiveRecord http://api.rubyonrails.org/classes/ActiveRecord/Base.html

Then they will fight to the death!...or sing! You'll just have to find out. Leggo...

Let's first talk syntax. Look at the different ways these crazies create tables.

Each of these creates the same thing, a table named rabbits with columns id, name, description, age, color, created_at and updated_at. The first glaring difference is that big ol' freak, ActiveRecord. Sequel and DataMapper both explicitly include the columns for id, created_at and updated_at while ActiveRecord just said naw, I already know. ActiveRecord basically creates these columns automatically, pretty handy!(well with the help of that timestamps macro).

Next difference is in how you run the code in these files. Sequel and ActiveRecord both use migrations. Migrations are basically the instructions on how to build the schema of the table. So usually the first migration run is the one that creates the table as you can see with Sequel and ActiveRecord. 

The Sequel migration file title is prefixed with a number that represents the order this migration was created/should be run, starting with 01. (ex: 01_create_donuts_table.rb). To run the Sequel migration from the command line just do this:

$ sequel -m migrations sqlite://[DB_Name].db

The ActiveRecord migration file title is created with the same line of thinking as the Sequel migrations but the prefixed is a timestamp. (ex: 201311231234_create_donuts.rb). Timestamps are a hard thing to manually create so you can use the "sinatra-activerecord" gem to create a rake task that creates the file for you. Like so...

$ rake db:create_migration NAME=[FileName] #Will create migration file in the db folder prepended with a timestamp.

Then run the migration with:

$ rake db:migrate 

For both Sequel and ActiveRecord these migrations are run in order, whether by timestamp or regular old fashion numbers.

In DataMapper though, you define the database structure in the class definition. When you want to make changes to schema you can put DataMapper.auto_upgrade! at the end of the file. It's like a quickie migration that only runs when there is a change to the schema.

No need to write separate migration files then run them as with Sequel and ActiveRecord. There's also DataMapper.auto_migrate! which is a destructive operation that drops the table and creates it with the model def, use with caution.

Speaking of class definitions, let's have a looksie at the differences there.

First let's talk about the similarities. It's a small thang but notice how all the models/classes are named in singular fashion? This is because, in general, migrations assumes the pluralized name of the class is the table name and thus creates that relationship in the background. We can get behind this convention since the class is the blueprint for one object and table represents many objects through it’s rows. (class Pie => table Pies)

Now we're already familiar with DataMapper's class/DB definition combo deal, so the real difference between these guys is how they each gain access to their respective data manipulation methods (create, find, save, etc.)

Sequel and ActiveRecord both inherit from their respective parents. While DataMapper includes the DataMapper::Resource module. All these techniques do the same thing, gives your class the methods needed to reach into the database and do things.

There's also some syntactical differences between the way these methods work but for the most part they perform as expected.

There ya go folks! That's the way programmers get away with doing databasey things without writing SQL. Of course, it's still very much a requirement to understand SQL in order to even work with these ORMs. In fact, I challenge ya'll to write your own ORM based a table of your liking. You can see my Dog table ORM here, which is currently being abstracted to work as a general ORM but feel free to browse past commits...I'm not shy.

Thanks for reading! Oh and you can check out example use cases for all the ORMs I mentioned above here.

Now I invite you to sing along below, to the tune of REM's - Losing my religion 

Ohhhh crap, that figures

Figures, the table dropped,

it was not me,

the lengths I had to go thru,

to set the schema ids,

oh no, I forgot to auto

Increment

Thas me in the corner,

coding to the moon-light,

using my migrations,

thank god I didn't start on the views,

maybe it was faith I didn't do it,

cuz, I didn't set those keys

to primary,

wiped my brow n started laughing

imagine I deployed this thing?,

The thought just makes me start to cry,

Every SQL

Every timestamped file

I'm adjusting my migrations,

This time with pair programming

Like a hurt, lost, blind fool...fool

Oh no, I start to blush,

Duped columns in a rush

Consider this,

Consider this, the developer I could be

Consider this,

if mistakes

don't get to me,

Breathe

fixed migrations, typed rake db

executed without a sound

I started tux,

Yes I am so happy

Inserted the new values

So, happy that I started to cry

Turns out it was all a dream,

It was all a dream

Take it as lesson, 

Never code by moon-light

and double check my migrations

Localhost all my views,

Then I hope this post influence,

Folks to be fluent

in other O-R-Ms

You've got DataMapper,

Sequel and ActiveRecord,

I think you should give them all a try,

Only dropped in the dream,

relax, sigh, nom french fries,

Only dropped in the dream,

in the dream 

just a dream, dream

Note: If you drop, migrations will only bring back the structure of your table, your data is as gone as yesterday...sowwie =(

Remembered Unique Values

If you run into TooManyTriesException whilst using the Sweatshop's unique method you might need to clean up after yourself. The Sweatshop keeps track of all records it spawns over time and the values it presents as unique values in an effort the maintain a blacklist of already used values.

I usually purge and repopulate tables prior to unleashing my assertions unto the code that query them. The unique_map, however; remembers all values for unique fields until the map is explicitly cleared or until the ruby instance is terminated. Even destroying your records won't clear entries from the unique_map

In order to get rid of the TooManyTriesExceptions I clear the unique_map in the helper I use to clean up after myself (most likely the afterhelper).

DataMapper::Sweatshop::UniqueWorker.unique_map.clear

Picking and the Record Map

It might occur to you that in some cases the pick method returns valid content while getting the same id from the database returns nil.

> p = Frog.pick # should return something to variable p #<Frog @id=2 @name="Kermit"> Frog.get(p.id) nil

The record map seems to be used by Sweatshop as a store of all items it has generated for pick-ing among a few other things I haven't looked into. In my tests I prefer the succinct pick notation instead of all.sample. I also have this habit of destroying the old and spawning some new records for every test just to start of with a clean slate with known properties required for that specific test. It is important, however; to be aware of the fact that the record_map does not keep track of the the table entries removed during its lifetime. Given this behavior you will need to clear the record_map after destroying some records unless you occasionally need to pick nonexistent entities.

Just like I clean up the unique_map, I clean up up my record_map in the test helper which does all the clean-up housekeeping.