Small Batch Code

Just because two functions/methods look alike doesn’t mean they are.

Hard rule of 3: perceived duplication needs to exist 3 times to be considered duplication. If there are only two occurrences, then drying them out can lead to the wrong level of abstraction.  This can make the code 1. cost more 2. more difficult to navigate and 3. too many more to list here.  I’ll have to create a post on that.  

Are they returning different data types? If so, it might not be duplication.

This week, I was tasked with creating a dashboard for IZ.  It meant that I got to dig around the main application, which I haven’t done much of yet, as I’ve been working primarily in separate services.

One of the reasons I look forward to going to work every day is that if someone has a cool idea that will be useful to the business I get to build it.  Yesterday, IZ had the idea of having a dashboard for a specific team to use.  He and JR were super enthusiastic about it, and threw around a lot of great ideas.  Their positive energy and enthusiasm for the project made me want to build it even more.

I had to figure out how we do Rails-magicie-things the AV way.  That wasn’t too bad.  The part that stung was using HAML.  It actually wasn’t that bad either, I’m just used to using erb and presenters for the view.

Front end lesson: if you render a partial in a Rails controller, the page/endpoint will not have the css.  But if you get rid of that little underscore, and render the path of the file you need, it will magically work.

iNotes from braveclojure.com/writing-macros and briangilbert.com/blog/2013/07/30/anatomy-of-a-clojure-macro/

1.  What is a Macro?

Macros take unevaluated data structures as args

Macros return some evaluated data structure

Can be used to write more readable code

the macro -> allows you to eliminate parenthesis

Macros allow make the language more extendable

2.  What are Quote, Syntax Quote and Unquote ?

Quote is like 'literally' the thing we are talking about.  It can be written (quote x), or shorthand 'x.

For example, 'x will return a x.

Syntax Quote will always evaluate to the namespace.

So, `x evaluates to   user/x.  `+ returns clojure.core/+ .

Unquote, a tilde,  will reverse the work of quote.  This is helpful when you have an expression, but want to quote most of it, but not all of it.  You could add a quote or syntax quote to the beginning of a statement that needs to be evaluated, and a simple tilde in front of something that does not need the quote.

3. What Does macroexpand do?

1. Finish up the rest of story 1165 for  exercism.io. Contributing to open source projects is important to me; its something I’d like to continue doing even thought there is no more Waza.

2. Continue to read/blog about The Data Warehouse Toolkit.  I should aim for publishing 1 blog a week.

3. Get more comfortable querying databases.

4. Continue to build my knowledge of Service Oriented Architecture.  The bigger goal is to lead a team; this will be a valuable skill.

5. Continue to grow clean code skills.  This means brushing up on design patterns, best practices, TDD.

6. Get to know metaprogramming.

1. What is a Fact Table? A fact table contains business facts & foreign keys which refer to candidate keys in the dimension table.

2. What is a Dimension Table? Think of them as the 5 points in a star schema or the boundaries/entry points to the fact table.  A dimension table is one of the set of companion tables to a fact table.  They may represent the hierarchical relationships in the business.  Each dimension is defined with a single primary key.

3. What is an attribute? The “BY” words.  So if business needs to know scores by student by grade, that means we have 2 attributes: student and grade.  Attributes are the primary source of query constraints, groupings and repost labels. 

4. Where lies the power of the DW? The power of a DW lies in the quality and depth of the dimension tables.

5. What makes the DW better in terms of this specific quality? The more time spent populating values, ensuring the quality of the values, & providing attributes with verbose business technology the better the DW is.

These are my notes taken from The Data Warehouse Toolkit: Dimensional Modeling by Ralph Kimball and Margy Ross.  Since I don’t know much about data tables, I’ll be pulling info from Quora, Wikipedia, and hopefully the brain of my department head.

1. What is a data warehouse?

The central repositories for integrated data used for reporting data and analysis. 

2. What are the 4 components of the DW?

a. Operational Source Systems: Located outside of the data warehouse.  We have no control over the content/format of the data.  Typically part of legacy systems.  Its main priorities are processing performance & availability. Queries against it are typically 1 record at a time.  Separation of Concerns: Not queried in the broad/interesting ways that data warehouses are queried.  Doesn’t keep much historical data, the DW does.  

b. Data Staging Area: Composed of 2 things; the staging area and the ETL.  The staging area is everything in between the oss and the data presentation layer.  Raw data is extracted from the oss and passed off to the data staging area.  the DSA is like a kitchen; it cooks the data and passes it off to be loaded into the data presentation area. ETL stands for extract, transform and load.  Extracting means that we read and understand the source data & copy it into the staging area.  Extracting sits between the OSS and the DSA.  Transformation happens IN the staging area.  Transforming data means  cleaning it, combining it, assigning it warehouse keys and more.  Loading means presenting the quality-assured dimensional tables to the bulk loading facilities of each data mart.  The data mart indexes the new data for query performance. Data is published after each data mart has been loaded, indexed, supplied, etc.  

c. Data Presentation: Where data is organized and stored.  Data is loaded from the DSA to the presentation area.  Data access tools access data from the presentation layer.  Its composed of many data marts.  A single data mart presents the data from a single business process.  3 Best practices: 1.  data is  presented, stored and accessed in dimensional schemas.  Dimensional modeling makes databases simple and understandable. 2. Data in this layer should be atomic.  3. Should adhere to the data warehouse bus architecture.

I started reading the Data Warehouse Toolkit to prepare for my new gig at AC.  I had a little experience working with database things while at 8th Light, but not much.  Which is great; I love learning something that I know little about.

So this is how I’m going to do it:

1. Read for 45 minutes at a time, with 15 minute breaks in between.  I’ll be taking notes in my notebook as I read.

2. Make note cards of important concepts.

3. Schedule naps.  Go over my notes before them.  

4. Go over notes before bed.

5. Wait at least 1 day, but no more than 3, before blogging about what I’ve learned. 

6.  Talk to JP about setting aside 10 minutes a week to talk specifically about the things that I’ve learned.

I’m going to be working with Data.  So I thought I’d take some time to understand it!  These are just my notes.

1. What is ETL?

Extract, Transform, Load.  Its the process of extracting data from multiple data services, cleaning it, and loading it into some warehouse.

2. What is Business Intelligence?

The set of techniques/tools used to transform raw data into useful information used by the business.

3. What is information? How is it kept?

Information is an asset.  Its kept in 2 forms: operational system of records (data in, usually 1 record at a time) and the data warehouse (data out).  

4. What problems should we be helping business to solve?

A lot of data needs to be analyzed and presented to them in a consistent way that shows business what data is the most important to them.   Business needs the data to make fact-based decisions.  The data warehouse needs to easily adapt to change.

5. What is slicing and dicing?

Tonight, I’m going to try something new.  I’m going to take the idea of Crossfit, and apply it to Katas.  I’m going to try it with Prime Factors and *hopefully* Roman Numerals.

The Rules:

1. For 55 minutes I’m going to try to get through as many katas as I can.

 2. I’ll time it at 11 minutes per kata.  This means I’ll have 5 rounds.

3. I can only move on once I’ve managed to finish a kata in 11 minutes or less.

1: Prime Factors:

1st Round: After 11 minutes I got only the first 3 tests passing.  Ouch.

2nd Round: Flew through first 4.  Got stuck on number 9, the last test.

3rd Round:  Got the solution just in time... and then my computer froze.  Double checked the solution with the kata I was following, and my computer still freezes.  Not sure about this one; will have to find a different solution later.

2. Roman numerals:

4th Round Round: WAT

5th Round: Took a break to study Nhu’s implementation before starting this round.  Finished a few minutes after the time was up, but I finished!

Tips: 

1. Know when the big changes in logic happen before approaching the problem.  For example, The Roman Numeral Kata can be solved, up to 5, by using if statements.  However, after that its best to define the pairs [num roman-num]: and recur through the rest of the problem. 

2. Test drive it the first time.  Comment out the tests after that.  This will cut back on having to remember the details of the test implementation.  No need to get stuck if you mistakenly assert that four is “VI”, or something small like that.

3. Keep the clock where you can see it.  Its motivating!!!

The katas I’d like to practice tomorrow: 

 Fibonacci

According to http://clojure.org/cheatsheet ,  mapcat falls under the list of functions meant to manipulate sequences.  Specifically, mapcat is used to get longer seqs.

mapcat takes a function and a collection.  Then it returns a map version of the concat of the function and collection.

From the REPL:

I’m practicing katas in preparation for an interview.  Right now I’m practicing the coin changer Kata, which happens to be more readable in Ruby and Java (I know thats subjective, but...).  Like most katas I’ve practiced so far, the coin changer solution comes quicker in Clojure than in other languages I’ve practiced them in.

I’m following Micah’s solution.  You can find it here:

https://vimeo.com/53519408

I tried coming up with a slightly different solution, using conj or concat.  But I realized that I didn’t really know how they were behaving, and which was better for the thing I wanted to achieve.  So here I am, figuring that out.

What are the differences between concat and conj?

1. Take a look at http://clojure.org/cheatsheet.  Conj and concat are both listed under seq.  Bur conj is also listed under lists, sets, vectors and colls.

conj     = generic operation for a coll

concat = used to get longer sequences.  

2. Understand the definitions.

conj = Returns a new coll with something added either to the beginning or the end of it.  Where is is added depends on what type of data structure the coll is.

concat = Returns a lazy seq.  This lazy seq is just the squishing together of 2 data structures.

3. Pop open the REPL & see how they behave.

https://gist.github.com/LNA/13178c620bde35721afb

This made me realize something about Clojure:

When deciding which core function to use, the data you’re putting in is just as important at the the data you’d like to get out.

I'm in the process of applying for 2 positions right now.  I thought I'd have more time to do things that I enjoy, catch up with people I haven't seen in a while, contribute to open source projects.  But getting ready for interviews and completing code challenges has taken up all of my time... and I love it.

So the intermission continues.  

Things will be put on hold until I've settled into a position.  I spend every day working on the challenges I've been given, making sure I ask myself if my work would be up to my mentors standards.  Or how craftsmen at 8th Light might nitpick, or notice things that I typically don't.  Gotta love imaginary feedback. 

I'm also taking time to review my old blog posts.  I'm glad MEbert had me write to many, because the notes I've collected over the past year are valuable.  

I'm re-watching some of Robert C. Martins's Clean Coder videos, and listening to old Ruby Rogues podcasts.  

My big goal for March is to obtain a position as a developer that will allow me to learn daily.  I've only looked at companies where TDD, agile, clean code, and other skills I've been taught will be valuable.  I want to continue to build on the things that I've learned since I started working with my mentor, and those companies are the places that I will be able to do so.

I've been working on my web server.  I have a method that looks like a simple conditional, filled with many else if statements that needs to be refactored.  Its pretty big, and I've been avoiding it since Saturday. 

https://gist.github.com/LNA/adb13dd541ea352ee7f8

I looked at Martin Fowlers refactoring, and asked a couple of craftsmen for their input.  An apprentice today made a suggestion on how to refactor the code, but none of my research or any given suggestions would work.  The code was a 13 line block on inconvenience, preventing me from making the class it lived in cleaner.

Why was it so hard to refactor the code? Even though it had been refactored to look like it was doing one thing, it wasn't.  Its doing too much.  

It looked as if it were saying something like:

"If you get a route, return the corresponding body".

But it was really saying:

I have two classes: ClassA and ClassB.

ClassA needs and instance of ClassB injected into it. ClassB is hard to test.  It might look something like this:

ClassA(ClassB classB)

I want to create a mock to help me test the behavior of ClassB.  

Java only wants ClassA to take in an instance of ClassB, because Java is stubborn, which is a good thing.

I can't inject ClassA with an instance of MockClassB.  Java knows that ClassB and MockClassB are two different types.

To make ClassB and MockClassB more alike, I can create an interface that they will both be forced to implement. 

Lets call it iMagicalInterface.

Now, you can inject any class into ClassA that implements that interface. This allows us to do this:

ClassA(iMagicalInterface classB)

ClassA(iMagicalInterface mockClassB) 

1.  With a spike, I want to see how the thing I'm building works, how the classes fit together, and how its going to interact with an external service (if there is one).  When its time to build the "real" thing, It will be easier to test drive.  I don't think its a good idea to test first when you don't have an idea of what the shape of the application is going to be.  It can lead you to this weird place where things are harder to change later.

2.  A spike has lost all sense of generalities.  For example, I won't refactor to a general interface (polymorphism, open closed).   Once I know that everything is working how its supposed to, I can read up on design patterns that I might want to implementand I will make my tests fit that specific implementation.   Plus, I can clearly see what belongs together and what doesn't when there is an annoying amount of duplication.  This prevents me from over abstracting things. 

I'm working on my Spike, my spiked version of the Java server.  When I go to localhost:5000/file1 I see three lines:

fooAllow: GET,HEAD,POST,OPTIONS,PUT Location: http://localhost:5000/ file1 contents

But all I want to see is the string "file1 contents", because the rest of the data showing up is a header.  You can't see headers.

Byron suggested that I navigate to the terminal, and type:

curl -v http://justinjackson.ca/words.html so that I could see the difference between what should be in my headers, and what I should be returning as html. 

It showed me this: