#reshape2

Daily aggregation of Big Data news from @cloudfeednet.

Chapter 2 - Practical Data Science Cookbook

Analytics guru Wolf helps Pistons cut through data thicket

Wolf, 34, is a partner with Chicago-based data science and design firm Datascope Analytics. The company has signed an agreement through August ...

Data Scientist

of developing high-value digital journalism, storytelling, and brand ...

Hadoop Developer/Architect

investment bank working on large-scale projects within the bank's big data ...

more

.

On an airplane the other day, I learned of a method called latent class (transition) analysis, and it sounded like an interesting thing to try in R. Of course, as with everything R, There is a Package for That, called poLCA, written by none other than Drew Linzer (of Votamatic fame) and Jeffrey Lewis.

I wasn't able to think of a good application for transition analysis specifically, but I did use Christopher's ANES data to estimate latent "types" of respondents. The example model illustrates a four-class model, and I'll leave it as an exercise for the interested reader to assign subjective names to each class.

This Gist also attempts to improve on the default plot both by eschewing the 3-D effect, and by putting classes, rather than variables, in direct comparison with one another. Also, for what it's worth, the plot code shows how to draw a bar plot when you have already computed counts or proportions -- use stat="identity".

Thanks for celebrating Advent with us, and for your feedback and support. We're taking a little break after tomorrow's post, but we'll be back better than ever next year!

We've recently seen how to parse XML for the data that goes into producing graphs such as the one on this page, comparing the Romney vs. Obama polling average.

Today's Gist shows how to approximately replicate that very figure, with two trend lines indicating each candidates' polling average, plus an area plot of the difference between the two. This is done in ggplot2 by way of a semi-hack, which uses the scales package to transform the y-axis so that part of the continuous scale is "cut out."

The figure above is built in 5 steps:

Plot the two competing trendlines

Add an area plot showing Obama's (possibly negative) lead

Write a function (scaleBreaker()) that takes a continuous vector and makes a break in it, between a lower and an upper threshold

Use this function to make a custom axis transformation with trans_new()

Add standard ggplot2 axis breaks and labels, apply the custom transformation, and draw a line to emphasize the disjunction

This post is in response to an is.R() reader's "Ask us anything" query. In short, the reader has several .CSV files of World Bank Data, in which each row is a Country, each column is a Year, and each separate file contains a different variable (like population, GDP, etc.).

Today's Gist illustrates how to use a simple loop to load, reshape, and then "stack" multiple data sets into a more usable form. The first part of the script creates and saves some random "World-Bank-like" files, which are then loaded and stacked in an iterative fashion.

Featuring the lovely "spectral" palette from Colorbrewer. This really just serves as a reminder of how to do four things I frequently want to do:

Make a heatmap of some kind of matrix, often a square correlation matrix

Reorder a factor variable, as displayed along the axis of a plot

Define my own color palette with colorRampPalette()

Use RColorBrewer, specifically the diverging "spectral" scheme

As a follow-up to yesterday's post on producing visually-weighted regression plots, here is some code which illustrates the production of similar plots, but using Zelig's convenient modeling and simulation functions.

This code was produced to assist a colleague, which just goes to show that the "Ask us anything" page really works!