Exploration

When piling dimensions onto a KDE, the possibility of correlations among the data is a striking concern. Incorporating multiple strongly correlated variables results in counting the underlying evidence explaining the results several times, possibly skewing your predictions if the evidence weighs in a different direction, and making them overconfident.

Performing a factor analysis (FA) or principal component analysis (PCA) on the data used to generate predictions prior to creating a KDE can thus not only reduce the dimensionality of your data, but also increase the accuracy of your predictions.

In order to demonstrate this method, I reused the dataset of LAUSD schools, testing the accuracy of % health risk predictions through cross validation (eliminating schools from the dataset, and then determining likelihood that a KDE generated from the remainder of the dataset assigns to the actual health risk of the eliminated school, using eliminated measures as the evidence). I iterated through a variety of variables, as well as methods of treating the data prior. The 10 most effective methods of predicting health risk of LAUSD middle school, and the probabilities when normalized to only those 10 (abreviations listed below):

Directly using %FRM and %WnHS: p = 21.2% that this is the best model.

PCAmle of API, %AAnHS, and %WnHS: p = 13.1%.

Two component PCA of API and %AAnHS: p = 13.0%.

PCAmle of %FRM and API: p = 11.1%.

PCAmle of API and %WnHS: p = 10.3%.

Two factor FA of %FRM, %AAnHS, and %WnHS: p = 7.9%.

Directly using API score, %AAnHS, and %WnHS: p = 7.0%.

Two factor FA of %FRM, API, and %HLS: p = 6.4%

Directly using API and %AAnHS: p = 5.9%.

Two component PCA of API and %WnHS: p = 4.1%.

PCAmle = PCA using Minka's mle heuristic for determining the number of components. %FRM = % of students on free or reduced meal plans. API = Academic Performance Index %AAnHS = %African American non-Hispanic students %AS = %Asian students %HLS = %Hispanic or Latino students %WnHS = %White non-Hispanic students

While in some cases, effective results were attained by using data directly, overall prediction accuracy was increased by removing correlations between the types of data. I plan to investigate the details of why the results are what they are, in hopes of building a predictive model.

Some of the results betray my expectations, eg: %FRM has a strong negative correlation with %WnHS (r^2 = 0.716). While this was the highest rated model, this may be analogous to building a PMF for coin flips and updating it 10 times per flip. If one gets lucky in their first 10 flips, landing 5 heads and 5 tails, the probability mass will very quickly and strongly converge on p(heads) = 0.5, making this PMF stand out in predictive power compared to another correctly updated only once per flip. But this is only because it converged far more rapidly than evidence would allow, and in this example due to luck it did so on the actually correct value.

When dealing with multidimensional KDEs, bandwidths are tricky to deal with. The kernels are symmetrical, so for them to provide a similar degree of smoothing in all dimensions, all dimensions must have a similar distribution.

I have been scaling each variable by dividing each value by the standard deviation. This is despite fears that it is blasphemous to incorporate the use of a parameter into what should be a nonparametric analysis - and that it is suboptimal to treat data with very non-normal distributions. That, while two non-normal distributions might have equal standard deviations, their spreads could still be different.

I'm still mulling over better solutions. If anyone has any ideas, suggestions, or thoughts - I would love to hear them!

But this just deals with making sure the bandwidth works effectively across different dimensions. Another concern is adjusting the bandwidth itself. 

The concern here is primarily one of getting as many useful numbers out of the KDE as I can. In particular, the "D:" density values, as seen in part 1.

With the kernel density estimator I have been using, the interior volume of n dimensions of bandwidth * probability equals 1, regardless of the number of dimensions. As I obtain the total density values of slices across the KDE by adding up the probabilities, this creates an inconsistency when changing the number of dimensions (n); eg modeling the situation with a single n+1-dimensional rectangular kernel, of width 0.1 and probability density D: 1 = (0.1)^n * D As the dimensionality of the data increases, D also increases by an order of magnitude. If width were instead 10, it would decrease. The simple solution is thus to restrict ourselves - if we are interested in the specific density values - to a width of 1.

In the case of Gaussian kernels, the bandwidth keeping density weights consistent across dimensions is 0.398942280401.

http://www.mglerner.com/blog/?p=28 Is a fantastic introduction to Kernel Density Estimates, doing a great job of illustrating the concept of bandwidth.

Unfortunately, because of privacy reasons data on school tests is hard to come by. Other data, like the percent of children on free or reduced meal plans in LAUPD middle schools (n=72) are readily available:

The distribution is not normal; it is skewed strongly to the left, and irregularly shaped. The bandwidth, determined by cross validation, was a little over 5, resulting in the probabilities bleeding a little over the 100% point.

While the resolution of the PMF was 6,001 data points, I scaled the probabilities so that they reflected probabilities of units with range 1. That is, the probability of 84-85% of the students in a school being on free or reduced lunch is roughly 0.04, or 4%.

In addition, "D: 71.99" is a calculation of the total density of the KDE the PMF accounts for. In this case, where there were 72 data points total, almost all of the data is accounted for in the above graph.

But, this is not what makes KDEs exciting! They are not limited to one variable; we can freely add extra dimensions. For example, adding information on the percentage of kids classified as being at health risk due to exceeding a cut-off BMI value:

(In this graph the probability values are somewhat arbitrary, but relative differences remain accurate.)

We can visualize the complex interaction between %HealthRisk and %FreeorReducedPriceMeals. Schools with less children in families requiring financial assistance also appear to have healthier children, but unfortunately the bulk of schools do not appear to fall into this came.

This relationship appears strong, as the probability density holds tightly against the line. But, we can do more than that - KDEs let us conveniently control for variables and create PMFs by simply using certain values.

We can see how the probability mass of health risk shifts based on the % of children on free or reduced meal plans. The approximate number of data points now corresponding to each line is much lower.

We can also present them as cumulative distribution functions (CDFs), which can clearly illustrate differences:

For example, while almost all schools with 50% of children on free or reduced meal plans have less than 40% of children at health risk, this is only true for about half of schools with 80% or more of children on such plans.

A powerful tool for making predictions, as well as visualizing variations in data!

If we flip a coin 100 times and get 59 heads and 41 tails, we may want to estimate the probability it lands heads between 45 and 55% of the time, as well as have a 95% confidence interval.

Using computers, this is simple. We can create a probability mass function of discrete probabilities, ranging from p = 0 heads through to p = 1.0 heads. For each hypothesis, the likelihood of the coin coming up heads = the hypothesis; the likelihood of tails is 1 - hypothesis.

Here I am:

Creating a probability mass function with 1,001 discrete elements, giving us accuracy to the thousandths place.

Updating each hypothesis with 59 heads and 41 tails.

Naively assuming equal prior probabilities for all hypothesis.

For each piece of evidence, after multiplying each prior probability by the likelihood, we can sum up all these probabilities to get a value for p(B) in Bayes theorem: the probability under any hypothesis. We can then divide each hypothesis by this value, normalizing the probabilities to a total probability of 1.

Following these steps with a little bit of python, I find a 95% confidence interval of the coin landing heads between 49.2 and 68.1% of the time. The probability that the coin lands heads between 45 and 55% of the time is only 20.9%.

Realistically, my personal experiences have been quite inconsistent with those of Rosencrantz and Gildenstern (are Dead), so I'd actually assume a much higher prior probability for the hypothesis that the coin lands heads 50% of the time than the extreme values, like 0 or 100% heads.

If we instead imagine a negative exponential function peaking at 0.5, such that p(0.5)/p(X) = (0.5/(0.5-absolutevalue(0.5-X)))^2, we get: 95% confidence interval: 48.6 - 67.1%, and 27.3% chance that the actual value lies between 45 and 55%.

Changing the priors did not have an overwhelming impact, as the large amount of evidence quickly overwhelms it. On the otherhand, if we start with naive priors and then a body of 1,000 coin flips that landed 50% heads to create the priors used for our 100 flips, we get a 95% confidence interval of 47.9 - 53.8%, and a 94.1% chance that it lies between 45 and 55%. The mountain of evidence shaping our priors is not so easily overturned!

  The takeaway here is that, when dealing with quantitative hypothesis, treating them as discrete (eg, 50.5 vs 50.6 vs 50.7% heads) rather than continuous (anywhere between 50.5 and 50.7% heads) makes updating a set of hypothesis a straightforward process: one must simply define a likelihood function. Which, in cases like the coin flip example, is particularly simple.

Quick notation note: p(A) means "the probability of A", and p(A|B) means "the probability of A, given B is true".

p(A) and p(B) both occurring only equals p(A) * p(B) if A and B are independent, such as when flipping coins. Often the probabilities of two events are dependent on one another, eg if it rained yesterday it is more likely to rain today, because rain tends to come in blocks with a front.

But, lets use a simple example: we have a bag of 3 brown and 2 red M&Ms, and eat two at random, one after the other. For the first M&M, p(first brown) = 0.6 and p(first red) = 0.4. Second M&M color probabilities, if the first M&M eaten was brown: p(second brown|first brown) = 0.5, p(second red|first brown) = 0.5 But if the first M&M we ate was red: p(second brown|first red) = 0.75, p(second red|first red) = 0.25 Thus, the probabilities that M&Ms 1. & 2. were colors: Case A: 1.brown & 2.brown = 0.3 Case B: 1.brown & 2.red = 0.3 Case C: 1.red & 2.brown = 0.3 Case D: 1.red & 2.red = 0.1.

While a little more complicated, it can also be fun to express this the other way. We can see above that the overall probabilities the second M&M is brown or red are 0.6 (Case A + Case C) and 0.4 (Case B + Case D), respectively. Given that the second was brown, p = 0.5 that the first was brown (Case A/0.6) and p = 0.5 that the first was red (Case C/0.6). Given that the second was red, p = 0.75 that the first was brown (Case B/0.4) and p = 0.25 that the first was red (Case D/0.4). We can express p(A) and p(B) both occurring as either p(A) * p(B|A), as in the first example, or p(A|B) * p(B), as in the second, and achieve the same result. p(A|B) * p(B) = p(A) * p(B|A) If we simply divide both sides by p(B), we get: p(A|B) = p(A) = p(B|A)/p(B)

Thus, Bayes Theorem has been derived. Ie, take A = hypothesis, and B = evidence, it reads: the probability of a hypothesis, given evidence, equals the prior probability assigned to that hypothesis times the likelihood of that evidence, given the hypothesis, divided by the probability of that evidence given all hypothesis.

Back to the M&M example! You know there were 3 brown M&Ms and 2 M&Ms in the bag. I ate one, and you couldn't see its color. You pull one out, and it is red. Hypothesis 1: The M&M I ate was brown. Hypothesis 2: The M&M I ate was red.

What are the probabilities you assign both hypothesis? p(H1|red) = p(H1) * p(red|H1)/p(red) p(H1|red) = 0.6 * 0.5/0.4 p(H1|red) = 0.75 The prior probability assigned to my M&M being brown was 0.6 (3 out of 5 M&Ms were brown). Given H1, there are then 2 brown and 2 red M&Ms, meaning p(red) is then 0.5, while p(red) overall = 0.6*0.5 + p(red|H2) = 0.3 + 0.4 * (1 red M&M/4 M&Ms total) = 0.4

The probability of H2 can be calculated similarly: p(H2|red) = p(H2) * p(red|H2)/p(red) p(H2|red) = 0.4 * 0.25/0.4 p(H2|red) = 0.25

While it is hard to glean from evidence what the territory looks like, we can draw alternative maps, representing the range of possibilities. From there, we simply ask how theoretically likely each map is, how likely the evidence is within that map, and how likely it is across all maps.

Maps are not discarded, but we thus adjust our favored map corresponding to which best explains our observations: which map best explains & predicts our observations vs the other maps?

Note that such heuristics as "non-falsifiable hypothesis"-important to traditional rationalists*-become unnecessary: if a million different possibilities are equally likely, each possibility has a pitifully low p = 10^-6 of occurring, punishing the hypothesis in direct correspondence to just how hard it is to falsify.

The map and territory are a metaphor for our understanding of reality, and reality itself. Our ideas and models of how the world works are all features on the map. The level of detail and simplification varies, and much of the territory remains unexplored.

For a map to be useful, it must reflect the actual features of the territory. If a bridge is out, it'd be useful to know, and when navigating a new city it had better have the streets in order.

This also lets us define what a "proper belief" is: something that would correspond to a feature on the map. To escape the metaphor, a proper belief controls our anticipations, directly making a prediction/statement on the condition of reality.

In this context, evidence allowing us to update the map must be something somehow corresponding to the territory. If we want to draw a map of New York, we'd better explore New York, or at least look at another map drawn by someone who explored/looked at a map drawn...

Proper methods of updating the map, to actually correspond with reality, reflecting its various contours and details rather than personal visions, is a question for the next section.

While not yet particularly revelatory, I will use it regularly throughout future posts where it is useful for illustrating points.

Lets try some logic puzzles: a) We have cards with a letter on one side, and a number on the other. Hypothetical rule: if a card has a vowel on one side, it has an even number on the other. You have 4 cards, you see: A, T, 4, and 7. To decide if the rule is true or false, what are the only cards you ought to turn over?

b) Given that all X are Y, and some Z are Y which of the following necessarily follow: 1. Some X are not Z; 2. All X are Z;  3. Some Z are X.

c) No nutritional things are inexpensive. Some vitamin tablets are inexpensive. True or False: Therefore, some vitamin tablets are not nutritional.

d) No addictive things are inexpensive. Some cigarettes are inexpensive. True or False: Therefore, some addictive things are not cigarettes.

e) This one is interactive! There is a rule defining sequences of numbers. 2, 4, 6 follows that rule. Message me with example sequences, asking if they follow the rule. I'll answer yes/no, as often as you'd like, and when ready you get a chance to guess at the rule.

I am starting a sequence of posts detailing the history of my love affair with statistics.

Skipping past a long series of arguments that began with good intentions and a flurry of citations, but ended with venom and a bitter disagreement -- I was left, wondering: when presented with the same evidence, how can we come to different conclusions? How ought we interpret evidence? Come to conclusions? Find truth?

What is truth?

In search of answers, I came across a seductive article, then a sequence of posts by Eliezer Yudkowsky on lesswrong: http://wiki.lesswrong.com/wiki/Rationality_materials

Reading up on sexism in the games industry. Any good links on the effects (if there are any) of representation?

Lots of interesting info on how representation affects legislation, but I haven't found any quantitative measures regarding representation and reported sexism.

I'm breaking the question down into two parts: 1. How female people are represented. 2. Representation matters. A lot. Part 1: How female people are represented. (Collins (2011) did a great job summarizing a lot of existing literature, so I'll draw from it freely for this section.)

The depressing state of the representation of female people in media is epitomized by this quote:

There was little discussion in any of the papers regarding content that might depict women particularly positively... Given what we know from the articles in these issues, there are likely to be few such portrayals, making it difficult detective work to track them down, and obtaining enough of them for a quantitative analysis even more taxing.

Positive portrayals of female people in the media are so few and far between, they could not analyze them.  In 1950, the 2.5:1 disparity in representation of men:women in prime time television mirrored the disparity in representation within the workforce. Today, the workforce ratio dropped to 1.2:1, but the prime time portrayal ratio hasn't budged.

In G-rated films, the ratio is 2.57:1. In music videos from music-oriented TV channels it is 3:1. In video games it is 6:1.

Female people are highly underrepresented in media, compared to their proportion of the population and role within the workforce.

However, the level of representation is only part of the picture – when they are represented, a dramatic inequity exists in terms of how. From Downs and Smith (2010), here are some statistics on characters from the 20 best selling console games:

Feature: % occurrence in women / % occurrence in men ||| % women in games rated E / % men in games rated E Sexually revealing clothing: 41% / 11% ||| 33% / 18% Partially / totally nude: 43% / 4% ||| 22% / 1% Unrealistic body proportions*: 25% / 2% ||| 44% / 1% Small waist**: 40% / 1% ||| 56% / 0% Breast size (voluptuous): 26% / na ||| 11% / na Inappropriate attire: 16% / 2% ||| 9% / 0%

Collins (2011) was careful to distinguish between sexual content, and sexualization. Female and male people can be sexual without being sexualized. The article notes popular press often confuses these, referencing the negative press against the positive image produced by Dove's “beauty has no age limit” campaign as an example. Objectification is a key difference.

Paek et al (2011) found that male people in the US were more commonly portrayed in professional (13.9 vs 5.4 %) and office (55.6 vs 45.9 %) roles. They also made up more of the prominent (38.9 vs 30.2 %) and assisting (27.8 vs 11.1 %) characters. The large majority of voice overs are done by male people (72.8 vs 27.2 %).

These are things we see all around us. That such a thing as the Bechdel test can even exist, let alone be failed, is kind of ridiculous. Yet, somehow the movies I watch fail time and time again. I can't recall seeing a movie that would have been failed by the male-equivalent.

Part 2: Representation matters. There is an interesting consequence to the skewed representation that I – as a white male – hadn't even realized: Those portrayed less frequently in media learn to identify with characters from the better represented group. That is, female and black people can both identify with white male characters far better than white males can relate with female or black characters (Collins 2011).

This brings about a blatant problem of the dominant group identifying (and perhaps sympathizing) less readily with minority groups than vice versa. The problem perpetuates itself: if the dominant group displays inflexibility in their preferences, while other groups do not, the casting decisions to maximize success at the box office become clear.

Grabe et al (2008) performed an expansive meta-analysis, finding that female people with greater exposure to media have decreased levels of body satisfaction, more deeply internalized the thin ideal, and more eating disorder symptoms.

Quantified examples of the influence of perceived gender roles on children’s behavior are plentiful. For example, Croft et al (2014) found that mothers' explicit gender role beliefs predicted children’s explicit gender role beliefs, that fathers' explicit gender role beliefs and mother's explicit self-stereotyping predicted children’s explicit self-stereotyping, that fathers' explicit gender role beliefs, implicit gender role attitudes, and contribution to domestic chores predicted stereotyping of children’s career aspirations.

Games probably more strongly modify our behavior than non-interactive types of media. Bandura (1977) outlines a theoretical framework of human-behavior modification. It argues we humans have substantially more sophisticated and long-term belief systems guiding our behavior than animals like Skinner's rats or Pavlov's dogs (Sorry, The Office: conditioning Dwight with Altoids probably wouldn't work without his express conscious awareness).

Through rationalizations, our beliefs and behaviors have substantial resistance to chance and logically disconnected events. Basically, we interpret the world through our understanding of reality.

Of the four sources that can modify self-perceptions (performance accomplishments, vicarious experience, verbal , and emotional arousal), performance accomplishments have the greatest influence. Thanks to their interactive nature, games can modify our beliefs about our abilities and potential more than other types of media.

Yang (2014) provides an interesting example of the impact of performance accomplishments. 141 white university students (65% female) played either WWE Smackdown vs. Raw (2010) or Fight Night Round 4, using 3rd person perspective. They played either a black or a white character, and then took a test of implicit associations between black and white faces with weapons such as guns and hand grenades (note: neither game involved use of such weapons). Playing a black character resulted in a higher probability of associating black faces with weapons (mean = 0.61, standard deviation = 0.53) than white faces (mean = 0.39, standard deviation = 0.35).

They also found that those playing a black character gave an average of 115% more hot sauce to someone who'd said they “strongly dislike” hot sauce. @_@

Games are a powerful tool for modifying behavior and attitudes. Unfortunately, rather than aiding underprivileged groups, they are being utilized as tools for perpetuating and reinforcing negative stereotypes & oppression.

Works Cited