#classical statistics

I'll get straight to the punch: the expected probability is actually (k + 1)/(n + 2). I'll sketch the proof later, but let me try to explain intuitively first. It's almost the same as k/n, but we have an added 1 on top and 2 on the bottom. These numbers are accounting for the fact that both failure and success are possible outcomes of these trials. It's as if we had two extra trials, with one failing and one succeeding, before we started doing our actual trials. These extra numbers will now take care of the case where k = 0 or n.

The proof: we'll be using Bayesian conditioning. It's actually pretty simple, but the equations get garbled when I type them up. I'll be schooling myself in some equation tools this summer, so hopefully this kind of stuff won't be an obstacle any more. (Also it wouldn't be a problem if I just stopped insisting on writing posts about statistics...) In the mean time, maybe...copy out the equations so you can see them nicely? LOL.

P(Xn+1 = 1 | Sn = k) = P(Xn+1 = 1, Sn = k) / P(Sn = k) = [P(Sn = k | Xn+1 = 1)P(Xn+1 = 1)] / [P(Sn = k | Xn+1 = 1)P(Xn+1 = 1) + P(Sn = k | Xn+1 = 0)P(Xn+1 = 0)]

Okay. Let's look at the numerator. The unconditional probability P(Xn+1 = 1) is just 1/2 because we really don't know anything about the experiment; so we assume failure and success with equal likelihood. What about P(Sn = k | Xn+1 = 1)? That's asking for the probability of getting k + 1 successes in n + 1 trials, with one success landing exactly on the last trial. That's (k + 1)/(n + 1) because it's equally likely to fall on any trial (n + 1) and we have k + 1 chances to hit that last trial. So the numerator is [(1/2)(n + 1)]/(k + 1).

Notice the first term in the denominator is identical to the numerator. The second term is similar: but what's P(Sn = k | Xn+1 = 0)? Well, now we just reverse our reasoning. We have n + 1 trials and n + 1 - k failures, with a failure landing squarely in our last trial. So following the same reasoning, we get (n + 1 - k)/(n + 1). Notice that P(Xn+1 = 0) is also 1/2 as above. So the denominator is (1/2)[(k + 1)/(n + 1) + (n + 1 - k)/(n + 1)]. So throwing all these rabbits back together (and cancelling the 1/2 and n + 1), we get:

(k + 1)/[k + 1 + n + 1 - k] = (k + 1)/(n + 2), exactly what we wanted! So...given that the sun has risen for like. 4000 years or more...what's the chance that it'll rise tomorrow? haha.

Statistical philosophy basically asks the question: what does it mean for an event to have a certain probability? And how do we assign probabilities to events in question? Classical statistics went the simple route and was largely based on symmetry--so six-sided dice, etc. Classical statistics predicted that the chance of getting 0 heads, 1 head, or 2 heads in two coin flips was 1/3, 1/3, 1/3: because there were three cases. Unfortunately, empirical results did not agree with these predictions; we saw one head coming up more frequently than none or two. Classical statistics also ran into limitations when there wasn't a clear symmetry.

Frequentism is the now popular convention for statistics. Frequentists believe that the inherent probability of an event is approximated by the proportion of times it happens in a series of experiments. The limit of this proportion, as the number of trials runs to infinity, is the exact probability. Frequentism is what we now instinctively fall to when dealing with probability: the chance of heads is 1/2, the chance of no heads in two flips is 1/4, the chance of rolling 5 is 1/6, etc. Of course, some people have issues with the idea of infinite trials, something that is not physically possible to perform. Critics of frequentism will argue that this definition of probability is unfounded because you can't construct infinite experiments.