Pain is a beautiful thing.

When I was in college, about ten or so years ago, I was a history major. I wanted to learn to dance, so I joined a swing dance club on campus. To my surprise, this club had about twice as many men as women (in high school, the last time I’d tried dancing, the ratio had gone the other way–lots of girls, and boys only that you could drag by their ears).

But apparently, there had been some kind of word spread specifically to the STEM guys that dance was a way that they could meet girls.

So anyway. I joined the swing dance club, and met a few guys. And at one point, when socializing with the guys outside of dance class, one of them asked me what my research was on. (I had already established that I was an honors history student doing a thesis, just as he had established that he was an honors… I’m not sure if he was CS or Math, but it was one of those.)

So I gave him the thumbnail sketch of my research. Now, to be clear, an honors senior thesis, while nothing like what a graduate student would do, was still fairly in-depth. I had to translate primary sources from the original late-Classical Latin. (My professor said, basically, that while there were plenty of translations of my source material, that I’d only be able to comfortably trust them if I had at least made a stab at a translation of my own. And he was right.) And there was so much secondary material, often contradictory, that I had been carefully sorting through.

But I was able to sift it into a three-sentence summary of my senior thesis work, you know, as one does.

So I gave him that summary, and then asked–since he was also an undergraduate senior doing an honors thesis–what his research was on.

“Oh,” he said, “you wouldn’t understand it.”

Reader, I went home in a frothing rage. Because I had thought we were playing one game–a game of ‘let’s talk about what we’re passionate about!’– and he had been playing another game, which was, one-upsmanship. I had done my best to give a basically understandable brief of my research–and he had used that against me. As if my research, my painstaking translation, my digging through archives and ILLs of esoteric works, my reading of ten thousand articles in Speculum (yes, the pre-eminent medievalist journal in North America is called Speculum, I’m sorry, it’s hilarious/sad but also true), and then my effort to sum it up for him, was nothing. Because his research into some kind of algorithm or other was just too complex for my tiny brain to conceive of. Because I just couldn’t possibly understand his work.

Now, the important note here is that the person I went home to was my senior year roommate. She was a graduate student–normally undergrads and graduate students couldn’t be roommates, but we’d been friends for years, and the tenured faculty-in-residence used his powers for good and permitted us to be roommates that year. Anyway. My senior year roommate was basically… in retrospect I think possibly an avatar of Athena. She was six feet tall, blonde, attractive in a muscular athletic way, a rock climber and racquetball player, sweet but sharp, extremely socially awkward, exceptionally kind even when it cost her to be kind, and an incredibly brilliant computer science major who spent most of her time working on extremely complicated mathematical algorithms. (Yes, I was a little in love with her, why do you ask? But she was as straight as a length of rope, and is now happily married, and so am I, so it worked out.)

(Still, yes, she is my mental image of Athena, to this day.)

Anyway, I came home in a frothing rage to my roommate, the Athena avatar. And I said, “He made me feel like such an idiot, that I could sum up my research to him but his research was just too smart for stupid little me.”

And she shut her book, and smiled at me, with her dark eyes and her high cheekbones and her bright hair, and said, “If he can’t explain his research to you, then he’s not nearly as smart as he thinks he is.”

Now I hesitated, because I’d be in college long enough to have sort of bought into the ridiculous idea that if you couldn’t dazzle them with your brilliance, you should baffle them with your bullshit. But she said, “Look, I’ve been doing work on computer science algorithms that have significantly complicated mathematical underpinnings. What do I do?”

And I said, “Genetic algorithms–that is, self-optimizing algorithms–for prioritization, specifically for scheduling.”

“Right,” she said. “You couldn’t code them because you’re not a computer scientist or a mathematician. But you can understand what I do. If someone can’t explain it like that, it isn’t a problem with you as a person. It’s a problem with them. They either don’t understand it as well as they think they do–or they want to make you feel inferior. And neither is a positive thing.”

So. There.

“Commander Joseph C. Clifton arrives at his hometown of Paducah , Kentucky, in a TBF/TBM Avenger, 1944. This item is part of a photograph album assembled by Commander Joseph C. Clifton during his service in World War II.”

I have posted about survivorship bias and how it affects your career choices: how a Hollywood actor giving the classic “follow your dreams and never give up” line is bad advice and is pure survivorship bias at work.

When I read up on the wikipedia page, I encountered an interesting story:

During WWII the US  Air Force wanted to minimize bomber losses to enemy fire. The Center for Naval Analyses ran a research on where bombers tend to get hit with the explicit aim of enforcing the parts of the airframe that is most likely to receive incoming fire. This is what they came up with:

So, they said: the red dots are where bombers are most likely to be hit, so put some more armor on those parts to make the bombers more resilient. That looked like a logical conclusion, until Abraham Wald - a mathematician - started asking questions: 

- how did you obtain that data? - well, we looked at every bomber returning from a raid, marked the damages on the airframe on a sheet and collected the sheets from all allied air bases over months. What you see is the result of hundreds of those sheets. - and your conclusion? - well, the red dots are where the bombers were hit. So let’s enforce those parts because they are most exposed to enemy fire.  - no. the red dots are where a bomber can take a hit and return. The bombers that took a hit to the ailerons, the engines or the cockpit never made it home. That’s why they are absent in your data. The blank spots are exactly where you have to enforce the airframe, so those bombers can return.

This is survivorship bias. You only see a subset of the outcomes. The ones that made it far enough to be visible. Look out for absence of data. Sometimes they tell a story of their own.

Japanese postcard showing the First Fleet underway, led by battleship Nagato.