Green traffic lights are a beautiful indescribable turquoise, the most intense turquoise you’ve ever seen. You’ll feel crazy once you see it, and want to run around telling everyone. Green traffic lights not only aren’t green, but they’re also exquisitely beautiful.

You utter a statement. Someone says, “Yet. Growth mindset.” This happens more and more. If you are very careful in your phrasing, you may hear, “So far,” instead. You speak in soft whispers, alone. Pessimistically. “Yet,” says a voice. “Growth mindset.” Silently, in a deserted office, you say nothing. “Growth mindset,” says the voice. Louder. It is growing.

You did not always live in the Bay Area. Once you began a circle of friends based around similar interests, somewhere else, far away. You made a pilgrimage. You did not return. You do not know if you have abandoned your friends to the wider world or if they will join you one day. You do not know what you hope they will do.

It has been six weeks since you met someone who was not a programmer. Perhaps everyone is a programmer. If we were in a simulation, everything would be programming.

You’re sure you’re imagining the blood on your friends’ teeth when they endorse killing babies. That was only a story. You’re so happy.

It is the Solstice. Something is wrong with the tech for the ISS footage. You wait in darkness. You have a light, but you are forbidden to ignite it.

You listen to Michael Vassar. You don’t remember traveling to this party or sitting on this beanbag. You don’t remember when he began to speak. He is still speaking. He sounds like madness and glory given lisping poetry, and you want to obey.

Someone is getting married. You speak dutifully of statistics when asked what you think of their wisdom. Someone toasts the Chosen One. You raise a glass that is lit with changing colors.

You are caught up with Harry Potter and the Methods of Rationality. You have always been caught up on Harry Potter and the Methods of Rationality. You were born caught up on Harry Potter and the Methods of Rationality. Your car clock ticks to the moment of its update and you drift into oncoming traffic as you scramble for your phone. You will always be caught up on Harry Potter and the Methods of Rationality.

You’re struggling with akrasia. You have insight into your condition. You pity people who don’t even know to call it “akrasia”. You have not done anything in months.

Your house has a name. Your house’s whiteboards have messages on them that none of you remember writing. Your house wants you to buy seaweed snacks and almonds. They’re paleo. Your house is concerned about diets of the ancestral environment. Your house is gluten-free.

You have not found a way to connect your hobby to rationalism yet. It itches. You are not whole. It is forbidden to post an article entitled Rationalist Hobby on Less Wrong. You lie awake at three in the morning, trying to create puns.

You stare at plastic pyramids. Your eyes are playing tricks on you. There is a koan with both a black and a white Go stone. Your eyes must be playing tricks on you.

You have a nightmare about committing the fallacy fallacy fallacy fallacy fallacy fallacy fallacy. You wake up. You haven’t, really. Probably. One is not a probability and you are not safe, never safe.

You see movement out of the corner of your eye. You dismiss it on priors. You still see it. You dismiss it on priors. You fear updating.

You are prepared for Omega to appear to you and present you choices. Ecstatic choices that will leave you laughing from the top of a heap of utility or in torment for all eternity. You are ready. You fear and yearn for this day in equal measure. You are ready and know exactly what you will do and so does Omega.

You want to be frozen when you die. Not sooner. You look around corners before you step past. Sometimes, you consider leaving your necklace at home. Not yet.

You aren’t afraid of paperclips. They can’t hurt you. They’re only paperclips. Don’t be afraid.

Politics is the mindkiller. You do not need to come up with thoughts about politics. They would only kill your mind. Rest now.

You have expanded your comfort zone. Today you have committed two felonies and eaten a motorcycle. You are comfortable, you remind yourself, as you flee the authorities. You are comfortable.

You abandon a sunk cost. You didn’t want to. It’s crying.

You donate to effective charities to signal virtue to your friends. They are impressed. You feel cold. Friendship isn’t about affection. You signal louder and louder to nothing and no one. You have never done anything real. You are not real.

Empiricism is an important virtue; it's great to try things and find what works for you, and there's a noble tradition of people doing that and blogging about it. We should be doing this more.

The problem is that experimenting on yourself can be pretty slow and tricky and inconvenient.

This post is about Taguchi arrays, which fix that by allowing you to get much more useful information from far fewer experiments. The method is commonly used in industry for process optimisation, but it isn’t complicated and is very easy to apply to DIY experiments. (Thanks to NightHawkInLight for making me aware of this!)

The default way kind of sucks

Suppose you're wondering what will work best to improve your sleep. You've got one of those sleep tracker devices that gives you an integrated Sleep Quality Score, so you've got a clear metric to optimise. Let's do some empiricism!

Now, what interventions to test? A lot of people say melatonin is good. Apparently 0.3mg is the ideal, but maybe 1mg works better, and it's more commonly available. Other people swear by magnesium. You can get magnesium glycinate easily, but some say magnesium l-threonate is better, although it's more expensive. You've heard exercising helps, but is cardio or strength better? And you've got some of those blue light blocking glasses, but they look kind of goofy. There’s a ton of things to test.

One thing at a time

The big piece of accepted wisdom here is to only change one thing at a time. If you change more than one thing and your sleep gets better, you won't know which thing did it. Or, one thing you change might help and the other might make things worse, so they cancel each other out and you learn nothing.

Changing one thing at a time, we need at least 8 experiments:

Baseline

0.3mg Melatonin

1mg Melatonin

Magnesium glycinate

Magnesium l-threonate

Cardio

Strength

Goofy glasses

In principle we could do this in 8 days, just trying each thing once.

But that's going to be really noisy. Just one measurement per item is hardly going to tell us anything; we should try each thing on more than one night to get more signal. Maybe do each one for two or three days? That's 16 or 24 days, which is kind of a lot. Give each thing a week and that’s 2 months of experimentation.

And also, we care about finding a good overall setup, not just which individual things work. Probably there are some interactions here, like, what if magnesium works really well, but only if you also exercise? We'd never find out using this experiment.

Test everything?

To really know, we'd want to try every combination. So, 3 levels of melatonin (none, 0.3mg, 1mg), times 3 options for magnesium (none, glycinate, l-threonate), times 3 exercise options (none, cardio, strength), times 2 for red glasses (yes or no), means... 54 days, to try each thing once. That's not great.

There's got to be a better way!

This is where Taguchi Arrays, aka Orthogonal Arrays come in!

You go to an online Taguchi array generator like this one, add in your factors and their levels, and it spits out something like this:

Before you start, it’s a good idea to randomly shuffle the order of the rows, so secular changes or cumulative effects don’t bias things.

Then you go through the rows one night at a time. Each night you follow the protocol listed, and each morning you record your Sleep Score for that protocol.

Now to analyse the results. The genius of orthogonal arrays is that for any pair of factors, every possible combination of their levels appears the same number of times. This means that you can safely compare averages.

So if for example we want to know how magnesium glycinate compares with magnesium l-threonate, we can take the average of the sleep scores of nights we took glycinate, and the average of nights where we took l-threonate, and just see which is better. This is safe to do, because all the other variables are equally represented in those sets, so they won't bias things. You can go through and check - the 3 glycinate nights have one of each of the melatonin levels, and one of each of the exercise levels, and two have glasses and one doesn't, and the exact same is true of the L-threonate nights, and of the "no magnesium" nights. And this nice property is also true for all of the possible values of all other pairs of factors.

So using just 9 nights of experiments (just one more than the original 8 night experiment which told us very little), we get at least three nights of data to average together for each level of each factor, rather than just one! And we're testing all possible combinations of levels for each pair of factors at least once. With just these 9 experiments, we can find out a huge amount of information, almost as much as we'd get from the full 54-day factorial experiment!

Limitations

Now, this works best for situations where you expect the result to be approximately a linear combination of the factors, with only simple interactions between pairs of factors. For example, if there were some major interaction between three factors, like if melatonin worked wonderfully but only if you do cardio and also don't wear light-blocking glasses, this wouldn't catch that. But real life experiments tend not to have much of that kind of thing.

And of course if you have a situation like, magnesium only starts to help with sleep on the third night in a row you take it, this won’t detect that. But that’s a problem with using ‘sleep score on the same night’ as the metric, not an issue with Taguchi methods.