The Decline Effect
In the 1980s, psychologist Jonathan Schooler conducted a study which led to the development of the theory of verbal overshadowing. In the experiment, college students were told to observe a video of a bank robbery, during which the viewers  were given a good look at the robber’s face. Half of the subjects were asked to give a verbal description of the robber immediately after the video was played; half were not. All of the subjects were then asked at a later time to describe the robber. Contrary to popular belief at the time, the test subjects who were initially asked to verbally describe the robber did much worse later on than the ones who did not describe the robber. The scientific community caught on to this idea quickly after its publication, and it has now been cited over 400 times- in short, it is an important and reputable concept in modern psychology. Yet Schooler noticed something troubling in the midst of his success- his experiment was proving impossible to replicate. Each time he tried to replicate the study, the effect size- that is, the number of positive test results- declined dramatically, first by thirty percent, then by another thirty. Schooler was completely baffled by this, as he could find no errors in his experimental methods. This decreasing support for scientific claims over time is known as “the decline effect,” and is actually a common occurrence in experiments across a variety of scientific fields. Faced with this mounting evidence and the frustration of his experimental evidence slipping away from him, Schooler started speculating as to why the decline effect was happening to him, and came up with some pretty radical ideas. Here is quoted from a recent radio interview: “I say this with some trepidation, but I think we can’t rule out the possibility that there could be some way in which the act of observation is actually changing the nature of reality.” That is, by conducting experiments, we are in fact changing the subject of the experiment itself.
This point of view echoes the tone of the first article I read about the decline effect: a piece published back in December in the New Yorker entitled “The Truth Wears Off.” Jonah Lehrer, the author, presents the decline effect as a mysterious and troubling development that calls into question the very nature of scientific pursuit. Is there something wrong with the scientific method? If seemingly solid experimental results gradually fade over time, how are we supposed to discern the difference between truth and falsity? Can we ever really “know” anything? The implications of this decline effect seem to be both disturbing and baffling.
However, Lehrer’s article doesn’t exactly hold up under close examination. In fact, it received an almost immediate backlash from the scientific community, expressing nothing short of outrage at the conclusions drawn in the New Yorker article. Here are some explanations that scientists have given to refute the mysterious and disturbing decline effect that Lehrer presents in his article:
1) Regression to the Mean: this refers to the statistical “averaging out” of data over time. For example, in the case of Schooler’s study, there might have been some factor at play which the experimenter was not aware of, but that nevertheless affected the experimental results; this could be anything from a demographics factor (age, race, gender), to something almost unnoticeable, such as the color of the walls in the room that the subjects were taking the test in. This factor could skew the data a certain way, showing results that seem significant but may simply be an anomaly. But, if this experiment is repeated a number of times after the initial experiment, the skewed data from the first experiment will even itself out, and effectively disappear. This may be a less intriguing, but perhaps more reasonable, explanation for the decline effect as it appears in experiments such as Schooler’s.
2) Publication and Psychological Biases- the apparent decline effect can also be attributed to the fact that not all experimental results have a balanced chance of being brought to public awareness. Science and medical journals are far more likely to publish experiments that have positive results that seem to be groundbreaking than negative ones that seem to rain on the scientific breakthrough parade. Therefore, an experiment that gets a lot of press may not have solid results at all, which would be apparent in any immediate replications of the study. However, these replications may not be publicized, and therefore the shakiness of the initial experiment is not exposed until years later, when it has been thoroughly cemented as scientific “fact”. Also, the experimenters themselves, being human after all, may unintentionally, perhaps even subconsciously, skew their own data to essentially see what they want to see. The availability error is a term describing a human being’s tendency to only see information that is most psychologically available to them. Say you need a car and purchase a Toyota Tercel; suddenly, there seem to be Toyota Tercels everywhere on the road when previously you hardly saw any. This is because, by way of ownership, the Tercel has become more psychologically available: you notice it more. The same can happen, despite their best efforts, to scientists. Say you are looking for evidence to support your theory that animals mate more frequently with members of their species that are symmetrical. You are immediately more inclined to notice animals with symmetry in your study, which means you may end up disregarding evidence you find that doesn’t support your theory. These are biases that only become apparent when they are replicated, perhaps by other scientists, years later.
3) Many rebuttals to Jonah Lehrer’s article have also cited this fact as an example of why the decline effect is intuitive, rather than mysterious: the decline effect does not happen in physics, the area of science that most resides over the hard rules and facts that guide our reality. Instead, it is happening in fields such as psychology, medicine, and ecology. The subjects that these fields study are constantly changing: it is their nature. The generation in which our parents were born are psychologically, and even in some respects physiologically different than our own. How can we expect drugs and pharmaceuticals that were effective fifty years ago to affect the next generation in exactly the same way? In this way, the decline effect is true, but it is not at all baffling when we take the time to think about it. The nature of human beings, and every other organism on this planet, is not to remain static. We are constantly changing and evolving, and science accounts for this. This is why subjects continue to be studied, and why experiments are replicated. Because the world changes, and we are just trying to keep up with it. There is nothing inherently wrong with the decline effect, it is simply part of the scientific process.
So, what does this leave us with? On one hand, we have an interpretation of the decline effect as a mysterious and troubling phenomenon, one that leaves us questioning the very nature of knowledge and reality. On the other, a considerably less seductive explanation whose collective response to Lehrer’s article seemed to be “duh. We know.” But although the decline effect seems to be less mysterious than some may spin it, what certainly remains true is that the world itself is still a mysterious place. Sciences such as ecology and psychology, as we said, run up against the decline effect more often. It does seem plausible that by observing ourselves, and the physical world around us, we are in fact changing what we study in subtle ways. We are, after all, part of what were are observing.  Spooky? Not quite. Fascinating? Definitely.
