Psych Major Ramblings

I wanted to study others and behaviours. Now I see through everyone and everything.

Here’s An Idea. Smith et al. 2016

Terrible Idea; a comment on Smith et al. 2016. Johnson 2016.

You’re Wrong Too; a response to Johnson 2016. Nelson 2016.

Source

Another person in the notes linked an additional article that contained the full study, and these are the questions that were included in the study:

A couple of problems right off the bat. The first one that jumps out is #6: "The Earth is between 5,000 and 10,000 years old." In this study, the correct answer would be that this is a statement of fact, because it is, well, stated as a fact. And according to the paper in question:

A fact is a statement that can be verified. It can be proven to be true or false through objective evidence.

That's lovely and all, but nowhere in this study does it say that the participants were informed that this is definition of "statement of fact" that the study was using. It's not exactly common knowledge that "statement of fact" and "fact" do not actually mean the same thing. A statement of fact is when someone says something that they believe is true. A fact is something that is true.

So, anyone who participated in this study who interpreted "statement of fact" to mean "fact" (and also wasn't a young-Earth conspiracy theorist) would have gotten this question wrong, because they know that Earth is much older than 10,000 years.

Another somewhat less obvious problem is in question 2, which the authors of the study address as such:

First, as noted by Firey (2018), item 2 perhaps could be categorized as a statement of opinion because the phrase “a significant portion” is subjective. We repeated Pew’s coding on our survey, and solid majorities on both surveys rated the item as a statement of fact. In retrospect, we would categorize the statement as borderline (see note 4 above). By the end of 2017, ISIS had lost 95% of its territory (Wilson Center, 2019). Because such an overwhelming loss would be difficult to deem insignificant, we retain the item as a statement of fact for present purposes.

Yes, I agree that nobody could reasonably argue that a 95% loss isn't significant. However, unless the study specifically recruited participants who were extremely knowledgable about statistics relating to ISIS activity, the participants have no way of knowing that 95% is the amount in question. That "significant portion" could be anything!

Like, let's say that ISIS had lost 25% of its territory. That's a lot! That's over 10,000 square kilometers that were no longer under ISIS control. So one could argue that this is a significant portion. But also, that's only a fraction of their territory. ISIS still has control over the majority of the territory they'd had previously - three times as much as the amount they'd lost. So one could also argue that this is an insignifant portion.

If I had been a participant in this study, I would have marked this as a statement of opinion. And thus I would have become part of the 95% who apparently can't differentiate between fact and opinion.

But, I was not a study participant. So, who was? What does this study say about its participants?

Well, here's what the study says about it:

Data are from a national online survey we designed. The survey was fielded by YouGov from March 9 to March 14, 2019. There are 2,500 respondents.

That's it. No information about how participants were recruited or invited to take the survey. Nothing about what quality assurance methods were used to make sure that participants were following instructions, or that computer error didn't interfere with the data.

Nothing about incentives given for the completion of the survey.

Oh, yes, that's something very important to note. You see, YouGov is not an academic research firm. It's a marketing research firm. It functions similarly to sites like Swagbucks or Survey Junkies. And this is how it markets itself to participants when you search for YouGov in Google:

That's exact marketing used by Swagbucks. I use Swagbucks. I'm a pretty active user, in fact. I answer surveys and play mobile games to collect points, then redeem them, usually for Wal-Mart gift cards so I can save on groceries. I know how sites like these work.

And I also know that when I just want to get a survey completed, I bullshit. I just click whichever multiple choice option is closest to my cursor until I'm done and can get the points.

Guess what else I do: I lie. Sometimes a survey is listed on the site that's worth a lot of points but is only looking for people of a certain demographic. And I take it anyway. I give them whatever info is needed to get my points and get those gift cards. I have had a grand total of one cup of coffee in my life and hated it. But whenever a survey asks me if I have ever bought a certain brand of coffee, I say yes, and I review it.

Oh yeah, that's another point: there is no demographic information given about this study's participants. We know that participants were at least asked for their age and political affiliation ("Multivariate models include controls for age and partisanship") but we, the readers, are never told what that spread turned out to be. We just have to take their word for it that YouGov's sampling methodology was sound.

If I had turned in this research paper for review, with this Methods section and this survey, my advisor would have slapped me.

So, what is my point? Why did I spend an hour writing this response to a tumblr post? Well, because the findings of this study may be bullshit, but this whole post brings to light another big problem in the world of media and journalism.

The University of Illinois published this sloppy-ass study that I would have been embarassed to hand in as a 10-point assignment. This survey made it through the internal review board, a process that I know from experience can take months and dozens of rounds of review. It was then peer reviewed by Harvard, the school that people point to as the epitome of academic prestige full of super geniuses, and added to their library. Then it was picked up by a Washington DC online newspaper and tweeted about by a member of the House of Representatives.

And finally, it was posted here on Tumblr, uncritically.

I don't know what percentage of Americans can't tell fact from opinion. And reading this study is not going to give me an accurate answer, because the study's design and conclusions the authors reached are a mess.

Excessive production of unusually thick mucus, as it turns out. It was inhibiting my ability to breath while I slept, which was harming my sleep quality. My ENT was impressed that my lungs were strong enough to breathe through it most of the time.

someone said that if you study it, then you're going to get a new perspective and understand why the field is like that and why you shouldn't be so doubtful of it, that it's not psychology's fault that some bad apples exist and ruin it for their pacients

i study it. it's the 4th year. im getting a master's degree in clinical psychology, just like them. but the more i study, the more in contact i am with it, the more i realize how unfair this whole system is, how, no matter how good your intention, the fact that it's based in outdated ideas is going to end up hurting people, one way or another. everytime i go to class, someone, the majority of people, is ready to point a finger at a child and blame them for making their parents' lives hell, especially if there's the slightest hint of autism. do they even think about how they might harm them?

At the same time, people with ADHD also have fears that are based on real events in their lives. People with ADHD nervous systems are consistently inconsistent. The person is never sure that her abilities and intellect will show up when they are needed. Not being able to measure up at the job or at school, or in social circles is humiliating. It is understandable that people with ADHD live with persistent fear. These fears are real, so they do not indicate an anxiety disorder.

OTC pain meds are not a devil’s bargain or a moral failing. They are a tool to reduce unneeded suffering. They do not destroy your organs if taken correctly, and there is no reward for the people who take the fewest pills in life.

@cautionramen I know the answer to that!

So, acetaminophen/ paracetamol/ Tylenol is kinda garbage on its own because it doesn't work directly on the site of pain. Instead, it affects the BRAIN- it temporarily dials down your brains pain receptors. (Your body can heal itself better when it isn't in pain.) (Weirdly, this also means it can cause a temporary drop in feelings of empathy and existential dread, because your brain doesn't really distinguish physical and emotional pain) Because of this, it has fewer side effects than NSAIDs, but is less powerful.

However, it's an amazing buff FOR NSAIDS (and opiods. It gets mixed with a lot of opiods too.) Because it makes your brain chill out, the other meds can work more effectively at a lower dose, which means fewer side effects and lower chance of overdose. (So taking 200mg of Tylenol WITH 200 MG of ibuprofen is safer and more effective than taking 400 of either of them alone.)

and from what I've been reading, what's shown in the video may not even be postural sway at all?

Postural sway, as far as I can tell, consists of generally tiny movements used to maintain balance, that just aren't as tiny in the case of this small sample of adults with ADHD. The tests in the study were performed on people standing still (on Wii balance boards.) I admittedly know practically nothing about this, but it seems to me like people who have issues maintaining balance while standing still are not going to be the people swooping and leaning dramatically as they walk past things.

Here's a (possibly) relevant passage fro the paper in the NIH link above

Overall, these data show that adults with ADHD manifest subtle abnormalities in postural sway. Other reports have linked hospital admissions from accidents with the presence of adult ADHD, and many of these patients were referred to as “clumsy” (Kaya et al., 2008). Though the reasons for such accidents are unknown, poor balance combined with ADHD inattention and impulsivity could play a role. These data could motivate further exploration of this explanation. Increasing awareness of this relatively subtle motor control abnormality could be of practical significance in helping to reduce accident rates in ADHD individuals.

I absolutely do what's shown in the video, but feel like I do it out of having mis-estimated where the thing is and deciding it would be easier to lean around it instead of getting my feet to change where I had already decided to put them, because that probably would make me stumble (so that does check the 'inattention' and 'impulsivity' parts)

Also, the way things are phrased in the screencap of search results led me, at least, to think they were saying that lower cerebellar volume would mean more sway, but it's the opposite. The paper in the NIH link has a paragraph where it kinda meanders off into the way cerebellar volume relates to ballerinas and musicians, concluding that maybe the smaller cerebellar volume is ~more efficient~

So I majored in psychology in undergrad, right, and when you take psych classes they almost always require you to do a certain number of hours of being an experimental research subject, right, because professors have to publish and need someone they can force into their studies without paying, right?

And like that's fine if it's 4 hours for one class, but when you're doing an entire psych major in two years like I did, that's a LOT of hours of research studies.

And we all dreamt of getting in those fun social psych experiments where they fuck with your head, right? And not the ones where they make you sit in front of a laptop and do math for an hour. (They made me DO MATH Y'ALL. "This is an experiment in how well you learn under certain conditions." Conclusion: you suck at this.)

Anyway, you'd get funny things like one time I turned up on campus on a freaking SATURDAY for a study and sat around in a courtyard with like 30 other people while nothing happened and all of us talking about like "hey, uh, do you think THIS is the study? Like to see if we leave?" But no, just no one turned up for us and we didn't get our freaking hours.

But one time I DID end up in one of those freaky social psych studies. So to be fair, I had insider knowledge, because again, psych major. So I signed up intentionally to my social psych profs study. Hoping at least it wouldn't be math.

And I arrive and am greeted by...the TA for my social psych class wearing a fat suit.

It's like IDK April or something and Texas and 80+ degrees and she's in a long skirt and a sweatshirt. And also she's MY TA. I recognize her. I even say "Oh, hi!" like I would when seeing someone I know on campus. And she greets me like she vaguely recognizes me. But, normally she is not approximately 280-300lbs.

So, I'm, like REALLY REALLY sure it's a fat suit. Like....99.5% sure.

But not 100%.

And so what the FUCK do you SAY to THAT? Well, obviously you don't say FUCKING SHIT. You pretend nothing is fucking weird, right? Because the very, very small chance that you are WRONG and this isn't a normally thin girl in a fatsuit but a real person who is shaped like that is still...NOMINALLY THERE and OMG what if you comment on it and you're WRONG?

So I get ushered into this little room and shown a bunch of pictures of people and asked to rate them on various things like competency and attractiveness, this is normal social psych survey stuff, except I'm SURE the experiment is does the person in the room with you and their appearance change your ratings.

And the whole time I'm distracted as FUCK, cause I'm just sitting there thinking "am I wrong and this isn't my TA somehow? no, no, pretty much ALMOST certain I'm right...etc." And "why would anyone be wearing a sweatshirt in this weather/building if they WEREN'T wearing a fatsuit?"

But MAYBE the point of the experiment is "will this person call out an obvious fake fat person?" And should I do so or not? Is this social pressure to conform and not speak the truth I know? Should I say something? I, uh, may have had an unknown and untreated anxiety disorder at this point in my life so, yeah I'm LOSING MY MIND and probably acting like a FREAK.

So anyway, eventually I decide okay, obviously you can't say anything because yeah...but you will be debriefed once this is over and you'll FIND OUT THE TRUTH. You won't have this lingering doubt in your mind when this is over because they will debrief you.

If you don't know human research, debriefing is when, after all the experiment data collection is over, they inform the subject of what the topic of the research was and explain any tricks or deception or anything to them. (We played distracting music to see how you did on the test...) If there was any potential distress involved it should be dealt with by examiners, etc. Usually in practice you are handed a slip of paper that explains the purpose of the study and what you did.

Usually you don't care and barely read it. But I was dying to be debriefed. I wanted her to be like "yeah I am wearing a fatsuit" and me to be like "lol, yeah I know, cause like, I know you right?" And if the secret purpose of the experiment was actually "will you say something" then I will be told that and get to explain why I didn't.

Like sometimes debrief also involves follow-up questions that helps determine why you gave responses you did or whether you should actually have your data thrown out for some outlier reason (like the person faking being fat is my fucking TA).

So we get done with all the questions and leave the small room. And we're in the antechamber and the TA is like "okay, cool, thanks bye" and directs me to the door. And doesn't hand me a debrief slip or mention debriefing AT ALL.

And now I'm fucking SPOOKED. It's a TRICK and they want you to SAY SOMETHING and you're going to try to leave without saying anything and they will then stop you and debrief you. So I wait, for like...several seconds, waiting for her to remember debrief and just get stared at so I go "oh okay" and like stutter-step my way to the door of the office and like open it and turn back and she's already gone, and so I like, step out into the quad and am like "what. the. fuck."

And I literally stand there like "should I go back in and ask to be debriefed?" Literally I knew enough to know that THIS IS THE PURPOSE OF DEBRIEFING to not leave subjects wondering about shit like this and not leave them with nagging doubts and questions.

And the only really mysterious experiment of my life just failed to debrief me.

But of course I don't do anything but walk slowly away.

And it's now 20+ years later and I never did find anything out. Except you can DAMN well bet I confirmed at my next class that 1) yeah that was TOTALLY my fucking TA, I was right and 2) NO she was actually really slim.

So I'm sure it was about how a fatsuit (or *cough* sorry "attractiveness") of an interviewer changes responses to surveys. But I'm STILL mad I wasn't debriefed because it's fucking annoying and violates HRB standards and I could have gotten them in serious trouble over that by reporting it. And also my data should have been thrown out.

The weirdest thing by far about the “Why didn’t they just ask a [person who experiences that type of marginalization/trauma/adverse situation]” response is that, well, they did. That’s literally what they’re doing when they conduct research on that topic. Sure, research is a more formal and systematic way of asking people about their experiences, but it’s still a way.

And while researchers do tend to have all kinds of privilege relative to the people who participate in their studies, many researchers are also pushed to study certain kinds of oppression and marginalization because they’ve experienced it themselves. While I never did end up applying to a doctoral program, I did have a whole list of topics I wanted to study if I ever got there and many of them were informed directly by my own life. The reason researchers study “obvious” questions like “does fat-shaming hurt people” isn’t necessarily because they truly don’t know, but because 1) their personal anecdotal opinion isn’t exactly going to sway the scientific establishment and 2) establishing these basic facts in research allows them to build a foundation for future work and receive grant funding for that work. In my experience, researchers often strongly suspect that their hypothesis is true before they even begin conducting the study; if they didn’t, they might not even conduct it.

That’s why studies that investigate “obvious” social science questions are a good sign, not a bad one. They’re not a sign that clueless researchers have no idea about these basic things and can’t be bothered to ask a Real Marginalized Person; they’re a sign that researchers strongly suspect that these effects are happening but want to be able to make an even stronger case by including as many Real Marginalized People in the study as financially/logistically possible.

Well, for starters, the placebo effect is a real thing, and also where do you think the idea came from in the first place?

Fifty years ago everyone knew, and would swear to you by their personal experience, that paddling kids with a wooden spoon never did them any harm and, in fact, was absolutely necessary if you wanted to raise kids that had any respect for authority. 

Right now there are hundreds of people out there training horses who know, from their extensive personal experience, that aversive (aka punishment-style) discipline is absolutely central to horsemanship. Of course, repeated actual studies show they’re wrong. But they still know it from their own experience. 

We KNEW that dieting worked! As a society, we KNEW it was calories in calories out, one to one ratio, dead simple, you could see it all the time why would you need to test it? Except it turned out that when we did, it turned out to be a WHOLE LOT MORE COMPLICATED THAN THAT. 

Out there right now are all kinds of cops who know, from their own experience, that aggressive, tough-on-crime, jail-sentences-for-all methods are the only ones that work. They know it. This is their whole lives, they’ve lived it!  … They also appear to be dead wrong, by the data. 

We KNEW, at one point, that cigarettes were GOOD for asthma. They cleared the tubes! We KNEW that the human uterine lining is meant to make a warm, nurturing nest for the fertilized ovum to settle into! We knew all kinds of damn things. 

For that matter, it’s common-sense obvious to any kid on the playground that things that are heavier will fall faster than things that are lighter. We knew that once too. And everyone with the slightest common sense (many people say) can TELL that the world is more violent and dangerous now than it was in the 1950s.

Except it turns out absolutely none of this is true. We were wrong. In all of those cases the common sense, obvious, “anyone who has any experience with these things knows that” answers were absolutely wrong. But we didn’t find that out until we did the work. 

So yes a lot - a LOT - of the time these things really totally are “I’m pretty damn sure what the outcome is, so I’m going to study it for those reasons.” But we also do this work so that when we turn out to be wrong, we find out. 

(My field works a lot with child-development stuff. The current big mess is “screen-time”. Everyone - including such bodies as the American Association of Pediatricians, and so on - KNOWS that screen-time for kids under a certain age is bad for them! 

In 1909 Robert Millikan and Harvey Fletcher measured the charge on the electron in what’s know as “the Millikan oil drop experiment” (sorry, Harvey). They got it wrong. As Feynman told it:

It’s a little bit off because he had the incorrect value for the viscosity of air. It’s interesting to look at the history of measurements of the charge of an electron, after Millikan. If you plot them as a function of time, you find that one is a little bit bigger than Millikan’s, and the next one’s a little bit bigger than that, and the next one’s a little bit bigger than that, until finally they settle down to a number which is higher.

Why didn’t they discover the new number was higher right away? It’s a thing that scientists are ashamed of—this history—because it’s apparent that people did things like this: When they got a number that was too high above Millikan’s, they thought something must be wrong—and they would look for and find a reason why something might be wrong. When they got a number close to Millikan’s value they didn’t look so hard.

The contrast with the scans of the eighteen chronic PTSD patients with severe early-life trauma was startling. There was almost no activation of any of the self-sensing areas of the brain: The MPFC, the anterior cingulate, the parietal cortex, and the insula did not light up at all; the only area that showed a slight activation was the posterior cingulate, which is responsible for basic orientation in space.

There could be only one explanation for such results: In response to the trauma itself, and in coping with the dread that persisted long afterward, these patients had learned to shut down the brain areas that transmit the visceral feelings and emotions that accompany and define terror.

Yet in everyday life, those same brain areas are responsible for registering the entire range of emotions and sensations that form the foundation of our self-awareness, our sense of who we are. What we witnessed here was a tragic adaptation: In an effort to shut off terrifying sensations, they also deadened their capacity to feel fully alive.

The lack of self-awareness in victims of chronic childhood trauma is sometimes so profound that they cannot recognize themselves in a mirror. Brain scans show that this is not the result of mere inattention: The structures in charge of self-recognition may be knocked out along with the structures related to self-experience.