Just Because You Don't Like a Study Doesn't Mean It Is Wrong
Last week, the blog world picked up an LA Times news story about a scientific study that had just been published in an academic journal, Communication, Culture and Critique. The title of the study: "Women (Not) Watching Women: Leisure Time, Television, and Implications for Televised Coverage of Women's Sports."
The LAT's headline: "Wives watch sports for husband's sake, study reports." A representative sarcastic blog headline: "Fun Fact: Girls Only Like Sports to Impress Boys!" And therein lies the complete breakdown in the communication chain.
Neither journalists nor bloggers are scientists (usually). Even many science beat writers at top-tier publications lack the scientific expertise to independently analyze and interpret the myriad academic studies that flow forth from the world of science. This is okay. Most people who went to school long enough to be scientists are scientists, not news writers. Someone who writes news stories or blog posts about scientific studies is not required to have a Ph.D. in every scientific field. They are only required to know how to find someone who does know the answers to relevant questions to speak to, or, even simpler, to be able to accurately translate the meaning of a study into language that everyone can understand. Nobody expects bloggers to be so scientifically savvy that they can independently spot scientific or mathematical errors. People do not turn to general-interest news writers and bloggers for savvy scientific analysis. This is not a job requirement for bloggers. You just have to be able to say what something says. You can joke about it. You can speculate on what it means. (Or, as I did, you can go steadfastly absurd in order to tacitly acknowledge that you are not a scientific authority.) But you can't make it say something that it doesn't.
Perhaps it was the LAT's definitive and wildly oversimplified headline that started the downhill unscientific snowball. The Atlantic Wire weighed in with "Do Wives Really Only Watch Sports Because of Their Husbands?" questioning whether a study with only 19 participants could say something definitive. (Spoiler: no, although that did not stop the LAT's headline writer.) Woman-centric blogs like Jezebel and XO Jane went on to mock the study as "stupid" and "TOTAL BULLSHIT."
Was it really possible that a 21-page double-blind, peer-reviewed academic study with 80 separate cited sources could be so TOTALLY BULLSHIT that a typical non-Ph.D. non-scientist writing on a blog could dismiss it out of hand simply by considering what they believed its conclusions to be? I decided to ask Erin Whiteside, a Ph.D and assistant professor of Communications at the University of Tennessee Knoxville, one of the study's two authors (and a "regular Gawker and Jezebel reader," sadly). You may be surprised to learn that it appears that the media and meta-media, through a combination of laziness, sensationalism, and ignorance, may have misstated the results of a scientific study!
"I would like to point out that nowhere in the paper do we assert that *All women watch sports to be with their husbands," Whiteside told me. "As you mention, feminist blogs have been critiquing the study, (and I would, too, based on what they think the findings say) but it is based in feminist theory and very much thinking about gender roles and the production of gendered subjectivities through everyday interactions (including watching sports) in the home. What we do say is that many of the women we talked to said they love watching sports because of the teams they favor, but *also like watching sports because it is a time to connect with their families. We expand on this point in the paper."
In the paper! The paper itself, it turns out, is much more nuanced and reasonable than it summary in the newspaper. It notes that while female participation in sports is up, women's sports leagues still have trouble drawing spectators. Why? What a good question for an academic research project, right? The paper delves into patterns of media coverage, gender theory, and academic theories in search of answers. The study's authors conducted focus groups with women in order to get a sense of what they themselves had to say. It is, all in all, a better-researched, better-sourced, and far more labor-intensive piece of work than any blog post that has ever been written on any scientific topic.
As to the much-criticized fact that the study used a relatively small number of women, Whiteside explained: "This is a qualitative project — hence group interviews with 19 women— as opposed to a wide-scale survey using a random sample. The strength of such an approach is it allows a researcher to go beyond describing a given social phenomena (e.g. women's sports lack a fan base) to explaining it. In fact, research that begins from the standpoint and everyday experiences of women is often considered a feminist approach to research (as opposed to male-centric modes 'discovery') because it privileges women's voices and experiences in the production of knowledge." Whiteside says that while standard numerical quantitative studies are based on set categories of identity (age, gender, etc.), "Our study thinks about how those identity categories are created... I would argue that identity categories are inherently restrictive and a product of power relations; once we assign an identity to a person, there is an expectation toward behavior. This is admittedly an abstract concept and difficult to report in a 300-word story, especially in a world in which measurable, objectively observed data is considered most worthy. This latter point reveals how epistemology (philosophy about what counts as knowledge) may figure into women's oppression. Although feminists hedge from assigning one method as THE feminist method, many would argue that qualitative research in the form of interviews, etc. can be feminist in that it situates the production of knowledge in women's everyday experiences and in their own voices."
Cruel irony. Not only were the purposes and conclusions of this study mischaracterized, but that mischaracterization led to widespread derision from feminist blogs over methodologies that were explicitly feminist in nature. It is useful, when we decide to offhandedly offer our uninformed analysis about a study, to reflect upon the fact that the author of the study has probably already considered all of these objections—which are, after all, usually little more than the first knee-jerk reaction that pops into our heads. Whiteside, for example, is a Ph.D and an academic and is of course aware that it is impossible to generalize from 19 women to the entire world. "Applying any research 'finding' to a wider population is only* appropriate when the study draws from a random sample that is sufficiently representative of that wider population– which we did not do, nor try to do," she says. "But, again, our goal was not to describe a social trend at a meta-level but to interpret it, something for which qualitative research provides an especially useful toolkit."
It is possible for scientific or academic research studies to be wrong. But they must be wrong for a reason. The fact that their conclusions just rub us the wrong way is not a reason. It is possible that our instincts have been proven wrong, by the science at hand. It is also possible, as we have seen, that we do not actually understand the science at hand—possibly because we're relying on a secondhand interpretation of it, which has been misstated or exaggerated or twisted in some way. Let's not kid ourselves into thinking that we have spotted what the professionals could not, due to our own incisive brilliance, as professional writers.
Sometimes it's better to just stick to jokes.
[Image by Jim Cooke]