Dec 102013
 
(Continued from part II: slippery slope)

(Ir)responsible Inference

In the article, “In Defense of ”Rape“ Fantasies,” Tracy Clark Flory quotes the Atlantic Wire:

“The connection between actual real-life violence and porn is blurry at best — India, which bans all forms of porn, has been in the news thanks to a rash of brutal rapes. Meanwhile, in the United States the incidence of rape declined 85 percent over a period of 25 years while access to pornography has increased.”

The inference is pretty clear. It suggests there’s an inverse relationship between pornography and sexual violence — as access to porn increases, sexual violence decreases.

In this case, the caveat — “The connection between actual real-life violence and porn is blurry at best” — wasn’t enough. It didn’t stop the inference from being carried on and reproduced by any number of bloggers, writers, and reporters. The direct quote has been reproduced by NPR, Wonkette, Salon, and who knows how many other sources have paraphrased it.

And it’s true. In the US, porn is up and rape is down. The problem is that the statistics show correlation, not causation. The statistics don’t prove any sort of relationship between the two. There are lots of other correlations that may also contribute to the decrease in rapes. Allow me to suggest a few:

In the US, incidence of rape declined 85% in the past 25 years…

…while efforts for victims advocacy have increased.
…while investigations and prosecutions of sexual assault have increased.
…while programs to educate people about rape myths have increased.
…while women’s groups have sought to raise awareness about sexual assault.

Any/all of the above may prove some correlation, but none of them prove causation.

The “Source”

After reading the quote in Salon, I wanted to find out where it came from.

In Salon, Flory cites the Atlantic Wire. The Atlantic Wire cites The New York Times. Here’s the full quote:

Further, the connection between actual real-life violence and porn is blurry at best — India, which bans all forms of porn, has been in the news thanks to a rash of brutal rapes. Meanwhile, in the United States the incidence of rape declined 85 percent over a period of 25 years while access to pornography has increased, The New York Times reported.

It sounds as if that juxtaposition of rape statistics in India vs. the United States came from the New York Times. It didn’t (at least not in the way the Atlantic Wire presented it). And it’s not even the New York Times, it’s a blog(?) hosted on the International New York Times website called India Ink.[1]

According to India Ink:

“The porn industry also has many defenders. The incidence of rape in the United States has declined 85 percent over a period of 25 years while access to pornography has increased, according to research by Anthony D’Amato and Glenn Reynolds, both law professors.”

Not quite. The “research” is by Anthony D’Amato, not Glenn Reynolds.

And I wouldn’t call it “research.” India Ink links to a ‘working paper’ titled “Porn Up, Rape Down,” by Anthony D’Amato.

It’s just over six pages (and includes tables and charts), and I strongly suggest reading it if you’re curious about it’s credibility. (In fact, please read it and tell me if his statements on correlation and causation are as confusing and misleading as I think they are.)

In the paper, D’Amato juxtaposes statistics on increasing access to pornography and decreasing incidence of rape. He uses this as a foundation on which to forward his own personal theory:

“There is, however, one social factor that correlates almost exactly with the rape statitistics [sic]. The American public is probably not ready to believe it. My theory is that the sharp rise in access to pornography accounts for the decline in rape. The correlation is inverse: the more pornography, the less rape.” (3)

Here, he calls it a correlation, but then he suggests one “accounts for” the other. That’s not how it works — correlation does not imply causation — but D’Amato implies it anyway.

Later on, he says it himself: “proof of correlation is not the same thing as causation. If autumn regularly precedes winter, that doesn’t mean that autumn causes winter” (4). He doesn’t explain, but instead, talks about a colleagues study on abortion and explains his interest in rape statistics (which, after reading another ‘article’ of D’Amato’s, seems like a grudge against the Meese Commission).

And it gets weirder from there.

“If pornography does not produce rape, I thought, then maybe it reduces rape. But no one apparently had any incentive to investigate the latter proposition. But the just-released rape statistics provide the necessary evidence.” (5)

He thought? Maybe it reduces rape? Evidence?

There it is. D’Amato states the statistics are “evidence” that pornography reduces rape. He states it, but that doesn’t make it true. D’Amato never proves causation.

Despite this, maddeningly, he closes his paper with this:

“I am sure there will be other explanations forthcoming as to why access to pornography is the most important causal factor in the decline of rape. Once one accepts the observation that there is a precise negative correlation between the two, the rest can safely be left to the imagination.” (6)

There it is again. “…pornography is the most important causal factor in the decline of rape.” (I have no idea what he means by “the rest can safely be left to the imagination.”)

In brief, D’Amato provides statistics that show correlation, states that correlation isn’t causation, and then calls it causation anyway.

For the record, I have no doubt the statistics he uses are credible, but his text isn’t, and it shouldn’t have been used to provide ‘support’ for the inference that more porn leads to less rape, an inference that, unfortunately, is all over the place.

(If you’re interested in the D’Amato credibility rabbit hole, you can read all of what I found here. In brief, “Porn Up, Rape Down” is a ‘working paper’ that has not been published by any credible journal. It hasn’t really been published at all — D’Amato simply uploaded it to a “research network.” It’s not been peer-reviewed or refereed. It contains typographical errors, misleading citations, and missing links. While one imagines D’Amato’s expertise is in some area of law, his paper doesn’t actually mention law. And on top of all that, he seems to have a personal grudge against the Meese Commission.)

More Criticism of D’Amato

I’m not the only one to see problems with D’Amato’s assertion.

Ann Bartow, of Feminist Law Professors, admits she isn’t “an expert in the social science research related to the causes of rape,” but suggests “[…] there are some things [D’Amato’s] paper doesn’t address.” She outlines seven issues/questions, and among them are the following:

“There is some evidence that watching violent acts increases acts of violence among spectators. I know there are concerns about these studies too, as with most social science research, but assuming for a moment that this is true, why would rape be so diametrically different? Why would watching people hitting each other lead to increased hitting, but watching penetrative sex lead to less pursuit of penetrative sex? Is there any evidence that watching performance of violent acts leads to less violence by observers?”
“Can all porn effect people the same way? Some porn depicts consensual sex, other porn depicts forced sex. If porn does effect behavior, wouldn’t different kinds of porn have different kinds of effects?”
“To extend the ”porn equals less rape“ thesis in a logical but very unappealing way: Would an increase in the availablity[sic] of child porn lead to less pedophilia? Any evidence of this? And if not, why would porn decrease some undesirable sexual behaviors but not others?”

Ampersand’s critique, “No, Porn Doesn’t Prevent Rape,” identifies three problems with D’Amato’s text:

  1. “During recent years, the NCVS [National Crime Victimization Survey] has found a steep decline in all violent crime, not just rape. It seems likely that whatever’s causing the decline in all violent crime measured by the NCVS, is also causing the decline in rape measured by the NCVS; but it seems unlikely that pornography reduces all violent crime.”
  2. “Particularly notable is this study, by the Bureau of Justice Statistics, which directly compared the NCVS’s methodology for measuring rape prevalence with modern ”best practice“ survey design – and found that the NCVS vastly undercounted rape.”
  3. “D’Amato has no measurement of porn prevalence other than internet access, nor does he do any real statistical analysis. In contrast, studies with sophisticated statistical analysis and more accurate measures of porn usage – such as the study published in Four Theories of Rape in American Society – tend to find that porn usage has little or no correlation with rape prevalence.”

My Point (or, The Bottom Line for Responsible Debate):

If you’re going to cite studies, statistics, make inferences, or suggest your readers draw their own, know what you’re referencing… particularly when it’s an important issue and particularly when the “evidence” isn’t clear.

So what of that ‘evidence’? The more I read, the more I believe there’s no way to reliably prove a direct, causal relationship, even if there is one.

With that said, I think opponents of the UK rape porn ban are being reductive in their repetition of what supports their position. While it’s true — there are no statistics to prove rape porn causes sexual assault — that’s only part of the truth. First, it seems finding evidence would be damn-near impossible. Second, there are other factors, cultural ones, that are potentially significant and worth considering.

In my next post, I’ll present some studies that address cultural factors. While they have significant limitations (ones I’ll point out myself), I believe they’re compelling enough to deserve consideration as part of the larger debate.

 


[1] According to the website’s “About India Ink” description: “This report on India from the journalists of The New York Times and a pool of talented writers in India and beyond provides unbiased, authoritative reporting on the country and its place in the world.”

 

  9 Responses to “part III: statistics and irresponsible inference”

  1. He seems eligible for one of those Ignoble Awards! You’ve also exposed how sloppy media coverage of science is (see also, Katie Couric and HPV vaccine). Any chance you used to do peer review for journals? :) Small words like explain, account, effect, impact – all tools in the bait and switch from correlation to causation. Personally I think the dramatic decline in rape is due to the nearly simultaneous drop in consumption of Fig Newtons and Mallomars… p<0.00001.

  2. Just a couple of comments from my fields, that have relatively little to do with the post series. Mostly, this post pushed some of my buttons!

    First, it is well established that childhood viewing of aggressive television leads to higher levels of aggressive behavior. Last I checked, there was far less evidence in adults. One of the primary effects I’m aware of in experimental manipulation of adults’ (non-violent) porn viewing is reduced satisfaction with their partners, at least in the short term.

    Second, it is certainly possible to make valid inferences of causation from observational data–and, as you cite above, longitudinal data are only part of the answer. However, the methods to do so and the assumptions involved are fairly technical. I have not read the article, but, from your presentation, I’m doubtful those methods were employed here.

    Third, the classic three cases of “the causation underlying a correlation” are that X causes Y, Y causes X, and something else causes both X and Y (and, again, in observational data, sequence in time in is not the key). The argument you’re responding to is that porn causes less rape (let’s say that’s X causes Y). It seems unlikely on the face that reduced rape increases porn viewing (Y causes X). The major alternative explanations to X->Y would therefore be third-variable causation of both. The note on the NCVS is suggestive that that’s at least part of the mechanism.

    In sum, media reports of science do almost nothing to report the subtleties, in whichever direction, and are usually based on, at best, press releases from the author’s institution. And there’s plenty to indicate that peer-reviewed articles are often also deeply flawed, though presumably significantly less so or less frequently than working drafts.

    My $0.37 of contribution.

  3. I appreciated this carefully researched post. Not “dumb” at all!

  4. Perhaps there are no reliable statistics as to whether or not rape porn actually causes criminal rape, simply because psychological motives cannot be measured emperically. They are rather like opinion polls. While opinion polls can give a particular politician or a party faction a rough idea what the opinion of the populance is, they cannot with any absolute certainty determine the outcome of an election. When it comes to human sexuality, psychological polls and statistics seem to have little reflection with reality, and hence much of the statistics are scewed, because they are based on moods-du-jour, lies, social pressure, or false interpretation of the subject interviewed.

  5. It’s just over six pages (and includes tables and charts), and I strongly suggest reading it if you’re curious about it’s credibility. (In fact, please read it and tell me if his statements on correlation and causation are as confusing and misleading as I think they are.)

    I did download and read that article and in it, he purports to show that the increased availability of pornography is a contributing factor in the decline of the rate of forcible rape. However, I found that the evidence he presented to back his theory to be tenuous, at best.

    The first chart shows the decline in the reported incidents of rape between 1973 and 2004, while the rise in the availability of pornography is shown in charts showing internet availability as compared to the rate of rape in eight states. The four highest in internet availability VS the rate of rape are compared to the for lowest with their rate of rape. It is on the basis of the comparison between these data that he reaches his conclusion.

    I have two problems with this. First of all, one of the states he sites (Alaska) is anomalous to his conclusion. It shows that in spite of a rise in internet usage, the rate of rape also rises and so, he states that it’s very small population makes it a poor candidate for a stand alone evaluation and combines it with the other states in order to demonstrate his point. Using that same reasoning, I submit that his sample, of only eight American states is too small to use for such a study. What about the rest of the United States? Would that same trend hold true if the rest of country were included in his results?

    For that matter, does the same hold true for the rest of the internet connected world? If the increased availability of porn results in a decline in the incidents of forcible rape in the United States, it stands to reason that such a trend would be reflected worldwide, yet he presents no evidence to that effect. I would conclude that for the kind of theory he is putting forth, the evidence he presents is statistically insignificant.

    Secondly, he seems make the assumption that internet usage = pornography. Using that same formula, I could just as easily make the case that email, social networking, or online game playing have produced the same results. This is not to say that pornography does not account for a significant amount of web traffic, it does. However, to use an increase in internet usage as any kind of “proof” that more pornography leads to fewer rapes is purely speculation.

    Mind you, I am not saying definitively invalidating the claim that the increased availability of pornography will reduce the indecent of rape, but if there is a case to be made, Mr D’Amato certainly hasn’t made it.

  6. Naturally no post of mine is complete without at least one typo.

  7. You are correct, that paper is essentially useless. Aside from the fact that correlation and causation are two different things, we need to contend with the ecological fallacy: states with lower Internet access (which we apparently think means less porn, evidence not cited) have higher reported rape rates? Okay. This doesn’t look into the individual reported rates among men who watch more porn vs. those that do not. I don’t see analysis by income, education, rural versus urban environment, any number of other potential confounders. Changes in data collection methods and oversights regarding factors that may cause victims of violent crime to underreport also require consideration. NCVS data is the best long-term data we have, but there is evidence that interview method and environment may affect report rates (Yu and Stasny, Bayesian Models to Adjust for Response Bias in Survey Data for Estimating Rape and Domestic Violence Rates from the NCVS, freely available here: http://arxiv.org/pdf/0801.3442.pdf). There’s pages more to be said, but I’m tired and it’s mostly quite visible to a critical reader. The D’Atamo paper presents no evidence whatsoever. If there is a connection, positive or negative, between porn of any kind and sexual violence, it has not been shown here.

  8. Clearly there are issues with that piece of research however there is other research based on studies done of Czechoslovakia. Sadly, because it is a genuine piece of research I can not access it without paying for it however I have seen a number or articles which cite sections from it.

    http://link.springer.com/article/10.1007/s10508-010-9696-y

    • This is the article you referenced:

      Diamond, Milton, Eva Jozifkova, and Petr Weiss. “Pornography and Sex Crimes in the Czech Republic.” Archives of Sexual Behavior 40.5 (2010): 1037–1043.

      In that same journal, the article immediately following it is a criticism of it’s findings:

      Kingston, Drew A., and Neil M. Malamuth. “Problems with Aggregate Data and the Importance of Individual Differences in the Study of Pornography and Sexual Aggression: Comment on Diamond, Jozifkova, and Weiss (2010).” Archives of Sexual Behavior 40.5 (2011): 1045–1048.

 Leave a Reply