Freak or not... bad science is bad science..

Scientists have jumped on the Economists who wrote Freakanomics because of their chapter on Global Warming in there follow up book SuperFreakonomics.  Here is a pretty good wrap up via The Economist Expertise, and "expertise":

STEVEN LEVITT and Stephen Dubner have reprised their Freakonomics roles in "SuperFreakonomics", which is due out in just a few days. As it happens, some chapters from the book are already in circulation, including one on "global cooling" which has drawn quite a bit of criticism, including responses from respected climate scientists and environmental economists. Mssrs Dubner and Levitt have attempted to respond, but I find the criticism of their work to be quite compelling; it appears that the authors made a number of outright errors and generally opted to present their case in a manner aimed more at provoking controversy than informing, which is highly irresponsible given the subject matter. (You can follow links here, here, and here.)

One interesting point that a number of critics have made is that the Freakonomists' writing seems to be vastly different in quality when using research that Mr Levitt has himself produced (as was the case in the first book) than when addressing topics he has not previously discussed. This isn't all that difficult to understand; Mr Levitt no doubt chooses his research topics based on things like the quality of data available rather than the likelihood of a particular question being "hot button". And there are also very different publication standards for academic work than there are for popular publications.

But that doesn't mean that there can't be reputational blowback from a disastrous popular publication. Given that, it's a shame that Mr Levitt opted to dedicate a chapter to a subject—climate change—on which he doesn't have subject-area expertise and on which topic he doesn't use his methodological expertise. Instead, he simply deploys his reputational "expertise". This may make him some money, but it won't come without real costs.

And it's not like he didn't have plenty of real academic work to use. Matt Yglesias directs readers to an interesting paper co-written by Mr Levitt on the use of minimax strategies in professional sports. And just today, this paper popped up at NBER (co-written by Steven Levitt and Roland Fryer):

We document and analyze the emergence of a substantial gender gap in mathematics in the early years of schooling using a large, recent, and nationally representative panel of children in the United States. There are no mean differences between boys and girls upon entry to school, but girls lose more than two-tenths of a standard deviation relative to boys over the first six years of school. The ground lost by girls relative to boys is roughly half as large as the black-white test score gap that appears over these same ages. We document the presence of this gender math gap across every strata of society. We explore a wide range of possible explanations in the U.S. data, including less investment by girls in math, low parental expectations, and biased tests, but find little support for any of these theories. Moving to cross-country comparisons, we find that earlier results linking the gender gap in math to measures of gender equality are sensitive to the inclusion of Muslim countries, where in spite of women’s low status, there is little or no gender gap in math. 

Very interesting stuff. With the reputation Mr Levitt built for himself through the first book and his New York Times blog, he could easily have made himself millions of dollars with a sequel focused on these kinds of questions. But instead he and Mr Dubner wrote a chapter that differs dramatically in style and method from what we've all come to understand as the Freakonomics way.

That is an thought-provoking phenomenon in and of itself. I suspect Mr Levitt could say something quite interesting about the incentives facing academics with popular brands, if he weren't quite so involved with a natural experiment of his own making just now.

Posted via email from Jim Nichols