The single worst survey question I’ve ever seen.

This makes me spit with rage. Some economists at (where else) George Mason University conducted this survey and claimed to find that liberals are less “economically enlightened” than conservatives.

What were the questions used to make this determination? Well, here’s my very favorite:

5) Third World workers working for American companies overseas are being exploited (unenlightened answer: agree).

That’s right. You count as not understanding economics, in the minds of these “scholars,” if you come up to a different position than they do on a hotly contested moral question of what counts as exploitation.

The only way you can think that that question will give you useful data on whether someone understands economics or not is if you genuinely can’t think of any way that a Pareto-superior transaction might be exploitative. Then, sure, there’s no debate about whether or not developing country labor is exploitative, because, hey, they were willing to take the deal, insert revealed preferences, done! Except that on that theory, hanging out in the desert and selling bottles of water for ten grand a cup is perfectly non-exploitative.

And even under that kind of dessicated theory of exploitation, you’d still have to have a bunch of contestable counterfactual beliefs about what options would be available to those workers without the existence of a multinational labor market.

I’m not saying that this market is uncontestably exploitative. I happen to think it is exploitative, but there’s room for debate. But coming down on the right-wing side on that debate does not economic competence make! One wonders if Joseph Stiglitz counts as “economically unenlightened” on the authors’ theory. Actually, who needs to wonder?

This is absolutely enraging. And totally unpublishable. Beautifully, it seems like it couldn’t get published in a real journal:

Mr. Klein is a professor of economics at George Mason University. This op-ed is based on an article published in the May 2010 issue of the journal he edits, Econ Journal Watch, a project sponsored by the American Institute for Economic Research.

Hint: if you have to publish in your own damn journal, maybe you should hesitate before putting it out? And did he bother to send it out for peer review? Somehow I doubt it.

The Wall Street Journal should be condemned for giving column inches to this deceptive “research.”

Paging Andrew Gelman!


13 Responses to “The single worst survey question I’ve ever seen.”

  1. Ashley Says:

    I’m glad to see you picked up on this – I found the whole article amusing, but that’s only because I’ve been conditioned to expect such garbage from “academia.” Dan saw my posting on Facebook and wanted to respond “worst survey design ever” but couldn’t be bothered to care enough.

  2. Paul Gowder Says:

    Seriously. Did they really think people wouldn’t notice?!

  3. Paul Gowder Says:

    Oh god, I have to quote this line from the actual “paper”:

    Even if one accepts that our handling of each of the eight economic questions tracks economic enlightenment, the set represents a baseline rather than the heights of economic wisdom. In other words, the most econ- omically enlightened mind would score no better than a solidly sensible mind on the eight questions, as they would both ace the 8-question exam.

    That’s right. You’re not even solidly sensible unless you agree that workers in developing countries aren’t exploited by multinationals!

  4. Julia Azari Says:

    Well, I’ll chime in from the qualitative perspective. WTF counts as “Third World” any more? And why are we aggregating all developing countries together? In some cases it may be exploitative, others not (this is what I actually believe, so I might be biased, but that is the glory of critiquing rather than designing and publishing a survey).

  5. Paul Gowder Says:

    Yeah, that’s another good point.

  6. Tomer Says:

    On days like this I’m happy you are my big sib, Paul.

    In addition to the point you make about forming hot contested normative issues as ones which have clear ’scientific’ answer thus begging the question as to who would have higher percentages of ‘enlightened’ responses (Hint: those who think what you think is ‘right’), i took the liberty (granted to me by various libertarian thinkers) to delve a little deeper.

    First, you won’t be surprised that the same research also claims that ‘enlightened’ responses are not more common among college graduates. Which means, education doesn’t make you a republican, which apparently is a bad thing.

    But second, and more intrestingly, I looked into this journal called Econ Journal Watch. Interestingly, it is funded by the American Institute for Economic Research, which claims to represent “no fund, concentration of wealth, or other special interests” since, as they say, “Experience suggests that information and advice on economic subjects are most useful when they come from a source that is independent of special interests, either commercial or political.” This of course, does not preclude them from celebrating Friedman Day (named after my distant cousin and late economist Milton Friedman), a.k.a “Freedom Tax Day”, which marks the day “we will be done paying for government” if “every dollar that every American earned starting on January 1 was used to fund government”. I guess populist, contested, partisan and inciteful political campaigns fall under the purview of “information and advice on economic subjects” from a source that is “independent of special interest, either commercial or political”.

  7. Paul Gowder Says:

    Tomer, thanks for the investigation! It’s amazing that someone who edits a hackish “journal” like that could still have tenure at a major department — oh, the distorting effects of politics.

  8. Steve M. Says:

    Wait, if Tomer is Paul’s older sibling, and Tomer is distantly related to Milton Friedman, then that means Paul Gowder is distantly related to Milton Friedman, which is awesome.

  9. Paul Gowder Says:

    Sadly, the “big sib” in question is not by blood but by departmental relationship…

  10. mtraven Says:

    Klein is a tool, but he also manages to put out some of the more thought-provoking stuff from the Masonomics school. I found his paper The People’s Romance (PDF) thought-provoking, if flawed.

    BTW his fellow Masonite Bryan Caplan makes exactly the same kind of self-indulgent error you point out here in his book on voting. Caplan’s output is a target-rich environment and spurred the coinage “libertardianism”.

  11. Steve M. Says:

    Damn. I suppose I’ll have to find some other way to vicariously live out my dream of being related to an important public intellectual.

  12. JSB Says:

    I’m surprised you didn’t mention anything about Question 4. I found it to be pretty bad too.

  13. JL Says:

    What if we think that American companies overseas are being exploited by Third World workers, who toil in unsafe conditions, work long hours, and take low pay? These workers are really exploiting American companies, who must deal with things like excessive turnover due to injuries and/or death, equipment broken from workers falling asleep on the job, and other risks.

Leave a Comment