The Ideological Turing Test

By Sheldon Greaves

The quote above is a less well-known ancillary to Sun Tzu’s more famous quote: “Know your enemy as you know yourself, and in one hundred battles you will not be in peril. This post is about intellectual discourse and dialogue, so I am frankly uncomfortable using military metaphors to describe it. The metaphors we use frequently map to how we really feel about something, and these days disagreements are laden with stakes of life or death (See, “culture wars”).

The Turing Test is a standard developed in 1950 by Alan Turing to determine whether a machine could be said to “think” or, more precisely, exhibit intelligent behavior on the same level as a human being. If the computer could communicate and respond in a way that was indistinguishable from a human being (for instance, through text messages), then the machine could be said to be intelligent.

Economist Bryan Caplan has suggested a variation on this idea that is a test for the intellectual rigor behind someone’s ideas.

The Ideological Turing Test is a concept invented by American economist Bryan Caplan to test whether a political or ideological partisan correctly understands the arguments of his or her intellectual adversaries: the partisan is invited to answer questions or write an essay posing as his opposite number. If neutral judges cannot tell the difference between the partisan’s answers and the answers of the opposite number, the candidate is judged to correctly understand the opposing side. 

Quoted in Praxtime by Nathan Taylor

One of the most difficult problems in the current rhetorical environment is that of misrepresentation. There are entire institutions whose sole existence is to misrepresent, to distort, to flat-out lie about the positions of their political opponents. Ironically, Caplan is a libertarian who admires and once subscribed to the Objectivism of Ayn Rand. Last year he published a book under the title The Case Against Education: Why the Education System Is a Waste of Time and Money. This, despite his own Ph.D. in economics and professorship at George Mason University. So I wonder whether he could himself pass such a test.

John Stuart Mill articulated this principle in terms somewhat lengthier than Caplan, but he describes the process, and its spirit, very well:

He who knows only his own side of the case, knows little of that. His reasons may be good, and no one may have been able to refute them. But if he is equally unable to refute the reasons on the opposite side; if he does not so much as know what they are, he has no ground for preferring either opinion. The rational position for him would be suspension of judgment, and unless he contents himself with that, he is either led by authority, or adopts, like the generality of the world, the side to which he feels most inclination. Nor is it enough that he should hear the arguments of adversaries from his own teachers, presented as they state them, and accompanied by what they offer as refutations. That is not the way to do justice to the arguments, or bring them into real contact with his own mind. He must be able to hear them from persons who actually believe them; who defend them in earnest, and do their very utmost for them.

On Liberty. 1869. Chapter II: Of the Liberty of Thought and Discussion

Are There Shortcuts?

Mill’s prescription sounds like a lot of work, but I submit that there are a few positions–moon landing denial, original sin, Donald Trump–that have enough problems that I, at least, can consider them as less than fully tenable. I will be the first to acknowledge, however, that it constitutes a slippery slope.

One shortcut is to consider the origins of the idea. For what purpose did it arise? Was it developed as a rhetorical or propaganda device, as was the case with Birtherism? Another is to consider whether the idea ever been successfully put into practice (See, “Libertarians”)? Who are the idea’s advocates? Is the idea embraced by people who don’t have the brains of a soda cracker? If the only advocates are Sean Hannity and Alex Jones, I submit that it can be safely discarded.

Who Does This?

One thing that strikes me in discussions of the Ideological Turing Test that it is pretty theoretical. It’s treated as a general idea, a way to determine whether someone knows what they’re talking about. In that sense, it’s a sound rule. I’ve discovered over the years that the party that displays the clearest and deepest knowledge of their opponent’s position almost invariably has the better argument.

But there is one place where the principles implicit in this idea are actually practiced on a daily basis, and that is in the Intelligence Community. As intelligence analysts consider a problem, often factions emerge around competing views–no real surprise there. But a 2009 “Tradecraft Primer” states that sometimes it makes sense to ask analysts to argue in favor of views the disagree with:

If opposing positions are well established, it can be useful to place analysts on teams that will advocate positions they normally do not support; forcing analysts to argue “the other side” can often make them more aware of their own mind-set.

A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis

Unfortunately, this kind of enlightened technique almost never happens in academia, and certainly not in most political discourse. In fact, a common conservative jab at liberals was to frame liberal willingness to consider the other side as a weakness. It suggested uncertainty (Which is true, but not necessarily bad) which further suggested a lack of conviction, and before you knew it, intellectual humility and open-mindedness signaled a lack of patriotism.

This may seem like wild hyperbole, but then I read the following in Psychology Today, “Does Liberal Truly Mean Open-minded?”: “If you Google “liberal means open-minded,” you get more than 5000 hits. Googling “conservative means open-minded” yields exactly one.”

I tried the latter test. Things must be getting better, because I got three hits.

Even if that test isn’t conclusive (it isn’t), it is very, very suggestive. It argues that not seeing the other point of view is practically a policy of certain parts of the political spectrum, more often on the right than the left. Consider how, after the election, there were multiple calls from the left for a need to better understand the Trump voter. Do you recall any calls from the right to understand the majority of Americans who voted for Clinton? Me neither.

The Ideological Turing Test is more than a useful tool for bullshit detection. It is an intellectual discipline, a call to intellectual humility. Its absence from so much of American public life does not bode well.


Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.