Scholars of international relations, like their comparative and American counterparts, commonly follow what Keith Krehbiel (1991, 15) describes as the “orthodox tenets of positive social science”; that is, deducing hypotheses from a verbal or formal theoretical model, testing them, and then drawing conclusions about the theoretical model. In War and Reason, for example, Bruce Bueno de Mesquita and David Lalman develop and test the international interaction game through which they hope to explain many theoretical puzzles in IR, including the empirical generalization that democracies rarely fight one another. Early on, they lay out their view of positive social science: “The science of modeling depends on the ability to extract testable, falsifiable relationships among variables that follow in a logically coherent fashion, so that the connection between the model’s structure and its empirical implications is clear and consistent” (Bueno de Mesquita & Lalman 1992, 22).

In summarizing their results, they write,

“We have suggested solutions to each of these and many other theoretical or empirical puzzles. In virtually every case the proposed solution has satisfied the following formula: first the solution has been formally deduced and proved in the context of our theory, then the hypothesized solution has been submitted to empirical scrutiny through the analysis of the historical record. We have endeavored to be explicit about the expectations in our empirical tests so that the conditions for falsification are clear” (Bueno de Mesquita & Lalman 1992, 270).

Bueno de Mesquita and Lalman, as well as Krehbiel, take a particular approach to “testing” theoretical models known as hypothetico-deductivism (falsificationism is one variant). H-D suffers from a number of defects, particularly when applied to political science. Deductions work in a particular way. Truth flows down a deductive system (if the premises are true, then the deductions must be true), but it does not flow up a deductive system (if the deductions are true, the premises may or may not be true). Now consider a theory in political science. If the theory is true, then the deductions drawn from it must be true. Testing is therefore irrelevant. If the theory is not true, then the deductions drawn from it may be true or may be false, and the logical connection between the theory and the deductions is broken. In this case, testing cannot tell us anything about the theoretical model. Either way, testing the deductive consequences of a theory tells us nothing informative about the theory itself. (Table 1 depicts these two possible states of the world.)

Of course, in international relations, as in the rest of political science, we routinely use false assumptions in our theories. We make assumptions, such as rationality or treating states as unitary actors, which simplify a far more complex reality.

What then should we do? In our recent book, A Model Discipline, we draw on an analogy between models and maps first made by the philosopher Ronald Giere. Think about maps for a minute. Maps are characterized by limited accuracy, partiality, and purpose-relativity. Take the old Boston subway map prior to the addition of the Silver Line as an example. (See Figure 3-1 of A Model Discipline.) The map has limited accuracy and is in many ways factually wrong; if you deduce a hypothesis from the map such as, “the neighborhood of Mattapan is south of the city of Braintree,” and test it, you would discover that the opposite is true. The map is also partial; it displays some features of the area and not others. Finally, it is purpose-relative. The map is useful for riding the subway, but it is of little use for anything else. Anyone attempting to use the map for walking or driving around the city will become hopelessly lost.

The limited accuracy and partiality of the subway map does not mean that it is somehow false. Whether the map is true or false is the wrong question; a map is an object, and objects cannot be false (or true). The right question is whether the map is useful. Subway officials can evaluate the map, not through deductive testing, but simply by handing it to subway riders and asking if it helps them negotiate the subway. Despite its many inaccuracies, the Boston map is really quite useful.

Models are like maps in that models have limited accuracy and are partial. In truth, we are aware of few political scientists who would disagree on these points. We are forever told that theoretical and empirical models make use of assumptions that “simplify” reality and include only one or two features of the political landscape. Only somewhat more controversially, we claim that models are purpose-relative in the same way that maps are. We show that theoretical models can be useful in one or more of four ways: as foundational models, organizational models, exploratory models, and predictive models. Empirical models can be used for prediction, measurement, and characterization. (We show that a fourth use of empirical models, theory testing, cannot be justified beyond the relative comparison of models.)

Again, much of this way of thinking is not particularly controversial. The implications for the practice of political science, however, are controversial. We have been bombarded in recent years with the claim that science consists of proposing a theory and testing it with data. Not only would that definition come as a surprise to many in the hard sciences, it does not comport with what we know about the nature of models. Theoretical models are not “tested” with data; theoretical models are “tested” with models of data (a category that includes qualitative data). Why should one limited accuracy, partial, and purpose-relative model “test” another limited accuracy, partial, and purpose-relative model? Theoretical models can be useful without being tested, and empirical models can be useful in roles other than testing. It is rarely necessary to include both kinds of models in a single paper.

That being said, there is a justifiable way of linking theoretical and empirical 3 models, and it has to do with the concept of explanation. Empirical models cannot provide autonomous explanations, while theoretical models can. Knowing that democracies do not usually fight one another and tend not to lose wars (Bueno de Mesquita, Smith, Siverson & Morrow 2003) does not provide an explanation. Theoretical models provide the explanatory “bite” that empirical models lack (see Chapter 6 of A Model Discipline for a discussion of what constitutes an explanation). Although choosing between rival explanations is not always necessary—complex events often have more than one explanation—it can be done with the tools of comparative model testing (Clarke 2007). Here the question is not whether an explanation is “true,” but which of a set of explanations is strongest given the available evidence.

Our argument has been misconstrued at times, so it is important to be clear. We believe that theoretical work is important. We believe that empirical work is important. Our deviation from current orthodoxy lies in our insistence that empirical models are not useful for testing theoretical models and that a new justification for linking the two is required. We hope that these ideas will help scholars of international relations think about their own work in new ways.

—

Kevin A. Clarke is Associate Professor of Political Science and David M. Primo is the Ani and Mark Gabrellian Professor, both at the University of Rochester. This piece is adapted from Clarke, Kevin A., and David M. Primo. 2012. “The Modeling Enterprise in Comparative Politics.” APSA Comparative Politics Section Newsletter 22(2): 8-9 and also draws from Clarke, Kevin A., and David M. Primo. 2012. A Model Discipline: Political Science and the Logic of Representations (New York: Oxford University Press).

References

Bueno de Mesquita, Bruce, Alastair Smith, Randolph M. Siverson, and James D. Morrow, The Logic of Political Survival (Cambridge: The MIT Press, 2003).

Bueno de Mesquita, Bruce, and David Lalman, War and Reason (New Haven: Yale University Press, 1992).

Clarke, Kevin A. 2007, “The Necessity of Being Comparative: Theory Confir- mation in Quantitative Political Science.” Comparative Political Studies 40 (7): 886–908.

Krehbiel, Keith, Information and Legislative Organization (Ann Arbor, MI: University of Michigan Press, 1991).

Further Reading on E-International Relations

• The Case for a New Constructivism in International Relations Theory
• The Rise of Paradiplomacy in International Relations
• Calling for Reflexive Solidarity in International Relations
• Post-Positivism Is Not Yet Normalized in International Relations
• Theory Synthesis in Sport and International Relations Research
• A Defence of Macro-History in International Relations