write 7 questions from the reading

political science question and need the explanation and answer to help me learn.

please read the materials and write four questions, good 4 questions please not summery the reading
and let me know where did you get the questions from such as the name of the reading and the page number
4 questions from this reading
Legacies of War & Intervention in El Salvador:
Moodie, E. (2011). El Salvador in the aftermath of peace: crime, uncertainty, and the transition to democracy. University of Pennsylvania Press. Chapter 1, pages 18 – 50.
External UrlWheeler, W. (2020, January 10). “How the US helped create El Salvador’s bloody gang war.” The Guardian.
====================================================================
3 questions from this reading
Gerring, J. 2012. Social Science Methodology: A Unified Framework. Cambridge: Cambridge University Press. [Chapters 11-12] pages 291 – 360.
======================================================================
please put each of them in different document so you have to send me two documnet one has 4questions and one has 3 questions
Requirements: 1 hour
Chapter 1Big Stories and the Stories Behind the StoriesYou have to go back. It’s urgent. As soon as we got to Cacaopera, they told me. They didn’t say who had called. Or why. All afternoon that day back in June 1994, I had been riding around in the bed of a rattling pickup truck with four or fi ve San Salvador friends who had invited me to their home village. I still have the pictures I took. Laughing children running alongside us on dirt roads shouting their greetings, “¡Salud! ¡Salud!” Grandmothers hanging wet socks off the sides of sad adobe houses shadowed by sagging palms. Foregrounds of brilliant fl owering trees whose names I didn’t know, backgrounds of thickly forested mountains where I imagined abandoned guerrilla camps. Somewhere along the way my borrowed baseball cap had fl own off. My hair was a tangled mess. My face was covered with a thick layer of grit. I don’t remember now why we arrived so late in the evening in Cacaop-era, in the mountainous northeastern department of Morazán. During the 1980s civil war, part of the region had been a guerrilla zone controlled by the People’s Revolutionary Army (ERP), one of the fi ve factions that united in late 1980 to launch an insurgency as the Farabundo Martí National Libera-tion front (FMLN). That particular day might have been the time the group of us slogged through the muddy paths of some nearby cantones (hamlets), stopping at door after door. We were seeking someone’s brother’s ex-wife’s family whose address we didn’t know to deliver a package carried by a visit-ing migrant friend of a relative back in New Jersey. By the time we reached Ernesto’s house, it was eight or nine p.m., long dark. That cryptic communi-cation demanding my urgent return had been delivered, Ernesto said, by la Niña Luisa, his elderly cousin.1 He said he thought that Luisa’s neighbor—with one of the few telephones in their part of the village—had brought her the message. By then, it was too late to knock on either of their doors to fi nd 12476-El Salvador in the Aftermath of Peace.indd 1812476-El Salvador in the Aftermath of Peace.indd 1811/25/09 10:53:04 AM11/25/09 10:53:04 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
Big Stories and the Stories Behind the Stories 19out anything more. And the local offi ce of the national telephone company (ANTEL), where many people made their calls, had closed for the night. All we knew was that I had to go back to San Salvador.I guessed the message was from María Eugenia (Maru), a daughter of Niña Luisa. I had met Maru, the sister of a friend in North Bergen, New Jersey, the year before, during my fi rst trip to the small Central American country. When I returned to El Salvador, she invited me to stay with her in the small cinderblock house her sister Gertrudis had bought with money earned cleaning other people’s houses in wealthy New York City suburbs.Why hadn’t Maru said anything more? The next morning I agonized the full fi ve hours back to San Salvador. I imagined worse and worse scenarios as the fi rst bus groaned and farted its way down the mountain pass to San Fran-cisco Gotera, as the next one barreled nonchalantly over the broken roads to San Miguel, and as the third creaked slowly over the Lempa Rivera and past the volcanos and fi elds near the towns of San Vicente and Cojutepeque. San Salvador’s sprawling metropolis of two million arrived in our nostrils before we saw it, in the thick exhaust of the traffi c jams on the Boulevard del Ejército (Army Boulevard, a continuation of the Pan-American Highway). There the trees morphed into factories, their walls splattered with a hybrid mix of early fl ashes of hieroglyphic gang graffi ti—“MS-13” (standing for Mara Salvatrucha, the now well-known transnational gang)—and the slo-gans echoing war resistance—“Calderón Sol es escuadronero” (“Calderón Sol is a death squad member”; the graffi ti refers to the former mayor of San Salvador, a member of the dominant, right-wing party ARENA, the Nation-alist Republican Alliance [Alianza Republicana Nacionalista], who had just been elected president). Then came the corrugated metal-roofed neighbor-hoods of the hunched cities of Ilopango and Soyapango. It was Saturday morning. The whole world was up, selling, watching, buying, mothers and children with mangos and plantains and avocados and pineapples arranged on banana leaves along the road, men and women with newspapers and brooms and fl owers walking among the cars and trucks and buses gunning at the stoplights. The fourth leg of the trip was on a dented Bluebird-type city bus that would have reminded me of my childhood rides to school through Indiana cornfi elds if it weren’t for the crucifi x and image of martyred Archbishop Óscar Arnulfo Romero gazing at the passengers from the front. I got out at the stop by the furniture factory near the Residencial Holanda, a newer working-class neighborhood named for the estate that once stood there. 12476-El Salvador in the Aftermath of Peace.indd 1912476-El Salvador in the Aftermath of Peace.indd 1911/25/09 10:53:05 AM11/25/09 10:53:05 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
20 Chapter 1Maru appeared at a neighbor’s door just as I was trudging up the last few steep meters. “Maru!” I shouted. “What happened!?” She ran down to me. “It’s okay, it’s okay.” She wasn’t one to yell her busi-ness in the streets. “It’s just—I was robbed. They even got my key. I couldn’t get into the house.” 2 I had the only spare.A Little MuggingLooking back, that urgent return to the city, that abrupt truncation of my palm-lined fi eldwork fantasy, became my arrival scene.3 The anxiety I felt, the not knowing what would come next, may be a common condition for an uneasy foreign fi eldworker. But it was also a widespread way of being for many nervous Salvadorans in the fi rst few years after the country’s civil war ended in 1992. So many people were conscious of not knowing what to expect back then. They recognized the loss of context, the failure of trusted categories, even if the categories had been built up from the perversions of war, even if the “pathologies of the state [had acquired] a life in the every-day.”4 (Now you probably could carry a book-fi lled backpack on the bus without fear—in the past university students were instantly suspected of subversion. But would you sit next to a baggy-pantsed kid with glaring eyes? Could he be one of those gang members who by the mid-1990s were mob-bing the mass media—or was he just a sullen teen mimicking MTV?) My forced return, then—and my need to know even as I dreaded what I might learn—is the fi rst story behind the story. It marked my settlement into the meddlesome, presumptuous, and often naïvely optimistic discipline of so-ciocultural anthropology in its self-doubting late twentieth-century version. It was also how I began to dwell, awkwardly, in and on the urban chaos and gritty charisma of San Salvador, El Salvador. And it was on that very day I joined Maru, and countless other Salvadorans, in telling and listening to crime stories while grasping at the murk of the aftermath of war. Or, per-haps better said, grasping at the murk of the aftermath of a peace declared to much fanfare just two years earlier.5Back in June 1994, I didn’t seize on that key to comprehending the post-war moment. I didn’t write much about Maru’s mugging in my fi eld notes. Why should I have been concerned with the affairs of pickpockets and mug-gers? The Salvadoran papers and TV may have been spreading more and more alarming information on what was being called delincuencia común 12476-El Salvador in the Aftermath of Peace.indd 2012476-El Salvador in the Aftermath of Peace.indd 2011/25/09 10:53:05 AM11/25/09 10:53:05 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
Big Stories and the Stories Behind the Stories 21(common crime). But, I reasoned—much like UN observers surveilling the early implementation of the peace accords—what transcendental meaning could there be in the theft of a few colones and a house key? 6 What conse-quence could a swift street robbery possibly have for someone like Maru, after her brother-in-law had slipped into el monte in the early 1980s (to the bush, as they said about those who joined the guerrilla forces) and her sis-ter had fl ed two thousand miles north from Cacaopera, Morazán, to North Bergen, New Jersey, leaving her at the age of fi fteen to take charge of her two small nieces? The big story, I believed, the story that mattered, was the war and its end. That was why I had fi rst visited El Salvador, after all, arriving with a bright-eyed delegation from New Jersey to see our Sister City, a re-settled village in a once wartorn rural area.But that little mugging had meant something to Maru. She had not wanted to tell anyone in Cacaopera. They worry too much about her already, she explained. But she talked in San Salvador. I must have heard her story fi ve or six times that week, as she repeated it to neighbors over coffee or to friends on the telephone she covered with a tiny yellow towel. I remember it wel1.7 It conformed to a kind of Salvadoran postwar “standard version” crime story, happening unexpectedly, and forcing changes in understand-ings and habits to adjust to a new climate of risk.8 Here is how it went: About six in the evening—sundown—at the Plaza Libertad in the city center, Maru got off the bus as usual in order to board another on her way home from classes at the University of El Salvador. (Afterward she found a new route, longer but thought safer.) A girl, a teenager, no one you would look at twice, bumped into her. Maru turned. A group of girls swarmed her. One scraped a blade against her stomach. She demanded cash. Maru protested. They grabbed her backpack. They ran.In the years after the peace accords, crime stories both like and unlike Maru’s moved through San Salvador, across the country, indeed over the globe as Salvadorans continued to migrate elsewhere. As these fragments of experience began to accumulate, to acquire patterns in social and indi-vidual memory, people started saying, “It’s worse than the war.” Whether they described muggings or kidnappings, the stories fi t into a series of bigger stories—the story of the democratic transition, the story of the war and the peace process, the story of twentieth-century postcolonial and anti-imperial resistance, the story of a new free-market global economic order. In order to understand processes of knowledge production in postwar El Salvador, emerging from a feeling of not-knowing the new situation, we must grapple with these big stories. They are the uneven foundation on which any sense 12476-El Salvador in the Aftermath of Peace.indd 2112476-El Salvador in the Aftermath of Peace.indd 2111/25/09 10:53:05 AM11/25/09 10:53:05 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
22 Chapter 1of the present is built—and through which conditions for possibilities of change emerge. After all, men and women can change the world, but not just as they please—not under circumstances they choose, but, as Marx put it, “under circumstances existing already, given and transmitted from the past.” And the past, in El Salvador, weighs like a nightmare.9 Some stories of the past constructed coherence in familiar histori-cal arcs. There is that of violent El Salvador, a place of chaos and carnage. Other stories have faded under the weight of hegemonic versions of events. The story of Salvadoran struggles for democracy, surging in key moments through the twentieth century, has usually been swallowed into a dominant historical plot of fi fty relentless years of authoritarianism leading inevitably to the war. In this chapter, we will explore some of the larger narratives in which many Salvadorans’ experiences are embedded—the big stories, and the stories behind the stories.10 Land, Coff ee, and a MassacreOne big story reverberates through twentieth-century Salvadoran history: the massacre of 10,000 indigenous and ladino (non-Indian) peasants who participated in a Communist-led uprising in 1932.11 The Matanza, it is called. That moment of horror, that nightmare, starts with the rise of coffee as El Salvador’s main export. The account usually given is one of dispossessed In-dians forced into landlessness and peonage by a greedy, rising oligarchy that wanted to insert El Salvador into the rising world economy of the late nine-teenth century. The “grain of gold,” or coffee, thrived on the rich volcanic soils where Indians had planted maize and beans. So the rich took the lands in the name of progress, and paid peasants (a pittance) to harvest the beans a few months a year. After the worldwide depression hit, the hungry masses rose up in desperation—only to be cut down by a savage military. This account is still the dominant narrative. The big story. But it gets it wrong. What happened is not merely a familiar Latin American tale of elite greed and repressive military violence. It is also a story of the fi rst experi-ences of popular sovereignty and democratic hopes in El Salvador. It is true that in 1881 and 1882 the Liberal government divided up corporate rights to communal lands. But the principal motive for this privatization was not just a land grab enabling the rise of the coffee aristocracy, as popular histories would have it (and as Marxist theories of primitive accumulation would in-sist). Historian Aldo Lauria-Santiago has shown that an important goal was 12476-El Salvador in the Aftermath of Peace.indd 2212476-El Salvador in the Aftermath of Peace.indd 2211/25/09 10:53:05 AM11/25/09 10:53:05 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
Big Stories and the Stories Behind the Stories 23to nudge peasants into a new economic order, to get them to participate in the expanding commodity market.12 Partition aimed in part to turn Indians (“perceived as having no interest in accumulating wealth”) into productive, individualistic entrepreneurs.13 The number of landholding peasants did in-crease. In the fi rst years after land reform, small farms were more common in many areas than haciendas or plantations. But through the 1920s the power of the agrofi nancial oligarchy grew in a land boom linked to increasing inter-national commodity prices. A rising class of coffee barons seized more and more land. Indebtedness of the smaller property holders also fed the land grab, especially after the decline in coffee prices that began in 1927.14 The state may have sought to modernize its peasants, prodding them toward productive roles in the market economy. But it also needed workers from which to extract the surplus labor that creates value. When it priva-tized the land, it also passed new criminal laws, and directed the (itinerant, mostly privately paid) police force to gather up vagrants and debtors and send them to work the fi elds. To justify these tactics, the country’s lead-ers pointed to the need to discipline the Indians, whose perceived lack of work ethic they blamed for El Salvador’s backwardness. The liberal rhetoric of individual freedom, in other words, was frequently lost in the quest for economic growth. When the National Guard was formed in 1912 as El Sal-vador’s fi rst permanent system of state vigilance, its primary mission was to defend agrarian interests. The more prosperous landowners billeted Guards-men on their estates.15 The country was run by a small group of landholding families—many (largely European and some Middle Eastern) immigrants of whose names still reverberate powerfully today. To them the national in-terest and coffee profi ts were one and the same. Then, in 1927, one member of the ruling elite challenged the repressive system of government in his country. As a candidate, Pío Romero Bosque, brother-in-law of the president, had been considered a safe choice by the Meléndez-Quiñonez political dynasty that had ruled for fi fteen years. But a large contingent of artisans and workers, listening to his words of change and progress, also supported him. Once elected, Romero Bosque quickly broke from the regime, quite likely in part for personal reasons.16 He started by lifting the state of siege that had dominated the decade. What followed was the rise of a civil society unprecedented in El Salvador’s history.17 Rural and urban workers began organizing openly. The emergent news media began to disseminate more diverse views. In 1930, Romero Bosque refused to anoint a successor. Five new politi-cal parties competed in the country’s fi rst (reasonably) free elections. Arturo 12476-El Salvador in the Aftermath of Peace.indd 2312476-El Salvador in the Aftermath of Peace.indd 2311/25/09 10:53:06 AM11/25/09 10:53:06 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
24 Chapter 1Araujo, a landowner with progressive ideals, won. His election raised hopes across the country. But the timing was bad, very bad. The October 1929 Wall Street crash had changed everything. El Salvador’s national income had dropped by half after coffee prices began falling. A third of the tenant farm-ers had been forced off their land. Popular unrest increased. Labor unions and student syndicates called for massive strikes. The chaos forced Araujo to declare martial law. The military overthrew his government in Decem-ber, and hardline General Maximiliano Hernández Martínez (known by his maternal surname, Martínez) took over. For six weeks after the coup, Indian and ladino peasants and workers, many affi liated with Communist Party-linked groups, organized a series of strikes and protests. On 22 January 1932, they launched a well-planned up-rising, attacking army garrisons, telegraph offi ces, and city halls and briefl y holding a dozen towns in the western, coffee-growing region.18 The rebels killed no more than fi fteen to twenty people. But Martínez responded with what has been described as hysteria. He ordered the immediate executions of insurgents. Soldiers killed far more than those who had participated in the uprising. Most of the victims were indigenous, though ladinos likely made up half the rebels.19 The massacre devastated indigenous communi-ties. It also demolished political dissent. Martínez’s rule began fi fty years of direct military rule characterized by coercive labor practices and repressive policing.Among those killed was a young man and Communist Party leader from a landowning family, Agustín Farabundo Martí, whose name would be taken up by the FMLN a half century later. The remembered event would also play a central role in the formation of ARENA, the right-wing party founded in 1980 that won the presidency in 1989 and stayed in power for twenty years. ARENA opened each new presidential campaign in the town of Izalco, the symbolic center of the massacre, where candidates declared, “Here we buried communism!” 20 The Matanza became a central reference in Salvadoran history, the big story, interpreted and reinterpreted according to the contingencies of the political moment. For the political left, it has served as a story of class warfare, demonstrating the ruthless nature of El Salvador’s bourgeoisie and military. More recently it has become a tale of racism and genocide. For the political right, the story of 1932 became a cautionary tale that the state would trundle out whenever deemed necessary—especially when agrarian reform, or any kind of redistribution of power or resources, threatened. The specter of communism condensed into an image of the peasantry rising up with 12476-El Salvador in the Aftermath of Peace.indd 2412476-El Salvador in the Aftermath of Peace.indd 2411/25/09 10:53:06 AM11/25/09 10:53:06 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
Big Stories and the Stories Behind the Stories 25machetes—or what has been called the indio comunista.21 Such views would harden in the 1960s and 1970s, merging threats of peasant organizations, workers’ unions, priests and nuns and guerrillas into an outlaw specter of criminals and terrorists. These fi gures inevitably haunted crime stories that circulated in a newly democratizing El Salvador of the 1990s.Democracy, Prosperity, and SecurityIn the immediate post-civil war years, social scientists and journalists dis-cussing the transition would lament El Salvador’s authoritarian political cul-ture. “El Salvador is not a case of re-democratization; there was no legacy of democratic institutions to draw on,” they asserted.22 For most of its history it has been ruled by “a self-serving oligarchy allied with a praetorian military,” they declared.23 To these experts the big story is a tragic one, defi ned by El Salvador’s lack. But a lack of democratic tradition of governance does not purge political feelings in the population. It does not preclude democratic desires. The continually thwarted struggle for democracy may be the biggest story in twentieth-century Salvadoran history.As the buildup to the 1932 uprising shows, the possibilities of liberatory politics have overtaken popular imagination at several critical junctures in Salvadoran history. Another important episode of democratic aspirations and political opening occurred twelve years after the Matanza. The people rose up to overthrow Martínez. This was the same time, 1944, that Guate-malans deposed their own dictator, Jorge Ubico, and launched a democratic decade known as “The Ten Years of Spring.” El Salvador enjoyed only fi ve months of such a spring. In May, Martínez stepped down under pressure from a coalition of state employees, professionals, and urban workers who collaborated in Gandhi-inspired actions of noncooperation, refusing to at-tend work or school. Then in October, the military staged another coup. The new government quashed the opposition and clamped down on dissent.24 The country had to wait twenty years for another upsurge of democratic desires. What incites democratic desires in a place with no fi rm democratic tra-dition? Indeed, what kinds of democracy have Salvadorans desired? This question is important for understanding expectations after the 1992 peace accords—for grasping how the Salvadoran story was supposed to “end.” Po-litical scientists tend to delimit democracy in their taxonomy of political forms as comprising regular, competitive elections with universal suffrage, 12476-El Salvador in the Aftermath of Peace.indd 2512476-El Salvador in the Aftermath of Peace.indd 2511/25/09 10:53:06 AM11/25/09 10:53:06 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
26 Chapter 1as well as political accountability of the state to the population. But, as his-torian Greg Grandin reminds us, when the United Nations was founded in 1945 democracy was broadly seen as entailing both human freedom and social equality. In his own research Grandin has worked with Guatemalans struggling against “daily traps of humiliation and savagery” of an exclu-sionary, exploitative society. In the mid-twentieth century, he argues, they shared “a commonsensical understanding of democracy not as a procedural constitutionalism but as the felt experience of individual sovereignty and social solidarity.” 25 The post-World War II global wave of anticolonial resistance inspired many Central Americans. In El Salvador it shaped political commit-ments of a generation of educated middle-class young people born about the time of the Matanza. Many of them came of age with the overthrow of Martínez—only to see the military coup fi ve months later. They then watched the decade-long rise of democracy in Guatemala—only to see it killed by a CIA-backed coup in 1954. The sensibilities born of these experi-ences are documented through the writings of a group of politically engaged artists called the “Committed Generation.” The poet Roque Dalton coined the name in an editorial in which he insists that art and literature must change the world. Dalton, a committed Communist Party member, would be arrested several times—charged with organizing student protests on one occasion, and found with the “communist propaganda” of a book of poetry by Cuban Nicolás Guillén on another—before going into exile in 1961.26 The 1959 Cuban revolution inspired political movements throughout the region. In 1961, a group of Nicaraguan intellectuals formed the Sandini-sta Front for National Liberation (Frente Sandinista de Liberación Nacional; FSLN); they spent nearly two decades planning for a popular revolution and insurrection that would lead to the overthrow of dictator Anastasio Somoza in 1979. The Guatemalan insurgency arose in that moment as well; in April 1961 students and members of the outlawed Communist Party marched in Guatemala City’s streets to protest their government’s participation in training Cuban exile mercenaries for the Bay of Pigs invasion. Three protes-tors were killed when the military opened fi re.While political ideas of democracy were growing and sometimes radi-calizing among activist Latin Americans, the concept was being deployed in rather different ways in powerful global institutions of the Cold War. De-mocracy there functioned as the simple antithesis to an ominous gray image of Soviet communism. Primarily as not-communism, it promoted individu-alism, property rights, and free-market economics. Promoting this perspec-12476-El Salvador in the Aftermath of Peace.indd 2612476-El Salvador in the Aftermath of Peace.indd 2611/25/09 10:53:06 AM11/25/09 10:53:06 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
Big Stories and the Stories Behind the Stories 27tive became an urgent task to the United States after the Cuban revolution (and Bay of Pigs failure) threatened its hegemony in its own “backyard.” So the Kennedy administration launched Alliance for Progress in Latin Amer-ica in 1961, linking “political freedom” with capitalist economic planning, development, and the fl ow of foreign investments. To broaden its appeal it tolerated some welfare-state features, such as limited agrarian reform, na-tionalized resources such as oil, and social security systems offering limited health care and pensions. In El Salvador, Col. Julio Adalberto Rivera, who ran unopposed for the presidency in 1962 (following a military coup two years earlier), quickly “molded himself as an Alliance for Progress Presi-dent.” 27 For a time the top-down modernizing vision encouraged by the United States produced strong statistics. The agro-export economy, not just coffee but also cotton and sugar, expanded. The industrial base grew, especially for manufacturing light products. The Central American Common Market (CACM) began operating in 1960, promoting an import-substitution model (in which fi nished goods were produced for a “domestic” market spanning Guatemala to Panama) and helping to further integrate the region into the global economy. In 1964 and 1965 El Salvador’s annual economic growth rate hit 12 percent, the highest in Latin America, raising living standards among most urban groups and the rural middle and upper classes—though not among the poor majorities.28 El Salvador’s space for democratic energy started to widen again. The electoral system changed in 1963 to permit proportional representation. This move gave opposition parties, led by the newly formed Christian Democrats, a voice in governance. State spending on education rose, with more and more literacy programs, grade schools, and universities. My friend Urias Betoel Escobar here in Illinois tells a story that shows the almost fl amboyant surge of hope of that moment: one day some strangers from the capital came to his school and administered tests to all the young people in rural Santa Rosa de Lima in the eastern part of the country. They chose him to be one of the fi rst music students at the Centro Nacional de Arte, though he had no training. Almost all their teachers were U.S. Peace Corps volunteers, who passed on to their students a passionate belief in possi -bilities for the future—it was 1968, after all. That, he laughs, was how a peasant boy from El Salvador became a trombone player in a symphony orchestra, in Costa Rica and then Colorado: the audacious optimism of that moment. That collective energy spilled across the nation. Political activism fi lled 12476-El Salvador in the Aftermath of Peace.indd 2712476-El Salvador in the Aftermath of Peace.indd 2711/25/09 10:53:06 AM11/25/09 10:53:06 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
28 Chapter 1the streets. It was fueled in part by a university population that quadrupled between 1960 and 1971 to more than twelve thousand. More and more labor unions formed, including the National Association of Salvadoran Educators (Asociación Nacional de Educadores Salvadorenos; ANDES-21 de Junio). Rural organizing reemerged for the fi rst time since the 1920s. The U.S. Agency for International Development (USAID) supported peasant orga-nizations, and the Christian Democrats and the Catholic Church launched rural cooperative programs and peasant leagues. The church, inspired by new ecclesiastical doctrines of social justice and a preferential option for the poor, sent priests and laypersons across the country. Christian base communities began to form, emphasizing reading the Bible for liberatory messages—and for the place of the poor in a big story of capitalist expansion and exploitation.Most of the rural poor were not recruited to play classical trombone. They still had few options. Their wages stagnated, even in the years of growth. Landlessness also rose through the 1960s, from 12 to 29 percent, partly because of population growth but mainly due to greater concentra-tion of land ownership.29 As many as 300,000 Salvadorans migrated north to squat in more sparsely populated Honduras. Most peasants who stayed in El Salvador remained tied to a coercive labor regime that demanded their ser-vices for brief, intense periods. The percentage of rural residents dependent on temporary wage work grew from 51 to 60 percent.30 Astounding social and economic inequality marked El Salvador. In a slim 1964 Life World Library volume on Central America, a chapter called “The Great Families” points out that only in El Salvador, among the six Cen-tral American nations, was there a millionaire class. The Catorce Grande [Big Fourteen] . . . live the lives of men of wealth. They have lavish homes in town and imposing estates in the country. They work in fancy, air-conditioned offi ces and drive to work in fl ashy sports cars. The women wear the latest dresses from New York, Paris and Rome. . . . Members of the Catorce Grande [have] done a great deal for the Sal-vadoran economy. El Salvador is a highly developed country by Central American standards. . . . Until recently, however, the Catorce Grande resisted social change. It took a military dictatorship, which seized power in 1960, to force them to pay their workers of minimum wage of seventy cents a day, to reduce slum rents and to make other reforms necessary. . . . Many of the sons of the other families have returned from schools in the U.S. with the idea that Indians and mestizos are human beings, not animals. However, this is still a minority view.3112476-El Salvador in the Aftermath of Peace.indd 2812476-El Salvador in the Aftermath of Peace.indd 2811/25/09 10:53:07 AM11/25/09 10:53:07 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
Big Stories and the Stories Behind the Stories 29Such attitudes constituted, and were constituted through, a form of dis-trust that only intensifi ed with the rising prosperity. The paramilitary net-works that traditionally policed the countryside increased even during this time of democratic opening. The government may have acceded to some political liberalization, but it hardly forgot the threat of subversion. It soon began to reorganize its intelligence activities with technical assistance from the United States. Anticommunism, after all, had been the impetus for the Alliance for Progress. And the Salvadoran oligarchy, raised on stories of the terror of the subversive rebellion and the Matanza, was nothing if not anti-communist. A paramilitary network known as the Democratic Nationalist Organization (ORDEN) was founded in the mid-1960s to surveil suspicious activities in the countryside (seen in practices such as peasant organizing). Betoel Escobar, the trombone player, recalls those years as times of politi-cal liberation—but also of danger. He and his fellow students—musicians, actors, and painters—were under constant suspicion as subversives for the simple fact of studying the arts, having long hair, singing. (Ironically, the left also distrusted them, since their teachers were gringos—sure to be spies.) He recalls how, the night of one large protest rally in 1971, no one could leave the school grounds—the National Guard had surrounded them, con-vinced they were plotting anarchy.32 He would go to Costa Rica the next year. He was not there when his father was taken hostage on their farm, when neighbors were murdered, their bodies left out on display. Eventu-ally such “intelligence activities” would be centralized under the National Security Agency of El Salvador (ANSESAL), cooperating closely with the agrarian elites described in the Life volume. Early death-squad actions in the 1970s would be traced back to ANSESAL.33Though police repression continued through the 1960s, after the civil war ended in 1992 many Salvadorans looked back on that distant decade almost fondly. “You could walk around late at night and nothing would hap-pen to you,” my father-in-law, Don Antonio, once told me. “Even rich people took the buses. President Rivera used to ride his motorcycle to church—the Don Rua—without security guards! The main problem I remember was the police. They would grab you on the street and make you pay to get out of jail, if you weren’t carrying [the paper showing you paid] your viabilidad (street usage) taxes. Once I had to pay seven colones and fi fteen centavos to get my brother out. That was two days’ pay!” 34In 1969 the fi ve-day war between Honduras and El Salvador broke out. Outside Central America, the confl ict became known as “football war,” because it erupted after hostile World Cup qualifying matches. The light-12476-El Salvador in the Aftermath of Peace.indd 2912476-El Salvador in the Aftermath of Peace.indd 2911/25/09 10:53:07 AM11/25/09 10:53:07 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
30 Chapter 1hearted dubbing of the event as an amusing fi t of pique between two pe-ripheral little places—the only kind of story that seemed to make subaltern states legible in the North (beyond proximity to Cold War interests)—belied the gravity of the situation.35 Honduras, resentful over imbalances in the CACM, had passed a law ordering deportation of Salvadoran settlers. Many of them were small farmers pushed from their own country by land scar-city. El Salvador responded with invasion, bombing the Honduran capital, Tegucigalpa. Several thousand died; 100,000 deported Salvadorans were left homeless.36 The land tenure problem intensifi ed. The CACM disintegrated. Industrial production slowed. Economic growth stalled. The big story of progress to democracy faltered.Repression, Rumors, and RevolutionThe 1972 presidential elections became a defi ning moment in many Sal-vadorans’ faith in the possibility of democracy. A progressive coalition is widely believed to have won. But the military party declared victory. Though thousands marched in protest (130,000 gathered in a rally in the Plaza Lib-ertad in the city center), opposition candidate Christian Democrat José Napoleón Duarte was arrested, tortured, and forced into exile. The fraud galvanized many activists. Some sought out the clandestine revolutionary organizations that had been forming among student groups, labor activists, and Christian peasant leaders. Roque Dalton returned secretly to his coun-try to join a new group supporting armed insurrection, the ERP. (In an act that still incites animosity among leftists in the country today, in May 1975 he would be murdered by his own comrades, some of whom accused him of being a spy.37) The rise of the Salvadoran insurgency has been traced by so-ciologist Paul Almeida to the same political dynamic we saw between 1927 and 1932. The liberalization of the 1960s sustained growth in civil society and social movements. In the subsequent political closure of the early 1970s, these groups radicalized. But this time there would be no single Matanza to silence their struggle.Was war inevitable? Was revolution the only story that could follow? The government, trying to stave off revolt, proposed modest land reform in 1976 (and would do so again several times). President Arturo Armando Mo-lina promoted it as an “insurance policy” against social upheaval.38 Land-owners fought the legislation with everything they had—including death squads, which targeted not only leftist activists but also Ministry of Agri-12476-El Salvador in the Aftermath of Peace.indd 3012476-El Salvador in the Aftermath of Peace.indd 3011/25/09 10:53:07 AM11/25/09 10:53:07 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
Big Stories and the Stories Behind the Stories 31culture offi cials. All efforts at reform failed, leading to deeper radicalization of not only peasants but also many members of the middle class. As the national crisis sharpened, the next year’s presidential elections, in 1977, saw even more fraud. This time, state security forces shot at the crowd protesting the results in the Plaza Libertad. Between fi fty and one hundred died.39 For many Salvadorans, the war had started by then. Some historians date the confl ict to the January 1981 FMLN “Final Offensive,” in which hun-dreds died. Others say it began with the 15 October 1979 military coup that overthrew the president, Gen. Carlos Humberto Romero. But people living through those events often recount episodes from the mid- or even early 1970s as part of the war. They might speak of kidnappings, such as those of businessmen Ernesto Regalado Dueñas in 1971 or Roberto Poma in 1977 (both died in guerrilla custody).40 They might name the 30 July 1975 protest march of university students through San Salvador, in which the National Guard and Treasury Police killed at least thirty-seven.41 Or they might point to the assassinations of priests. Between 1976 and 1977 more than twenty-fi ve Catholic priests were imprisoned, tortured, or murdered.42 As one say-ing went at the time, “Be a patriot—kill a priest.”In the 1970s, slogans and rumors circulated as information, as clues to what was happening and what would come next. Any sense of vague-ness about when the war became the big story may have something to do with the lack of verifi able facts. Before the war there had been very little independent journalism in El Salvador. The mass media ignored the ma-jority of the population. Few peasants and urban workers could afford the products advertised.43 (Roque Dalton wrote, “Freedom of the Press/for Don Napoleón Viera Altamirano/and the Dutrizes . . . /is worth several million dollars: . . . what they receive every day from the big/retailers industrialists and ad agencies.” 44) The only television news program in the country, Tele-prensa, largely dedicated itself to “society news.” After all, only the elites and the bourgeoisie—not “the masses”—could afford televisions.45 Most people listened to radio.As guerrilla strength grew, censors choked the already limited dissemi-nation of political news. “The repression was brutal,” the founders of the FMLN’s clandestine Radio Venceremos recall. “The written media became ineffective. If you had a (revolutionary) fl yer with you, this could cost you your life.” 46 Between January 1980 and June 1981, seventeen news offi ces and radio stations would be bombed or machine-gunned, twelve journal-ists would be killed, and three would be disappeared.47 Seeking to broadcast their views, guerrillas would carry out lightning takeovers of radio stations, 12476-El Salvador in the Aftermath of Peace.indd 3112476-El Salvador in the Aftermath of Peace.indd 3111/25/09 10:53:07 AM11/25/09 10:53:07 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
32 Chapter 1arriving with cassettes and guns and demanding the managers play their messages. In early 1980, National Guard major Roberto D’Aubuisson, rising star of the extreme right, organizer of death squads, and founder of the ARENA party, sent his own messages through mass media. He made several televi-sion spots in which he named priests, activists, and others as “part of El Salvador’s terrorist conspiracy” planning to “deliver the country to a totali-tarian regime.” D’Aubuisson’s targets feared for their lives. Some went into exile, some disappeared, some were found dead. The day after D’Aubuisson denounced prominent Christian Democrat Mario Zamora, on 22 February 1980, six armed, masked men broke into his home and shot him ten times.48 Archbishop Óscar Arnulfo Romero, whose increasingly radical masses were broadcast on diocese radio, would be next.49Murder of the ArchbishopIt was, fi nally, the martyrdom of Romero that left no question about war in El Salvador. His murder was a truly big story—one that continues to be told, traveling through Vatican City as theologians and cardinals consider petitions for his sainthood. In 1980, the story brought the entire world’s at-tention to Central America.Romero became archbishop in February 1977, a week before the mili-tary fi red at the protesters in the Plaza Libertad. He said nothing then. Two weeks after that, on 12 March, his friend Father Rutilio Grande was mur-dered along with a little boy and a seventy-two-year-old man. They had been traveling from Aguilares, a town where peasants had begun to participate in political organizations, to celebrate mass in El Paisnal, a rural community just north of San Salvador.Grande’s assassination transformed the archbishop. At that moment he embraced the preferential option for the poor enunciated in the 1968 Latin American Bishops’ Conference in Medellín, Colombia. After the assassina-tion, this new archbishop, originally selected for his conservatism, went to Aguilares to collect the bodies of the victims. He said, “I have the job of picking up the trampled, the corpses, and all that the persecution of the church dumps along the road on its way through.” 50 Romero brought stories of those bodies into his sermons every week. He “put their lives in his mouth.” He became their voice, la voz de los sin voz (the voice of the voiceless). His sermons were full of what many people, 12476-El Salvador in the Aftermath of Peace.indd 3212476-El Salvador in the Aftermath of Peace.indd 3211/25/09 10:53:07 AM11/25/09 10:53:07 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
Big Stories and the Stories Behind the Stories 33especially the poor majority, saw as the reality of El Salvador, despite all state efforts to project a story of a modernizing democracy. Salvadoran Je-suit theologian Jon Sobrino writes: “In front of those who wanted to por-tray what they called the ‘true’ image of the country, Monsignor Romero answered . . . that it had all been built with blood. ‘What good are beautiful highways and airports, all these beautiful skyscrapers, if they are fashioned with the clotted blood of the poor, who will never enjoy them?’ ” 51 In his homilies the archbishop listed names of disappeared people. He read the names of fathers and daughters found discarded on roadsides, he repeated the names of kidnap victims. As he predicted, he became one of those names when he was murdered on 24 March 1980, shot in the chest while saying mass at the chapel in the Hospital Divina Providencia in San Salvador. The day before on the radio he had pleaded with soldiers to put down their arms rather than kill their brothers. To their leaders, he said, “In the name of God, in the name of this suffering people whose cries rise up to Heaven more urgently with each day that passes, I beseech you, I beg you, I order you to stop the repression!” 52 During his funeral mass a week later, a bomb went off in the plaza outside the Metropolitan Cathedral. Snipers shot into the crowd of 50,000. Forty died and two hundred were wounded. The investigation carried out by the Commission on the Truth for El Salvador, more than twelve years later, would conclude that D’Aubuisson had planned the murder. More than 8,000 others died in the war that year.53 Some victims were noncombatants, such as the three hundred Chalatenango villagers murdered in May by National Guard and ORDEN paramilitaries who suspected them of being guerrillas. They died fl eeing across the Sumpul River into Hondu-ras. The Honduran military refused to let them onto the other side.54 Other victims were open members of the political opposition, such as six leaders of the Democratic Revolutionary Front (FDR). They were abducted from a San Salvador school one November morning by a group of Treasury Police offi cers. They were tortured before being murdered and dumped into public streets.55 Though both incidents hit the international headlines, it was the December 1980 murder of three North American nuns and a religious lay worker by the National Guard that got the most global attention.56 Just at that time, in the last months of 1980, the FMLN was forming as a coalition of fi ve guerrilla factions: the Popular Liberation Forces (FPL); Peo-ple’s Revolutionary Army (ERP); National Resistance (RN), a group that had broken off from the ERP after Roque Dalton’s murder; Central American Workers’ Party (PRTC); and Communist Party. It would claim to have more 12476-El Salvador in the Aftermath of Peace.indd 3312476-El Salvador in the Aftermath of Peace.indd 3311/25/09 10:53:08 AM11/25/09 10:53:08 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
34 Chapter 1than 5,000 militia when in January 1981 it launched the “Final Offensive,” named to emulate the May–July 1979 Sandinista Front for National Libera-tion action in neighboring Nicaragua. The FSLN overthrow of the dictator Anastasio Somoza had inspired the Salvadoran guerrilla combatants. After they failed to repeat the Sandinista story—they could not spark an urban popular uprising—they turned to the countryside to raise their army. Some people joined out of outrage at the increasing repression, whether against themselves or family members or priests. Others became guerrillas seek-ing vengeance against local landlords or ORDEN. While some combatants were forcibly recruited by the FMLN, many concluded that revolution was the only way to make change, after the failure of the reform movements of the previous decade. Political scientist Elisabeth Wood identifi es a “pleasure of agency” in the moral commitment and emotional engagement of Salva-doran peasants in the insurgency.57 The guerrilla forces, supported by a vast underground network of civilians who gave them food and supplies, even-tually took about a quarter of Salvadoran territory as “liberated zones.” At its height in 1984, some estimates suggest the FMLN had as many as 12,000 combatants. The Salvadoran armed forces would also grow tremendously at that time. In one of his last acts as U.S. president, on 14 January 1981, Jimmy Carter restored military aid that had been suspended the previous month after the murders of the North American churchwomen. Carter pointed to evidence of Nicaraguan aid to the Salvadoran rebels as a reason to dis-patch another $5 million to the country.58 Over the next eleven years, the United States would send $6 billion in economic, military, and covert aid. The money would go to increase the size of the military from 15,000 to 60,000; to augment military power with helicopters and gunships; and to restructure and professionalize the forces, creating groups such as the Rapid Deployment Infantry Battalions specializing in counterinsurgency tactics. Within a year, on 11 December 1981, the fi rst of those groups, the Atlacatl Battalion, would carry out the largest atrocity of the war, in northern Mora-zán. It killed more than a thousand men, women, and children in an event known as the El Mozote massacre.59 Hundreds of international reporters had arrived in the country by then.60 The place became a proving ground for would-be war correspon-dents, much like Iraq or Afghanistan some twenty years later. At the Univer-sity of Michigan in 2000 I met a journalist on a year’s fellowship who, hearing about my research, told me, “Oh, yeah,” he’d “done Salvador.” Stories were 12476-El Salvador in the Aftermath of Peace.indd 3412476-El Salvador in the Aftermath of Peace.indd 3411/25/09 10:53:08 AM11/25/09 10:53:08 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
Big Stories and the Stories Behind the Stories 35not hard to come by, even as the Salvadoran and U.S. governments tried to deny or downplay military atrocities (most famously in the cover-up of El Mozote). Between 1979 and 1983, as many as 40,000 Salvadoran civilians, including many students and labor and religious activists, were killed.61 The seeming clarity of the struggle—suffering peasants and workers rising up against arrogant oligarchs and generals who refused even the most modest land reform (or, conversely, communists bent on world domination invad-ing a country determined to defend its “freedom”)—caught writers’ and poets’ and fi lmmakers’ imaginations and inspired a broad-based solidar-ity movement in the United States and Europe.62 Many tried to tell the big story themselves. Carolyn Forché wrote a famous poem about dining with a colonel who poured a pile of dried-peach-half-like human ears from a gro-cery sack and said, “As for the rights of anyone, tell your people they can go fuck themselves.” 63 Joan Didion and Oliver Stone produced their separate Salvadors, a dread-fi lled novelesque essay and a frenetic Hollywood movie.64 Italian Gillo Pontecorvo, seeking a project for which he felt as much passion as he had about his anticolonial classic The Battle of Algiers, came close to directing a fi lm on Romero.65This was not the news the U.S. administration wanted to get out. By then the story of saving El Salvador (and redeeming U.S honor after Vietnam) had become an all-consuming project for the young neoconservative idealists who surrounded newly elected President Ronald Reagan. They aimed not only to empower the military to defeat the guerrillas, but also to establish a legitimate government through democratic elections. “The country was brought under U.S. tutelage in a manner unprecedented in Central Ameri-can history, except perhaps for the U.S. occupation of Nicaragua in earlier decades of the twentieth century,” sociologist William Robinson would later declare. “U.S. intervention thoroughly penetrated and transformed Salva-doran society—apart from those areas under FMLN control—from govern-ment ministries, to social service institutions, the private sector the mass media, and civil society organizations.” 66The United States did not immediately fi nd suitable allies in its efforts. The Christian Democrats—or the remnants left after repression and defec-tions to the insurgency—became the “moderate” alternative to the extrem-ists in ARENA. José Napoleón Duarte had returned from exile in Venezuela after the 15 October 1979 coup. He joined the governing military junta the following March. With massive fi nancial support from U.S. sources, he won the 1984 presidential elections against D’Aubuisson. Some analysts date the 12476-El Salvador in the Aftermath of Peace.indd 3512476-El Salvador in the Aftermath of Peace.indd 3511/25/09 10:53:08 AM11/25/09 10:53:08 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
36 Chapter 1Salvadoran transition from this moment, even as it took seven more years and the loss of about 35,000 more lives to end the war.Paradoxical Spaces for StoriesAs Ignacio Martín-Baró, a Jesuit priest and sociologist at the Central Ameri-can University (UCA), would write in 1989—shortly before his murder—the U.S.-led counterinsurgency project starting in 1983 actually opened up public dialogue in El Salvador. It offered a “paradoxical space for com-munication” in which many new stories could be told.67 The newly elected Duarte needed to bolster his democratic credibility in both Salvadoran and world eyes. He began by loosening restrictions on the mass media. What followed would be called a news “boom.” It started with a state-run news program that often clashed with the nation’s business sector but also cri-tiqued the government. In 1985 a commercial program called Noticiero Al Día debuted on Channel 12, owned by independent and politically unaf-fi liated businessman Jorge Zedan. It immediately drew an audience—and advertising dollars. It carried “testimonios by the population affected by the combat . . . [and] the voices of unions, students, combatants.” Many such reports were brought in by a rising young journalist named Mauricio Funes who in 2009 would be elected president on the FMLN ticket. Much of his appeal would later be rooted in a reputation for integrity (he was never a guerrilla and only joined the party when he became a candidate).68 On the program, the outspoken Jesuit priest Ignacio Ellacuría, rector of the UCA (he would be killed in 1989 along with Martín-Baró) would debate rising ARENA activists. Reemergent civil-society organizations, such as labor unions and church human rights groups, would be invited to speak. When the profi tability of Zedan’s Channel 12 venture became clear, more news outlets followed. They began to compete with each other for stories of war. The commodity of representation of violence had found a domestic market. By 1987 one survey showed that three-quarters of San Salvador’s population informed themselves by television news. Five years earlier the majority got their news from the radio.69 The war inspired a new kind of knowledge production in El Salvador, in which big stories could potentially be challenged through networks of news reporting. The thousands of foreign correspondents who had come for the war made an impression on young Salvadorans. Local reporters worked for the internationals as assistants, and with them as colleagues. Often, the 12476-El Salvador in the Aftermath of Peace.indd 3612476-El Salvador in the Aftermath of Peace.indd 3611/25/09 10:53:09 AM11/25/09 10:53:09 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
Big Stories and the Stories Behind the Stories 37guerrillas would speak only to international reporters, who would tip off their local colleagues to events. Despite all horrifi c stories of the war’s sav-agery, then, for most of the 1980s El Salvador was never fully engulfed in what some social scientists and writers have described as a totalizing “cul-ture of terror” or “culture of fear.” 70 People wanted to know; navigating the atmosphere of not-knowing the state promulgated in its efforts to maintain political control, Salvadorans worked to decipher the traces and fragments and aporias and absences that circulated in rumors and news reports. Many of them knew how to read (or how not to read) the right-wing newspaper El Diario de Hoy or to listen to the dominant radio news outlet YSKL, perhaps in tandem with the clandestine Radio Venceremos. “Reality” was not fully up for grabs—even as most news media self-censored while the military watched over them.71 This was the paradox of the war. More knowledge was being produced than ever before about, and within, their country. The war forced the development of a news media infrastructure that had never ex-isted before.72 Of course, in the aftermath of disappearances and kidnappings, in the sites of ongoing repression, a more existential not-knowing envelops every-day living. Anthropologist Linda Green wrote of “fear as a way of life” dur-ing ethnographic fi eldwork among Mayan widows in highland Guatemala in the 1980s. More than 200,000 Guatemalans were killed in the civil war between 1960 and 1996, when peace accords were signed in that country. Green vividly portrays how among the Mayan women she knew fear ex-panded far beyond the coordinates of formal politics, much as it did among rural Salvadorans. Violence and militarization suffused people’s embodied interactions with the world. Green points to the power of ambiguity, of not-knowing why some people were targeted, in maintaining domination over a population’s feelings and activities.73 Susan Coutin infers the same kind of political subjectivity among Central Americans applying for refugee status in the United States in the 1980s and 1990s. During the Salvadoran and Guatemalan civil wars, she writes, “continual violence, surveillance, and interrogation made the causes of persecution unclear and defi ned average people as potentially subversive.” 74 This uncertainty—the inability to dis-cern precisely what actions put one at risk—is a key tactic used in state ter-rorism.Spectacular acts of intimidation—massacres, abductions, torture—decreased in El Salvador after 1984, as the U.S. strategy of democracy pro-motion and low-intensity warfare began to show results. Human rights training seemed to limit more egregious violence (as some offi cers connected 12476-El Salvador in the Aftermath of Peace.indd 3712476-El Salvador in the Aftermath of Peace.indd 3711/25/09 10:53:09 AM11/25/09 10:53:09 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
38 Chapter 1with major abuses were removed from their positions). The guerrillas, too, changed their strategies, breaking into small units and retreating to the countryside to engage in smaller-scale maneuvers. President Duarte pro-posed peace dialogues, and held several summits with the FMLN and the military. Still, the Christian Democrats lost support with continued confl ict and a deteriorating economy. Many people became disillusioned with Du-arte’s inability to deliver promised land reform. The powerful right-wing business sector also fi ercely opposed him, sure he was pro-communist. The October 1986 earthquake exacerbated the crisis, causing $1 billion in dam-age and destroying more than 50,000 homes. ARENA took the majority in the legislative assembly in 1988, and the presidency in 1989.One more horrifi c big story would circulate globally before the war ended. On 11 November 1989, fi ve soldiers from the Atlacatl Battalion shot and killed Martín-Baró, Ellacuría, four of their fellow Jesuit priests, and their housekeeper and her sixteen-year-old daughter, at their residence in the Pastoral Center on the UCA campus. The UN Truth Commission, noting the “nightmare image” of a bullet hole in a portrait of Monsignor Romero, called it “the fi nal outburst of the delirium that had infected the armed forces and the innermost recesses of certain government circles.” 75 The Jesuits were massacred in the midst of a major guerrilla attack in San Salvador. The November 1989 offensive became the most powerful ex-perience of war for many Salvadorans living in the capital. People from poor and working-class barrios of Soyapango have told me of watching hundreds of guerrillas pass by silently at dawn, of sharing their food with the com-pas, and of having to evacuate en masse with white fl ags after days without water or food. Wealthy Escalón residents have shared the shock of seeing dead bodies outside their gates—and the grim amusement with which they watched guerrilla combatants from the countryside peer curiously into their cabinets, confusing cat food and human food. People from other neighbor-hoods have described listening to bombs as they huddled inside their homes with friends and family—and then hearing rumors about the murders of the Jesuits. The news media played a potent role in that event. The knowledge it produced would be simultaneously infused with the not-knowing tactic of state terror. In fact the government took over the airwaves, declaring a state of emergency. Channel 12’s Noticiero Al Día went black rather than broad-cast military-vetted information. At the same time some radios broadcast call-in programs dominated by anonymous voices demanding the murder of “the terrorists.” For days reports insisted that guerrillas had killed the 12476-El Salvador in the Aftermath of Peace.indd 3812476-El Salvador in the Aftermath of Peace.indd 3811/25/09 10:53:10 AM11/25/09 10:53:10 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
Big Stories and the Stories Behind the Stories 39priests (a sign left near the bodies said, “The FMLN has executed the spies who turned on them. Victory or death. FMLN”; some crime storytellers I met nearly a decade later insisted that that version of events was true, even though two army men had been convicted of the murders in 1991 76).Three days into the siege, the FMLN called for a general uprising. The people did not respond as the insurgents hoped. The military, with its air power, drove the guerrilla units out of the capital. The offensive forced all the parties to recognize a stalemate. The FMLN could not win, though the of-fensive proved its continuing capacity to infl ict damage on centers of power. The military could not win, even though ARENA’s Alfredo Cristiani had taken the presidency six months earlier. ARENA, now dominated by a fac-tion of fi nancial elites, wanted to make a deal; many of its leaders were more interested in getting on with business in a globalizing economy than with fi ghting. The U.S. Congress, pressured by constituents outraged by the kill-ing of the Jesuits, threatened to cut military assistance. The new president, George H. W. Bush, was not sure he wanted to keep up a Cold War confl ict in the face of rapidly changing geopolitics. The Berlin Wall had fallen on 9 November, 1989, just days before the San Salvador offensive. The FMLN and the Salvadoran government would meet over a period of nearly two years to negotiate the end of the war, their discussions medi-ated by the United Nations. On 16 January 1992, in Chapultepec Castle in Mexico City, they signed the peace agreements.Stories Behind the Big (Success) StoryThis big story usually ends with the pronouncement of peace. Since that moment when the guerrillas and the government offi cials shook hands in front of the cameras, the United Nations has repeatedly pointed to the Sal-vadoran peacekeeping and human rights monitoring missions as “paradig-matic” and “pioneering.” After the United Nations Observer Mission in El Salvador (known as ONUSAL, its Spanish acronym) shut down in 1995, the Security Council passed a resolution recognizing ONUSAL’s feat in guiding El Salvador as it “evolved from a country riven by confl ict into a democratic and peaceful nation.”77 The Salvadoran showcase buttressed UN Secretary-General Boutros Boutros-Ghali’s vision of an activist United Nations, pro-moting peace through democracy around the world.78 Academic experts and international relations types generally have con-curred that El Salvador offers one of “the most dramatic and positive recent 12476-El Salvador in the Aftermath of Peace.indd 3912476-El Salvador in the Aftermath of Peace.indd 3911/25/09 10:53:10 AM11/25/09 10:53:10 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
40 Chapter 1political achievements in the Western hemisphere.” 79 Such authoritative voices have largely set the tone for the production of knowledge about the Salvadoran story. Political scientist Terry Lynn Karl, for example, labeled it a “negotiated revolution” and pronounced the Central American country’s experience as a model: “Future policy for dealing with regional confl icts can benefi t from the experience of El Salvador.” 80 The reasons for El Salvador’s feats vary in analysts’ accounts, but that the case is a successful one is rarely in doubt. Elisabeth Wood, for example, privileges insurgents’ counterhege-monic agency as crucial in driving the government to the negotiating table. She calls El Salvador’s transition a case of democracy “forged from below via a revolutionary social movement.” 81 William Robinson, in contrast, sees a peace pushed by a transnational, technocratic faction of elites, who formed a “polyarchic democracy” in which a select group of leaders rule and mass participation is limited to voting.82 But what happened next? How was it that so very soon after such a suc-cessful conclusion to a globally storied confl ict, so many Salvadorans were saying it was “worse than the war”? Why did life in peace feel (more) risky and dangerous for the majority of people in the country? 83 What kept them from recognizing El Salvador’s achievement the way the experts did? One could answer, sincerely, “It’s complicated.” Experts in the many public seminars on violence I attended in El Salvador during the 1990s unfurled impressive multicausal charts, with arrows pointing to factors varying from unemploy-ment to international drug traffi cking to endemic acculturation to violence to broken families to lack of a history of dialogue to post-traumatic stress to indexes of economic inequality. In the United States, sensationalist media reports became fascinated with a small number of deportees with criminal records, some of whom formed transnational gangs (the most famous being the Mara Salvatrucha). All these factors undoubtedly play a part. But other stories behind the story are rarely recounted. And an important one insists that the lauded peace accords hardly represent a “negotiated revolution.” In fact the agreements maintained El Salvador’s unjust structures of power. Anthropologist Leigh Binford declared, succinctly, “The 1992 peace accords ended the shooting, but they did not initiate an era of greater social and economic justice.” 84 Indeed, as we know, they did not even end the shooting. Risk and danger just took new forms in postwar El Salvador.The public management of risk may have seemed to be at the core of what was negotiated in nearly two years of meetings between the FMLN and the government in 1990 and 1991. And perhaps it was. Military and po-lice reforms dominated the talks. But not in the sense most observers (and 12476-El Salvador in the Aftermath of Peace.indd 4012476-El Salvador in the Aftermath of Peace.indd 4011/25/09 10:53:10 AM11/25/09 10:53:10 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
Big Stories and the Stories Behind the Stories 41perhaps many participants in the peace agreements) had imagined. In the simplest terms, the FMLN guerrillas had gone to war with an emancipatory politics proposing to radically restructure the economy so that Salvadorans would not have to live desperate lives constantly at risk of disaster. They could not do so, either in battle or at the negotiating table. In the end, it was ARENA that would radically restructure the economy. It was ARENA that would launch the “liberation movement”—but for elites’ unconstrained accumulation of capital, not the masses’ rights to inclusion.85 Individual risk-taking and everyday insecurity, intensifi ed but thoroughly familiar ele-ments of late modernity’s “risk climate,” long part of most poor people’s ontologies, became sanctifi ed features of economic existence in El Salvador after 1992.86 Shock DoctrineThe peace accords marked not a beginning but a midpoint in the imple-mentation of a series of market-oriented reforms in governance commonly known as “structural adjustment” toward a neoliberal economy. When the United States intervened in the war, it aimed to do more than merely defeat the guerrillas, more than just “save” El Salvador from communism. In fact it wanted to knock out the oligarchs as well as the rebels. It aspired to modern-ize the country, principally by remaking it as a model free-market democ-racy in which capital gained maximum mobility. The public struggles over human rights and democratic advances during the twelve-year civil war had masked processes pitched toward economic globalization. El Salvador offers a clear case of what journalist Naomi Klein calls the “Shock Doctrine,” in which societies reeling from some kind of shock are reengineered for neoliberal economics.87 Klein’s prime contemporary exam-ples are wartorn Iraq and post-Katrina New Orleans. But as she points out, the myth of the democratic triumph of free-market capitalism had earlier taken hold elsewhere, fi rst in Chile, then through much of Latin America. For El Salvador, William Robinson points to the year 1983, one of the blood-iest in the civil war, as the beginning of the transition to a new mode of gov-ernance. At that moment the FMLN guerrillas were on the verge of winning. Masses of Salvadorans were fl eeing the chaos as internally displaced persons or refugees and migrants. That same year, a new constitution was written, preparing the country for elections. However, the year 1983 was crucial not just because of the bloodshed, or 12476-El Salvador in the Aftermath of Peace.indd 4112476-El Salvador in the Aftermath of Peace.indd 4111/25/09 10:53:11 AM11/25/09 10:53:11 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
42 Chapter 1the new constitution; it became a pivotal moment because the Foundation for Social and Economic Development (FUSADES) think tank was founded. FUSADES, funded by USAID, aimed to establish a consensus for the devel-opment of neoliberal social and economic policies.88 The U.S. stance was that economic liberalization should accompany the counterinsurgency pol-icy. To that end, in 1985 it began to attach conditions to support to the Salva-doran government, demanding structural adjustment toward a more open economy.89 In 1989 ARENA’s Cristiani won the presidency. His government, following a blueprint drawn up by FUSADES, immediately launched a full-blown economic liberalization program. It started by eliminating price controls, deregulating interest rates, and cutting public spending, especially in public services such as education and health care. Next it reprivatized the banks, granting autonomy to the Central Reserve Bank, a move that allowed the fi nancial sector to concentrate its power.90 Even before the war ended, El Salvador was well on its way to becoming a model—not of peace, at least as imagined by revolutionaries and solidarity movements, but of neo liberalism.Neoliberalism fi rst refers to the tenets of a Smithian market economy. It is linked to theories of Milton Friedman and a generation of University of Chicago economists who opposed interventionist Keynesian economic theories and believed fervently in the democratic possibilities of a radically free market.91 Its earliest state practice emerged in Chile, where Friedman advised dictator Augusto Pinochet. But since the early 1980s it has taken on several new modes of meaning. The mode promoted by the International Monetary Fund (IMF) and the World Bank through the 1980s and 1990s fo-cused on privatizing public entities, encouraging investment, and eliminat-ing trade barriers. This is an orientation many Latin Americans know well. It echoes the impulses of liberals of the late nineteenth and early twentieth centuries. El Salvador’s privatization of communal lands in 1881 and 1882, encouraging peasants to enter the market, perfectly exemplifi ed such classic economic liberalism. The rise of neoliberalism shows once again capital’s creative and fl exible possibilities. It may be a global force but it constantly articulates itself with local conditions.Neoliberalism began to take on additional qualities in the early 1990s, aiming not just to free the markets but also to substitute the market for both state and society. Since the Cold War ended, neoliberalism has intertwined with the very idea of democracy. “Democracy,” anthropologist Julia Paley tells us, “is now so deeply embedded in a prolonged moment of economic and philosophical liberalism that democracy (as ideology, as experience, as 12476-El Salvador in the Aftermath of Peace.indd 4212476-El Salvador in the Aftermath of Peace.indd 4211/25/09 10:53:11 AM11/25/09 10:53:11 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
Big Stories and the Stories Behind the Stories 43expectation, as policy) is co-produced with market economics, a phenom-enon neatly captured by the phrase ‘free market democracies.’ ” 92 (Or have we been duped into calling it democracy? After all, “A democratic election can enhance a country’s credit rating, regardless of how national decisions are made,” as anthropologist of Guatemala Diane Nelson cracks.93)Even as neoliberalism linked to democracy as a political form, in its more philosophical mode neoliberalism recast governing activities as non-political problems in need of technical solutions. It sought rationalizing, optimizing outcomes that maximized effi ciency rather than expanded citi-zenship rights. Furthermore, in a constructivist mode, neoliberalism of the post-Cold War moment did not assume, as in classic liberalism, a natural tendency toward the market. Rather, it aimed to instigate this ostensibly apolitical market rationality in all spheres of society. In particular it has worked as a “technology of self,” fashioning the kind of “free” person desired by the market economy. It aims to produce neo-liberal citizen-subjects as unmoored, ahistorical, and risk-taking yet ratio-nal, private agents.94 Citizens are reenvisioned as “autonomous, individual bearers of rights who ‘must entrepreneurially fashion their overall personal development through wider relations to the marketplace.’ ” 95 In different contexts this project has played out differently, though always in the logic of global capitalism. In the United States, efforts to inculcate individual mar-ket orientations were exemplifi ed by President Bill Clinton’s 1996 welfare re-form in which moralistic “individual responsibilization” became the goal.96 In Guatemala, such subject formation has targeted indigenous peoples in a form of “neoliberal multiculturalism” which, by granting cultural rights while denying equality, has effectively shaped forms of resistance.97 In El Salvador, neoliberal governance has shaped and been shaped through an entrepreneurial culture focused on the informal sector, as well as by both high migration rates and high crime rates. I recall during 1995 and 1996 after massive public-sector layoffs every-one commenting on the proliferation of tiendas (small shops often in peo-ple’s houses) in every neighborhood, as many former state employees used their small buyouts to start their own businesses. Everywhere, people were selling chocobananos (frozen chocolate-covered bananas) out of new freez-ers. Or they were fl ipping pupusas (the national dish, fat tortillas stuffed with meat, beans, and/or cheese) on new grills in garages converted to picnic-table-furnished eateries. Or they were importing used car parts from North American junkyards. Or they had managed to nab some kind of job with one of the many new nongovernmental organizations unevenly carry-12476-El Salvador in the Aftermath of Peace.indd 4312476-El Salvador in the Aftermath of Peace.indd 4311/25/09 10:53:11 AM11/25/09 10:53:11 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
44 Chapter 1ing out some of the functions of a recalibrated state. The decline of govern-ment services and jobs intersected with increased remittances migrants sent home, encouraging more and more individualistic economic orientations. At the same time, as we will see in this book, the rise in crime and the shar-ing of crime stories describing new, individual ways to confront postwar risks also helped to shape this new postwar rationality.This account of legal, political, and administrative intention should draw attention to another aspect of neoliberalism of the late twentieth and early twenty-fi rst centuries. It is not free, in the sense understood through the concept of laissez-faire economics, in which state intervention is minimized in order to let the market do what it will. It requires concerted institutional efforts to develop. Such efforts include those imposed from the outside, as in the conditions attached to loans from the World Bank or stipulations in regional trade agreements. Within national economies, these efforts in-clude the restructuring of the state regulatory apparatus. This neoliberalism also hardly marks a hallowed “end-of-history” form of human organization when fused with liberal democracy, as was declared to much fanfare soon after the fall of the Berlin Wall in 1989.98 It is just another phase in capi-talist expansion. If this historical articulation was not clear to economists before, by 2007 the global economic crisis and massive state intervention that followed seemed to predict the end of the neoliberalism as an ideologi-cally radical free-market economic form. But of course the history of capital reveals it as creative, expansive, fl exible, and ever articulating to changing historical conditions.99Under ARENA even before the war ended, El Salvador’s new economic policies quickly drew praise. In March 1991, the journal Business America boasted, “The El Salvadoran economy . . . has made remarkable strides in the last eighteen months thanks to a bold new stabilization and structural adjustment program.” The journal claimed that the nation’s growth rate had reached 3.4 percent in 1990, the highest in many years. It also pointed to an unquantifi ed “boom” in the agricultural sector, and vague “respectable gains” in manufacturing, commercial and service sectors. It encouraged in-vestors to explore the “wide range of Salvadoran government investment incentives and the renowned work ethic of Salvadoran laborers.” 100 In an immediate post-Cold War world awash with new capitalists seeking invest-ments for their cash, El Salvador looked like a good bet. Such news brought more international support to the country. The World Bank provided struc-tural adjustment loan packages in 1990–91 and 1993–94. The United Na-tions Development Programme would later call El Salvador’s structural 12476-El Salvador in the Aftermath of Peace.indd 4412476-El Salvador in the Aftermath of Peace.indd 4411/25/09 10:53:12 AM11/25/09 10:53:12 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
Big Stories and the Stories Behind the Stories 45adjustment process “one of the most aggressive of Latin America.” 101 By 2001, El Salvador would fi nally dollarize its economy, allowing for an even freer fl ow of investments and migrant remittances but effectively raising the cost of living for most Salvadorans, who would say, “We’re paid in colones but we have to buy in dollars.”The signing of the peace accords thus both marked a kind of artifi cial rupture (since the crucial economic transitions predated and presupposed an eventual “peace,” long before the details were negotiated) and, in a soft parallel to the atmosphere in postsocialist states, allowed an accelerated shift to the transnational fi nance model of capital accumulation. Of course El Salvador had its particular differences, most notably the rising tide of mi-grant remittances, which held up the economy by pumping cash into con-sumer services and generating high savings rates in the banks.102 Were these neoliberal, or market, rationalities entering common sensical ways of knowing the world in El Salvador? How did people understand and represent to each other ongoing experiences of subjectifi cation, of political-economic oppression, of social injustice? How and when might they recog-nize legitimations of continuing, or growing, inequality? 103 These are some of the questions that must follow the grand pronouncement of peace at the end of the story: what happened next?Postwar Crime WaveIn March 1994, my friend Maru had voted with a million and a half other Salvadorans in what were called the country’s “elections of the century.” The hyperbole referred in part to the simultaneous competition for all public offi ces—for the president, assembly deputies, and mayors. These were the fi rst contests in which the FMLN participated as a legalized political party. Thousands of international observers as well as ONUSAL monitored the voting. Though they found many irregularities, they certifi ed the contest as legitimate, if “mediocre,” conducted under “appropriate conditions in terms of freedom, competitiveness and security.” 104 On 1 June 1994, shortly before the truncated trip to Cacaopera that opened this chapter, Maru and I had stood together at the edge of what was then called the Hermano Lejano monument on the Boulevard de los Heroes, waving tiny paper blue-and-white Salvadoran fl ags with her cheering uniformed fi fth-grade students.105 We waited hours for the newly elected president, Armando Calderón Sol of ARENA (for whom she had not voted, Maru assured me), to pass by in a 12476-El Salvador in the Aftermath of Peace.indd 4512476-El Salvador in the Aftermath of Peace.indd 4511/25/09 10:53:12 AM11/25/09 10:53:12 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
46 Chapter 1cacophonous motorcade. Reviewing my photographs years later, I was sur-prised to see that the portly new president had not merely passed by but de-scended from his limousine and mingled with the people. He had stood just two feet away from me. Perhaps these snapshot moments encapsulate the “success” praised by the United Nations: after a brutal civil war there had been a nonviolent competition for power in which the winner could safely step out of his armored vehicle, his bodyguards in relatively inconspicuous positions, and greet crowds.A year after the hopes signaled by Calderón Sol’s inauguration, in 1995—just as the UN Security Council was making pronouncements about Salva-doran success—statistics suggested a cruelty in that optimism. El Salvador’s postwar violence levels matched, and perhaps surpassed, war carnage. The attorney general’s offi ce would report 7,877 intentional homicides, a mur-der rate of 138.9 per 100,000 population—more than the annual wartime violent death rate, the highest in the Americas and second only to South Africa in the world. These dramatic fi gures have since been questioned (one recent estimate suggests that the rate was about 80 per 100,000).106 But for most Salvadorans, usually skeptical of government data, the high fi gure held unique truth-value. It was bolstered by headline after headline, all lament-ing the crime wave, the crime increase, the crime surge. Such media narra-tion helped frame the stories Salvadorans told and circulated of their own and others’ experiences with everyday insecurity. Crime stories amplifi ed uncertainty; talk of crime can be contagious, compelling endless analy-sis, obsessive interpretation, and surplus anxiety.107 A poll just a year after the war ended found that 73 percent of the respondents called violence the country’s principal problem—violence in the form of “crime, thievery, lack of authority, robbery, violations and gangs”—and that 89 percent believed crime had increased since the peace accords.108 These survey results would be replicated through much of the decade, the frustration doubled by the incapacity of the public security bodies (whether this was due to state restructuring or the endemic weakness of the state was unclear). By 1996, 45 percent of the respondents to a national survey agreed that people had the right to take justice into their own hands because the government did not provide justice and security. The same pro-portion approved of the social cleansing of criminals by a clandestine orga-nization called the Sombra Negra (Black Shadow). Two years later more than half of those polled supported such vigilantism.109 As José Miguel Cruz, the independent director of the University Institute of Public Opinion, wrote 12476-El Salvador in the Aftermath of Peace.indd 4612476-El Salvador in the Aftermath of Peace.indd 4611/25/09 10:53:12 AM11/25/09 10:53:12 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
Big Stories and the Stories Behind the Stories 47in 1997, “Concern about the so-called ‘problem of crime’ has reached such a level that Salvadorans have become more alarmed about crime than they were worried about the war in the second half of the 1980s.” 110 “Peace,” then, may have meant absence of war—but it did not feel “better” or more secure than before. It didn’t mean the end of violence. And it didn’t mean justice. “Those who were excluded from judicial protection at the beginning of the war,” writes legal scholar Margaret Pop-kin, who spent years in El Salvador supporting judicial reform, “still found themselves essentially without recourse, both for what happened to them and their families during the war and for cases that have arisen in the post-war period.” 111 The judicial system’s actions only exacerbated distrust in the state: statistics even fi fteen years after the war ended suggest that only 3 percent of cases that made it to the courts ever ended in sentences.112 The vio-lence of the postwar period, apparently unmoored from any sense of deep motivation, any possibility of redemption, would become mere scandal.113 It seemed to delegitimate past struggles, robbing meaning from earlier deaths in which people were portrayed as “assassinated as witnesses to faith” in Monsignor Romero’s words.114 Martyrdom such as Romero’s, celebrated in the Christian theology of the popular churches, lost at least some of its sense, and the political killings their fi erce purpose.115It was precisely after the war, when peace was declared, that Salvadoran mass media and academics and policymakers (left and right) began to frame violence as the problem (rather than a symptom of ideological opposition, or of communism, or of a repressive state apparatus). Barely a day passed without some newspaper columnist agonizing over the seemingly apolitical matters of national and citizen security. Estudios Centroamericanos, the aca-demic journal of the UCA, dedicated a full issue to “The Culture of Violence in El Salvador.” Much of the research was funded by the Inter-American Development Bank (IADB).116 The IADB is one of a number of institutions that sponsored forums on violence in postwar El Salvador during the 1990s: others included the World Bank, the Pan-American Health Organization, and the United Nations Development Programme, as well as national power brokers such as FUSADES. While debates about violence and policing grew in the early postwar years, the Salvadoran government itself was quite anx-ious at that particular moment to dismiss the disorder as an internal issue: to state as a state that postwar criminality was a domestic problem, unre-markable, not of concern to the international human rights community or to global investors. Political leaders’ efforts to assert the state as central-12476-El Salvador in the Aftermath of Peace.indd 4712476-El Salvador in the Aftermath of Peace.indd 4711/25/09 10:53:13 AM11/25/09 10:53:13 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
48 Chapter 1ized entity, exercising what Max Weber called the state’s “monopoly on the legitimate use of physical force in a given territory,” stumbled throughout the next decade.117 It was only when ARENA fell behind the FMLN in the pre-election polls in mid-2003, after several years of economic decline, that it launched a very public anticrime initiative, Plan Mano Dura (Operation Iron Fist), targeting gangs. It fi nally declared everyday crime—at least that embodied in defi ant young men who did not fi t into the neoliberal economic formula—a dire threat to postwar democratic accomplishments.In the mid-1990s, many people I knew in El Salvador, and countless media commentators and state offi cials, thought of the overwhelming sense of everyday danger as a postwar phenomenon rather than as a regional (or global) issue. It has since become clear that crime spikes and surges of violence were not just Salvadoran phenomena in that historical moment. By the middle of the fi rst decade of the 2000s, “everyday violence, public insecurity and deteriorating rule of law” had become central concerns in much of Latin America.118 This rise in criminality has coincided with de-mocratization and structural adjustment, not just in places like El Salvador and Brazil and Haiti, but across the world, in South Africa and Russia and elsewhere. This millennial crime wave is more than fallout of a post-Cold War economic transition. After all, as anthropologists John and Jean Comaroff contend, criminal economies are often the “most perfect expressions of the unfettered principle of supply and demand.” 119 Other analysts link the problem of violence to economic globalizing processes, noting that most “national” economies have been superseded by transnational capital. They point to processes of fragmentation across the planet, to massive disloca-tions in the wake of collapsed imperial orders or disintegrating postcolonial states, and to a “violence of lumpenproletarianization in many of the world’s urban zones.” 120 Indeed theories of market effi ciency tell us that some dis-order is necessary. The market thrives on fl exibility, insecurity, and anxiety, on unevenness of knowledge—seeking the territory or product that no one knows yet, that isn’t already wired into global fl ows, and exploiting that dis-junction. Freedom, in post-Cold War democracies, is all about risk. Danger and disorder are not unintended but part and parcel of a market state, just as in late modernity crisis is “not merely . . . an interruption, but a more or less continuous state of affairs.” 121 Marx and Engels told us a long time ago of the connection between capitalism and crisis: “Constant revolutionizing 12476-El Salvador in the Aftermath of Peace.indd 4812476-El Salvador in the Aftermath of Peace.indd 4811/25/09 10:53:13 AM11/25/09 10:53:13 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
Big Stories and the Stories Behind the Stories 49of production, uninterrupted disturbance of all social conditions, everlast-ing uncertainty and agitation distinguish the bourgeois epoch from all ear-lier ones,” as they wrote in 1848. “All that is solid melts into air, all that is holy profaned.” 122Hope and AngstbereitschaftWhat can a little mugging mean in the midst of a dramatic reordering of social structure, at the turn of a historic epoch? Recalling Maru’s bus-stop assault more than a dozen years later, even knowing now that this kind of criminality would dominate discourse within El Salvador and across the Americas through much of the 1990s, I am still struck by its banality in the context of her own life story. During the months we lived together in 1994, Maru sometimes spontaneously shared glimpses of her experiences in the war era. This was a strong, stubborn young woman who during the 1980s often traveled between Cacaopera, where her family lived in wartorn Mora-zán, and San Salvador, where she worked as a primary school teacher in the mornings and studied for a university degree in education in the after-noons and evenings. She boasted about challenging soldiers at the frequent military checkpoints on her trips—crossing the Lempa River, leaving the military stronghold of San Francisco Gotera—as they demanded all the bus passengers’ papers and sometimes more. “The soldiers thought we were all guerrillas,” she told me. She re-counted one occasion when she had registered (as required) with the mili-tary in order to get permission to bring back cement and nails to repair her mother’s house. The sergeant in charge at one of the stops insisted she must be a guerrilla since “only terrorists need those kinds of things.” Another time a soldier tried to accost her sexually as she sat in the back of the bus. She yelled at him—and then refused orders to get off the bus. Whatever had or had not happened in her past, I believe that what dis-tinguished the assault in June 1994—what made it an unexpected shock—was that Maru simply hadn’t been prepared. Perhaps she lacked the right kind of Angstbereitschaft, or readiness for anxiety, in Freud’s lexicon. Per-haps her “knowledge and sense of power vis-à-vis the external world,” as Freud put it, limited her capacity to respond at the moment, her anxiety diffuse, paralyzing.123 During the 1980s, as the war continued for so many years, she believed she had felt that she was ready for almost anything, that 12476-El Salvador in the Aftermath of Peace.indd 4912476-El Salvador in the Aftermath of Peace.indd 4911/25/09 10:53:13 AM11/25/09 10:53:13 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
50 Chapter 1she knew what might happen, where it might happen, how she might avoid it, and what she might have to do if she couldn’t. At least in retrospect she believed this, despite long experience living in the murk of a terrorist state. After all, she had survived. But this new mode of alarm—amid so many other transformations after the 1992 peace accords—in some strange way confounded her. And so she, like countless other Salvadorans, told her post-war crime story again and again.12476-El Salvador in the Aftermath of Peace.indd 5012476-El Salvador in the Aftermath of Peace.indd 5011/25/09 10:53:14 AM11/25/09 10:53:14 AMMoodie, Ellen. El Salvador in the Aftermath of Peace : Crime, Uncertainty, and the Transition to Democracy, University of Pennsylvania Press, 2012. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/knowledgecenter/detail.action?docID=3441901.Created from knowledgecenter on 2023-10-01 02:51:52.Copyright © 2012. University of Pennsylvania Press. All rights reserved.
9/30/23, 7:52 PMHow the US helped create El Salvador’s bloody gang war | Gangs | The Guardianhttps://www.theguardian.com/news/2020/jan/10/how-the-us-helped-create-el-salvadors-bloody-gang-war1/11Members of the MS13 gang wait in their cell at Chalatenango prison in El Salvador. Photograph: JoséCabezas/ReutersThe story of El Salvador’s gang problem is astudy in shortsighted thinking and DonaldTrump’s policies threaten to make a badsituation even worseby William WheelerFri 10 Jan 2020 01.00 ESTNewsOpinionSportCultureLifestyleIsrael Ticas is racing down the highway, drumming his hands on thewheel of “The Beast”, a tall, boxy police truck that he aims at thesmall, bustling town of San Luis Talpa, about 25 miles south of ElSalvador’s capital, San Salvador.A decades-long veteran of the security forces, Ticas’s first job was as an artistin the counter-terrorism unit, sketching suspected guerillas during thecountry’s 1979–1992 civil war. The experience left him equally as distrustfulof the rightwing generals he had served as of the guerrilla commanders whowould join them among the political elite at war’s end. In most ways, thecountry has never quite recovered since. In 2015, homicides in El Salvadorrivalled the most violent peak of the civil war, and it ranks consistentlyamong the world’s most violent nations. Before long, Ticas spots a body bythe roadside. “It’s fresh,” he observes. “With clothes on.” It hasn’t beenstripped or dismembered. The victim, he says, was likely shot at that spotduring the night.The long readHow the US helped create El Salvador’s bloody gang war
9/30/23, 7:52 PMHow the US helped create El Salvador’s bloody gang war | Gangs | The Guardianhttps://www.theguardian.com/news/2020/jan/10/how-the-us-helped-create-el-salvadors-bloody-gang-war2/11Ticas calls himself a “lawyer for the dead”. A self-taught forensiccriminologist, he locates and digs up the bodies of victims of gang killings,and in so doing, he documents the crimes of the country’s notorious maras,or gangs. On this hot March morning in 2018, his finger is wrapped thick withgauze – a few days earlier, he pricked it on a thorn covered in fluids fromdecomposing bodies. His belt is adorned with a skull-and-crossbonespattern. As always, he carries a pistol in a handbag at his side.But we aren’t here for the body by the roadside. Instead,we stop outside a two-storey concrete building where men in blue-and-whitecamouflage uniforms armed with assault rifles are milling about. Oursecurity detail piles into a Toyota Hilux, and we follow them zig-zagging outof town and into the surrounding sugar cane fields, the convoy kicking up abright cloud of swirling dust. Our destination is a site used by members ofthe local MS-13 gang to rape, torture and execute people. The victims includecivilians, rivals from the Barrio 18 gang, and their own members who breakinternal codes of discipline. After a few minutes, the convoy stops at aparched basin beside the fields, a spot where a river runs during the wettermonths.As the river rises and falls in the jungle terrain, Ticas explains, the landswells and crumbles. So the topography has changed since the site was inuse, several years ago, and his informant has struggled to remember whereall the bodies are buried. Still, Ticas has managed to find 11 of the 21 bodieshis informant says are buried here. The attorney general gave Ticas threemonths to work the location, and today is the deadline. He thinks he can findone more before his time is up and he has brought the informant here tohelp.Ticas’s informant is a lanky young man who wears a balaclava to hide hisface. The night of the murder was his initiation, when he received a call andwas summoned to the site. When he arrived, he was told to dig a hole: awoman would be killed. The woman and her partner had recently moved totown, and the gang suspected the couple had problems with MS-13elsewhere. After an “investigation”, the gang “disappeared” her partner.Grief-stricken, the woman confronted them, screaming at them in the street,threatening to tell the police. They decided to kill her as well. A civilian wasinstructed to get the woman drunk in her home, just up the road from theburial site. Then she would be brought to the informant. His job, theinformant was told, would be to cut off her head – “to prove you have balls”.But one of the gang members rushed the job and struck her in the back of thehead with a machete. She wandered around the house in a stupor, like azombie, smearing her blood on the walls. So he struck her again. And again.And again.Ticas asks him if the victim died in her house or whether they finished heroff at the burial ground. “She was in agony,” the informant says, but notdead. They removed her clothes and dragged her here, then began to chopher up.
9/30/23, 7:52 PMHow the US helped create El Salvador’s bloody gang war | Gangs | The Guardianhttps://www.theguardian.com/news/2020/jan/10/how-the-us-helped-create-el-salvadors-bloody-gang-war3/11Ticas and his team shovel out the topsoil until they reach hard-packed earth,then sweep away the dust with brooms. He surveys the crust, looking for apatch of discoloured soil, a sign that something has been altered. With hisfingers he traces the boundaries of what he sees in the dirt. His men digdown a layer around its perimeter, then level the ground flat. He draws theoutline again and they dig a layer deeper. Gradually, an oval silhouetteappears, the result of soil that has been dug up, oxygenated and repacked.Ticas works the site laterally, instructing his men to dig a trench beside thecavity. They sift the dirt they extract, looking for any clues the perpetratorsor the victim might have left behind.Ticas moves around the grave in a dizzying pattern, fishing out roots androcks, working his way around the hole as if he is playing pool. The cavity isroughly the shape of the African continent. In the lower right corner, aboutwhere Tanzania might be, is a fist-sized hole. He reaches elbow-deep into itand feels what he knows by touch to be a human pelvic bone. It most likelybelonged to a woman. The hole was formed by the decomposition of thefleshy mass around her hips. Over several hours, he combs away the dirt,exposing a human skeleton. Its head is bent backward, as if in supplication.“It’s weird,” says the informant. He was sure they had buried her deeper. Thelimbs seem largely intact, with bits of tattered clothing around them. Ticasclears away dirt from the skull. He uses a turkey baster to clean the scalp,then fishes out broken shards from its face. “Talk to me,” he mutters to thebones. “What do you want to tell me?” After reconstructing her neck,vertebra by vertebra, Ticas gathers her ribs into a pile by the spine. He notesthe slash marks on her breastbone.Something else is amiss. The informant’s victim would have sufferedmachete wounds to the back of the head, but this cranium is intact. Instead,the front of the skull shows signs of being hacked repeatedly. Ticasconcludes that it belongs to a different woman altogether – a name that wasnot on the informant’s list. It is the third body they have found here that theinformant knew nothing about.Get the Guardian’s award-winning long reads sent direct to you every Saturday morningSiEtildd“We haven’t even found a quarter of the fucked-up things these assholeshave done,” says a member of the police crew keeping watch over Ticas andhis team. He, too, wears a mask, with an assault rifle slung over his shoulder,in case the gang’s spotters are watching. Lately, the gang has beendisappearing off-duty members of the police and military and their families.Israel Ticas in his office, standing in front of images of his team members at work and a map of ElSalvador marking points of reported violent crimes. Photograph: Esteban Félix/AP
9/30/23, 7:52 PMHow the US helped create El Salvador’s bloody gang war | Gangs | The Guardianhttps://www.theguardian.com/news/2020/jan/10/how-the-us-helped-create-el-salvadors-bloody-gang-war4/11The murders that occurred here happened in the middle of a truce that thegovernment negotiated between the rival gangs, which was credited withhalving the homicide rate. But the reality, the informant says, is that it taughtthem to hide their victims in clandestine graves such as these. Ticas was notformally trained in forensics, and many of the techniques he uses hediscovered himself. But he is not the only one learning in the process.First, he noticed that the gangs had begun dismembering corpses so theywould fit into smaller holes, making them tougher to spot. Later, they beganstabbing the corpses in the stomach and throat before burying them in orderto release gases trapped inside, so the decomposition process would leave aneven smaller cavity. As they worked to cover up their crimes with increasingsophistication, they even joked that they were making it a challenge forTicas, the informant tells him.The informant had lived in the US for a decade when, in 2013, he wassuddenly deported after missing a court appointment, he says. As soon as hearrived at his family’s house in El Salvador, members of MS-13 showed up athis doorstep. Everyone here must collaborate, they told him. He started as alookout, but before long they said he knew too much about them and wouldhave to join the gang. Today, at 24, he has already committed 31 murders, heclaims. His manner is earnest and agreeable. But Ticas tells me the informantwould just as soon murder us all. “We have a working relationship,” Ticassays. “But he’s a psychopath.”A few months earlier, the informant fell out of favour. His first offence wasunauthorised drinking – members have to ask for permission beforeconsuming alcohol, since intoxication renders them unreliable. Then shortlyafter, he survived a police ambush. The gang assumed he was a collaborator,and they tried to kill him, though he survived again. So he went to the policeand said he could give them information on 20 murders. So far, 105 arrestshave been made because of his cooperation.In addition to revealing where the bodies are buried, the informant mustname names and testify against his former associates. Unlike in the US,where he would probably be offered witness protection, in El Salvador helives on his own, even though the gang would like nothing more than to findand kill him, which they will likely succeed in doing if he doesn’t leave thecountry when the case is finished.Ticas says the gangs appreciate his efforts: some day it could be theirmothers to whom he gives closure. But it’s not hard to imagine his workputting him in their crosshairs. In one scene in The Engineer, a documentaryabout Ticas, a gang member says that if they ever catch him offguard, theywill bury him in one of the graves he has been excavating.But for now, today was a good day. Ticas even thinks he knows the identityof the victim they have found. At the start of this case, the daughter of amissing woman came to him asking for his help. “Have faith,” he told her.“God will help me find your mother.” Each corpse that goes undiscovered isanother family that will never get closure. “It’s days like this that I know thatGod does miracles,” he says.Donald Trump delivering his first State of theUnion address in 2018, during which hementioned MS13. Photograph: Pool/Reuters
9/30/23, 7:52 PMHow the US helped create El Salvador’s bloody gang war | Gangs | The Guardianhttps://www.theguardian.com/news/2020/jan/10/how-the-us-helped-create-el-salvadors-bloody-gang-war5/11The story of El Salvador’s gang problem is a study in shortsightedthinking – from governments in Washington and San Salvador, onboth sides of the political spectrum – that has backfireddisastrously. In his first State of the Union address, PresidentDonald Trump railed against “the savage gang MS-13”, and called onCongress “to finally close the deadly loopholes that have allowed MS-13, andother criminals, to break into our country.”The gang is the president’s favourite public menace to invoke in his bid toconvince US voters that illegal immigration constitutes an urgent crisis and athreat to national security (second only, perhaps, to an “invasion” ofmigrants in caravans seeking asylum in the US – a great many of whom,ironically, are trying to flee the gang’s reach).Rather than a problem to be deported away, however, the reality of the gangis considerably more complex. Born out of the ecology of Los Angeles’s fiercegang warfare, MS-13 was founded in the 1980s by Salvadoran refugees whohad been hardened in a brutal civil war still raging at home. In time, the gangexpanded to include other nationalities, and it spread to other Americancities. Today, in the US, it numbers no more than 10,000 members andfunctions mostly – its penchant for sensational violence aside – like anaverage American street gang, fighting to control neighborhood turf and localdrug sales.In the late 90s, the Latino gangs of LA found an export mechanism: inresponse to MS-13’s growing clout and amid Bill Clinton’s immigrationcrackdown, the US began deporting foreign-born residents convicted ofwide-ranging crimes. Thousands of convicts were sent back to the NorthernTriangle each year – the neighboring Central American countries of ElSalvador, Honduras and Guatemala. Among them were members of MS-13and their LA rivals, the Eighteenth Street gang, or Barrio 18. In a regionreeling from endemic poverty, wars and political violence, the struggle forsurvival and dominance of these Americanised gangsters produced asociological phenomenon.El Salvador had small, disorganised neighbourhood gangs before. But,according to a popular view in El Salvador, these mass deportations changedeverything in the country. Many have come to believe that the US got rid ofits problem at El Salvador’s expense. The state’s institutions had been guttedby conflict, poverty and corruption. The deportees came back from thestreets of LA with tattoos and baggy clothes and brought along with themgang culture, urban warfare tactics and criminal networks from prison. TheSalvadoran youths, a generation of jobless foot soldiers who made easyrecruits, flocked to their banner. The maras have since drawn threegenerations into an escalating cycle of conflict that offers no easy escape.Today, the countries of the Northern Triangle, where the maras predominate,rank among the world’s highest murder rates and account for 75% of themigrants arriving at the southern US border. The maras, in this analysis, arethe primary and most urgent problem facing countries such as El Salvador.El Salvador’s government and its law enforcement have been quick tosupport this view. According to Salvadoran government numbers, there are60,000 gang members and some 10% of the population are dependent on orotherwise tied to the gangs – in a country of just over 6 million.It is not difficult to understand why the authorities are eager to depict ElSalvador’s violence as the original sin. Doing so has allowed the Salvadoranregime to blame the cause not only on a gang culture imported from the US,but on often simplified notions of crime that have little to do with difficultand costly political solutions. Making the gangs the focus of the country’stroubles allows the government to put off engaging with more urgent anddeep-seated problems such as corruption, lack of state institutions andinequality. Politicians have, to much fanfare, introduced violent and
9/30/23, 7:52 PMHow the US helped create El Salvador’s bloody gang war | Gangs | The Guardianhttps://www.theguardian.com/news/2020/jan/10/how-the-us-helped-create-el-salvadors-bloody-gang-war6/11repressive anti-gang measures, often prior to key elections. But evidencesuggests that the gangs’ power has only grown as a result. The maras, so goesthe conventional wisdom, are a crime problem, best countered with severepolice and even military force.The reality is more complicated. The country’s violence is not only the resultof American-imported crime. It was always determined by the legacy of ElSalvador’s civil war and the underlying inequality that had precipitated it,but was never resolved by its outcome. For both of these factors, the USindeed bore considerable responsibility. But neither would be remediedalone by police killing of mareros or the mass imprisonment of gangmembers. If anything, US assistance to Salvadoran regimes to help tackleroot problems that had been exacerbated by the war and its aftermath werein order. Successive Salvadoran governments, with US support, have donelittle, if anything, to address these issues – and have more often made theseproblems worse.The maras will not simply be killed off or arrested away. Neither will theconsequences of their continuing evolution be walled off behind nationalboundaries, increasingly intertwined as they are with the currents of illicitsupply and demand that tie producers to the US, the world’s largest marketfor illegal drugs. As US-led interdiction efforts in Mexico, Colombia and theCaribbean have pushed trafficking routes into Central America – now thetransit corridor for an estimated 88% of US-bound cocaine – the maras havecome into closer contact with brutal Mexican trafficking organisations suchas the Sinaloa cartel and the Zetas, for whom they work as contractors andhired guns.Meanwhile, behind the noisy spectacle of family separations and thedeployment of the US military to its southern border, the Trumpadministration has quietly enacted a wide range of calibrated policy changesto dramatically ramp up the deportation machinery it inherited, and tochoke off immigration across the board. To name just a few: the US hasremoved domestic violence or persecution by gangs – conjoined crises in theregion – as grounds for asylum in the US. It has ended the “temporaryprotected status” that has allowed hundreds of thousands of CentralAmericans to remain in the US legally for years.El Salvador is one of the countries that is most dependent on remittancesfrom abroad, and the fate of some 200,000 of its citizens now hangs beforeUS courts as they decide whether Trump has the authority to revoke theirlegal status. Most extraordinarily of all, the Trump administration signedagreements last year with El Salvador, Guatemala and Honduras that requireUS-bound migrants who pass through these countries to apply for asylumIsrael Ticas taking photographs of the remains of an unidentified woman in Colon, 20km west ofSan Salvador. Photograph: AFP via Getty
9/30/23, 7:52 PMHow the US helped create El Salvador’s bloody gang war | Gangs | The Guardianhttps://www.theguardian.com/news/2020/jan/10/how-the-us-helped-create-el-salvadors-bloody-gang-war7/11The long readI hope you appreciated this article. Before you move on, I was hoping you wouldconsider taking the step of supporting the Guardian’s journalism. From Elon Musk to Rupert Murdoch, a small number of billionaire owners have apowerful hold on so much of the information that reaches the public about what’shappening in the world. The Guardian is different. We have no billionaire owner orshareholders to consider. Our journalism is produced to serve the public interest – not profit motives.And we avoid the trap that befalls much US media – the tendency, born of a desire toplease all sides, to engage in false equivalence in the name of neutrality. Whilefairness guides everything we do, we know there is a right and a wrong position inthe fight against racism and for reproductive justice. When we report on issues likethe climate crisis, we’re not afraid to name who is responsible. And as a global newsorganization, we’re able to provide a fresh, outsider perspective on US politics – oneso often missing from the insular American media bubble. Around the world, readers can access the Guardian’s paywall-free journalismbecause of our unique reader-supported model. That’s because of people like you.Our readers keep us independent, beholden to no outside influence and accessible toeveryone – whether they can afford to pay for news, or not.If you can, please consider supporting us just once from $1, or better yet, support usevery month with a little more. Thank you.ContinueRemind me in Novemberthere first (and allowing the US to send back anyone who failed to do so). Ineffect, this could make it virtually impossible for Central Americans to seekasylum in the US.The human toll of all these changes will be devastating. Of those affected,many will remain in the US, working and living in the shadows. Others willbe forced back to the countries of their birth and will meet violent ends.Many more will return, both clients and cargo of the human-smugglingnetworks now controlled by organised crime. If history is a guide, the gangswill only emerge stronger as a result. Adapted from State of War: MS-13 and El Salvador’s World of Violence byWilliam Wheeler, published on 14 January by Columbia Global Reports Follow the Long Read on Twitter at @gdnlongread, and sign up to the longread weekly email here.Betsy ReedEditor, Guardian USSingleMonthlyAnnual$5 per month$7 per monthOther
9/30/23, 7:52 PMHow the US helped create El Salvador’s bloody gang war | Gangs | The Guardianhttps://www.theguardian.com/news/2020/jan/10/how-the-us-helped-create-el-salvadors-bloody-gang-war8/11The Audio Long Read‘The Eurocentric fallacy’:the myths that underpinEuropean identity podcast2d ago‘We are just gettingstarted’: the plasticeatingbacteria that could changethe world3d agoThe Audio Long ReadFrom the archive: ‘MamaBoko Haram’: one woman’sextraordinary mission torescue ‘her boys’ fromterrorism podcast4d agoMore from HeadlinesUS CongressGovernment shutdownavoided as Senate passesstopgap funding bill6h agoSerbiaGovernment pulls sometroops back from Kosovoborder after warning fromUS5h agoJamaal BowmanDemocrat pulled fire alarmon Capitol Hill beforeHouse vote5h agoMost viewed
9/30/23, 7:52 PMHow the US helped create El Salvador’s bloody gang war | Gangs | The Guardianhttps://www.theguardian.com/news/2020/jan/10/how-the-us-helped-create-el-salvadors-bloody-gang-war9/11
9/30/23, 7:52 PMHow the US helped create El Salvador’s bloody gang war | Gangs | The Guardianhttps://www.theguardian.com/news/2020/jan/10/how-the-us-helped-create-el-salvadors-bloody-gang-war10/11
9/30/23, 7:52 PMHow the US helped create El Salvador’s bloody gang war | Gangs | The Guardianhttps://www.theguardian.com/news/2020/jan/10/how-the-us-helped-create-el-salvadors-bloody-gang-war11/11
SocialScienceMethodologyAUnifiedFrameworkSecondeditionJohnGerring’sexceptionaltextbookhasbeenthoroughlyrevisedinthissecondedition.Itoffersaone-volumeintroductiontosocialsciencemethodologyrelevanttothedisciplinesofanthropology,economics,history,politicalscience,psychology,andsociology.Thisneweditionhasbeenextensivelydevelopedwiththeintroductionofnewmaterialandathoroughtreatmentofessentialelementssuchasconceptua-lization,measurement,causality,andresearchdesign.Itiswrittenforstudents,long-timepractitioners,andmethodologists,andcoversbothqualitativeandquantitativemethods.Itsynthesizesthevastanddiversefieldofmethodologyinawaythatisclear,concise,andcomprehensive.Whileofferingahandyoverviewofthesubject,thebookisalsoanargumentabouthowweshouldconceptualizemethodologicalproblems.Thinkingaboutmethodologythroughthislensprovidesanewframeworkforunder-standingworkinthesocialsciences.JohnGerringisProfessorofPoliticalScienceatBostonUniversity,whereheteachescoursesonmethodologyandcomparativepolitics.HehaspublishedseveralbooksincludingCaseStudyResearch:PrinciplesandPractices(CambridgeUniversityPress,2007),andACentripetalTheoryofDemocraticGovernance(CambridgeUniversityPress,2008).HeservedasafellowoftheSchoolofSocialScienceattheInstituteforAdvancedStudy(Princeton,NJ),asamemberoftheNationalAcademyofSciences’CommitteeontheEvaluationofUSAIDProgramstoSupporttheDevelopmentofDemocracy,asPresidentoftheAmericanPoliticalScienceAssociation’sOrganizedSectiononQualitativeandMultimethodResearch,andwastherecipientofagrantfromtheNationalScienceFoundationtocollecthistoricaldatarelatedtocolonialismandlong-termdevelopment.HeiscurrentlyafellowattheKelloggInstituteforInternationalAffairs,UniversityofNotreDame(2011–12).Downloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:19 BST 2012.http://ebooks.cambridge.org/ebook.jsf?bid=CBO9781139022224Cambridge Books Online © Cambridge University Press, 2012
StrategiesforSocialInquirySocialScienceMethodology:AUnifiedFramework(secondedition)EditorsColinElman,MaxwellSchoolofSyracuseUniversityJohnGerring,BostonUniversityJamesMahoney,NorthwesternUniversityEditorialBoardBearBraumoeller,DavidCollier,FrancescoGuala,PeterHedström,TheodoreHopf,UskaliMaki,RoseMcDermott,CharlesRagin,ThedaSkocpol,PeterSpiegler,DavidWaldner,LisaWedeen,ChristopherWinshipThisnewbookseriespresentstextsonawiderangeofissuesbearinguponthepracticeofsocialinquiry.Strategiesareconstruedbroadlytoembracethefullspectrumofapproachestoanalysis,aswellasrelevantissuesinphilosophyofsocialscience.ForthcomingTitlesMichaelCoppedge,ApproachingDemocracy:TheoryandMethodsinComparativePoliticsThadDunning,NaturalExperimentsintheSocialSciencesDianaKapiszewski,LaurenM.MacLeanandBenjaminL.Read,FieldResearchinPoliticalScienceJasonSeawright,Multi-MethodSocialScience:CombiningQualitativeandQuantitativeToolsCarstenQ.SchneiderandClaudiusWagemann,Set-TheoreticMethodsfortheSocialSciences:AGuidetoQualitativeComparativeAnalysisDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:19 BST 2012.http://ebooks.cambridge.org/ebook.jsf?bid=CBO9781139022224Cambridge Books Online © Cambridge University Press, 2012
SocialScienceMethodologyAUnifiedFrameworkSecondeditionJohnGerringDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:19 BST 2012.http://ebooks.cambridge.org/ebook.jsf?bid=CBO9781139022224Cambridge Books Online © Cambridge University Press, 2012
cambridgeuniversitypressCambridge,NewYork,Melbourne,Madrid,CapeTown,Singapore,SãoPaulo,Delhi,Tokyo,MexicoCityCambridgeUniversityPressTheEdinburghBuilding,CambridgeCB28RU,UKPublishedintheUnitedStatesofAmericabyCambridgeUniversityPress,NewYorkwww.cambridge.orgInformationonthistitle:www.cambridge.org/9780521132770©JohnGerring2012Thispublicationisincopyright.Subjecttostatutoryexceptionandtotheprovisionsofrelevantcollectivelicensingagreements,noreproductionofanypartmaytakeplacewithoutthewrittenpermissionofCambridgeUniversityPress.Firstpublished2012PrintedintheUnitedKingdomattheUniversityPress,CambridgeAcataloguerecordforthispublicationisavailablefromtheBritishLibraryISBN978-0-521-11504-9HardbackISBN978-0-521-13277-0PaperbackAdditionalresourcesforthispublicationatwww.cambridge.org/gerringCambridgeUniversityPresshasnoresponsibilityforthepersistenceoraccuracyofURLsforexternalorthird-partyInternetwebsitesreferredtointhispublication,anddoesnotguaranteethatanycontentonsuchwebsitesis,orwillremain,accurateorappropriate.Downloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:19 BST 2012.http://ebooks.cambridge.org/ebook.jsf?bid=CBO9781139022224Cambridge Books Online © Cambridge University Press, 2012
Thereisnoroyalroadtoscience,andonlythosewhodonotdreadthefatiguingclimbofitssteeppathshaveachanceofgainingitsluminoussummits.KarlMarx,“PrefacetotheFrenchEdition,”Capital(299),quotedinLevi(1999:171)Tohavemastered“method”and“theory”istohavebecomeaself-consciousthinker,amanatworkandawareoftheassumptionsandtheimplicationsofwhateverheisabout.Tobemasteredby“method”or“theory”issimplytobekeptfromworking,fromtrying,thatis,tofindoutaboutsomethingthatisgoingonintheworld.Withoutinsightintothewaythecraftiscarriedon,theresultsofstudyareinfirm;withoutadeterminationthatstudyshallcometosignificantresults,allmethodismeaninglesspretense.C.WrightMills,TheSociologicalImagination(1959:120–121)Surely,inaworldwhichstandsuponthethresholdofthechemistryoftheatom,whichisonlybeginningtofathomthemysteryofinterstellarspace,inthispoorworldofourswhich,howeverjustifiablyproudofitsscience,hascreatedsolittlehappinessforitself,thetediousminutiaeofhistoricalerudition,easilycapableofconsumingawholelifetime,woulddeservecondemnationasanabsurdwasteofenergy,borderingonthecriminal,weretheytoendmerelybycoatingoneofourdiversionswithathinveneeroftruth.Eitherallmindscapableofbetteremploymentmustbedissuadedfromthepracticeofhistory,orhistorymustproveitslegitimacyasaformofknowl-edge.Buthereanewquestionarises.Whatisit,exactly,thatconstitutesthelegitimacyofanintellectualendeavor?MarcBloch,TheHistorian’sCraft([1941]1953:9)Downloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:19 BST 2012.http://ebooks.cambridge.org/ebook.jsf?bid=CBO9781139022224Cambridge Books Online © Cambridge University Press, 2012
ContentsDetailedtableofcontentspageixListoffiguresxviListoftablesxviiPrefacexix1Aunifiedframework1PartIGeneral252Beginnings273Arguments584Analyses74PartIIDescription1055Concepts1076Descriptivearguments1417Measurements155PartIIICausation1958Causalarguments1979Causalanalyses21810Causalstrategies:XandY25611Causalstrategies:beyondXandY29112Varyingapproachestocausalinference327viiDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:24 BST 2012.http://ebooks.cambridge.org/ebook.jsf?bid=CBO9781139022224Cambridge Books Online © Cambridge University Press, 2012
PartIVConclusions35913Unityandplurality36114Settingstandards379Postscript:Justifications394Appendix:Afewwordsonstyle402Glossary407References444Index492viiiContentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:24 BST 2012.http://ebooks.cambridge.org/ebook.jsf?bid=CBO9781139022224Cambridge Books Online © Cambridge University Press, 2012
DetailedtableofcontentsPrefacepagexixThepresentvolumexxAcknowledgmentsxxiv1Aunifiedframework1Theproblemofpluralism3Aunifiedframework(Table1.1)11Clarifications14Exclusions16Terminology18Examples20Democracy21Vouchers21Advicetothereader22PartIGeneral252Beginnings27Generalgoals(Table2.1)28Discovery28Appraisal30Tradeoffs32Findingaresearchquestion37Studythetradition38Beginwhereyouare40Getoffyourhometurf41Playwithideas43Practicedis-belief45ixDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:32 BST 2012.http://ebooks.cambridge.org/ebook.jsf?bid=CBO9781139022224Cambridge Books Online © Cambridge University Press, 2012
Observeempathically47Theorizewildly48Thinkahead50Conductexploratoryanalyses52Concludingthoughtsonbeginnings543Arguments58Criteria(Table3.1)60Truth60Precision61Generality61Boundedness64Parsimony66Coherence68Commensurability68Relevance694Analyses74Definitions(Figure4.1)75Researchdesignversusdataanalysis78Criteria(Table4.1)80Accuracy81Validity,precision,uncertainty(Figure4.2)82Internal/externalvalidity84Sampling86Representativeness86Size(N)88Levelofanalysis90Cumulation91Standardization91Replication92Transparency94Theoreticalfit95Constructvalidity95Severity96Partition100xDetailedtableofcontentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:32 BST 2012.http://ebooks.cambridge.org/ebook.jsf?bid=CBO9781139022224Cambridge Books Online © Cambridge University Press, 2012
PartIIDescription1055Concepts107Thequandaryofdescription110Concepts112Criteriaofconceptualization(Table5.1)116Resonance117Domain119Consistency121Intensionandextension:tradeoffs(Figure5.1)123Fecundity124Differentiation127Causalutility130Strategiesofconceptualization(Table5.2)131Surveyofplausibleconcepts132Classificationofattributes(Table5.3)133Definition:concepttypes134Minimal135Maximal136Cumulative(Table5.4)137Discussion1386Descriptivearguments141Strategies(Table6.1)142Indicators142Syntheses143Typologies144Simpletypology145Temporaltypology145Matrixtypology(Table6.2)146Taxonomy(Table6.3,Figure6.1)147Configurationaltypology(Table6.4)147Sequentialtypology150Associations151Trend151Network151Correlation153Conclusions153xiDetailedtableofcontentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:32 BST 2012.http://ebooks.cambridge.org/ebook.jsf?bid=CBO9781139022224Cambridge Books Online © Cambridge University Press, 2012
7Measurements155Criteria158Overallgoals:reliability/precisionandvalidity(Figure7.1)160Strategies(Table7.1)163Levelsofabstraction164Structure165Aggregation167Scales(Tables7.2and7.3)167Objectives172Approaches:deductiveandinductive173Ethnography175Surveysandexperiments177Cross-referencing179Causalrelations181Corruption:adetailedexample184Expostvaliditytests191PartIIICausation1958Causalarguments197Definitions198Asimplecausalgraph(Figure8.1)200Causalcriteria(Table8.1)202Clarity204Manipulability207Separation212Independence213Impact214Mechanism2159Causalanalyses218Causaleffects219Varietiesoftreatmenteffects(Table9.1)220Varietiesofcausalrelationships(Table9.2)224Departuresfromthetreatmenteffect225Anelaboratedcausalgraph(Figure9.1)228Criteria(Table9.3)231xiiDetailedtableofcontentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:32 BST 2012.http://ebooks.cambridge.org/ebook.jsf?bid=CBO9781139022224Cambridge Books Online © Cambridge University Press, 2012
Treatment232Exogeneity232Variation233Simplicity234Discrete-ness235Uniformity236Evendistribution237Strength238Proximity238Scaleability239Outcome240Variation240Sample242Independence242Comparability(Figure9.2,Table9.4)24610Causalstrategies:XandY256Summary(Table10.1)257Randomizeddesigns(Table10.2)258Examples262Example1:employmentdiscrimination262Example2:corruptioncontrol263Example3:historicelectioncampaigns264Example4:genderandleadership265Example5:democracypromotion267Obstacles268Internalvalidity269Externalvalidity271Conclusions273Nonrandomizeddesigns273Regression-discontinuity(RD)designs(Figure10.1)275Paneldesigns279Cross-sectionaldesigns283Longitudinaldesigns(Table10.3)28511Causalstrategies:beyondXandY291Conditioningandconfounding:aprimer292Conditioning(Figure11.1)293Confounders(Figure11.2)294xiiiDetailedtableofcontentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:32 BST 2012.http://ebooks.cambridge.org/ebook.jsf?bid=CBO9781139022224Cambridge Books Online © Cambridge University Press, 2012
StrategiesofcausalinferencethatreachbeyondXandY299Conditioningonconfounders(Figures11.3and11.4)299Instrumentalvariables(Figure11.5)304Mechanisms(Figure11.6)306Alternateoutcomes(Figure11.7)310Causalheterogeneity(Figure11.8)315Rivalhypotheses(Figure11.9)316Robustnesstests319Causalreasoning321Theassignmentproblemrevisited324Beyondrandomization32612Varyingapproachestocausalinference327Causal-processobservations328Causesofeffects333Necessary/sufficientcausalarguments(Table12.1)335Discussion337Qualitativecomparativeanalysis(QCA)(Tables12.2,12.3,12.4,Figure12.1)342cs-QCA(Table12.2)343fs-QCA(Tables12.3and12.4,Figure12.1)346Discussion350IVConclusions35913Unityandplurality361Qualitativeversusquantitative362Culturalismversusrationalism366Modelsofcausality368Thecovering-lawmodel368Thepotential-outcomesmodel369Pluralismandmonism373Aunifiedaccount37614Settingstandards379Ceterisparibus379Trade-ups381Multimethodresearch382xivDetailedtableofcontentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:32 BST 2012.http://ebooks.cambridge.org/ebook.jsf?bid=CBO9781139022224Cambridge Books Online © Cambridge University Press, 2012
Settingstandards386Thresholdtests387Best-possible,allthingsconsidered389Postscript:Justifications394Thepracticeofsocialscience398Appendix:Afewwordsonstyle402xvDetailedtableofcontentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:32 BST 2012.http://ebooks.cambridge.org/ebook.jsf?bid=CBO9781139022224Cambridge Books Online © Cambridge University Press, 2012
Figures4.1Time-seriescross-sectiondatasetpage774.2Reliability(precision)andvalidity825.1Intensionandextension:tradeoffs1236.1Ataxonomyintree-diagramformat1497.1Ameasurementdiagram1607.2Histogramof“Polity”scaleofdemocracy1718.1Asimplecausalgraph2009.1Anelaboratedcausalgraph2299.2Causalcomparability:twosimpleillustrations24810.1Illustration:theregression-discontinuitydesign27611.1Basicprinciplesofconditioningillustrated29311.2Atypologyofconfoundersusingcausalgraphs29511.3AcomplexDGPillustrated30111.4Theintractableproblemofcolliders30311.5Instrumentalvariablestrategy30411.6Mechanismicstrategies30711.7Alternateoutcomes31011.8Causalheterogeneitystrategy31511.9Rivalhypothesesstrategy31712.1Relatingconfigurationstooutcomeswithfs-QCA348xviDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:32 BST 2012.http://ebooks.cambridge.org/ebook.jsf?bid=CBO9781139022224Cambridge Books Online © Cambridge University Press, 2012
Tables1.1Theframeworksummarizedpage152.1Generalgoalsofsocialscience283.1Arguments:generalcriteria604.1Analysis:generalcriteria815.1Criteriaofconceptualization1175.2Strategiesofconceptualization1315.3Aclassificationoffundamentalattributes:“Democracy”1355.4Cumulativedefinition:“Democracy”1386.1Descriptivearguments1426.2Amatrixtypology:regimetypes1466.3Ataxonomyintabularformat1486.4Aconfigurationaltypology:ideal-typeandradialcategories1507.1Measurementstrategies1647.2Typologyofscales1687.3Asinglescalewithmultipleinterpretations:“Electoralcontestation”1708.1Causalarguments:criteria2039.1Treatmenteffects:anoncomprehensivetaxonomy2239.2Causalrelationships:apartiallist2269.3Causalanalysis:criteria2329.4Violationsofcausalcomparability:apartiallistofconfounders25110.1Strategiesofcausalinference25710.2Atypologyofrandomizeddesigns25910.3Atypologyoflongitudinalresearchdesigns28612.1Necessary-and-sufficientcausalpatterns33912.2cs-QCAtruth-table34412.3Codingmembershipincausalfactorsandconfigurationswithfs-QCA34712.4fs-QCAtruth-tablewithconsistencyscores349xviiDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:43 BST 2012.http://ebooks.cambridge.org/ebook.jsf?bid=CBO9781139022224Cambridge Books Online © Cambridge University Press, 2012
Downloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:43 BST 2012.http://ebooks.cambridge.org/ebook.jsf?bid=CBO9781139022224Cambridge Books Online © Cambridge University Press, 2012
Cambridge Books Onlinehttp://ebooks.cambridge.org/Social Science MethodologyA Unified FrameworkJohn GerringBook DOI:http://dx.doi.org/10.1017/CBO9781139022224Online ISBN: 9781139022224Hardback ISBN: 9780521115049Paperback ISBN: 9780521132770ChapterPreface pp. xix-xxviChapter DOI:http://dx.doi.org/10.1017/CBO9781139022224.001Cambridge University Press
PrefaceThenaturalsciencestalkabouttheirresults.Thesocialsciencestalkabouttheirmethods.HenriPoincaré1Inaverycrucialsensethereisnomethodologywithoutlogos,withoutthinkingaboutthinking.Andifafirmdistinctionisdrawn–asitshouldbe–betweenmethodologyandtechnique,thelatterisnosubstitutefortheformer.Onemaybeawonderfulresearcherandmanipulatorofdata,andyetremainanunconsciousthinker…theprofessionasawholeisgrievouslyimpairedbymethodologicalunawareness.Themoreweadvancetechnically,themoreweleaveavast,unchartedterritorybehindourbacks.GiovanniSartori2Thefieldofsocialsciencemethodologyhasbeenhyperactiveoverthepastseveraldecades.Methods,models,andparadigmshavemultipliedandtrans-formedwithdizzyingspeed,fosteringaburstofinterestinaheretoforemoribundtopic.Onesignofthegrowingstatusofthisfieldisthescholarlyvituperationitinspires.Termssuchasinterpretivism,rationalchoice,post-structuralism,constructivism,randomization,positivism,andnaturalismarenotjustlabelsforwhatwedo;theyarealsofightingwords.Meanwhile,venerabledebatesoverpower,class,andstatusseemtohavesubsided.Itisnotthatwenolongertalkaboutthesesubjects,orcareaboutthem.Yetthereappearstobegreaterconsensuswithintheacademyonnormativepoliticalissuesthantherewas,say,inthe1960sand1970s.Weareallsocialdemocratsnow–forbetter,orforworse.Debatescontinue,especiallyovertheroleofrace,gender,andidentity.However,theydonotseemtobeaccompaniedbyagreatdealofrancor.Thus,overthepastfewdecadesmethodologicaldisagreementshavelargelydisplaceddisagreements1AttributedtoPoincarébyBerelsonandSteiner(1964:14).SeealsoSamuelson(1959:189).2Sartori(1970:1033).xixDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:54 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.001Cambridge Books Online © Cambridge University Press, 2012
oversubstantiveissuesaspointsofconflictatconferences,atfacultymeetings,andoneditorialboards.Methodology,notideology,seemstodefinethemostimportantcleavageswithinthesocialsciencestoday.3Readersdisturbedbythisdevelopmentmayfeelthatthereisaltogethertoomuchmethodologyinhabitingthesocialsciencestoday–toomuchdiscussionabouthowtogetthere,andnotenoughaboutwhat’sthere.TheymaybepartialtoC.WrightMills’admonition:“Methodologists,gettowork!”Thisisconsistentwiththepleaforaproblem-centeredsocialscience,onedirectedtowardsolvingproblemsofpublicconcernratherthantheapplicationofparticularmethods.4Thequestionnaturallyarises,howisonetogotowork?Itisunlikelythatthisquestionisbestansweredinapurelyinductivemanner.V.O.Keypointsout,“Methodwithoutsubstancemaybesterile,butsubstancewithoutmethodisonlyfortuitouslysubstantial.”5Arguably,thebestwaytoensurethatsocialscienceremainsproblem-orientedistocultivateadeepknowledgeofmeth-odologyandalargetoolkitofmethods.Onlyinthisfashioncanonebesurethatsubstantiveproblemsoftheoreticalconcernandeverydayrelevancearedrivingouragendas,ratherthanasearchforvenuestoapplythemethoddujour.ThestakesinourcurrentMethodenstreitareindeedhigh.Atissueisnotmerelywhowillmakeitintothefirst-tierjournalsandwhowillmaketenure,butalsotheshapeandfocusofthesocialsciencesinthetwenty-firstcentury.Thewinnersofourcurrentmethodologicalwarswilldeterminethesortoftrainingthatisofferedtostudents,thesortofadvicethatisofferedtopolicy-makers,andthesortofguidancethatisofferedtothelaypublic.Socialsciencematters–perhapsnotasmuchaswemightlike,butagooddealnonetheless.Andbecauseofitsprominentplaceinshapingthecourseofsocialscience,methodologymatters.ThepresentvolumeThisbookisadramaticallyrevisedandexpandededitionofabookthatappearedadecadeago.6Theoverallargumentremainsintact.However,3In1958,V.O.Keyadmonishedthemembersofthedisciplineofpoliticalscienceforhavingclosedtheirminds“toproblemsofmethodandtechnique”(p.967).Thesamecouldcertainlynotbesaidtoday.4Mead(2010);Shapiro(2005);Smith(2003).SeealsodiscussionofrelevanceinChapter3.5Key(1958:967).6Gerring(2001).Thisvolumealsodrawsonothermanuscriptsandpublicationswrittenoverthepastdecade,e.g.,Gerring(1997,1999,2005,2007,2008,2009,2010);GerringandThomas(2011);GerringandYesnowitz(2006);GerringandBarresi(2003).xxPrefaceDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:54 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.001Cambridge Books Online © Cambridge University Press, 2012
Ihavereformulatedthesubtitlealongwithmanyofthelower-levelarguments,addedagreatdealofnewmaterial,andre-writtenvirtuallyeveryparagraph.Allthingsconsidered,itprobablydeservestobeconsideredanewbook.Inanycase,IhopethatthereaderofthisbookwillfindanimprovedrenditionofSocialScienceMethodology.Beforeenteringthenarrative,itmaybeworthwhileoutliningafewgeneralcontrastsbetweenthisvolumeandothersonthemarket.First,Itakesocialscienceasmyprimaryunitofanalysis.Socialscience,Ibelieve,isnotsimplyanoffshootofthenaturalsciencesorthehumanities.Itis,rather,adistinctiverealmofinquirywithasomewhatdistinctivesetofnormsandpractices.Thus,ratherthanfocusingonaparticulardiscipline,oronscienceatlarge,thisbookaddressesallfieldswhoseprimaryfocusisonhumanactionandsocialinstitutions.Thisincludesanthropology,arch-aeology,business,communications,demography,economics,education,environmentaldesign,geography,law,politicalscience,psychology,publicadministration,publichealth,publicpolicy,socialwork,sociology,andurbanplanning.Frommyperspective,themethodologicalissuesfacedbythesefieldsaresoremarkablysimilarthattheydeserveaunifiedtreatment.Insofarasthebooksucceeds,itmayhelptorestoreasenseofcommonpurposetotheseoftenestrangedfields.Second,Iattempttospeakacrosscurrentmethodological,epistemological,andontologicaldivides–interpretivistversuspositivistversusrealist,quanti-tativeversusqualitative,andsoforth.WhilerecognizingthepersistenceofthesecleavagesIdonotwishtoreifythem.Indeed,theyareoftendifficulttodefine,andinthisrespectareuninformative.7Forexample,tosaythataresearchdesignis“qualitative”or“quantitative”istosayverylittle,asmostissuesofmethodologicaladequacyarenotaboutsheernumbersofobserva-tions(Chapter13).Here,aselsewhere,abstract,philosophicaldiscussionsoftenhavetheeffectofobscuringmethodologicalissues,whichbecomeclearonlywhenframedinahighlyspecific,focusedmanner(andthendonotalwaysfitneatlywithinlargerframeworks).Third,thebookapproachessocialsciencemethodologythroughproseratherthanthroughnumbers.Althoughthetopicpertainsequallytoqualita-tiveandquantitativeresearch,thelanguageofthebookislargelyqualitative.Anarrativeapproachhascertainadvantagesinsofarasonecancoveragreat7ThedistinctionbetweenqualitativeandquantitativemethodsisdiscussedinBradyandCollier(2004);GerringandThomas(2011);GlassnerandMoreno(1989);Hammersley(1992);MahoneyandGoertz(2006);McLaughlin(1991);Shweder(1996);Snow([1959]1993);Seealsoentryfor“Qualitative”intheGlossary.InterpretivismisdiscussedinGerring(2003).xxiPrefaceDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:54 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.001Cambridge Books Online © Cambridge University Press, 2012
dealofmaterialinarelativelyconciseandcomprehensiblefashion.Moreover,manymethodologicalissuesarenotmathematicalinnature;theyhingeonconcepts,theories,researchdesigns,andothermattersthatarebestarticulatedwithnaturallanguage.Evenso,Imakeapointofreferencingstatisticalprocedureswhereverrelevantsoastofacilitatethetransitbetweentheworldofnumbersandtheworldofprose.Itishopedthatthebookwillbeenjoyableandinformativeforthosewhoareschooledinquantitativemeth-ods,aswellasthosemorefamiliarwithqualitativeresearch.8Fourth,thebookaimstoaddressthesubjectofsocialsciencemethodologyinwaysthatwillbeusefultopractitioners.Weshouldremindourselvesthatthereislittlepointinstudyingmethodologyifthediscoveriesofthisfieldaresharedonlyamongmethodologists.RatherthanhighlightingargumentswiththeliteratureIhavesoughttoplacetheseargumentsinfootnotes,inappen-dices,orhaveomittedthemaltogether.Chapters,sections,andtablesareorganizedtofacilitateeasyaccessandreference.Specializedvocabularyisavoidedwhereverpossible,andanextensiveglossaryisincludedtotrytosortoutthelexicon.Finally,thebookplacesthesubjectofsocialsciencemethodologyinabroadhistoricalandintellectualcontext.Itishelpfultorememberthatmostofthequestionswefindourselvesgrapplingwithtodayareiterationsofclassicmethodologicaldebates.Manywereaddressedasfarbackas1843,whenJ.S.Millpublishedthefirsteditionofhispath-breaking,SystemofLogic.SomegobacktoAristotle.Arguably,theintroductionofnewmethodshashadrela-tivelylittleimpactontheunderlyinglogicofsocialscienceanalysis.Thesamedifficultiescropupindifferentcircumstances.Thismayserveascausefordismayorcontentment,dependingonone’sorientation.Frommyperspec-tive,itisanotherindicationthatthereissomethingcentraltothesocialsciencesthatdistinguishesourenterprisefromothers.Wearedefined,toasignificantdegree,byourmethodologicalpredicaments.“God,”noteCharlesLaveandJamesMarch,“haschosentogivetheeasyproblemstothephysicists.”9Whattheauthorsmeanbythisprovocativecommentisnotthatitiseasytopracticephysics,butratherthatitisfairly8Althoughonehearsagooddealofrhetoricnowadaysaboutunitingqualitativeandquantitativemethodologies,thisformidabletaskappearstobeeasiertorecommendthantorender.Hence,thegeneralabsenceoftextsthatspeakmeaningfullytobothaudiences.ButseeBradyandCollier(2004);Firebaugh(2008);Goertz(2006);King,Keohane,andVerba(1994);Lieberson(1985);Ragin(1987,2008);Shadish,Cook,andCampbell(2002).Forfurthercommentsonthequalitative/quantitativedivideseeChapter13.9LaveandMarch(1975:2).xxiiPrefaceDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:54 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.001Cambridge Books Online © Cambridge University Press, 2012
apparentwhenonehasobtainedaresultinthisfield.Theimplicationsofthisfactarefar-reaching.Thenaturalscientistcanaffordtocultivateamethod,confidentthathisorherresults,ifsignificant,willberecognized.Thesocialscientist,bycontrast,mustjustifynotonlyhisorherfindingsbutalsohisorhermethod.Ourblessingandourcurseistobeimplicatedinthesubjectsthatwestudyandtostudysubjectswhoaresubjects,inthefullKantiansense.Asaconsequence,thoseworkinginthesocialscienceshaveharderproblems,methodologicallyspeaking.Wedisagreeonmorepoints,andonmorebasicpoints,andspendmuchmoretimedebatingthesepointsthanourcousinsinthenaturalsciences.Indeed,methodologyiscentraltothedisciplinesofthesocialsciencesinawaythatitisnottothenaturalsciences.(Thereisnofieldof“methodology”inphysicsorbiology.)ClarkGlymourobserves,“Exactlyinthosefieldswhereimpressiveanddominantresultsaredifficulttoobtain,methodologicalconsiderationsarelikelytobemostexplicit,andinnovationsinmethodarelikelytooccurmostoften.”10Inrecentyearsdatahavebecomeavailableonawiderrangeoftopicsandquantitativetechniqueshavebecomeevermoresophisticatedandmoreacces-sibletolayresearchers(viauser-friendlydatapackages).However,thegapbetweenwhatwedoandwhatwemeantodohasnotdiminished.“Methods”and“ontology”stillliefarapart.11Ibelievethattodogoodworkinthesocialsciencesrequiresmorethanmasteringasetoftechniques.Itrequiresunderstandingwhythesetechniqueswork,whyoneapproachmightbemoreappropriateforagiventaskthananother,andhowagivenapproachmightbeadaptedtodiverseresearchsituations.Goodworkinthesocialsciencesisnecessarilycreativework,andcreativeworkrequiresbroadgrounding.12Thegoalofthisbook,therefore,istoexplorethelogicofinquirythatguidesworkinthesocialsciences,aswellasthepragmaticrationalethat,Iclaim,underpinsthesenorms.Methodsareinseparablefrommethodology;wecanhardlyclaimtounderstandonewithoutdelvingintotheother.Thisworkisconcerned,therefore,withwhatsocialscientistsdo,whattheysaytheydo,andwhattheyoughttobedoing.Thesethreeissues,together,constitutesocialsciencemethodology.10Glymour(1980:291).11Hall(2003).12“Morethanotherscientists,”notesMiltonFriedman([1953]1984:236),“socialscientistsneedtobeself-consciousabouttheirmethodology.”xxiiiPrefaceDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:54 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.001Cambridge Books Online © Cambridge University Press, 2012
AcknowledgmentsCommentsandsuggestionsonvariousiterationsofthismanuscriptweregenerouslyprovidedbyArnabAcharya,PaulDragosAligica,JulianArevalo,NeilBeck,StephenBird,TaylorBoas,BobBullock,TomBurke,DaveCampbell,DinoChristenson,DavidCollier,MichaelCoppedge,PearsonCross,Pierre-MarcDaigneault,ThadDunning,ColinElman,TuliaFalleti,JonFarney,MarshallGanz,GaryGoertz,KristinGoss,SteveHanson,AndyHarris,DavidHart,DanielHidalgo,PeterHoutzager,AlanJacobs,MichaelJohnston,ElizabethKaknes,OritKedar,MarkusKreuzer,DougKriner,DanKryder,MarcusKurtz,DavidLyons,JimMahoney,MichaelMartin,AmyMazur,PatrickMello,RobMickey,DougMock,JairoNicolau,NathanNunn,BetsyPaluck,PaulPierson,HowardReiter,NealRichardson,BenoîtRihoux,IngoRohlfing,KateSanger,CarstenSchneider,JaySeawright,RudySil,Svend-ErikSkaaning,ThedaSkocpol,DawnSkorczewski,LaurelSmith-Doerr,CraigThomas,JohnWilliamson,andJoshuaYesnowitz.Moreinformal–butnolessuseful–wereconversa-tionsandemailexchangeswithNikBlevins,BenCampbell,RussFaeges,GarrettGlasgow,LincolnGreenhill,CathyHarris,SamanthaLuks,JeffMiron,JimSchmidt,LauraStoker,StromThacker,NedWingreen,andChrisWinship.Iwasalsofortunatetohavetheinputofparticipantsatvarioustalksatwhichportionsofthemanuscriptwerepresented:attheUniversityofConnecticut,BostonUniversity,theUniversityofCaliforniaatBerkeley,UniversityofMassachusettsAmherst,theUniversityofVirginia,andtheInstituteforQualitativeandMultimethodResearch(currentlysituatedattheMaxwellSchool,SyracuseUniversity).ThebookisvastlyimprovedduetocommentsreceivedfromreviewersforCambridgeUniversityPressandfromseveralgenerationsofstudentsinmygraduatemethodscourseatBostonUniversity.IoweaspecialroundofthankstoBearBraumoeller,PatrickJohnston,EvanLieberman,andDavidWaldner,whoblessedthemanuscriptwithextensivecriticismandwhoseenthusiasmformethodsandbreadthofinterestspromptedmanydiscussionsandmorethanafewrevisions.MydebttoDavidCollierwillbeapparenttoall.Lateintheday,AdamGlynnandIbegantoworktogetheronalternativeapproachestocausalinference,understoodthroughcausalgraphs.HisxxivPrefaceDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:54 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.001Cambridge Books Online © Cambridge University Press, 2012
contributiontotopicsaddressedinChapter11warrantsspecialcredit.Forhiscreativityandhisvastknowledge,Iamgrateful.AfinalacknowledgmentbelongstoallthepublishedworkonmethodologythatIborrowfrom.Althoughitwouldbetedioustolistauthorsbyname,thelengthybibliographyandcrowdedfootnotesserveasanexpressionofmygratitude.xxvPrefaceDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:54 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.001Cambridge Books Online © Cambridge University Press, 2012
Downloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:35:54 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.001Cambridge Books Online © Cambridge University Press, 2012
Cambridge Books Onlinehttp://ebooks.cambridge.org/Social Science MethodologyA Unified FrameworkJohn GerringBook DOI:http://dx.doi.org/10.1017/CBO9781139022224Online ISBN: 9781139022224Hardback ISBN: 9780521115049Paperback ISBN: 9780521132770Chapter1 – A unified framework pp. 1-24Chapter DOI:http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge University Press
1AunifiedframeworkThosesciences,createdalmostinourowndays,theobjectofwhichismanhimself,thedirectgoalofwhichisthehappinessofman,willenjoyaprogressnolesssurethanthatofthephysicalsciences,andthisideasosweet,thatourdescendantswillsurpassusinwisdomasinenlightenment,isnolongeranillusion.Inmeditatingonthenatureofthemoralsciences,onecannothelpseeingthat,astheyarebasedlikephysicalsciencesontheobservationoffact,theymustfollowthesamemethod,acquirealanguageequallyexactandprecise,attainingthesamedegreeofcertainty.NicolasdeCondorcet1Thereis…progressinthesocialsciences,butitismuchslower[thaninthenaturalsciences],andnotatallanimatedbythesameinformationflowandoptimisticspirit.Cooperationissluggishatbest;evengenuinediscoveriesareoftenobscuredbybitterideologicaldisputes.Forthemostpart,anthropologists,economists,sociologists,andpoliticalscientistsfailtounderstandandencourageoneanother…Splitintoindependentcadres,theystressprecisioninwordswithintheirspecialtybutseldomspeakthesametechnicallanguagefromonespecialtytothenext.Agreatmanyevenenjoytheresultingoverallatmosphereofchaos,mistakingitforcreativeferment.EdwardO.Wilson2Thesubjectofthisbookisthesetofdisciplinesknownasthesocialsciences(whichinearliertimeswouldhavebeenreferredtoasthemoralorhumansciences).Bythisismeantascientificstudyofhumanactionfocusingonelementsofthoughtandbehaviorthatareinsomedegreesocial(nonbiolo-gical).“Theobjectofthesocialsciences,”writesHansMorgenthau,“isman,notasaproductofnaturebutasboththecreatureandthecreatorofhistoryinandthroughwhichhisindividualityandfreedomofchoicemanifestthem-selves.”3Wherevernurturemattersmorethannature,orwheresomesignifi-cantdecisionalelementisinvolved,weareontheturfofsocialscience.(Thisdoesnotmeanthatgeneticdispositionsareeliminatedfromconsideration;1Condorcet(writingin1782),quotedinScott(1998:91).2Wilson(1998:198).3Morgenthau(1955:441).SeealsoAlmondandGenco([1977]1990).1Downloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
indeed,theycompriseanactiveresearchagendainthesocialsciencestoday.4However,onepresumesthatanyoutcomeofinteresttothesocialsciencesisnotentirelybiologicallydetermined;theremustbeasignificantcomponentofchoice.5)Atthesametime,andinmarkedcontrasttothehumanities,mostresearch-ersinthesocialsciencestaketheirmonikerseriously.Theyaspiretoscience–whichistosay,theyintendtostudyhumanactioninasystematic,rigorous,evidence-based,falsifiable,replicable,generalizable,nonsubjective,transparent,skeptical,rational,frequentlycausal,andcumulativefashion.6Afundamentalpremiseofthisbookisthatthesocialworldcanbestudiedinascientificmanner(understoodthroughtheforegoinglistofattributes).Thisdoesnotmeanthattheinstrumentsofscienceprovidetheonlymethodofinsightintohumanbehavior.Theclaim,rather,isthatscienceoffersavalidapproachtoexplanationandunderstandingandthatthisapproachisproperlylocatedwithintheacademicdisciplinesknownasthesocialsciences(Chapter14).Socialsciencethustakesitscuesfromitscomponentterms,socialandscience.WhilethesetermsoftenseemtobeintensionwithoneanotherIwanttoproposethatthistensioncanalsobeaproductiveone,generatinginsightsintooursubjectthatmightnotbeapparenttoalayobserver.Sodefined,socialscienceencompassesthedisciplinesofanthropology,archaeology,business,communications,demography,economics,education,environmentaldesign,geography,law,politicalscience,psychology,publicadministration,publichealth,publicpolicy,socialwork,sociology,andurban4AlfordandHibbing(2008);Carey(2002);FowlerandSchreiber(2008);FreeseandShostak(2009);InstituteofMedicine(2006).5Thismattersquitealottotheconductofscientificinquiry,justifyingthefocusofthisbookonsocialscience,notscienceingeneral.InclaimingadistinctionbetweensocialscienceandnaturalscienceIamnot,ofcourse,assertingacleardichotomy;indeed,manydisciplinesstraddlethedivideandtherearemanyfeaturesofsciencethataresharedbyallscientificenterprises,whetherfocusedonnaturalorsocialphenomena.However,thedistinctionisimportantbecausethenatureofthephenomenaaresodifferentthattheyoftenrequireratherdifferentapproaches.Asignofthiscanbefoundinthedefinitionofthekeyterm“experiment.”Whileinnaturalsciencethetermisusedlooselytorefertoanymanipulatedtreatment,insocialscienceithascometohaveamuchmorespecificdefinition:atreatmentthatisrandomized(andprobablymanipulated)acrosstreatmentandcontrolgroups.Thisisbecausecontrolgroupsaregenerallynecessaryinordertoovercomepotentialconfoundersinasocial-sciencesetting,whiletheyareoftenunnecessaryinnatural-sciencesettings(e.g.,whentwofluidsarecombinedinabeakerandtheresultisimmediatelyobservable).SeeCooketal.(2010:109).6Ihavepurposefullystrungtogetheralltheadjectivesthatarecommonlyappliedto“science,”forminganideal-typedefinition(seeChapter5).Foracompendiumofdefinitionsfromprominentwriterssee:www.gly.uga.edu/railsback/1122sciencedefns.html.ForworkaddressingthemeaningofscienceinamorenuancedandextendedfashionseeLaudan(1983)andSchaffer(1997).Evidently,thereisdisagreementoverhowtodefinescience,andovertheutilityofthescientificideal–howeverdefined.Forcriticalviews,seeBarnesandBloor(1982);Feyerabend(1975);Harding(1986,1987);LatourandWoolgar(1979);Woolgar(1988).2Socialsciencemethodology:aunifiedframeworkDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
planning,alongwithvariousoffshootsofthesedisciplines.Ofcourse,thesocialsciencelabeldoesnotencompassallpractitionersofalltheaforemen-tioneddisciplines,forsomepractitionersareengagedinstudyingaspectsofhumanbehaviorthatarelargelybiological(e.g.,cognitivepsychology),andothersdonotacceptthegoalofscience,orhaveadifferentviewofitthanispresentedhere.(Forexample,theymightquestionthepossibility,orthepay-off,ofgeneralizingabouthumanbehavior.7)Ishallhavesomethingtosayaboutobjectionstosocialsciencelateron(seeChapter14).Forthemoment,itshouldbestressedthatmyunderstandingofsocialsciencewillnotpleaseeveryone,andthoseunhappywiththepointofdepartureareunlikelytobehappywiththepointofarrival.Somemayregardmyperspectiveonthesocial-scientificenterpriseasundulypositivistic.Othersmayregarditasnotpositivisticenough.(Muchdependsonone’sdefinitionofthatvexedterm,positivism,discussedbrieflyintheGlossary.)TheproblemofpluralismAnybookpurportingtoaddressthebroadrubricofsocialsciencemustcometotermswiththemanydivisionsthathauntthesefields,andtherepercussionsofthosedivisions.Asearlyas1938,JohnDeweycomplained:“Oneofthechiefpracticalobstaclestothedevelopmentofsocialinquiryistheexistingdivisionofsocialphenomenaintoanumberofcompartmentalizedandsupposedlyindependentnon-interactingfields.”8Arguably,socialscienceisnotasingleendeavor,butrathermanydifferentendeavors,eachwithitsownpeculiarities,asaverredbyE.O.Wilsonintheepigraphtothischapter.Thesocialsciencesaredivided,firstofall,amongtheseparatedisciplines:anthropology,archeology,etc.Althoughscholarsoccasionallycrosstheseborders,suchcrossingsarearduousandoftenproblematic.Itisnosurprise,then,thatforthemostpart,anthropologistsassociatewithotheranthropol-ogists,andeconomistswithothereconomists.Whethersustainedbymethod-ologicaldifferences,organizationalincentives,orsimpleinertia,academicstendtosticktotheirowntribe.7Thisistrue,forexample,forthosewhoembraceapoststructuralistorpostmodernistperspective(Norris1997;Rosenau1992).Amoredifficultquestionofclassificationconcernsculturalanthropology,history,andotherfieldsorsubfieldswithaninterpretivistbent.Theyareclearlysocialandempirical,buttheyarealsoleeryofscience–especiallythescientificquesttogeneralizeaboutpatternsofhumanbehavior.Inthisrespect,theymayfallsomewhatoutsidetheframeworkdescribedinthisbook.Examplesofself-consciouslyscientificmethodologyappliedtoculturalanthropologycanbefoundinBrimandSpain(1974)andRadcliffe-Brown([1948]1957,1958),butarelessfrequentincontemporarywork.8Dewey(1938:509).ContemporaryworkonthisissueincludesEastonandSchelling(1991).3AunifiedframeworkDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
Thesocialsciencesaredivided,second,amongsubdisciplinaryfields.TheAmericanPoliticalScienceAssociationcurrentlyrecognizesforty-oddsec-tions(e.g.,federalismandintergovernmentalrelations,lawandcourts,legis-lativestudies,etc.),theAmericanEconomicsAssociationseveralhundred.Similardivisionsmaybefoundelsewhere.Thesecubbyholesdefinecourses,jobs,conferences,journals,andscholarlyactivitygenerally.Theycomprisethedefactoboundariesofmostacademiclives.9Thesocialsciencesaredivided,third,amongspecificsubstantiveproblems.Somestudythewelfarestate,othersstudyethnicconflict,andothersstudymarketbehavior.Aproblem-centeredapproachtosocialsciencepresumesthat,becausetherearemanyproblems,eachwithitsownspecificmethodologicalobstaclesandopportunities,therearemanywaysofgoingaboutbusiness.10Thesocialsciencesaredivided,fourth,amongtheoreticalframeworks,eachwithitsownimplicitorexplicitmethodology.Behavioralism,conflicttheory,ethnomethodology,exchangetheory,institutionalism,interpretivism,ordin-arylanguage,rationalchoice,structural-functionalism,symbolicinteraction-ism,systemstheory(cybernetics),andtheresearchschoolsassociatedwithFreud,Marx,andWebereachoffertheirownresearchparadigm.11Thesocialsciencesaredivided,finally,andperhapsmostimportantly,bytheirmethods.Themethodologicaltooloneemploys,forexample,experiments,time-seriesanalysis,factoranalysis,formalmodels,surveyresearch,archivalresearch,ethnography,qualitativecomparativeanalysis,andsoforth,helpstodefineoneasascholarandprobablyalsoaffectshowoneviewsthesocialworld.12Beyondthesefinedivisionsliesoneover-archingcleavagebetween“quants”and“quals,”thatis,betweenthosewhoarecomfortablewithstatisticalanalysisandmathematicalmodelsandthosewhopreferthetime-honoredexpedientsofinformallogicandnaturallanguage.Thisdivision,inevidenceforwelloveracentury,continuestoprovokeandoffend.Asthereaderisnodoubtaware,quantoidsandqualtoidshavedevelopeddifferentlanguagesanddifferentapproachestotheirtopics.Theyareaccustomedtoarguingwitheachotherorignoringeachother.139Almond(1990b).10Shapiro(2005);Smith(2003).11Collins(1985);Parsons(2007);SilandDoherty(2000);Tang(2010).12MosesandKnutsen(2007).13Animpressionexistsamongsomequantitativiststhattheircolleagueswritingprose(particularlythosewritinggoodprose)arecompensatingforalackofrigor.“Ifyoucan’tmeasureit,”goestheunstatedpremise,“thatain’tit.”Acorrespondingimpressionexistsamongsomequalitativiststhattomeasuresomething–to“reduceittoavariable”–istoimpoverishourunderstandingofaphenomenon.“Ifyoucanmeasureit,”goestheircredo,“thatain’tit.”Kaplan(1964:206)attributesthisdictumtomembersof4Socialsciencemethodology:aunifiedframeworkDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
Divisionswithinthecontemporarysocialsciencesarethereforedeepandcomplex,involvingdisciplinary,subdisciplinary,problem-based,theory-based,andmethod-basedcleavages.Fromtheobviousfragmentationofthesocialsciencestodayitisasmallsteptoapluralisticsocialsciencemethodology.Thisacceptswhatis,byallappearances,anirrevocablefactontheground.RichardMillerargues:thereisnoframeworkofempiricalprinciplesdeterminingwhatcountsasanexpla-nationinallsocialsciences.Rather,thereareparticularframeworksforparticularfields.Eachspecificframeworkis,inturn,highlycomplex,withcomponentsservingmanyfunctions.Whetheratruehypothesisexplains,orwhetherahypothesisshouldbeacceptedasexplaining,inlightofgivendata,isdeterminedbyfactsspecific,say,tothestudyofpowerstructuresorinvestmentdecisions.14Methodologicalpluralismhasanappealingairtoit,suggestingtoleranceforapproachesemployedbyotherscholarsandpragmatisminselectingone’sownapproachtoatopic.Beagoodcraftsman,C.WrightMillsadvisesusinafamouspassage:Avoidanyrigidsetofprocedures.Aboveallseektodevelopandtousethesociologicalimagination.Avoidthefetishismofmethodandtechnique.Urgetherehabilitationoftheunpretentiousintellectualcraftsmanandtrytobecomesuchacraftsmanyourself.Leteverymanbehisownmethodologist;leteverymanbehisowntheorist:lettheoryandmethodagainbecomepartofthepracticeofthecraft.15Thereareevidentlymanywaystodogoodsocialscience.Methodsmaybestatisticalornonstatistical,large-Norsmall-N,historicalornonhistorical,andsoforth.Theoriesmaybeusefulforoneproject,anduselessforanother.MuchdependsonthenatureoftheevidenceavailableandthenatureofthequestiontheUniversityofMichiganfaculty(satirizingtheanti-quantoids).ThesameopposingsentimentscanalsobefoundinstatementsutteredlongagobyLordKelvin(“Whenyoucannotmeasureit,whenyoucannotexpressitinnumbers,yourknowledgeisofameagreandunsatisfactorykind”)andJacobViner(“Whenyoucanmeasureit,whenyoucanexpressitinnumbers,yourknowledgeisstillofameagreandunsatisfactorykind”).QuotedinBerelsonandSteiner(1964:14).SeealsothewordsofRobertFogel(onthequantside)andCarlBridenbaughandArthurSchlesinger,Jr.(onthequalside),recordedinLandesandTilly(1971:12).Anti-quantificationistmanifestosmaybefoundinWinch(1958)andWolin(1969).Forotherexamples,includingstatementsbyDanielBoorstin,CarlBridenbaugh,BarringtonMoore,ArthurSchlesinger,andE.P.Thompson,seeFischer(1970:94–96).Forhistoricalbackgroundonthequal–quantdistinctionseeSnow([1959]1993).CurrentstatementsonthesubjectincludeBradyandCollier(2004);GerringandThomas(2011);GlassnerandMoreno(1989);Hammersley(1992);MahoneyandGoertz(2006);McLaughlin(1991);Shweder(1996);Snow([1959]1993).ForfurtherdiscussionseeChapter13.14Miller([1983]1991).SeealsoCartwright(2007);Hall(2003);Hitchcock(2007);Little(1991);Miller(1987);Reiss(2009);Roth(1987).15Mills(1959:224),quotedinEldridge(1983:37).5AunifiedframeworkDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
underinvestigation.Itwouldbefolly,therefore,toproposeauniformmethodortheoreticalframeworkforallofsocialscience,orevenforasinglediscipline.Inanycase,specializationisnecessaryinorderforsocialsciencetothrive.Perhaps,then,thecurrentfragmentationofsocialscienceisthehappyoutcomeofdifferentscholarsdoingwhatthey,individually,dobest.Perhapsweoughttoregarddiversityasamarkofdisciplinarymaturityratherthanasamarkofconfusionanddisarray.Inaddressingthisquestion,Ishallinvokeadistinctionbetweenmethodsandmethodology.16Thefirstreferstoaspecificprocedureforgatheringand/oranalyzingdata.Thesecondreferstothetasks,strategies,andcriteriagovern-ingscientificinquiry,includingallfacetsoftheresearchenterprise.Whilemethodreferstotheparticularchoicesmadeinagivenstudy,methodologyreferstothelargerandmorepersistentfeaturesofthescientificenterprise.Methodspluralismiseasytojustify,andimpossibletoavoid.However,therearereasonstodoubtthewisdomofmethodologicalpluralism.Beneaththediversityofmethodsthereis–oratleastoughttobe–amethodologicalconsensus.17Considersomeofthepracticalquestionsthatwefaceinthequotidianworkofsocialscience.How,forexample,shouldwechooseourmethodsandtheoreticalframeworks,andhow,atthesametime,mightwejudgetheproductofourchoices?Itisapparentthatthesequestionsareuncleartomanysocialscientists,eventothoseworkingwithinthesamesubfield.CharlesLindblomrelatesthefindingsofarecentreviewofliteraturecoveringasmallsubfieldofpoliticalscience.Progresshasbeennotable,theauthorreports.Yet,Lindblomdiscoversthattheseclaimswerenotpositedbydetailingfindingsbutratherbyallegingthatpoliticalscientistshad“illuminated,”“wereconcernedwith,”“gavespecialemphasisto,”“devel-opedinsights,hypotheses,andanalyticalcategories,”“codified,”“stressedtheimpor-tanceof,”“examinedthesignificanceof,”“placedinthecontextof,”“treatedvariablestheoretically,”“producedgoodwork,”“werefruitful,”“appliedconceptsandmodelsto,”“vastlyimprovedourunderstanding,”“dealtwith,”and“increasedthelevelofrigor.”18Thereviewer’smethodologicaldifficultiesarecharacteristicofthesocialsciencesatlarge.Withinmanydisciplines–and,afortiori,acrossdisciplines–weseemtohavenoclearwayofchartingprogress.16Sartori(1970).17MyperspectiveechoesthatofarecentbookeditedbyHenryBradyandDavidCollier(2004),subtitledDiverseTools,SharedStandards.18Lindblom(1997:257).6Socialsciencemethodology:aunifiedframeworkDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
Althoughsomestandardsapplyonlytoparticularfieldsortopicstheremustalsobestandardsapplyingtosocialscienceatlarge.Otherwise,wecannotmakedecisionsamongavailablemethodsandtheoreticalframeworks.Onwhatbasisdoesthemethod-pluralistchoosehisorhermethod?Itdoesnotmakesensetoarguethatnormsoftruthshouldbefield-specificorsteepedinaparticulartradition.Forifstandardsoftruthareunderstandableonlywithinthecontextofspecificfieldsortheoreticaltraditionsthereisnowaytoadjudi-cateamongcontendingviews.Wheretraditionsaredeemedtobeincommen-surable,whateverscholarsinasubfielddecidetobelievebecomes,byvirtueofthatfact,true(aslongasscholarsdonotviolatetheirownnorms).ThissortofepistemologicalrelativismisnotwhatMiller,Mills,andothersintend,butitdoesseemtobeanecessaryconclusionifoneistoaccepttheassertionthatmethodologicalnormsarefield-specific.Whileitisreasonabletocultivateadiversityoftools,itisunreasonabletocultivateadiversityofmethodologicalstandards.19Adiscoveryinsociologyoughttobeunderstandable,andappraisable,bythosewhoarenotsociolo-gists;otherwise,itcannotclaimthestatusoftruth,asthattermisgenerallyunderstood.“Thetheoreticalaimofagenuinediscipline,scientificorhuma-nistic,istheattainmentoftruth,”writesE.D.Hirsch,“anditspracticalaimisagreementthattruthhasprobablybeenachieved.Thus,thepracticalgoalofeverygenuinedisciplineisconsensus–thewinningoffirmlygroundedagreementthatonesetofconclusionsismoreprobablethanothers.”20Norwillitsufficetoconcludethatmethodologiesmustbeappropriateto“context.”21Whichcontexts,andhowmany,willbeprivileged?Andhowmightonejustifyone’schoiceoftoolsandargumentswithinagivencontext?Itisallverywelltosay,ashard-nosedpractitionersarewonttosay,thattheproofisinthepudding(i.e.,thatwecanjudgesocialscienceworkonlybyitsproduct,notitsmethod).Butiftheproofisinthepudding,bywhatstandardsshallwejudgethepudding?Noescapeispossiblefrombroaderinterdisciplinarystandardsiftheenter-priseofsocialscienceistoproveusefultohumanity.Indeed,therationaleforaprofessionalcasteofscholars,financedatpublicexpense,breaksdownifwedenytransdisciplinarystandards.Naturally,scholarlyconsensusisnotalwayspossible.Butsurelytherearecertainthings–craniology(phrenology),forexample–thatmaysafelybeexcludedfromconsideration.Andifcraniologyisrejected,wemustappealtosometransdisciplinarystandardsindoingso.Notethatifknowledgeacrossdisciplinesistrulyincommensurable,weare19BradyandCollier(2004).20Hirsch(1967:viii–ix).21SeevanFraassen(1980).7AunifiedframeworkDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
compelledtoleavethequestionofcraniologytothecraniologists.Inthisvision,socialscienceisdefinedsimplybywhatsocialscientistsdo;thefactofbeingacraniologistisself-justifying.Whileonecanignoremethodology,onecannotchoosenottohaveamethodology.Inteaching,inresearch,andinanalyzingtheworkofcolleagues,scholarsmustseparatethegoodfromthebad,thebeautifulfromtheugly.Insodoing,broadercriteriaofthegood,thetrue,andthebeautifulnecessarilycomeintoplay.Socialscienceisanormativeendeavor.Likemembersofanycom-munitysocialscientistscreateandenforcenorms,rewardinggoodbehaviorandpunishing–orsimplyignoring–badbehavior.Thegate-keepingfunctionsoftheacademycannotbeabolishedbyawistfulappealtodiversity.Forsocialsciencehasalimitedsupplyofgoods,suchasjobs,funding,journals,books,andpublicattention,whichmustbeallocatedaccordingtosomerationale,hope-fullyarationalewecanallagreeupon.Finally,asamatterofgoodscholarship,writersinthesocialsciencesoughttobeabletoconversewithoneanother.22Economistsinterestedinpoliticaleconomyshouldbecognizant–andshouldseektoincorporate,whereverpossible–workinpoliticalscience.Andviceversa.Whilecross-disciplinaryresearchisoneofthemostfertileareasofresearchinthesocialsciencestoday,itisnotascommonasitshouldbe.Theproblemposedbyacademicparochialismstemsfromthefactthattheworldofhumanendeavor,whichitisthebusinessofsocialscientiststostudy,isremarkablyinterconnected.“Thedomainoftruth,”notesAbrahamKaplan,“hasnofixedboundarieswithinit.”23Itisdifficult,forexample,tounderstandfeaturesofapoliticalsystemwithoutunderstandingsomethingabouttheeconomicsystem.Yetifpoliticalscientistsandeconomistsconducttheirworkwithdifferentvocabulariesandareguidedbyanarrowconceptionofmethod,theywillnothavetheintellectualequipmenttoshareinsights.Theymaynotreadeachother’sworkorunderstanditwhentheydo,evenwhenworkingonrelatedtopics.Becausethevariousmethodsandtheoriesthatpopulatethesocialsciencesarenotcurrentlyunifiedbyasinglemethodology,cumulationofknowledgeisimpeded.Itisobviousthatknowledgecannotprogressunlessthereissomesharedgroundonwhichsuchknowledgecanrest.24Evenargumentsdemand22Hayek(1956:462–463;quotedinRedman1991:epigraph)onceremarked,“Thephysicistwhoisonlyaphysicistcanstillbeafirst-classphysicistandamostvaluablememberofsociety.Butnobodycanbeagreateconomistwhoisonlyaneconomist–andIameventemptedtoaddthattheeconomistwhoisonlyaneconomistislikelytobecomeanuisanceifnotapositivedanger.”SeealsoWilson(1998).23Kaplan(1964:4).24Fordiscussionofwhat“progress”mightmeaninthiscontext,seeLaudan(1977).FordiscussionoftheimportanceofsharedstandardsseeBradyandCollier(2004).8Socialsciencemethodology:aunifiedframeworkDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
acommonframeofreference;withoutsuchsharedground,theyaremerelystatementsofposition.Inthelattercircumstance,sciencedegeneratesintoachorusofyeasandnaysreminiscentofMontyPython’sinfamous“ArgumentClinic”(excerptedintheepigraphtoChapter3).Thisbookdoesnotdelveintothesociologicalaspectsofsocialscience.Evenso,itisworthreflectingbrieflyonsocialscienceasaprofessionalactivity,withdistinctivenorms,habits,rewards,andsanctions.DonaldCampbell’scom-ments,synthesizingearlyworkbyRobertMerton,areworthquotingatlength.Science,writesCampbell,requiresadisputatiouscommunityof“truthseekers”…Thenormsofscienceareexplicitlyanti-authoritarian,anti-traditional,anti-revelational,andpro-individualistic.Truthisyettobediscovered.Oldbeliefsaretobesystematicallydoubteduntiltheyhavebeenreconfirmedbythemethodsofthenewscience.Persuasionistobelimitedtoequalitarianmeans,potentiallyaccessibletoall:visualdemonstrationsandlogicaldemonstrations.Thecommunityofscientistsistostaytogetherinfocuseddisputation,attendingtoeachother’sargumentsandillustrations,mutuallymonitoringand“keep-ingeachotherhonest,”untilsomeworkingconsensusemerges(butmutualconformityinbeliefperseisrejectedasanacceptablegoal).25Campbellnotesthatthisisadifficultbalancingact,requiringbothindividu-alism(everyonemustthinkforhim-orherselfandrefusetoengageinherdbehavior)andcollectivism(everyoneinthecommunitymustfocusonsimilarproblemswiththeaimoffindingconsensus).Inordertogetproponentsofdifferentmethodsandtheoriesontalkingtermsweneedtoprovideacommonframeworkbywhichargumentsandevidencecanbeevaluatedandalternativemethodsunderstood.Ifeachhassomethingtocontribute(asthephrasegoes),thenweoughttobeabletoexplainwhatthesecontributionsare.Whether,inpointoffact,normsexistthatmightprovidegroundsforjudgmentsofadequacyacrossthesocialsciencesisthequestiontakenupinthefollowingchapters.Forthemomentitissufficienttonotethatthenormativeargumentfornormsisstrong.Thereisnoprofitinincommen-surability.26Totheextentthatacademicsemployidiosyncraticorfield-specific25Campbell(1988:290).26IncommensurabilityisatermthatenteredthelexiconofphilosophyofsciencewiththeworkofThomasKuhn.Itrefers(broadlyandambiguously)toaconditionwherepersonsareunabletounderstandoneanotherbecauseoftheirdifferentideological,theoretical,ormethodologicalcommitments.Itisaveryoldproblem,ofcourse.Baconnoticedthaterrorwasthelikelyresultwhenever“argumentorinferencepassesfromoneworldofexperiencetoanother”(quotedinWilson1998:10),aconditionwewouldnowlabelincommensurability.Itshouldbenotedthatpluralismanduniformityaremattersofdegree.Allbutthemostrabiddeconstructionistswilladmitthattherearesomegeneralperspectivesontruthandknowledgethattiethesocialsciencestogether.SeeLaudan(1983,1996);Wallersteinetal.(1996:92–93);andWilson9AunifiedframeworkDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
theoreticalframeworks,webecomeislandsinaboatlessarchipelago.Knowledgewillnotcumulate.Progress–defineithowyouwill–isimpeded.Tobesure,theneedforagreementvariesbytopic.Thosesubjectsfirmlyembeddedinthepast–those,thatis,withfewcontemporaryramifications–canperhapsaffordawiderarrayofviews.Yet,forallsubjects,socialscientistsshouldalwaysstriveforagreement,andthegreateragreementtheyachieve–ceterisparibus–themoreusefulthatfieldofstudyislikelytobe.Whethertheissueisadeclarationofwaroracapital-gainstax,citizensandpolicymakerslookforscholarlyconsensus.Profoundscholarlydisagreementoverthesemattershamperspublicaction.Howcanwejustifytheexpenditureofmillionsofdollarsofpublicfundsiftheeffectivenessofapolicyisopenlyandrepeatedlychallengedbyexperts?Indeed,supportforsocialwelfareprogramshasbeenunderminedbysuggestionsfromprominentpolicyexpertsthattheseprogramsarenotachievingtheirintendedpurposes.27Similarly,supportforanti-missiledefensesystemshasbeenweakenedbyexperttestimonyquestioningthetech-nologicalviabilityofthesevisionaryweapons.28Citizensarerightfullyloathtosurrendertheirearningsinordertopayforprogramsthatcannotdemonstrateworkability,ajudgmentwerelyonexpertstoprovide.Underthecircumstances,itisnotveryusefulifthesocialsciencecommu-nitygeneratesfourteendifferentperspectivesonvouchersordemocracy(twokeyexamples,introducedbelow,thatwillguidemuchofourdiscussioninthisbook).Ifthisistheendresultofacademicendeavor,wehavenotadvancedveryfaroversheerintuition.Perhapswehaveincreasedour“understanding”ofthesemattersbylookingatthemfromsuchvariedperspectives.However,ifwehavenowayofadjudicatingbetweenconflictingvisions–ifdissensusreignssupremeamongacademicswhostudythesematters–thenwehavelittletoofferpolicymakersorthegeneralpublic.Ofcourse,scholarlydissensusmaysimplybeareflectionoftheuncertainnatureofthephenomena.Consensusisusefulonlyifitiswarrantedbytheevidence.Evenso,thereisnoadvantageincultivatingdiversityperse.Onemightapplauddifférance(aDerrideanneologism)inthehumanities,butnot,Ithink,inthesocialsciences.29Scholarsinanthropology,archaeology,(1998)forfurtherdefensesofaunified(“objective”)methodology.SeeHollisandLukes(1982)andLaudan(1983,1996)forgeneraldiscussionsofrelativism.Forargumentsinfavorofunifyingthe“qualitative”and“quantitative”dimensionsofsocialsciencemethodology,seeLazarsfeldandRosenberg(1955:387–391)andKing,Keohane,andVerba(1994).Fordoubtsonthisscore,seeMcKeown(1999).27Murray(1984).28LakoffandYork(1989).29Thequestforconsensusmightalsobereferredtoasaquestforobjectivity.Thetroublewiththismuchabusedtermisthatitfosterstheillusionthatsuchagreementwillariseunproblematicallyfromanempiricalrealityinsofarasweviewthatrealityneutrally(withoutprejudice).Myargumentfor10Socialsciencemethodology:aunifiedframeworkDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
business,communications,demography,economics,education,environmen-taldesign,geography,law,politicalscience,psychology,publicadministra-tion,publichealth,publicpolicy,socialwork,sociology,andurbanplanningoughttostriveforagreement.AunifiedframeworkIftherearegoodreasonstoseekmethodologicalconsensusacrossthefields,problems,andtheoriesofsocialscience,howmightonecraftsuchacon-sensus?Whatprinciplesmightprovidegroundsforagreement?Moretothepoint,howcanoneconstructaframeworkthatisusefulforpractitionerswhoarelearningtheircraft?Thisbookisahighlysyntheticendeavor,buildingself-consciouslyonaprodigiousliteraturespanningphilosophyofscienceandmethodstextsoverthepasttwocenturies.Itspurposeistointegrateextantmethodologicalrulesandnormsintoaunifiedframeworkthatisconcise,precise,andcomprehen-sible.InsofarasIamsuccessful,thenarrativeoughttoappearasacompen-diumofcommonsense.Yet,Ialsostakesomeclaims.Likeallmethodologytexts,thisbookisbothareflectionof,andanargumentabout,thesubjectmatter.30Inquiryofascientificnature,Istipulated,aimstobecumulative,evidence-based(empirical),falsifiable,generalizing,nonsubjective,replicable,rigorous,skeptical,systematic,transparent,andgroundedinrationalargument.Therearedifferencesofopinionoverwhether,ortowhatextent,sciencelivesuptothesehighideals.Evenso,thesearetheidealstowhichnaturalandsocialscientistsgenerallyaspire,andtheyhelptodefinetheenterpriseinageneralwayandtodemarcateitfromotherrealms.Ofcourse,theseidealsarealsorathervague.What,exactly,doesitmeantobe“rigorous,”or“rational”?Thechallengebeforeusistoreformulatetheseabstractidealsinanoperationalfashion.Thisrequiressomedisaggregation.agreementisgroundedinthepragmaticneedforagreement,ratherthaninaparticulartheoryofknowledge–empiricist,inductivist,verificationist,falsificationist,etc.30Itisworthpointingoutthatanyworkonmethodology–exceptperhapsforthemostassiduouslyhistorical–treadsonnormativeground.Whyelsewouldonewrite,orread,atreatiseonthissubject,ifnottodiscoverashouldorashouldnot?Anotherwayofstatingthispointistosaythattherelevanceofmethodologicalstudystemsfromitscapacitytoorientanddirectresearchinthefield.Apurelydescriptivestudy,assumingsuchabookcouldbewritten,islessinterestingbecauseittakesnopositionsonthemethodologicalbattlesoftheday.Moreover,asapracticalmatter,abookthattraversedthisterritorywhilegrantingequalcoveragetoeverymethod,practice,andpremisewouldbecometoolargeandtooheterogeneoustobeofassistancetopractitioners.Thus,Ihaveself-consciouslyexcludedordownplayedcertaintendenciesthatseemed,tomywayofthinking,idiosyncraticorunproductive.11AunifiedframeworkDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
Yetsincetheobjectiveistoprovideaunifyingschemathedisaggregationcannotgotoofar.Thebookisthereforepitchedatameso-level,inbetweentheabstractionsofphilosophyofscienceandthespecificrulesthatdefinevariousmethods.Myapproachcentersontheidentificationofbasictasksofsocialscience,strategiesenlistedtoachievethosetasks,andcriteriaassociatedwitheachtaskandstrategy.ThesearelaidoutschematicallyinTable1.1.(Tasksareinboldfont;strategiesandcriteriaarelabeledassuch.)Notethateachstrategyisalsodefinedbyasetofcriteria,thoughthesearegenerallynotlistedinTable1.1forreasonsofspace.Furtherexplicationisprovidedinlaterchaptersandtables,asindicatedinthefinalcolumnofthetable.Thegoalofthebookistouncoverthesharednormsthatgovernactivity–implicitlyorexplicitly–inthecommunityofsocialscientists.Whatmakesaworkofsocialsciencetrue,useful,orconvincing(“scientific”)?Whydowepreferonetreatmentofasubjectoveranother?Whatreasonsdowegivewhenweacceptorrejectamanuscriptforpublication?Thesearethesortsofground-leveljudgmentsthatdefinetheactivityofmethodology.Withthesejudgments,Ihopetoidentifythethreadsthattieourmethodologicalintui-tionstogetherintoarelativelyunifiedframeworkacrossthediversedisciplinesofsocialscience.31FollowingtheorganizationofTable1.1,Ishallnowtrytosummarizethemainargumentsofthebookinverybroadstrokes.(Readersshouldnotbedisturbedifthenarrativeisnotcrystalclear,asalloftheconceptslistedherewillreceivefurtherexplication.)PartI:thefirstpartofthebookintroduceselementsofthesocialscienceenterprisethataregeneralinpurview.Chapter2beginswithadiscussionoftwocriteriathatinformeveryscientificinvestigation:(a)discoveryand(b)appraisal.Ithenofferadviceforresearchconductedinanexploratorymode,wherethegoalistodiscoveranimportantresearchquestionandhypothesis.(Thisistheonlysectionofthebookwhich,byreasonofitsratherunstructuredsubjectmatter,departsfromtheframeworkpresentedinTable1.1.)Chapter3laysoutcriteriathat,Iargue,pertaintoallsocialscienceargu-ments:(a)truth;(b)precision;(c)generality;(d)boundedness;(e)parsimony;(f)coherence;(g)commensurability;and(h)relevance.31Criteriaarecentraltothepresentframework,whichinanearlierrendition(Gerring2001)wasreferredtoasa“criterial”framework.FollowingCavell(1979:9),criteriaare“specificationsagivenpersonorgroupsetsuponthebasisofwhich…tojudge…whethersomethinghasaparticularstatusorvalue.”12Socialsciencemethodology:aunifiedframeworkDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
Chapter4discussescriteriapertainingtothetestingstageofresearch,thatis,researchdesignanddataanalysis.Thesefallintofourgeneralcategories:(a)accuracy;(b)sampling;(c)cumulation;and(d)theoreticalfit.PartII:thesecondpartofthebookisfocusedondescription,thatis,onempiricalpropositionsthatanswerwhat,how,when,whom,orinwhatmannerquestions.Chapter5isfocusedonconcepts,thelinguisticcontainersbywhichwemakesenseoftheworld.Iarguethatempiricalconceptsinthesocialsciencesstrivetoachieve(a)resonance;(b)domain;(c)fecundity;(d)differentiation;(e)causalutility;and(f)operationalization(atopicpostponeduntilChapter7).Inachievingthesegoals,ageneralstrategyofconceptualizationisintroduced,beginningwithasurveyofplausibleconcepts,continuingwithaclassificationofattributesforeachchosenconcept,andendingwithoneofthreeapproachestodefinition:minimal,maximal,orcumulative.Chapter6outlinesvariousstrategiesofdescriptivegeneralization.Iarguethatthesesortsofargumentsmaybeusefullycategorizedas(a)indicators;(b)syntheses;(c)typologies;or(d)associations,eachwithvarioussubtypes.Chapter7isfocusedonthetaskofmeasurement.Inthisquest,multiplestrategiesmaybeemployed.Iarguethatallstrategiesencounterthefollowingchoicesandchallenges:(a)levelsofabstraction(high,medium,low);(b)concept/measurementstructures(set-theoretic,additive,fuzzysets);(c)aggregationtech-niques(Booleanlogic,weightings);(d)scales(nominal,ordinal,interval,ratio);(e)objectives(grouping,discrimination);(f)approaches(deductive,inductive);(g)cross-referencing;(h)ethnography;(i)surveysandexperiments;and(j)causalrelations.PartIII:thethirdpartofthebookfocusesoncausation,thatis,onempiricalargumentsthatanswerwhyquestions.(Howquestionsliesomewhereinbetweendescriptionandcausation.)Thissortofargumentpositsagenerativerelationshipbetweenacausalfactor(X)andanoutcome(Y).Chapter8beginsbyintroducingadefinitionofcausalityandreviewingthediversityofcausalarguments.Diversitynotwithstanding,Iarguethatallsocialscienceargumentsstriveforcommoncriteriaincluding(a)clarity,(b)manipulability,(c)separation,(d)independence,(e)impact,and(f)amechanism.Chapter9takesupthequestionofcausalanalysis.Theseissuesmaybedividedintothreecategories–(a)thetreatment,(b)theoutcome,and(c)thesample–eachwithmultiplecriteriathatapplytothattask.Chapters10and11explorevariousspecificstrategiesofcausalanalysis.Thesearedividedintothreebroadrubrics–(a)randomizeddesigns,13AunifiedframeworkDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
(b)nonrandomizeddesigns,and(c)methodsthatmovebeyondXandY–eachwithmultipleoptions.ThefirsttwoareexploredinChapter10andthelatterinChapter11.Chapter12,theconcludingsectionofPartIII,attemptstoshowhowtheframeworklaidoutinprecedingchaptersintegratesdiverseapproachestocausalinference.PartIV:thefourthpartofthebookelaboratesanddefendstheframework.Chapter13returnstotheproblemofunityanddiversity,reviewingthequalitative–quantitativedebate,theculturalist–rationalistdebate,andthedebateamongcontendingcausalparadigms.Chapter14reconsidersseveralnaggingconcerns:howtheframeworkhandlesconflictsamongcontendingtasks,strategies,andcriteria;howitsetsreasonablestandardsforresearch;andhowitmightbejustified.Iarguethatthetasks,strategies,andcriteriasummarizedinTable1.1arethosethat(a)bestfulfilltheproclaimedgoalsofsocialscience(understandingsocialactioninascientificmanner)and(b)guidetheworkofsocialscienceinwaysthatarelikelytobeofusetopolicymakersandthelaypublic.ClarificationsSeveralclarificationsmustbeinsertedbeforewecontinue.First,thereisthematteroflengthanddetail.Tosome,Table1.1mayseemanundulylongandcomplicatedlaundrylist.Toothers,itdoubtlessappearsshortandreductivist.Indeed,entirebookshavebeenwrittenaboutsomeofthesubjectsthatIoutline(rathercavalierly,itmustseem)inapageortwo.Idonotclaimtohaveendeddiscussiononthesepoints.Myclaimissimplytohavecoveredthisterritoryasthoroughlyaspossibleatthislevelofanalysis.Anintermediatelevelofanalysisischosensoastoaffordusthebenefitsofbreadthandparsimony,withsomesacrificeofdepth.Readershungryformorediscus-siononvarioustopicsareencouragedtofollowthetrailoffootnotes.Second,thereisthematterofhowtounderstandeachelementofthetaxonomy.Traditionalmethodstextshavesoughttoidentifycategoricalrulesthatdefinegoodresearch.Bycontrast,Iregardeachtaskandcriterionasamatterofdegree.Achievingdiscovery,forexample,isnotabinaryissue(oneeitherdiscoverssomethingneworonedoesnot).Rather,allresearch–ifitisworth-whileatall–hassomeelementofnovelty:itissayingsomethingnew.ThesamemightbesaidforalltheothertasksandcriterialistedinTable1.1.(Strategiesaremorelikelytobecategoricalinnature,thoughevenheredifferencesofdegreeareoftenencountered.)14Socialsciencemethodology:aunifiedframeworkDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
Table1.1TheframeworksummarizedI.GENERALTableOverallCriteriaDiscovery;Appraisal2.1ArgumentsCriteriaTruth;Precision;Generality;Boundedness;Parsimony;Coherence;Commensurability;Relevance3.1AnalysesCriteriaAccuracy(validity,precision,uncertainty,internal/externalvalidity);Sampling(representativeness,size,levelofanalysis);Cumulation(standardization,replication,transparency);Theoreticalfit(constructvalidity,severity,partition)4.1II.DESCRIPTIONConceptsCriteriaResonance;Domain;Consistency;Fecundity;Differentiation;Causalutility;Operationalization5.1StrategiesSurveyofconcepts;Classificationofattributes;Definition(minimal,maximal,cumulative)5.2ArgumentsStrategiesIndicators;Syntheses;Typologies(simple,temporal,matrix,taxonomic,configurational,sequential);Associations(trend,network,correlation)6.1MeasurementsCriteriaReliability(precision);ValidityStrategiesLevelsofabstraction(high,medium,low);Structure(set-theoretic,additive,fuzzysets);Aggregation(Booleanlogic,weightings);Scales(nominal,ordinal,interval,ratio);Objective(discrimination,grouping);Approach(deductive,inductive);Ethnography;Surveys/experiments;Cross-referencing;Causalrelations7.1III.CAUSATIONArgumentsCriteriaClarity;Manipulability;Separation;Independence;Impact;Mechanism8.1AnalysesCriteriaTreatment(exogeneity,variation,simplicity,discrete-ness,uniformity,evendistribution,strength,proximity,scalability);Outcome(variation);Sample(independence,comparability)9.3StrategiesRandomizeddesigns(pre-test/post-test,post-testonly,multiplepost-tests,roll-out,crossover,Solomonfour-group,factorial);Nonrandomizeddesigns(regressiondiscontinuity,panel,cross-sectional,longitudinal);BeyondX&Y(conditioningonconfounders,instrumentalvariables,mechanisms,alternateoutcomes,causalheterogeneity,rivalhypotheses,robustnesstests,causalreasoning)10.115AunifiedframeworkDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
Third,thereistheproblemofconflictsacrossthetasks,strategies,andcriteria.Achievingadequacyalongonedimensionmayinvolveasacrificealonganotherdimension:tradeoffsareubiquitous.Thismeansthateverytask,strategy,orcriterionmustbeunderstoodwithaceterisparibuscaveat.Parsimonyisdesir-able,allotherthingsbeingequal.Coherenceisdesirable,allotherthingsbeingequal.Andsoforth.Thisdoesnotmeanthat“anythinggoes,”butitdoesimplythatseveralapproachestoagiventopicareoftenmethodologicallyjustifiable,andthis,inturn,offersastrongprimafacieargumentformultimethodresearch.Thejobofthemethodologist,inanycase,istoarriveatabest-possibleresolu-tionofconflictingtasks,strategies,andcriteria(Chapter14).Tosumup,thepurposeoftheframeworkistoofferarelativelyparsimo-niousandcomprehensivereviewofissuesthatcropupintheprocessofdesigningandevaluatingsocialscienceresearchsothatmethodologicalintui-tionsaresharpened,workismorereflective,andcumulationeasiertoachieve.Whatevermethodologicalagreementispossibleinsocialsciencemustbeprovidedbyafoundationonwhichwecanall(moreorless)agree.Suchaframework,Ibelieve,ispresentalreadyinoureverydayjudgmentsaboutgoodwork,strongarguments,andsolidevidence.Bycontrast,consensusisnotlikelytoarisethroughourconversiontoasingletheoreticalparadigmormethod,inauguratingthatheavenlystateknownas“normalscience.”Wearenotlikelytowakeuponemorningtofindourselvesalldoinggametheory,orhermeneutics.Fortunately,agreementontheories,models,andmethodsisnotnecessary.Indeed,itwouldprobablybefoolhardyforsocialscientiststoallpursuethesamequestions,ortopursuequestionsinthesameway.However,knowledgegatheredwithdiversetoolswillcumulateifweareabletoputdiverseevidencetogetherinacommonframework.Progressisarealisticgoalaslongasweunderstandthatlastingprogressismorelikelytooccurinsmallstepsthaninrevolutionary(“paradigmatic”)leaps.Ifaunifiedframeworkwillnotresolveallourstrifeitmayatleastpointthewaytoamoreproductivestyleofdebate:whereargumentsmeeteachotheroncommonground,wheretheadvantagesanddisadvantagesofdifferentapproachestoaproblemcanbespecifiedandevaluated,andwherecumulationcanbeassessed.ExclusionsAlthoughcomprehensiverelativetootherapproachestothesubject,thepresentframeworkdoesnotencompassallaspectsofsocialscience.Letmenoteseveralomissions.16Socialsciencemethodology:aunifiedframeworkDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
First,thetasks,strategies,andcriteriaencompassedinTable1.1donotpayexplicitattentiontopredictiveinference.Predictionmaybeunderstoodeitherasforecastingintothefutureand/oraspointpredictionsforparticularcases(inthepast,present,orfuture)derivedfromgeneralcausalmodels.Ofcourse,insofaraseitherofthesesortsofinferencebuildsongeneraldescriptiveorcausalmodelsonemightsaythattheyareextensionsofsubjectsdiscussedinthisbook.However,Ishallhavenothingexplicittosayabouthowonereachespredictiveinferences.Second,theframeworkisonlyperipherallyconcernedwithargumentsfocusednarrowlyonsingleeventsoroutcomes(sometimesreferredtoasidio-graphic,purelydescriptive,singular-causal,ortoken-causal).Thisstemsfromtheinitialdefinitionofscience,understoodasageneralizingactivity.32Ofcourse,knowledgeofspecificeventsmayassistinreachingconclusionsaboutalargerpopulationofcases.Justasalargesampleofunitsreflectsonabroaderpopula-tionsomightasmallsampleconsistingofasingleunit,studiedintensively.Assuch,casestudyresearchfallswithintherubricofageneralizingscienceandthuswithintherubricofthisvolume.Thatsaid,thisvolumedoesnotdelvedeeplyintocase-basedstylesofdescriptiveandcausalinference,atopicaddressedelsewhere.33Third,theframeworkdoesnotextendtopragmatic,logistical,orethicalconcerns.Quiteoften,onechoosesaresearchdesignbecauseitismorecon-venientforonetodoso,orperhapsbecauseitisimpossibletodootherwise.Forexample,onemaylackthelanguageskillstostudysomethingelse.Politicalorculturalbarriersmaypreventonefromgatheringadditionalinformation.Evidenceitselfmaybescarce.Fundingopportunitiesmaybelimited.And,ofcourse,timeisalwayslimited.Ethicalconsiderationsmayalsoconstrainone’sabilitytodevelopasolutiontomethodologicaldifficulties.Itisworthremindingourselvesthatsocialresearchissubjecttothesameethicalnormsthatgoverneverydaylife(vaguethoughthesemaybe).Inaddition,thereareconsiderationsthatpertainspecificallytoresearchconductedonhumansubjects.Here,theimmediateimpactofapieceofresearchmustbebalancedagainsttheanticipatedlong-termimpactofthatresearchonthegeneralpublicandonthecourseofpublicpolicy.Sometimes,32Onemightaddthatthereisalsorelativelylittleonecansaymethodologicallyaboutthedescriptionorexplanationofaparticularevent.Tobesure,ahighdegreeofexpertiseisnecessaryinordertoreachadeterminationonacontestedquestionoffact;suchjudgmentsdonotcomeeasy.Butthisexpertisedoesnotusuallylenditselftogeneralcriteriaofinference.Itisbasedinsteadonhighlycontextualknowledgeaboutparticulartimes,places,andpeoples.Lieberson(1985);Thompson(1978);Winks(1969).33Gerring(2007).SeealsoBennett(2010);GeorgeandBennett(2005).17AunifiedframeworkDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
long-termbenefitsoutweighshort-termcosts;sometimes,theydonot.Unfortunately,thesearenotissuestowhichneatformulasapply;hence,theangstthataccompaniestheworkofInstitutionalReviewBoards(IRBs),assignedwiththedifficulttaskofapprovingresearchonhumansubjectsinuniversitiesthroughouttheworld.34Practicalconsiderationssuchasthesearenotmethodologicalintheusualsenseoftheterm.Onecouldhardlyargue,forexample,thatagivenresearchdesignhasabettergraspofthetruthbecauseitischeaper,easier,ormoreethical.Thismighthavebeenthereasonbehindastudy’ssuccessorfailure,butitcannotbethegroundsuponwhichweacceptorrejectatheory.Ifanotherresearchercomesalongwithmoretimeandmoney,betterlanguageskills,betteraccesstokeycases,orasolutiontoapersistingethicalobstacle,heorshewillbeabletoconstructabetterresearchdesign.Itisthelatter–goodnessinresearchdesign–thatweareprimarilyconcernedwithinthistext.However,thefactthatgoodnessinresearchdesignisconditionedbypecuni-ary,ethical,social,andpoliticalrealitiesmustenterultimatelyintoourjudg-mentofastudy’scontributiontoknowledge.Ifwetooknocognizanceofsuchmatterswemightfindourselvesstudyingonlythosetopicsthatareconvenient,unethical,ordata-rich.Thus,Idonotwanttodownplaytheimportanceofpracticalconsiderationsintheconductofsocialresearch.Theyareneglectedinthistextonlybecausethereislittlethatonecansayabouttheminthegeneralsense,andbecausethesesortsofconstraintsaregenerallyapparenttotheresearcher.35(ForfurtherthoughtsonthisissueseeChapter14.)TerminologyDistressingly,thevocabularyassociatedwiththesubjectofmethodologyisriddenwithambiguity.Keytermssuchas“positivism,”“qualitative,”“mechan-ism,”“experiment,”“causality,”“exogeneity,”“heterogeneity,”“validity,”and“identification”meandifferentthingsindifferentresearchtraditionsandindifferentresearchcontexts.Evenwithinthesametraditionandthesamecontexttheymaymeandifferentthingstodifferentpeople.3634Kelman(1982)offersgeneralreflectionsonresearchethicsinthesocialsciences.Mazur(2007)andSalesandFelkman(2000)discussresearchonhumansubjects.Paluck(2009)andWood(2006)investigateethicaldilemmasoffieldresearch,withspecialfocusonareasofintenseconflict.35OnpracticalconstraintsseeBarrettandCason(1997);Lieberman,Howard,andLynch(2004);vanEvera(1997).36Toaidindisambiguating,JudeaPearlrecentlycalledforanew“WikiGlossaryofCausalTerminology”(www.mii.ucla.edu/causality).18Socialsciencemethodology:aunifiedframeworkDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
Moreover,becauseofthehighdegreeofoverlapinconnotationacrossnear-synonyms,thereisalmostalwaysmorethanonewaytoexpressasinglethought.Anissueofgeneralitymightalsobearticulatedasoneofbreadth,comprehensiveness,domain,extensivity,externalvalidity,population,range,scale,orscope.Avariablecouldalsobereferredtoasaconcept,condition,dimension,factor,indicator,measure,metric,orunidimensionaldescription.Andsoforth.Eachofthesetermshasaslightlydifferentconnotation–andinsomecontexts,averydifferentdenotation.Aprimeexampleoflexicalabundanceisprovidedbyrecentworkoncausalinference,whichmaybeunderstoodfromtheperspectiveofphilosophyofscience(e.g.,byDavidLewis,JohnMackie,andWesleySalmon),fromtheperspectiveofresearchdesign(e.g.,byexperimentalistssuchasDonaldCampbellandcolla-borators),fromtheperspectiveofstatistics(e.g.,byDonaldRubinandcollabora-tors),andfromtheperspectiveofcausalgraphs(e.g.,byJudeaPearlandhiscollaborators).Eachofthesetraditionshasdevelopedahighlyspecializedvoca-bulary.However,allareconcernedwithasimilarsetofissues;thus,onefindsaroughequivalenceofprinciplesacrossthesetraditions(whichIhavetriedtocaptureinasparsimoniousafashionaspossibleinthethirdpartofthebook).Movingbeyondissuesofterminology,itshouldbepointedoutthatmeth-odologicalissuesarerarelyseparateanddiscrete.Theincorrigiblequalityofoursubjectisitsholism:everythingisenmeshedineverythingelse.Thus,althoughatask,strategy,orcriterionmaybedefinednarrowlyinthetextitwillquicklybecomecleartothereaderthatnoissueisentirelyself-contained.Aproperunderstandingrequiresustosituateeachelementwithinthebroadrubricofsocialsciencemethodology.This,itself,isastrongargumentforacomprehensive,book-lengthtreatment,whichaffordsthespacetodiscussinterconnectionsacrosstopics.However,itshouldalsoalertthereadertothefactthat,likecookiedough,oursubjectcanbesubdividedinmanydifferentways.Thatis,thetasks,strategies,andcriteriasetforthinTable1.1couldbenamedandarrangeddifferently.Writersonmethodologicalsubjectsareforcedtomakechoicesaboutwhichtermstohighlightandwhichtoignore,andhowtodefinethechosenterms.Nolexiconissacrosanct.37Inmakingchoices,Ihavegivenpreferencetotermsanddefinitionsthatpromisetotravelwidelyacrossmethodologicalanddisciplinarycleavagesandthatdivideupthesubjectinamannerconsistentwiththegoalsoftheproposedframework.Sometimes,thisinvolvestheadoptionofaterm37Indeed,thecategoriessetforthinTable1.1aresomewhatdifferentfromthoseemployedinthepreviouseditionofthisbook(Gerring2001).19AunifiedframeworkDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
originallydevelopedinaspecializedrealm.(Sometimes,Ihavebeenforcedtocoinnewtermsthatdonothavecommoncurrency.)Inanycase,Ihavedonemybesttomakeconnectionsacrossvariedlexicons.Wherevernear-synonymsorrivaldefinitionsexistIhavetriedtoclarifythesesimilaritiesanddifferences.Importanttermsareitalicizedatthepointinthetextwheretheyaredefined.AGlossaryprovidesdefinitionsforallkeyterms,notingnear-synonyms.Bywayofconclusion,itshouldbestressedthattheseemingarbitrarinessofourmethodologicallexicondoesnotmeanthatissuesofvocabularyarepointless.Tothecontrary,theyareindispensable,forthealternative–apurelymathematicallexiconencompassingallmethodologicalissues–doesnotexist,andprobablywouldneverbeadequatetothetask.Itrustthatquibblesoverterminologyorclassificationwillnotimpugntheutilityoftheframework.ExamplesBecauseoftheopacityofourmethodologicallexiconanydiscussionofmethodologicalissuesdependscruciallyuponaplentifulsupplyofspecificexamples.Itistheseexamples–ofworkthathasbeenconductedonasubjectormightbeconductedonthatsubject–thatoftenservetoclarifyapoint.InchoosingexamplesIhavetriedtocrossdisciplinaryboundariessoastoillustratethepervasivenessofvariousmethodologicalissuesthroughoutthesocialsciences.Ofcourse,spacelimitationsprecludediscussionofmultipleexamplesforeachargument,soreaderswillhavetointuithowtheargumentlinksupwithworkintheirownspecializedfieldorsubfield.Frequently,Ihavechosenolder,“classic”studiesthatarelikelytobefamiliartoreaders,eveniftheyhavebeensupersededbymorerecentwork.Discussionofaparticularworkdoesnotimplyanendorsementofitsfindingsormethods.Examplesarechosentoillustratespecificmeth-odologicalpoints;thatisall.Tomaintainconsistency,Ioftenreturntotwocentralexemplars,democ-racyandvouchers.Thesesubjectsarerelevanttomanysocialsciencedisci-plinesandhavealsoarousedagooddealofscholarlycontroversy.Thefirstexemplifiesworkwheretheunitofanalysisisverylarge(e.g.,nation-states)andthetheoreticalframeequallygrand.Thesecondexemplifiesworkondiscretepolicyinputswhereindividualsorsmallgroupsformtheunitsofanalysisandthetheoreticalframeiscorrespondinglysmall.Together,thesetwotopicsprovideamplefodderformethodologicalillustration.Followingisaverybriefreviewofthesecomplexissues(forfurtherelucidationthereaderisreferredtotheworkcitedbelowandtodiscussionthroughoutthetext).20Socialsciencemethodology:aunifiedframeworkDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
DemocracyDemocracyisafamiliartopic,thoughalsoaperplexingone.Scholarshavedifficultydefiningthisconceptand,evenonceissuesofdefinitionaredispensedwith,problemsofmeasurementremain.Thus,descriptivequestionsremainsomewhatunsettled:wefeelconfidentinidentifyingcountriesthatareveryautocratic(e.g.,NorthKorea)andthosethatarehighlydemocratic(e.g.,Sweden);butthereisagooddealofdisputeabouthowtoconceptualizemanyofthecountriesthatlieinbetween(e.g.,Russia,Turkey,orIran).38Underthecircumstances,itisnotsurprisingthatscholarsalsowrestlewiththecausesofdemocracy(whydosomecountriesdemocratize,andconsolidate,whileothersremainautocraticoronlyintermittentlydemocratic?)39andthecausaleffectsofdemocracy(doesregimetypeaffectpoliticaloutcomesandpolicyoutcomes?).40Withrespecttothecausaleffectsofdemocracy,onetheoryknownasthe“democraticpeace”willbediscussedatvariouspointsinthetext.Thishypoth-esis,whichharksbacktoImmanuelKant’sessayon“PerpetualPeace”(1795),proposesthatdemocraciesneverfightwarswithoneanother(thedeterministicversion)oraremuchlesslikelytofightwarswithoneanother(theprobabilisticversion).Variousreasonshavebeenproposedforthisapparent“law”ofinter-nationalrelations.Notonlythecausaleffect,butalsothemechanismsthatmightbeatworkremainopentodebate.Andyetthereisastrongempiricalregularityandanumberofplausiblemechanismsthatmightaccountforit.Notsurprisingly,thehypothesishasattractedagreatdealofinterestfromacademicsandpolicymakers.41VouchersThequestionofvouchersisamuchmorespecificphenomenonthanregimetype.Assuch,itiseasiertodefineandtomeasure,thoughlessgrandintheoreticalsweep(acommontradeoffintheworkofsocialscience).Evenso,descriptivequestionsremain.Diversestudiesproclaimingtobeabout“vou-chers”sometimesmaskdivergentpolicyinterventions.Andthequestionofpolicyimpactisbynomeansresolved.4238CoppedgeandGerring(2011);MunckandVerkuilen(2002).39Berg-Schlosser(2007);Coppedge(forthcoming);Geddes(2007).40GerringandThacker(2011);Mulligan,Gil,andSala-i-Martin(2004).41Brown,Lynn-Jones,andMiller(1996);Elman(1997).42Theliteratureonthiscontroversialsubjectisvast.RecentworkincludesChubbandMoe(1990);FullerandElmore(1996);HowellandPeterson(2002);Hoxby(2003);KruegerandMhu(2004);Ladd(2002);Neal(2002);Smith(2005).SeealsoMorganandWinship(2007)formethodologicalcommentaryaboutvouchersasanissueinsocialscienceresearch.AlthoughmuchofthecurrentacademicliteratureiscenteredontheUSexperience,thepolicyhasbeenimplementedonamuchwiderscaleinother21AunifiedframeworkDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
Vouchersaremostcommonlydiscussedinthecontextofeducationalpolicy,thoughtheyarealsoapplicabletootherpolicyareas(e.g.,housing,food,medicalcare).Vouchertheoriesgenerallycenteronamarketplacemodelofserviceprovision.ProponentsofschoolvouchersbeginningwithMiltonFriedman43believethatthebestwaytoimprovethequalityofprimaryandsecondaryeducationistointroducecompetitionintoschoolsystems,whichareusuallydominatedbyasinglegovernmentprovider.Inthisfashion,educationwouldbesubjectedtotherigorsofthemarketplaceandparents(andtheirchildren)wouldbeabletoexercisechoiceamongschools.Itisargued,further,thataneducationalmarketplacemaybeachievedwithoutsacrificingtheidealoffree,universaleducationthroughthemediumofgovernment-provided“vouchers”thatstudentscanredeemtowardtuitionatschoolswithinadistrict–whetherpublicorprivate(thoughgenerallywithsomeconditionsandqualifi-cations).Advocatesclaimthatthissystem,ifproperlyinstituted,willleadtoimprovedschoolquality,improvedstudentperformance(asmeasured,e.g.,bystandardizedtests),andperhapsadditionalbenefitsaswell(e.g.,parentandstudentsatisfaction,narrowingtestscoregapsbetweenblackandwhitestu-dents,andsoforth).Criticsaredubious.AdvicetothereaderInapproachingthisratherlargebookthereadermaywishtoreadselectively.Thisisadvisableforthosewithextensivebackgroundknowledge,whomaybefamiliarwithsomesubjectsandlessfamiliarwithothers.Suchreadersmaybrowsethedetailedtableofcontentsortheindexinordertoidentifysubjectsofinterest.Selectivereadingisalsoappropriateforbeginnersinsocialsciencemeth-odology,whomaywishtofamiliarizethemselveswithessentialelementsfirst,leavingmorecomplexissues–includingargumentswiththeliterature–forlater.Inthisfashion,itishopedthatthebookwillberenderedaccessibletoallreaders–beginners,intermediate,andadvanced.Ofcourse,confusionsmayarisefromselectiveperusalsofthetext.WithinPartIIandPartIIItopicsarecloselyintertwinedandthereforebestapproachedasaset,notonachapter-by-chapterbasis.Moreover,thefourpartsofthebookarecumulative,buildinguponeachother.GeneralcriterialaidoutinPartIapplytobothdescriptiveandcausaltasks.Andsincecausalcountries,includingChile,Colombia,andSweden.VouchersarebynomeansanexclusivelyAmericanpreoccupation.Carnoy(1998);ChakrabartiandPeterson(2008);GauriandVawda(2004).43Friedman(1955).22Socialsciencemethodology:aunifiedframeworkDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
argumentsbuildondescriptivearguments,thecriterialaidoutinPartIIalsoapplytoPartIII.Recallthatsocialscienceisaholisticenterpriseandthegoalofthisbookistoencompassthatactivityinareasonablycomprehensivefashion.Bitsandpiecesofsocialsciencemethodologymaynotmakesense–andmayevenbemisleading–ifwrenchedoutofthelargercontextinwhichtheyaresituated.Thiscaveatmustbeborneinmindbythosewhochoosetoreadselectively.Additionalresourcesforreadersandinstructorsarelocatedon-lineatCambridgeUniversityPresswww.cambridge.org/gerring.Thisincludes(a)tablesandfiguresfromthebook(inseparatefiles,foreasydownloadingandprinting);(b)apowerpointpresentation(foruseinlectures);(c)questions,exercises,assignments,andadvice(forinstructorsincorporatingthetextintotheirmethodologycourse);and(d)syllabifrominstructorswhohaveusedthisbookintheircourses.23AunifiedframeworkDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
Downloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:09 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.002Cambridge Books Online © Cambridge University Press, 2012
PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:15 BST 2012.http://ebooks.cambridge.org/ebook.jsf?bid=CBO9781139022224Cambridge Books Online © Cambridge University Press, 2012
Downloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:15 BST 2012.http://ebooks.cambridge.org/ebook.jsf?bid=CBO9781139022224Cambridge Books Online © Cambridge University Press, 2012
Cambridge Books Onlinehttp://ebooks.cambridge.org/Social Science MethodologyA Unified FrameworkJohn GerringBook DOI:http://dx.doi.org/10.1017/CBO9781139022224Online ISBN: 9781139022224Hardback ISBN: 9780521115049Paperback ISBN: 9780521132770Chapter2 – Beginnings pp. 27-57Chapter DOI:http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge University Press
2BeginningsDuringmycareerinscience,nownearlyahalfcenturyinduration,Ihavegrownmoreandmoreawarethatsuccessinscience,parallelingsuccessinmostcareers,comesnotsomuchtothemostgifted,northemostskillful,northemostknowl-edgeable,northemostaffluentofscientists,butrathertothesuperiorstrategistandtactician.Theindividualwhoisabletomaneuverwithproprietythroughtheworldofsciencealongacoursethatregularlyputshimorherinapositionofserendipityisoftentheonewhoexcels.JackOliver1Broadlystated,thegoalofscienceistodiscovernewthingsabouttheworldandtoappraisethetruth-valueofextantpropositionsabouttheworld.Considerourexemplars,democracyandvouchers,introducedinChapter1.Wewanttouncovernewthingsabouttheprocessofdemocratizationandtheimpactofvouchersonschoolperformance.Atthesametime,wewanttotestextanttheoriesaboutthesetwosubjects.Socialsciencemay,therefore,beunderstoodasatwinquestfordiscoveryandforappraisal,assummarizedinTable2.1.2Thechapterbeginsbyintroducingthesegoals,followedbyareviewoftheirimplicationsformorespecificmethodologicaltasks.Thenextsectionapproachesthegoalofdiscoverythroughtheconcretetaskoffindingaresearchquestion.Sincetheremainingchaptersofthebookassumethataresearchquestion–perhapsevenaspecifichypothesis–hasbeenidentified,thischapterfunctionsasaprologuetotherestofthebook.1Oliver(1991:ix).2ThiscontrastcanbetracedbacktoReichenbach(1938),whodistinguishedbetweena“contextofdiscovery”anda“contextofjustification.”SeealsoHanson(1961);McLaughlin(1982);Nickles(1980);Popper(1965);Zahar(1983).Critics(e.g.,Schiemann2003)notethatthedistinctionisnotadichotomy,i.e.,thetwogoalsaredifficulttoseparateinpractice.Myclaim,however,isnotthattheycompriseacrisptypology.Rather,Iclaimthattheyaretwofundamentalgoalsofsciencethatimposesomewhatdifferentmethodologicalstrategiesandcriteriaupontheactivityofscience.27Downloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
Discovery“Anauthorislittletobevalued,”saysHumeinhischaracteristicallybluntfashion,“whotellsusnothingbutwhatwecanlearnfromeverycoffee-houseconversation.”3Weshouldlikeanargument,andcorrespondingempiricalanalysis,tocontributesomethingnoveltoourunderstandingofatopic.Agoodpieceofresearchisonethatisinnovative,onethatmakesanovelcontribu-tion–usuallyunderstoodwithrespecttothekeyhypothesisorgeneraltheory.Ofcourse,some“discoveries”arenotreallynew,orarenotasinnovativeastheypurporttobe.Authorssometimesslighttheaccomplishmentsofothers,formulatetheirargumentagainstaridiculousnullhypothesis(a“strawman”argument),overstatetheaccomplishmentoftheirownwork,oradoptneolo-gismsthatrepackageoldwineinnewbottles.Ourcontemptforvariousspeciesofpseudo-innovationconfirmsthegeneralpoint:goodresearchshouldpushthefrontiersofknowledgeforward.Inthisquest,researchersaregenerallyforcedtoadoptanexploratoryapproachtotheworld.Newterritoryisentered,orestablishedterritoriesareinterrogatedforunexpectedpatterns(anomalies).Newexplanationsaretestedorinventedoutofwholecloth.Discoveryrequiresanaggressiveandcriticalengagementwiththestatusquo.Thisischaracteristicofinitialphasesofresearch.Butitisalsothegoaltowhichalltopresearchersaspire,foreveryonewishestosituatethemselvesonthefrontiersofknowledge.Inthewordsofonescientist,“theonlyinterestingfieldsofsciencearetheoneswhereyoustilldon’tknowwhatyou’retalkingabout.”4Inthissense,weareall–always–beginners.Considerthequestionofdemocratization,introducedinChapter1.Howandwhydosomestatesdemocratize,whileothersdonot(orareunabletosustainthosegains)?Thisisnotaneasyquestiontoanswer–somemightTable2.1Generalgoalsofsocialscience1.Discovery(conjecture,exploration,innovation,theoryformation)Isitnew?2.Appraisal(assessment,demonstration,evaluation,justification,proof,testing,verification/falsification)Isitfalsifiable?3Hume(1985:254).4I.I.Rabi,quotedinRoot-Bernstein(1989:407).28PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
arguethatitisnotamenabletogeneraltheory–butitisundoubtedlyanimportantone.5Innovationatthedescriptivelevelconcernswaysinwhichthebroadtopicofdemocratizationmightbeproductivelyconceptualizedandmeasured.Isthereacriticalmomentoftransitionatwhichtheprocessofdemocratizationisachieved?Isthereapointofconsolidationbeyondwhichreversalsareunlikely?Aretheredistinctivesequencesbywhichdemocratizationoccurs?Howshoulddemocracy,anditsvarioussubtypes(illiberaldemocracy,elec-toraldemocracy,competitiveauthoritarianism),bedefined?Thesearejustafewofthedescriptivequestionsthathaveoccupiedscholarsinrecentyears.Atthecausallevel,scholarshavefocusedonthepossiblepreconditionsforsuccessfuldemocratization.Arecertainauthoritarianregimetypesmorelikelytodemocratizethanothers?Doestheexistenceofmineralwealth(e.g.,oilordiamonds)inacountrymakedemocracylesslikely?Towhatextentdoesacountry’scolonialexperiencecoloritspropensityforachievingandmaintain-ingademocraticformofrule?Howmuchimpact(ifany)doeseconomicdevelopmenthaveondemocratic/authoritarianoutcomes?Inamoregeneralvein,onecanidentifycertaincharacteristictypesofcausalinnovation.Sometimes,anewfactor,X,isproposedasacontributingcauseforawell-studiedoutcome,addinganewvariabletoexistingmodels.Thatwoulddescribemostoftheexampleslistedinthepreviousparagraph.LesscommonisthetheoreticaleclipseofexistingtheoriesaboutYwithanewcausalframework.Thus,DaronAcemogluandJamesRobinsonhaveproposedthatdemocratizationcanbeunderstoodasadistributionalstrugglebetweenthehavesandthehave-nots.6Athirdtypeofcausalreformulationconsistsinworkingbackfromanestablishedcausalfactor,X,tosomepriorcause(X1)thatexplainsX,andtherebyY(reframingXasacausalmechanism).Thus,itmightbearguedthatgeographiccircumstances(e.g.,climate,soilquality,diseasevectors,accesstodeep-waterportsandnavigablerivers)affectedpatternsofcolonizationandresourceextrac-tion,withlastingeffectsonthedistributionofwealthandpower,and,ultimately,onacountry’spropensitytodemocratize.7AfourthtypeofinnovationfocusesonthecausalmechanismslyingwithinanestablishedX/Yrelationship.Inthisfashion,agooddealofworkhasbeendevotedtothecausallinksbetweenresourcewealthandauthoritarianrule.MichaelRosssummarizes:5ForrecentreviewsoftheliteratureseeBerg-Schlosser(2007);Coppedge(forthcoming);Geddes(2007).6AcemogluandRobinson(2005).7ThisfollowsthelineofargumentinitiatedbyAcemoglu,Johnson,andRobinson(2001);SokoloffandEngerman(2000).29BeginningsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
A“rentiereffect”…suggeststhatresourcesrichgovernmentsuselowtaxratesandpatronagetorelievepressuresforgreateraccountability;a“repressioneffect”…arguesthatresourceswealthretardsdemocratizationbyenablinggovernmentstoboosttheirfundingforinternalsecurity;anda“modernizationeffect”…holdsthatgrowthbasedontheexportofoilandmineralsfailstobringaboutthesocialandculturalchangesthattendtoproducedemocraticgovernment.8Astudyfocusedoncausalmechanismstypicallyculminatesinanewexplana-tionforwhyXcausesY(inthiscase,whythereisa“resourcecurse”).Ifnoplausiblecausalmechanismcanbediscovered,suchastudymightalsoservetodisconfirmtheentirehypothesis.Afifthtypeofinnovationfocusesonthepopulationofaninference(itsbreadthorscope).Onemightarguethattheconnectionbetweenresourcewealthandauthoritarianismisapplicableonlytothedevelopingworld,andnottoadvancedindustrialcountries(e.g.,Norway).Oronemightattempttoextendtheambitofthetheorytoapplytodifferenttimeperiods(e.g.,Greekcity-states)ordifferentphenomena(corporategovernance).Evidently,therearemanywaystoinnovate,whichistosay,therearemanytypesofdiscoveries.Thisisbecausetherearemanytypesoftheories,andeachtheoryhasmultipleparts–anissueweshallattempttodisentangleinthecomingchapters.AppraisalThesecondover-archinggoalofscienceistoensurethatthetruth-valueofpropositionsabouttheworldcanbetestedrigorously.“Thecriterionofthescientificstatusofatheoryisitsfalsifiability,orrefutability,ortestability,”assertsKarlPopper.9Thisprocess,incontrasttothegoalofdiscovery,mustbehedgedaboutwithrules.Otherwise,weshallneverbeabletoreachconsensusonanythingandthegoaloftruth(whichpresumesthepossibilityofreachingconsensus)dissipates.Fortunately,theprocessofappraisalismoreamenabletogeneralprinciplesthantheprocessofdiscovery.Andthis,inturn,helpstoexplainwhyithasbeenanabidingpreoccupationofmethodologists.(Itisvirtuallythesumtotalofthefieldofmethodology,astraditionallyconceived.)Withrespecttotheconstructionofarguments,itmaybeappropriatetobeginbyrepeatinganoldstory(perhapsapocryphal)aboutaphysicsdoctoral8Ross(2001:327–328).SeealsoDunning(2008a).9Popper(1965:37).Arguably,Popper’s([1934]1968)classictreatise,TheLogicofScientificDiscovery,wasmis-named.Itoffersnotalogicofdiscovery,butratheralogicoftesting.Inanycase,Iprefertheterm“appraisal”ratherthan“falsifiability,”asthelatterpresumesacertainapproachtotestingthatmaynotbeentirelyjustified.30PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
defense.Attheconclusionoftheproceedings,oneoftheexaminersexcoriatesthehaplesscandidatewiththefollowingremark:“ThisistheworstthesisIhaveeverread.Itisnotevenwrong.”Thesignofanonfalsifiableproposition,Popperpointsout,isthatvirtually“anyconclusionwepleasecanbederivedfromit.”10Itmaybetruebydefini-tion,butitisnottruebyanystandardsthatonemightsubjecttoempiricaltest.Popperchargedthatanumberofhighlyinfluentialtheories,includingMarxismandFreudianism,sufferedthisfatalflaw.Theycouldnotbeprovenordis-proven.Theywereneitherrightnorwrong.Asithappens,MarxismandFreudianismarestillwithus,alongwithWeberianism,realism(atheoryofinternationalrelations),rationalchoiceandahostofotherdifficult-to-appraisetheoreticalframeworks.Inthenaturalsciences,aswell,explanationssuchasstringtheorypersist,despitetheirseem-ingnonfalsifiability.Itwouldappearthatbroadandambiguousframeworksaresometimesuseful,evenwhentheycannotbeclearlyappraised.Indeed,appraisalisbynomeanstheonlycriterionofagoodargument.Thatsaid,thereisnearuniversalrecognitionthatfalsifiabilityisavirtuousideal–onetobestrivenfor,evenwhenconditionsdonotseemtobepropitious.Popperalsorecognizedthatfalsifiabilityisnotadichotomousmatter(either/or)butratheramatterofdegrees.Sometheoriesaremorefalsifiablethanothers.Indeed,noneoftheexamplesmentionedaboveareentirelyresistanttoempiricalrefutation.Andeventhemosttractabletheoriesputupsomeresistance.Generallyspeaking,anargumentismostfalsifiableinsofarasitisopera-tional,parsimonious,generalinpurview(offeringalargeterritoryforempiri-caltesting),wellbounded(sothatthepopulationofaninferenceisclear,anddefensible),coherent(internallyconsistent),clearwithrespecttocounter-factualsandcomparisons,andrelyingonasfewassumptionsaspossible.Additionalissuesariseduringthetheory-testingphaseofresearch.Forexam-ple,oneismoreinclinedtobelievearesultifasolid“partition”hasbeenmaintainedbetweentheconstructionoftheargumentanditssubsequenttesting;thisensuresthatthereisminimalwiggle-roomtoadjusttheargumenttosuittheresultsofatestortoadjustthetesttosuitthehypothesis.Goodtestsare“severe”;badonesarepermissive.Withrespecttocausalanalysis,themoststringenttestsareusuallyexperimentalinnature.Andsoforth.Agreatwealthoffactors–manymorethanPopperexplicitlyconsidered–contributetotherigorwithwhichahypothesisisappraised.Theseareexplored10Popper([1934]1968:92).31BeginningsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
insubsequentchapters.Someofthesecriteriaareintrinsictotheformalstructureoftheargument;othersrelatetotheproceduresusedtotestthatargument.TradeoffsArguably,allthetasks,strategies,andcriteriaintroducedintheremainingchaptersarewaysofachievingorinstantiatingeitherdiscoveryorappraisal.Theseprimalgoalsinformeverymethodologicalendeavor.Complicatingmatters,however,thesemethodologicalgoalsareoftenintensionwithoneanother.Ontheonehand,researchersareencouragedtoseekouttheunknown.Thisrequiresanexploratoryapproachtotheempiricalworld,forthereisnosystematicprocedurefordiscoveringnewthings.Andthenewerthething(themorerevolutionary),thelessrule-boundistheprocedure.PaulFeyerabendmakesthispointforcefully:Theideaofamethodthatcontainsfirm,unchanging,andabsolutelybindingprin-ciplesforconductingthebusinessofsciencemeetsconsiderabledifficultywhenconfrontedwiththeresultsofhistoricalresearch.Wefindthen,thatthereisnotasinglerule,howeverplausible,andhoweverfirmlygroundedinepistemology,thatisnotviolatedatsometimeorother.Itbecomesevidentthatsuchviolationsarenotaccidentalevents,theyarenotresultsofinsufficientknowledgeorofinattentionwhichmighthavebeenavoided.Onthecontrary,weseethattheyarenecessaryforprogress.Indeed,oneofthemoststrikingfeaturesofrecentdiscussionsinthehistoryandphilosophyofscienceistherealizationthateventsanddevelopments,suchastheinventionofatomisminantiquity,theCopernicanRevolution,theriseofmodernatomism(kinetictheory;dispersiontheory;sterochemistry;quantumtheory),thegradualemergenceofthewavetheoryoflight,occurredonlybecausesomethinkerseitherdecidednottobeboundbycertain“obvious”methodologicalrules,orbecausetheyunwittinglybrokethem.11Theprocessofdiscoveryisinherentlyanti-nomothetic–or,asFeyerabendwouldsay,anarchic.12Fromthisperspective,traditionalscientificmethodologyistoorespectfulofexistingtheoreticalconstructsandmethods.Scientistsneedtogetoutsidetheironcageofnormalscience–toaplacewheretheprocesses11Feyerabend(1975:23).12Feyerabend(1963,1975).AlthoughFeyerabendtookaradicalstanceagainstscience(astraditionallyunderstood),hisworkisdigestiblewithintheframeworkoftraditionalphilosophyofscienceifapproachedasacorrectivetoanaive,Popperian(“positivistic”)viewofthescientificprocess.MuchofwhatFeyerabendhadtosayappliedwithparticularforcetothecontextofdiscovery(thoughherejectedtheutilityofthediscovery–appraisaldistinction).32PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
ofexplorationandtestingaremutuallyintertwinedanddifficulttodisentangle.Here,theoriesarenotalwaysneatlyandcleanlyfalsifiable.Ontheotherhand,researchersarerightlyencouragedtodevelopriskypropositionsandhardtests,soastoassistinthetaskofappraisal.Thisistheconservativemomentofscience,personifiedbyKarlPopper.Here,thereareplentyofrules(oratleastgeneraltasks,strategies,andcriteria)toguideone’sresearch.Thefalsificationistconsidersthegreatestsinsofsocialsciencetobethoseofcommission,ratherthanomission.Thevirtueofgoodscienceistokeepquietwhenthetruthisambiguous–nottosaymorethanoneknowswithareasonablelevelofcertainty.(Indeed,Poppercounselsagainsttheuseoftheterm“truth”underanycircumstances.)Onlyinthisfashionwilltheproductsofsciencebedistinguishablefromconjectures,thestock-in-tradeofpoliticians,journalists,andcocktail-partyprognosticators.Onlyifthefieldisclearofnonsensewillthelong,slowprocessofscientificcumulationoccur.Manysocialscientistshaveembracedthisaustere,taciturnviewofscience(atleastrhetorically).Here,theprimaryjobofthemethodologististovigilantlyguardthegatesofscience,ensuringthatnounauthorizedentrantsareadmitted.Contratheorthodoxy,Iwillinsistthatatleasthalfthebattleofscienceliesinidentifyinginterestingproblemstosolve.Indeed,findingtherightquestionmaybemoreimportantinthelongrunthanfindingtherightanswerforalessinterestinghypothesis.Fromthisperspective,goodscienceisnotjustamatterofrigorbutalsoofinsight(or,ifyoupreferamorereligiouslytingedmetaphor,ofinspiration).Notethattheoreticaldevelopmentcouldnotoccur,orwouldoccuronlyveryslowlyandhaltingly,ifresearcherskepttheirPopperianblinderson–limitingthemselvestopre-formedhypothesesandyes/noempiricaltests.Aconstructivemethodologyshouldenableresearcherstothinkaboutproblemsinnewways;itshouldnotfocusnarrowlyandobsessivelyontesting.Tobesure,thereisplentyofammunitionforprotagonistsinbothcamps.Therearethosewhofeelthatthereisaltogethertoomuchtestingandnotenoughtheory(ornotenoughgoodtheory),andthatoureffortsshouldthereforebefocusedonthelatter.Andtherearethosewhofeelthatthereistoomuchtheory(ortoomanytheories)andnotenoughtesting,andthatoureffortsshouldbefocusedonthelatter.Whichsideofthisdebateoneadoptsdependsuponhowmuchconfidenceonehasineitherventure.Ifoneisconfidentinone’sabilitytocraftbettertheoriesandcorrespondinglyskepticalofourabilitytotestthem,onehewstothediscoverycamp.If,ontheother33BeginningsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
hand,oneisskepticalaboutobtaininglastingtheoreticaladvancesandrela-tivelyoptimisticaboutdevisingnewandbettertests,onefindsoneselfintheappraisalcamp.Thisisnotadebatethatwecansettle;Isimplynotetheissueforreaderstoconsider.Theharder,andsurelythemoreimportant,questionishowinnovativeoneoughttobeinthechoiceoftopic.Again,therearetwopositions,eachofwhichhascompellingpointstomake.Somebemoanthelackoftheoreticalambitionfoundamongthecurrentgenerationofscholars,presumablybyreferencetoanearliergenerationof“BigThinkers.”AdamPrzeworskiwrites:TheentirestructureofincentivesofacademiaintheUnitedStatesworksagainsttakingbigintellectualandpoliticalrisks.Graduatestudentsandassistantprofessorslearntopackagetheirintellectualambitionsintoarticlespublishablebyafewjournalsandtoshyawayfromanythingthatmightlooklikeapoliticalstance.Thisprofessionalismdoesadvanceknowledgeofnarrowlyformulatedquestions,butwedonothaveforumsforspreadingourknowledgeoutsideacademia.13Itisprobablytruethatmembersoftoday’sgenerationaremoreapttoacceptthenormsandextanttheoriesofthedisciplinethanthe1960sgeneration,whichperhapsqualifiesthemaslesstheoreticallyambitious.Probably,theyarelesspoliticallyengaged–thoughthisisnotnecessarilyconnectedtointellectualcuriosity.Alternatively,onemightarguethatthisgenerationhasfocuseditsenergiesinamoreproductivefashionthanpreviousgenerations.Indeed,manyofthe“BigTheories”propoundedinthesocialsciences–thenandnow–aredifficulttodigest.Ifatheoryisnotfalsifiable,ordoesnotcumulatewellwithothertheories(eithersubsumingthemortakingitsplacebesidethem),itisunlikelytomoveafieldforward.Insum,thequestionofhowtheoreticallyambitiousoneshouldbeisdifficulttoanswerinthegeneralsense.Oneshouldbeexactlyasambitiousasonecanbe,whileretainingtouchwiththeempiricalrealityunderinvesti-gation.Thegoalsoftheoreticalinnovationmustbebalancedbythequestfortheoryappraisal.Indeed,fromPopper’sperspective,thegoalsofdiscoveryandappraisalareentirelycompatiblewithoneanother.“Boldconjectures”canbecombinedwithstrenuouseffortsat“refutation.”14Sometimesthisispossible,andtotheextentthatitis,itdefinesthesummumbonumofscience.13QuotedinSnyder(2007:20).14Popper([1934]1968,1965).34PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
Evenso,thetensionbetweendiscoveryandappraisalseemsrathermoreintrinsicandirresolvablethanPopperwaswillingtoadmit.Considerthatifone’sprimarymotivationisthediscoveryofnewtheories,thenresearchersmusthavelatitudetoproposebroadandabstracttheorieswithoutclearlytestablehypotheses.Insofarashypothesesaregeneratedandtested,thistestingprocessshouldbeopen-ended–involvingnumeroushypothesesandacontinualprocessofadjustmentbetweentheoryandevidence–before,during,andaftertheresearchisconducted.Itisnotsurprisingthatresearchofthe“soakingandpoking”variety(whetherqualitativeorquantitative)isnotveryconvincing–thoughitmaybequiteprovocative,andmaylead,downtheline,tomoreconvincingdemonstrationsoftruth.Insofarasone’sprimarymotivationistotestthetruth-valueofanexistingtheoryone’smodeofproceduremustbequitedifferent.Here,atheoryshouldbeframedinaspreciseamanneraspossiblesothatitissuesspecific,testablepredictions.Theprocessoftheorydiscoveryandappraisalshouldbesegregatedfromoneanotherasmuchaspossible,sothereislittleroomforsubjectiveinterventionsinthetestingprocessorposthocaltera-tionsofthetheory.Inallrespects,theoryandresearchdesignshouldbe“risky,”allowingmanyopportunitiesforatheorytofail.Theproblemwiththisstyleofresearchisequallyapparent.Iftakenseriously,Popper’sinjunctionswouldseverelyconstrainthetypeoftheoriesadmissibletothecanonofsocialscience.InadditiontoMarxismandFreudianism,whichPopperexplicitlycondemned,itwouldalsoraisedoubtsaboutWeberiantheories,socialcapitaltheory,evolution-basedmodels,theoriesofinternationalrelations(e.g.,realism,liberalism,idealism/constructivism),rational-choicemodels,andmanyothersaswell.Withinthenaturalsciences(Popper’shometurf),thedemandforfalsifiabilitywouldpresumablyforceonetorejectstringtheoryandotherhighlyabstractandscarcelytestablecomponentsofmodernphysics.Popperiansmightrespondthat,whatevermessinessmightbeinvolvedintheprocessofdiscovery,atsomepointtheoriesoughttobeissuedinfalsifiableform.Thismerelybegsthequestion:atwhatpointshouldthisbe?Notethatmostofthetheoreticalframeworksmentionedpreviouslyhavebeenextantforacenturyormore,andappeartobenoclosertoadefinitiveempiricaltest.Indeed,broadtheoriesrarelyfallwhentheyfailempiricaltests.Thesefailures,contraPopper,canusuallybeexplainedaway(perhapsbyadhocadjustmentsofthetheory),ortreatedaspartoftheerrorterm.1515Gorski(2004);Lakatos(1978).35BeginningsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
ToadoptaphrasefromDouglasMacArthur:oldtheoriesneverdie,buttheysometimesfadeaway.Specifically,theymeettheirdemisewhenamorecompel-lingtheoryisproposed,onewhichattractsresearchersformerlycommittedtothelong-establishedtheory.Gradually,theoryBeclipsestheoryA.TheprocessisLakatosian(involvinggrandtheoreticalframeworks)ratherthanPopperian(involvingmiddle-rangepropositions).Inthisrespect,progressattheoreticalandempiricallevelscannotbeseparatedfromoneanother.Andinthisrespect,again,itmayappearthatourenergiesarebetterfocusedonthegenerativecomponentofsciencethanonthefalsifiability–verifiabilitycomponent.Marx,Freud,andWeberoughttobeouravatars,notthethousandsofassembly-linesocialscientistswhospendtheirlivestestingmiddle-rangetheories.Ishallconcludebyreturningtothecentralpoint:goodsciencemustembraceboththegoalofdiscoveryandthegoalofappraisal.Onewithouttheotherisnotserviceable.Indeed,scienceadvancesthroughadialecticofthesetwobroadresearchgoals.Inthelanguageofstatisticaltests,theemphasisofexploratoryanalysisisonavoidingTypeIIerrors(acceptingafalsenullhypothesis),whiletheemphasisoffalsificationisonavoidingTypeIerrors(incorrectlyrejectingatruenullhypothesis).InKuhnianterms,theconflictbetweentheorydevelopmentandtheory-testingmaybeunderstoodasacontrastbetween“revolutionary”(paradigm-breaking)scienceand“normal”(paradigm-constrained)science.Althoughthetermsareperhapsinappropriatelyapocalyptic,thecontrasthighlightsarecurrenttensioninthefieldofscience,wheresomelabortoinventnewtheorieswhileotherslabortotestthosetheories.1616Onewayofnegotiatingthisdisputeistoexaminethespecificcircumstancesofapieceofresearchtoseewhichsortofapproachiswarranted.Afalsificationistprocedureislikelytobejustifiablewhereverresearchonatopicisabundant,theprincipalhypothesisiswelldefined,experimentalmethodscanbeapplied,TypeIerrorsareofgreaterconcernthanTypeIIerrors,onehasreasontobeespeciallyconcernedaboutthepersonalbiasesandpreconceptionsoftheresearchers,aneutraloversightbodyisavailabletomonitorresearchonatopic,andresearchfundingisplentiful–inthesecases,hypothesis-generationandhypothesis-testingareappropriatelysegregated,andrigidrulesofprocedureoughttobeapplied.Popper,notFeyerabend,shouldbeourguide.Andyet,theseconditionsareoftenabsent–especiallyinthesocialsciences.Giventhisfact,thereislittlepointindressingupourresearchasifitfitstherequirementsofPopperianscience.Notethatsocialsciencejournalsfrequentlyinsistuponthepresentationofapriorihypotheses(“suggestedbytheliterature”),whichwillthen(thewritercharacteristicallymovesintothefuturetense)be“testedagainstthedata,”evenwhentheproceduresactuallyfollowedinthecourseoftheresearchareblithelyexploratory.Nothingisgained–andagreatdealmaybelost–bypresentingourfindingsinthismisleadingfashion.Recognizingthis,thedisciplinesofsocialscienceneedtodoabetterjobofdistinguishingworkthatistheory-testingfromworkthatis–rightly,andjustifiably–theory-generating.Bothshouldbehonored,insofarascircumstances(outlinedabove)warrant.36PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
FindingaresearchquestionMostofthisbookisdevotedtoproblemsofappraisalonceaspecifichypothesishasbeenidentified.Thisfollowsstandardpracticeamongmethodologicaltexts.However,afewwordsontheproblemoftheorydevelopmentareinorder.Howdoesonegoaboutidentifyingafruitfulresearchquestionand,ultimately,aspecificresearchhypothesis?Thisistheveryearlyexploratoryphase,whenonequiteliterallydoesnotknowwhatoneislookingfor,orat.Arguably,itisthemostcrucialstageofall.Nothingofinterestislikelytoemanatefromresearchontopicsthataretrivial,redundant,ortheoreticallybland–nomatterhowstrongtheresearchisfromafalsificationistperspective.Methodologistsgenerallyleavethistasktotherealmofmetaphor–bells,brainstorms,dreams,flashes,impregnations,lightbulbs,showers,sparks,andwhatnot.Thereasonforthislackofattentionisperhapstobefoundinthefactthatbeginningsareinherentlyunformulaic.Therearefewrulesorcriteriaforuncoveringnewquestionsornewhypotheses.Methodologistsmayfeelthatthereisnothing–nothingscientificatanyrate–thattheycansayaboutthisprocess.KarlPopperstatesthematterforthrightly,asusual:“Thereisnosuchthingasalogicalmethodofhavingnewideas,”hewrites.“Discoverycontains‘anirrationalelement,’ora‘creativeintuition.’”17However,sayingnothingatallmaybeworsethansayingsomethingunsystematic.Therestofthischapterthereforedepartsfromtheformatadoptedelsewhere.WhatIhavetoofferismoreinthecharacterofahomilythanaframework.Itreadslikeanadvicecolumn.Iurgethereadertostudythetradition,beginwhereyouare,getoffyourhometurf,playwithideas,practicedis-belief,observeempathically,theorizewildly,thinkahead,andconductexploratoryanalyses.Asaresult,thechapterisriddenwithshouldsandshouldnots.Iapologizeinadvancefortheratherdidactictone.18Myadviceislargelycommonsensicalandbynomeanscomprehensive.Itcannothelpbutreflectmyownviewsandexperiences,thoughIhavedrawnextensivelyonthewritingsofotherscholars.19Nonetheless,itmayhelpto17QuotedinKing,Keohane,andVerba(1994:129).18Withregardtomyownbonafides,letmenotethatinthisparticularareaofresearch(“startingout”)Icanperhapsclaimspecialauthority.Overthepasttwodecades,Ihavefoundmyselfcontinuallystartingafreshwithnewtopics,someofwhich(perhapsinevitably)haveturnedouttobelessenlighteningthanothers.19Theliteraturerelevanttothischapteremanatesfromresearchontheconjoinedsubjectsofdiscovery,innovation,andexploration,aswellasfromadvicecolumnsinnewslettersandintroductorytextbooks.37BeginningsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
orientthosewhoaresettingoutontheirfirstjourney,orwhowishtobeginagain.StudythetraditionThequestionofinnovativenessnecessarilyhingesonthetraditionofworkthatalreadyexistsonasubject.Thisisnotasubjectiveprior;itisoneestablishedbyafieldofscholarsworkingonatopicovermanyyears,anditshouldbeapparentinthepublishedworkthattheyhaveproduced.(Ifnot,theinquirymustbecarriedoutthroughpersonalcommunicationwithestablishedscholarsinafield.)Considerthestateofthefieldonatopic.Whatarethefrontiersofknowl-edge?Whatdowe–collectively,asadiscipline–know,andwhatdon’tweknow?Consideralsotheprobablelocationofthisfrontieradecadefromnow,extrapolatingfromcurrentscholarlytrends.Whatwillthecutting-edgebethen?Keepinmindthatthemostactiveresearchfrontiersareusuallymovingfrontiers;thetraditionasitexiststodaymaybequitedifferentwhenyoufinishyourresearch.Soabetterquestion(thoughamoredifficultone)is,whatwillthecutting-edgebeinadecade?Idoubtifanyonehashappeneduponareallyinterestingresearchtopicsimplybyreadingareviewoftheextantliterature.However,thisisanefficientmethodofdeterminingwherethestateofafieldliesandwhereitmightbeheaded.Beawarethatbecauseofthelengthoftimerequiredbythepublicationprocess,themostrecentworkonasubjectisusuallytobefoundinconferencepapersorpaperspostedonpersonalwebsites.Nowadays,theseareeasytolocatethroughsearchengines.YourfirstrecoursemightbeGoogleratherthanJSTOR.Inexposingoneselftotheliteratureonatopiconemustguardagainsttwocommonresponses.Thefirstistoworshipthosethathavegonebefore;thesecondistosummarilydismissthem.Respectthetradition–don’tflagellatetheforefathers.Thereisnothingsojejuneasareversalofhierarchies(“They’rewrongandI’mright”).Butdon’tbeawedbythetraditioneither.TrysteppingRegrettably,thisliteratureisfocusedmostlyontheorganizationalcontextofdiscovery(e.g.,bysocialpsychologistsandsociologists)andondiscoverywithinthenaturalsciences,wheretheconcepthasitscounterpartinthenotionofaclear“finding.”Inthesocialsciences,wheredefinitivefindingsarescarceandcumulationmoredubious,theconceptofdiscoverycarriesamoreambiguousmeaning.Withthiscaveat,thefollowingworksproveduseful:Koestler(1964);Luker(2008);McGuire(1997);Mills(1959:195–226);Oliver(1991);Root-Bernstein(1989);Snyder(2007).SeealsoAbbott(2004);Fleck([1935]1979);Freedman(2008);Geddes(2003:27–45);Hanson(1958);King,Keohane,andVerba(1994:14–19);Kuhn([1962]1970);Langleyetal.(1987);Most(1990);Root-BernsteinandRoot-Bernstein(1999);Useem(1997);Watson(1969).Onthecreativeactofconstructingformalmodels,seeCartwright(1983);Hesse(1966);LaveandMarch(1975).38PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
outsidethecategoriesthatareconventionallyusedtodescribeandexplainasubject.BythisImeannotsimplyarguingagainstthecommonwisdom,butalsothinkingupnewquestions,newissues,thathavenotbeenwellexplored.Insofarasnewtheoreticalparadigmsare“revolutionary,”thisiswhattheyconsistof.Asyouperusetheliterature,beconsciousofwhatexcitesyouandwhatbothersyou.Whichissuesareunder-explored,orbadlyunderstood?Wheredoyoususpecttheauthoritiesinafieldarewrong?Whatquestionshavetheyleftunanswered?Whatquestionsdoyoufindyourselfaskingwhenyoufinishreading?Wheredoesthislineofresearchlead?Sometimes,typicallyinaconclusionorareviewarticle,scholarswillreflectself-consciouslyuponthefuturedirectionofresearch;this,too,canbeuseful.Inanycase,youshouldnotlimityoureventualreviewoftheliteraturetoonlythemostrecentpublications.Ofinterestisnotonlythefrontierbutthehistoryofasubject.Thus,acomplementarystrategyistodelveintothe“classics”–thefoundingtextsofafieldorsubfield.20Thisisuseful(particularlyifyouhaveneverdoneso)becauseitsometimespromptsonetothinkaboutfamiliarsubjectsinnewways,becauseclassicworkstendtobeevocative(andthusraisequestions),becauseadifferentvocabularyisoftenemployed,andbecauseitisareminderthatsomethingshave,infact,beendonebefore.Thislastpointiseducationalintworespects:itwarnsusthatwemaybeabouttoreinventtheproverbialwheelanditinformsusofwaysthatperceptionsandconclusionsaboutafamiliarsubjecthavechangedwithinadiscipline(andwithinsocietyatlarge)overtime.Everysubjecthasanintellectualhistoryanditisworthwhilefamiliarizingyourselfwiththishistory,notmerelytofindapithyepigraphbutalsotoinformyouranalysisofaproblem.AsC.WrightMillsbeganhisstudyofelites,heconsultedtheworksofLasswell,Marx,Michels,Mosca,Pareto,Schumpeter,Veblen,andWeber.21Incommentinguponthisexperience,Millsreports:Ifindthattheyofferthreetypesofstatement:(a)fromsome,youlearndirectlybyrestatingsystematicallywhatthemansaysongivenpointsorasawhole;(b)someyouacceptorrefute,givingreasonsandarguments;(c)othersyouuseasasourceofsuggestionsforyourownelaborationsandprojects.Thisinvolvesgraspingapointandthenasking:HowcanIputthisintotestableshape,andhowcanItestit?HowcanIusethisasacenterfromwhichtoelaborate–asaperspectivefromwhichdescriptivedetailsemergeasrelevant?20Snyder(2007).21Mills(1959:202).39BeginningsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
Noteverytopicisblessedwithsucharichheritage;butsomeare,andthereitisworthpausingtoread,andtothink.BeginwhereyouareWithquestionsofmethodCharlesSandersPeircepointsout,“Thereisonlyoneplacefromwhichweevercanstart…andthatisfromwhereweare.”22Theeasiestandmostintuitivewaytoundertakeanewtopicistobuilduponwhatoneknowsandwhooneis.Thisincludesone’sskills(languages,technicalskills),connections,lifeexperiences,andinterests.23Hopefully,achosentopicresonateswithyourlifeinsomefashion.Thisisoftenasourceofinspirationandinsight,aswellasthesourcefromwhichsustainedcommitmentmaybenourishedandreplenishedoverthelifeofaproject.C.WrightMillswrites:Youmustlearntouseyourlifeexperienceinyourintellectualwork:continuallytoexamineandinterpretit.Inthissensecraftsmanshipisthecenterofyourselfandyouarepersonallyinvolvedineveryintellectualproductuponwhichyoumaywork.Tosaythatyoucan“haveexperience,”means,foronething,thatyourpastplaysintoandaffectsyourpresent,andthatitdefinesyourcapacityforfutureexperience.Asasocialscientist,youhavetocontrolthisratherelaborateinterplay,tocapturewhatyouexperienceandsortitout;onlyinthiswaycanyouhopetouseittoguideandtestyourreflection,andintheprocessshapeyourselfasanintellectualcraftsman.24Becausethebusinessofsocialscienceistoinvestigatetheactivitiesofpeople,anypersonalconnectionswemighthavetosuchpeoplemayserveasusefulpointsofleverage.Thehermeneuticactiseasedifonecanestablishsomepersonalconnection–howeverdistantorimaginative–withthegroupinquestion.25Sometimes,ourconnectionwithatopicismotivatedmorebyideasthanbypersonalconnections.Wearenaturallydrawntosubjectsthatareeitherhorrifyingoruplifting(orboth).Indeed,manyresearchprojectsbeginwithsomenotion–perhapsonlydimlyformulated–aboutwhatiswrongwiththeworld.Weallhavebeesinourbonnetsandthisnormativemotivationmaybevitaltoourinsightintothattopic.Whatreal-lifeproblem,relevanttoyourdiscipline,bothersyou?2622Kaplan(1964:86),paraphrasingCharlesSandersPeirce.23FinlayandGough(2003);Krieger(1991);Mills(1959);Snyder(2007).24Mills(1959:196).25Gadamer(1975)referstothisasafusionofhorizons–usandtheirs(theactorsweareattemptingtounderstand).26GerringandYesnowitz(2006);Shapiro(2005);Smith(2003).40PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
Thedesiretoredresswrongsalsohelpstokeepsocialsciencerelevanttotheconcernsoflaycitizens.Weallbegin,onemightsay,ascitizens,witheveryday(“lay”)concerns.Overtime,wecometoattainadegreeofdistancefromoursubject,quascholars.Thus,dotherolesofcitizenandscholarengageindialoguewithoneanother(Chapter14).Ofcourse,attheendofaprojectonemusthavesomethingtosayaboutatopicthatgoesbeyondassertionsofrightandexcoriationsofwrong.Thetopicmustbemadetractableforscientificinquiry;otherwise,thereisnopointinapproachingitasascientificendeavor.Ifonefeelsthatthetopicistooclosetothehearttoreflectuponitdispassionately,thenitisprobablynotagoodcandidateforstudy.Asaprobe,askyourselfwhetheryouwouldbepreparedtopublishtheresultsofastudyinwhichyourmainhypothesisisprovenwrong.Ifyouhesitatetoanswerthisquestionbecauseofnormativepre-commitmentsyoushouldprobablysettleonanothersubject.Asageneralrule,itisimportanttoundertakequestionsthatonefeelsareimportant,butnotprojectsinwhichonehasespeciallystrongmoralorpsy-chologicalpredilectionsforacceptingorrejectingthenullhypothesis.27Thus,onemightbemotivatedtostudytheroleofschoolvouchersbecauseoneisconcernedaboutthequalityofeducation.Butoneprobablyshouldnotunder-takeastudyofvouchersinordertoprovethattheyareagood/badthing.GetoffyourhometurfWhiletheprevioussectionemphasizedtheimportanceofbuildinguponone’spersonalprofile(skills,connections,druthers),itisalsovitalforscholarstostrayfromwhatissafe,comfortable,andfamiliar–theirhometurf.Considerthattheacademyisnotnow,andlikelyneverwillbe,arepresenta-tivecross-sectionofhumankind.Atpresent,thedenizensofsocialsciencearedisproportionatelywhite,Anglo-European,and(still,thoughdecreasingly)male.Theywillprobablyalwaysbedisproportionatelyprivilegedinclassback-ground.Evidently,ifmembersofthesedisciplinesrestrictthemselvestotopicsdrawnfromtheirpersonalexperiencelittleattentionwillbepaidtotopicsrelevanttoexcludedgroups,especiallythosethatarelessprivileged.Themoreimportantpointisthatadvancesinknowledgeusuallycomefromtransgressingfamiliarcontexts.Afterall,localknowledgeisalreadyfamiliartothosewholiveit.Whatevervaluemightbeaddedcomesfromtransportingcategories,theories,andwaysofthinkingacrosscontexts,inthehopethatnew27Firebaugh(2008:ch.1).41BeginningsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
perspectivesonthefamiliarwillbecomeapparent.Agoodethnography,itissometimessaid,renderstheexoticfamiliarorthefamiliarexotic.Thesamemightbesaidofsocialscienceatlarge.Trytothinklikeastrangerwhenapproachingatopicthatseemsobvious(fromyour“hometurf”perspective).Likewise,donotbeafraidtoexportcategoriesfromyourhometurfintoforeignterritory–notwillfully,anddisregardingallevidencetothecontrary,butratherasanoperatinghypothesis.Sometimes,theforeign-madeshoefits.Indeed,noveldescriptiveandcausalinferencesoftenarisewhenanextantconceptortheoryistransplantedfromoneareatoanother.Forexample,theconceptofcorporatismaroseinitiallyinthecontextofCatholicsocialtheoryasanalternativetostatesocialism.Itwaslateradoptedbyfascistregimesasawayoflegitimatingtheircontroloverimportanteconomicandsocialactors.Morerecently,ithasbeenseenasakeytoexplainingthedivergenttrajectoriesofwelfarestatesacrosstheOECD,andforexplainingthepersistenceandresilienceofauthoritarianruleinthedevelopingworld.28Thereareendlesswaysofadaptingoldtheoriestonewcontexts.Sometimesthesetransplanta-tionsarefruitful;othertimes,theyarenot.Mostimportant,trytomaintainaconversationwithdifferentperspectivesonyoursubject.Whatwouldso-and-sosayaboutX?Ifthisdoesnotdriveyoumad,itmayserveasahelpfulformoftriangulationonyourtopic.Anothersortofboundarycrossingisthatwhichoccursacrossdisciplines,theories,andmethods.Thetrendofthecontemporaryeraseemstobetowardevergreaterspecialization,andtobesure,specializationhasitsuses.Itisdifficulttomastermorethanoneareaofwork,giventheincreasinglytechnicalandspecializedtechniquesandvocabularydevelopedwithineachsubfieldoverthepastseveraldecades.Makingacontributiontoafieldnecessitatesadeepfamiliaritywiththatfield,andthisrequiresaconcentratedfocusovermanyyears.Yetitisworthreflectinguponthefactthatmanyoftheworksthatweregardtodayaspath-breakinghavebeentheproductofexoticencountersacrossfieldsandsubfields.Indeed,allfieldsandsubfieldsweretheproductoflong-agotransgressions.Someonemovedoutsidetheircomfortzone,andothersfol-lowed.Notealsothatthesocialsciencesarenotdividedupintodiscreteandwell-definedfields.So,tryreadinginside,andoutside,yourareaoftraining.Talktopeopleindistantfields.Seehowtheyrespondwhenyoudescribeyourquestions,andyourprojectedresearch,tothem.Bewareofcultivatinganarrowexpertise,forthisisapttoleadtoworkthatistheoreticallycircumscribedor28Collier(1995);Schmitter(1974).42PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
mundane.Ifallacademicworkistheftofonesortoranother,oneiswelladvisedtostealfromdistantsources.Anotherwordforthissortoftheftiscreativity.PlaywithideasTheliteratureoninventionanddiscovery–pennedbysciencewriters,philo-sophersofscience,andbyinventorsthemselves–isinconsensusononepoint.Originaldiscoveriesareusuallynottheproductofsuperiorbrainpower(i.e.,theabilitytocalculateorreason).RobertRoot-Bernsteinisemphatic:Famousscientistsaren’tanymoreintelligentthanthosewhoaren’tfamous.[Moreover,]I’mconvincedthatsuccessfulonesaren’trightanymoreoftenthantheircolleagues,either.Ibelievethatthearchitectsofsciencearesimplymorecurious,moreiconoclastic,morepersistent,readiertomakedetours,andmorewillingtotacklebiggerandmorefundamentalproblems.Mostimportant,theypossessintellectualcourage,daring.Theyworkattheedgeoftheircompetence;theirreachexceedstheirgrasp…Thus,theynotonlysucceedmoreoftenandoutofallproportion;theyalsofailmoreoftenandonthesamescale.Eventheirfailures,however,betterdefinethelimitsofsciencethanthesuccessesofmoreconventionalandsafescientists,andthusthepioneersbetterservescience.29Thekeyquestion,asRoot-Bernsteinframesit,is“Howcanonebestsurviveontheedgeofignorance?”30OnewayofansweringthisquestionissuggestedbyRichardHofstadter,whodescribesintellectuallifeasacounterpointofpietyandplayfulness.Thefirstreferstothesomberanddoggedsearchfortruth.Thesecond,whichsavestheenterprisefromdogmatismandwhichmaybelessobvious,istheintel-lectual’scapacitytoplay:Ideally,thepursuitoftruthissaidtobeattheheartoftheintellectual’sbusiness,butthiscreditshisbusinesstoomuchandnotquiteenough.Aswiththepursuitofhappiness,thepursuitoftruthisitselfgratifying,whereastheconsummationoftenturnsouttobeelusive.Truthcapturedlosesitsglamor;truthslongknownandwidelybelievedhaveawayofturningfalsewithtime;easytruthsareabore,andtoomanyofthembecomehalf-truths.Whatevertheintellectualistoocertainof,ifheishealthilyplayful,hebeginstofindunsatisfactory.Themeaningofhisintellectuallifeliesnotinthepossessionoftruthbutinthequestfornewuncertainties.HaroldRosenbergsummedupthissideofthelifeofthemindsupremelywellwhenhesaidthattheintellectualisonewhoturnsanswersintoquestions.29Root-Bernstein(1989:408).30Root-Bernstein(1989:408).43BeginningsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
EchoingHofstadter’sdescription,onemightsaythattherearetwodistinctmomentsinanyresearchproject.Thefirstisopen-ended,playful;here,awidevarietyofdifferentideasaregeneratedandgivenatrialrun.Thesecondisfilledwithzealandpiety;here,onegripstightlytoasingleideainthequesttodevelopitintoafull-blowntheoryandtestitagainstsomeempiricalreality.Thisconformstothedistinctionbetweendiscoveryandappraisalintroducedabove.Whatevertheshortcomingsofthisdichotomy,thereisnoquestionthattheacademicendeavorrequiresacrucialshiftofattitudeatsomepointintheenterprise.Sinceweareconcernedherewiththeinitialphase,weshalldwellontechniquesofplayfulness.Althoughtheartofdiscoverycannotbetaught(atleastnotinthewaythatthetechniqueofmultipleregressioncanbetaught),itmaybehelpfultothinkforamomentaboutthinking.Theactofcreationismysterious;yetthereseemtobeafewpersistentfeatures.ArthurKoestler,synthesizingtheworkofmanywriters,emphasizesthatdiscoveriesareusually“alreadythere,”inthesenseofbeingpresentinsomebodyofwork–thoughperhapsnotthebodyofworkwithwhichithadheretoforebeenassociated.Todiscoveris,therefore,toconnectthingsthathadpreviouslybeenconsideredseparate.Todiscoveristothinkanalogically:Thisleadstotheparadoxthatthemoreoriginaladiscoverythemoreobviousitseemsafterwards.ThecreativeactisnotanactofcreationinthesenseoftheOldTestament.Itdoesnotcreatesomethingoutofnothing;ituncovers,selects,re-shuffles,combines,synthesizesalreadyexistingfacts,ideas,faculties,skills.Themorefamiliartheparts,themorestrikingthenewwhole.Man’sknowledgeofthechangesofthetidesandthephasesofthemoonisasoldashisobservationthatapplesfalltoearthintheripenessoftime.YetthecombinationoftheseandotherequallyfamiliardatainNewton’stheoryofgravitychangedmankind’soutlookontheworld.31Whatframeofminddoesthisrequire?Howdoesonethinkanalogically?Thistrickseemstohavesomethingtodowiththecapacityto“relinquishconsciouscontrols,”toblockouttheacademicsuperegothatinhibitsnewthoughtsbypunishingtransgressionsagainstthetradition.32Aboveall,onemustfeelfreetomakemistakes:Justasinthedreamthecodesoflogicalreasoningaresuspended,so“thinkingaside”isatemporaryliberationfromthetyrannyofover-preciseverbalconcepts,oftheaxiomsandprejudicesengrainedintheverytextureofspecializedwaysofthought.Itallowsthemindtodiscardthestrait-jacketofhabit,toshrugoffapparentcontradictions,to31Koestler(1964:119–120).32Koestler(1964:169).44PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
un-learnandforget–andtoacquire,inexchange,agreaterfluidity,versatility,andgullibility.Thisrebellionagainstconstraintswhicharenecessarytomaintaintheorderanddisciplineofconventionalthought,butanimpedimenttothecreativeleap,issymptomaticbothofthegeniusandthecrank;whatdistinguishesthemistheintuitiveguidancewhichonlytheformerenjoys.33Itmightbeaddedthatwhatalsodistinguishesthegeniusandthecrankisthattheformerhasmasteredthetraditionofworkonasubject.Thegenius’liminalmomentsarecreativebecausetheytakeplaceonafoundationofknowledge.Inordertoforget,andthencerecombinefeaturesofaproblem,onemustfirstknow.Theanalogyofdiscoverywithadream-liketrance,althoughitbordersonsilliness,maynotbefaroff.Koestlerwrites:Thedreamerconstantlybisociates–innocentlyasitwere–framesofreferencewhichareregardedasincompatibleinthewakingstate;hedriftseffortlesslyfrommatrixtomatrix,withoutbeingawareofit;inhisinnerlandscape,thebisociativetechniquesofhumouranddiscoveryarereflectedupsidedown,liketreesinapond.Themostfertileregionseemstobethemarshyshore,theborderlandbetweensleepandfullawaken-ing–wherethematricesofdisciplinedthoughtarealreadyoperatingbuthavenotyetsufficientlyhardenedtoobstructthedreamlikefluidityofimagination.34Ithasoftenbeensuggestedthatthemindworkssemi-consciouslyonproblemsoncetheyhavebeenidentified,andwhensufficientmotivationispresent.Atthisstage,onebecomespossessedbyaquestion.Practicedis-beliefOnecannotthinkwithoutwords,butsometimesonecannotthinkwellwiththemeither.Sometimes,ordinarylanguageservestoconstrainthought-patterns,reifyingphenomenathatarescarcelythere.Whenwedefine,EdmundBurkecommented,“weseemindangerofcircumscribingnaturewithintheboundsofourownnotions.”35Languagesuggests,forexample,thatwhereareferentialtermexistsacoherentclassofentitiesalsoexists,andwheretworeferentialtermsexisttherearetwoempiricallydifferentiableclassesofentities.Sometimesthisistrue,andsometimesitisnot.Justbecausewehaveawordfor“socialmove-ment”doesnotmeanthatthereareactuallyphenomenaouttherethataresimilartoeachotherandeasilydifferentiatedfromotherphenomena.Dittofor“socialcapital,”“interestgroup,”andvirtuallyeveryotherkeyconceptinthe33Koestler(1964:210).34Koestler(1964:210).35QuotedinRobinson(1954:6).45BeginningsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
socialsciencelexicon.Wordsdonotalwayscarvenatureatitsjoints.Sometimes,theyarehighlyarbitrary(“constructed”).Afortiori,justbecausewehaveawordforsomephenomenondoesnotmeanthatcasesofthisphenomenonallstemfromthesamecause,orthesamesetofcauses.Itisnotevenclearthatthesamecausalfactorswillberelevantforallmembersoftheso-namedsetofphenomena.Thereadermightrespondthat,surely,conceptsaredefinedthewaytheyarebecausetheyareusefulforsomepurposes.Precisely.Butitfollowsthatthesesameconceptsmaynotbeusefulforotherpurposes.Andsinceone’sobjectiveatthisstageoftheresearchgameistothinkunconventionally,itisimportanttocallintoquestionconventionallanguage.Forheuristicpurposes,tryassum-inganominalistperspective:wordsaremerelyarbitrarylexicalcontainers.Asanexercise,putbracketsaroundallyourkeyterms(“socialmovement”).Tryoutdifferentvisions;seeifanyofthemarepersuasive.(Thisisagoodexample,incidentally,ofthedifferingcriteriaapplicabletothediscoveryandappraisalmomentsofscience.Anominalistperspectiveonconceptsisproblematicwhenthewriterturnstothetaskofformalizinghisorherresearch.Here,theusualcounselistoavoidneologism,unlessabsolutelyrequired[Chapter6].)Anothertechniqueforthinkinganewaboutasubjectistoconsiderthetermsthatforeignlexiconsorancientlexiconsimposeuponaconcept;oftentheywillhavedifferentconnotationsorsuggestdifferentdistinctionsamongphenomena.Aparallelskepticismmustbeextendedtonumbers,whichalsonaturalizephenomenathatmay,ormaynot,gotogetherinthesuggestedfashion.Here,theclaimismorecomplicated.First,theuseofanumberisexplicitlylinkedtoadimension–forexample,temperature,GDP,numberofautoaccidents–thatisthoughttoberelevantinsomeway.Moreover,theimpositionofanumericalscalepresupposesaparticulartypeofrelationshipbetweenphe-nomenawithdifferentscoresonthatvariable–nominal,ordinal,interval,orratio(Chapter7).Butisitreally?Morebroadly,isthisthedimensionthatmatters(forunderstandingthetopicinquestion)?Orarethereotherdimen-sions,perhapslessreadilyquantified,thatprovidemoreaccurateorinsightfulinformation?Anothersortofconventionalwisdomiscontainedinparadigm-cases.Thesearecasesthat,byvirtueoftheirtheoreticaloreverydayprominence,helptodefineaphenomenon:thewayItalydefinesfascism;theHolocaustdefinesgenocide;theUnitedStatesdefinesindividualism;Swedendefinesthewelfarestate;andtheSovietUnion(formanyyears)definedsocialism.Paradigm-casesexistinvirtuallyeveryrealmofsocialscienceinquiry.Theyoftenprovidegoodpointsofentryintoatopicbecausetheyareoverloaded46PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
withattributes;theyoperateinthisrespectlikeideal-types(Chapter6).Yetbecausetheyanchorthinkingonthesetopics,theyarealsothought-constraining.Andbecausetheyarealsoapttobesomewhatunusual–forexample,extreme–examplesofthephenomenoninquestion,theymaypresentmisleadingdepic-tionsofthatphenomenon.Withrespecttowords,numbers,andparadigm-cases–nottomentionfull-blowntheories–itisimportanttomaintainaskepticalattitude.Perhapstheyaretrueanduseful,perhapsonlypartiallyso,oronlyforcertainpurposes.Inordertotesttheirutility,tryadoptingtheSocraticguiseofcompleteignorance(perhapsbetterlabeledasthoroughgoingskepticism).Oncehavingassumedthispose,youarethenfreetoposenaivequestionsofsources,ofexperts,andofinformants.Itisacannystrategyandcanbeextraordinarilyrevealing–particularlywhen“obvious”questionscannotbereadilyanswered,orareansweredinunexpectedways.ObserveempathicallyOnetechniqueofdiscoveryisempathic,or(toinvokethephilosophicaljargon)hermeneutic.36Here,oneemploysobservationaltechniquestoenterintotheworldoftheactorswhoareengagedinsomeactivityofinterest–playingball,draftingabill,murderingopponents,castingavote,andsoforth–inordertounderstandtheirperspectiveonthephenomenon.Ofcourse,thisiseasierwhentheactorsareourcontemporariesandcanbestudieddirectly(i.e.,ethno-graphically).Itisharder,andyetsometimesmorerevealing,iftheactionstookplacelongagoorareremovedfromdirectobservationandmustbereconstructed.Inanycase,non-obviousperceptionsrequireinterpretation,andthisinterpretationshouldbegroundedinanassessmentofhowactorsmayhaveviewedtheirownactions.Considerthattheprocessofunderstandingbeginswithanabilitytore-createorre-imaginetheexperiencesofthoseactorswhoseideasandbehaviorwewishtomakesenseof.Somehowalinkmustbeformedbetweenourexperientialhorizonsandthehorizonsofthegroupwewishtostudy.Thismayinvolveaformofrole-playing(whatwouldIdoinsituationXifIwerepersonY?).Somelevelofsympathywithone’ssubjectsisprobablyessentialforgaininginsightintoaphenomenon.Thismaybedifficulttomusterifthesubjectisgrotesque.NoonewantstoempathizewithNazis.Butthehermeneuticchallengeremains;somewaymustbefoundtoenterintothelivesandperceptionsofthese36Gadamer(1975).47BeginningsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
importanthistoricalactorsinordertoexplaintheiractions,howeverstrangeandrepellant.Althoughthosewhoidentifywiththeinterpretivistlabelarenotalwaystheoreticallyinclined,wemaygrantthatmanyofthosewhoidentifyas“theor-ists”haveatonetimeoranotheremployedinterpretivetechniques(onthesly).Inanycase,thistechniqueneednotbemonopolizedbyafewspecialistpracti-tioners(“interpretivists,”“ethnographers,”etc.).Itisagamewecanallplay–indeed,mustplay,ifwearetobesuccessfulsocialscientists.TheorizewildlyRatherthanworkingsingle-mindedlytowardOneBigIdea,youmightconsiderthebenefitsofworkingsimultaneouslyalongseveraltracks.Thisway,youavoidbecomingoverlycommittedtoasingletopictooearly.Youcanalsocomparedifferenttopicsagainstoneanother,evaluatingtheirstrengthsandweaknesses.“Justhavelotsofideasandthrowawaythebadones,”advisesLinusPauling.37Atthesametime,youshoulddoyourbesttomaintainarecordofyourideasasyougoalong.38Takealookatthisideadiaryeverysooftenandseewhichprojectsyoufindyourselfcomingbackto,obsessingabout,inquiringabout.Theobjectiveshouldbetokeepyourmindasopenaspossibleforaslongaspossible(giventhepracticalitiesoflifeandscholarlydeadlines).“Letyourmindbecomeamovingprismcatchinglightfromasmanyanglesaspossible.”39Historiansofnaturalscienceidentifyproductivemomentsofsciencewiththesolvingofanomalies–featuresoftheworldthatdonotcomportcomfor-tablywithexistingtheories.40Iftheseanomaliescanbesolvedinamorethanadhocmanner,thefrontiersofknowledgearepushedforward.Perhapsevenanew“paradigm”ofknowledgewillbecreated.Onemayquestionwhethersocialscienceisripewiththeoreticallytractableanomalies.Somewouldsaythatitexistsentirelyofanomalies;therearenounsolvedintersticestofill,onlyadeepabyssofhighlystochasticbehaviorthatisresistanttotheorizingofanysort.Itseemsclearthatmostsocialsciencefieldsarenot–ornotyet–intherealmofKuhniannormalscience.Still,wefocusourenergies,quiterightly,onareasthatarethoughttobelesswellexplained.Whethertheseareunderstoodasanomaliesoras“areasofdeeper-than-usualignorance”hardlymattersforpresentpurposes.37QuotedinRoot-Bernstein(1989:409).38Mills(1959:196).39Mills(1959:214).40Kuhn([1962]1970);Lakatos(1978);Laudan(1977).48PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
Anothertechniquefortheorizingwildlyistojuxtaposethingsthatdonotseemtofitnaturallytogether.Theorizingoftenconsistsofdis-associatingandre-associating.Oneversionofthisistoexamineafamiliarterrainandthinkaboutwhatitresembles.Whatis“X”anexampleof?CharlesRaginreferstothisas“casing”asubject.41Anothertacticistoexamineseveraldiverseterrainsinordertoperceivesimilarities.(Cancolonialism,federalism,andcorporatismallbeconceptualizedassystemsof“indirectrule”?42)Athirdversionistoexamineafamiliarterrainwiththeaimofrecognizinganewprincipleoforganization.Linnaeusfamouslysuggestedthatanimalsshouldbeclassifiedonthebasisoftheirbonestructures,anewprincipleofclassificationthatturnedouttobeextraordinarilyfecund.43Intherealmofsocialscience,scholarshaveprovidedorganizationalschemesforpoliticalparties,bureaucracies,welfarestates,andothersocialphenomena–thoughfew,itmustbenoted,haveprovenasfruitfulorasenduringastheLinnaean.Ofcourse,areorganizationofknowledgebywayofclassificationneednotbeeternalorubiquitousinordertoproveusefulforcertainpurposes.Eachre-classificationmayhavedistinctuses.Athirdtechniqueforlooseningthetheoreticalwheelsistopushaconven-tionalideatoitslogicalextreme.Thatis,consideranexplanationthatseemstoworkforaparticulareventorinaparticularcontext.(Itmaybeyouridea,orsomeoneelse’s.)Nowpushthatideaoutwardtoothersettings.Doesitstillwork?Whatsortofadjustmentsarenecessarytomakeitwork?Orconsiderthelogicalramificationsofatheory–ifitwerefullyimplemented.Whatwouldthetheoryseemtorequire?Theoriesaretestedwhentheyarepushedtotheirlimits,whentheyaretriedoutinverydifferentcontexts.Root-Bernsteinobservesthatthisstrategyleads,attheveryleast,toaninvestigationoftheboundariesofanidea,ausefulthingtoknow.Alternatively,itmayhelpustoreformulateatheoryinwaysthatallowittotravelmoresuccessfully,thatis,toincreaseitsbreadth.Athirdpossibility,perhapsthemostexciting,isthatitmayleadtoanewtheorythatexplainsthenewempiricalrealm.44Intheorizingwildly,itisimportanttokeepalistofallpossibleexplanationsthatonehasrunacrossintheliterature,orintuited.Aspartofthiscanvas,onemightconsidersomeofthemoregeneralmodelsofhumanbehavior,forexample,individual(akarational)choice,exchange,adaptation(akaevolution),diffusion,andsoforth.45Sometimes,theseabstractmodelshaveapplicationstoveryspecificproblemsthatmightnotbeimmediatelyapparent.(Howmightthe41Ragin(1992).42Gerringetal.(2011).43LinsleyandUsinger(1959).44Root-Bernstein(1989:413).45LaveandMarch(1975).49BeginningsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
topicofromancebeunderstoodasanexchange?Asanadaptation?Asaproductofdiffusion?)Onceachieved,thislistofpossibleexplanationsforphenomenonYcanthenberearrangedanddecomposed(perhapssomepropositionsaresubsetsofothers).Recallthattheoreticalworkofteninvolvesrecombiningextantexpla-nationsinnewways.Yourlistofpotentialexplanationsalsocomprisesthesetofrivalhypothesesthatyouwillbeobligedtorefute,mitigate,and/orcontrolfor(empirically)inyourwork.Soitisimportantthatitbeascomprehensiveaspossible.Inordertofigureouthowtocorrectlymodelcomplexinterrelationshipsitisoftenhelpfultodrawpictures.(Ifoneissufficientlyfluentingraphicdesign,thismaybehandledonacomputerscreen.Fortherestofus,pencilandpaperareprobablythebestexpedients.)Layingoutideaswithboxesandarrows,orperhapswithVenndiagramsordecisiontrees,allowsonetoillustratepotentialrelationshipsinamorefree-flowingwaythanispossiblewithproseormath.Onecan“think”abstractlyonpaperwithoutfallingpreytotheconstraintsofwordsandnumbers.Itisalsoahighlysynopticformat,allowingonetofitanentireargument,inall(ormost)ofitscomplexity,ontoasinglesheetorwallboard.ThinkaheadAllelementsoftheresearchprocessareintimatelyconnected.Thismeansthatthereisnosuchthingasagoodtopicifthattopicisnotjoinedtoagoodtheoryandaworkableresearchdesign.So,thechoiceofa“topic”turnsouttobemoreinvolvedthanitfirstappears.Ofcourse,alltheelementsthatmakeforasuccessfulpieceofresearchareunlikelytofallintoplaceatonce.Andyetoneisobligedtowrestlewiththem,even–onemightsay,especially–attheveryoutset.Recallingtheelementsofyourtopic–containing,letussay,atheory,asetofphenomena,andapossibleresearchdesign–itisvitaltomaintainadegreeoffluidityamongallthesepartsuntilsuchtimeasyoucanconvinceyourselfthatyouhaveachievedthebestpossiblefit.Bewareofprematureclosure.Atthesametime,toavoidendlesscyclingitmaybehelpfultoidentifythatelementofyourtopictowhichyoufeelmostcommitted,thatis,thatwhichislikelytomakethegreatestcontributiontoscholarship.Ifthiscanbeidentified,itwillprovideananchorinthisprocessofcontinualreadjustment.Considertheinitialdecisionofatopicasaninvestmentinthefuture.Aswithanyinvestment,thepay-offdependsuponlotsofthingsfallingintoplace50PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
oversubsequentyears.Onecanneveranticipateallthepotentialdifficulties.Butthemoreonecan“game”thisprocess,thebetterthechanceofapay-offwhentheresearchiscompleted.Andthebetterthechancethattheresearchwillbecompletedatall.(Reallybadideasareoftendifficulttobringtofruition;themoretheyadvance,themoreobstaclestheyencounter.)Althoughtheprospectmayseemdaunting,oneisobligedtothinkforwardatthe“gettingstarted”stageofresearch.Trytomapouthowyourideamightwork:whatsortoftheorywilleventuate,whatsortofresearchdesign,andsoforth.Ifeverythingworksoutasanticipated,whatwillthecompletedthesis/book/articlelooklike?(Thisbringsustothetopicsentertainedintherestofthebook,thatis,whataregoodconcepts,descriptiveinferences,causalinferences,andresearchdesigns?)Anobviousquestiontoconsideriswhat“results”astudyislikelytogenerate.Regardlessofthetypeofstudyundertakentherewillpresumablybesomeencounterwiththeempiricalworld,andhencesomesetoffindings.Willtheevidencenecessarytotestatheory,orgenerateatheory,beavailable?Willthemainhypothesisbeborneout?Sometimes,thefailuretorejectanullhypothesismeansthattheresearcherhasverylittletoshowforhisorherresearch.Conventionalwisdomhasprevailed.Othertimes,thefailuretoproveahypothesiscanbequiteenligh-tening.46Sometimes,atopicissonew,oraresearchdesignsomuchmorecompellingthanothersthatcamebefore,thatanyfindingisinformative.Thisisidealfromtheperspectiveofthescholar’sinvestmentoftimeandenergy,asitcannotfailtopayoff.Inanycase,itmaybehelpfultoinquireofthosewhoknowasubjectintimately(experts,keyinformants)whattheythinkyouwillfindifyoupursueyourprojectedlineofresearch.Whatistheirbesthunch?Andhowwouldtheyrespondtoafailuretorejectthenullhypothesis?Woulditbepublishable?Wouldtherejectionofyournullhypothesisbepublishable?Thisisanevenmoreimportantquestion,anditisnotalwaysapparenttothenoviceresearcher.Thatwhichseemsnoveltoyoumayseemlessnoveltothosewhohavelaboredinafieldformanydecades.And,bythesametoken,thatwhichseemsobvioustoyoumaybesurprisingtoothers.Thus,youarewelladvisedtomarket-testvariousfindings.Considerhowyouranticipatedfindingsmightbesituatedwithintheliteratureonatopic.Howwilltheybeperceived?Whatwillbetheirvalue-added?Willtheybeconsideredmorecompellingthanotherextantwork46Thisraisesthequestionofhowoneoughttodefinea“null”hypothesis;butletusleavethismatterinabeyance.51BeginningsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
onthesubject?Willtheystandthetestofcurrentscholarshipandthetestoffuturescholarship(the“testoftime”)?Intest-drivingyourideayoushouldalsokeepacloseeyeonyourself.Seeifyouroralpresentationoftheprojectchangesasyouexplainittofriendsandcolleagues.Atwhatpointdoyoufeelmostconfident,ormostuncertain?Whendoyoufeelasifyouarebull-shitting?Theseareimportantsignalswithrespecttothestrengthsandweaknessesofyourproposal.Indeed,theprocessofpresenting–asidefromanyconcretefeedbackyoureceive–mayforceyoutoreconsiderissuesthatwerenotinitiallyapparent.ConductexploratoryanalysesWhenthetimeisright,considerconductinganexploratoryprobe.Thisshouldbeconstructedsoastobeasefficientaspossible–requiringtheleastexpendi-tureoftime,energy,andmoney.Youneedtogetafeelforyoursubject,andwhatthedatamightsay;thereisnopretenseofdrawingfirmconclusions.Sometimes,thebestwaytothinkthroughaproposalistoimplementtheideainaschematicfashion.Onetime-honoredapproachistheexploratorycasestudy,enablingonetogainmorein-depthknowledgeofoneorafewcasesthatarethoughttoexemplifykeyfeaturesofatopic.Here,onefindsanumberof(moreorlesswell-known)varieties.47Atypicalcaseisonethatexhibitstraitsthataredeemedtobehighlyrepresentativeofthephenomenonofinterest.Itmaybeusefulasacluetowhatisgoingonwithinothersimilarcases.Anextremecaseisonethatexhibitsanextreme(orrare)valueonarelevant(XorY)parameter.Whenunderstoodagainstthebackdropof“normal”cases(lyingnearertothemean),anextremecaseofferssupremevariationontheparameterofinter-est;thismayofferinsightsintowhatisgoingonacrossthelargerpopulation.Asampleofdiversecasesarethosethatexhibitarangeofvariationononemoreortherelevant(X,Y,orX/Y)parameters.Withonlyasmallsetofcases,thisprovidesawayofexploringalltheavailablevariationthatalargerpopulationoffers.Adeviantcaseisonethatexhibitsanunexpectedoutcome,accordingtosomesetofbackgroundassumptions.Thisiscommonlyusedtoopenupnewavenuesofinquiry,awayofidentifyinganomalies.Amost-similarsampleofcaseshavesimilarbackgroundcharacteristics,butexhibit47Gerring(2007:ch.5).52PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
differentoutcomesalongsomeparameteroftheoreticalinterest.Thisallowstheresearchertogeneratehypothesesaboutthepossiblecausesofanout-comethatvariesacrossotherwisesimilarcases.48Anotherexploratoryapproachallowsonetoprobealargersampleofcasesinamoresuperficialfashion.Theresearchermightbeginwithanexistingdataset(towhichadditionalvariablesofinterestcanbeadded).Ortheresearchermaytrytoconstructhisorherown“truth-table,”focusinguponasmallnumberofcasesandvariablesofinterest.Supposeoneisattemptingtodeterminewhysomecountriesinsub-SaharanAfricahavedemocratizedwhileothershavenotinthedecadessinceindependence.Onewouldbeginbycodingthedependentvariable(autocracy/democracy),andproceedtoaddpossiblyrelevantcausalfactors–economicgrowth,urbanization,landlockedstatus,colonialhistory,andsoforth.Someofthesefactorsmightbebinary,whileotherscouldbecodedcontinuouslyorreducedtoabinaryformat(e.g.,high/low).Someofthesefactorsarelikelytobeeasytocode(“objective”),whileothersmayinvolveconsiderablejudgmentonthepartofthecoder(“subjective”).Inanycase,thissimpledata-reductiontechniqueallowsonetoincorporatealargenumberofhypothesesandtoeye-balltheirfitwiththeevidenceacrossasmall-ormedium-sizedsample.Thekeypointoftheseadventuresindataexplorationistorevealnewhypothesesandtoexposeone’shunchestopreliminarytests,asquicklyaspossible.Donotbeafraidtodealinstylizedfacts–roughguesstimatesabouttherealityunderconsideration.Moresystematictestingprocedurescanwaitforalaterstageoftheprocess.Dataexplorationshouldbeunderstoodasaseriesofplausibilityprobes.49Ofcourse,thepointatwhichtheoryexplorationseguesintotheory-testingisneverentirelyclear-cut.Anymethodofexplorationisalso,tosomedegree,amethodoftesting,andviceversa.Theexpectation,inanycase,isthatonceakeyhypothesishasbeenidentifieditwillbesubjectedtomorestringentteststhanwereemployedinitsdiscovery.TheemphasisofresearchshiftssubtlybutimportantlyfromavoidingTypeIIerrors(failingtorejectafalsenullhypothesis)toavoidingTypeIerrors(incorrectlyrejectingatruenullhypoth-esis),asdiscussed.48Thesevariedcase-selectionstrategiescanbeimplementedinqualitative(informal)orquantitative(formalized)ways.Thelatterrequiresalargesampleofpotentialcasesandrelevantdataontheparametersofinterest.StatisticaltechniquesforselectingoneorafewcasesfromalargesampleareexploredinGerring(2007:ch.5).49Eckstein(1975).53BeginningsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
ConcludingthoughtsonbeginningsPublishedworkinthesocialsciencespresentsamisleadingappearanceoforderandpredictability.Theauthorbeginsbyoutliningageneraltopicorresearchquestion,thenstatesageneraltheory,andfromthencetothespecifichypothesisthatwillbetestedandhisorherchosenresearchdesign.Finally,theevidenceispresentedanddiscussed,andconcludingthoughtsareoffered.Thisisnothingatallliketheprogressofmostresearch,whichis,bycompar-ison,circuitousandunpredictable–hardlyeverfollowingastep-by-stepwalkdowntheladderofabstraction.Onereasonforthisisthatknowledgeinthesocialsciencesisnotneatlyparceledintodistinctresearchareas,eachwithspecificandstablequestions,theories,andmethods.Instead,itischaracterizedbyahighdegreeofopen-endedness–inquestions,theories,andmethods.Anotherfactoristhecircularityoftheenterprise.Eachelementofsocialscience–theresearchquestion,theory,hypothesis,keyconcepts,andresearchdesign–isinterdependent.Thisisbecauseeachelementisdefinedintermsofalltheothers.Thus,anyadjustmentinoneelementislikelytorequireanadjust-mentallaround.AssoonasIchangemytheoryImayalsohavetochangemyresearchdesign,andviceversa.ThereisnoArchimedeanpoint.Thismeansthattherearemanypointsofentry.Onemightbeginwithageneraltopic,aresearchquestion,akeyconcept,ageneraltheory,aspecifichypothesis,acompellinganomaly,anevent,aresearchvenue(e.g.,asite,archive,ordataset),amethodofanalysis,andsoforth.Accordingly,someresearchisproblem-orquestion-driven,someresearchistheory-driven,someresearchismethod-driven,andotherresearchisphenomenon-driven(moti-vatedbythedesiretounderstandaparticulareventorsetofevents).Theseareobviouslyquitedifferentstylesofresearch–eventhough,attheendoftheday,eachstudymustbeheldaccountabletothesamemethodologicalcriteria(summarizedinTable1.1).Oncebegun,thecorrectprocedureisdifficulttodiagraminaseriesoftemporallydiscretesteps–unlessoneimagineshoppingto-and-froandback-and-forthinaratherfreneticfashion.Empiricalinvestigationisnecessarilycontingentonpre-formedconceptsandtheories,aswellasourgeneralnotionsoftheworld;yetfurtherinvestigationmayalterthesenotionsinunpredictableways.Insodoing,wereviseourconceptionofwhatwearestudying.Inthisrespect,socialscienceoffersagoodexampleoftheso-calledhermeneuticcircle.5050Hoy(1982).54PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
Toreiterate,thereisnorightorwrongplacetostart.Allthatmattersiswhereyouendup.Andyet,whereoneendsuphasalottodowithwhereonestartsout,soitisnotincidental.Scholarsarerightlywaryoftheconsequencesofchoosingabadtopic–onethat,letussay,promisesfewinterestingsurprises,haslittletheoreticalorpracticalsignificance,oroffersinsufficientevidencetodemonstrateapropositionabouttheworld.Nomatterhowwellexecutedthatresearchmightbe,littlecanbeexpectedfromit.Moreover,changingtopicsmidstreamiscostly.Onceonehasdevelopedexpertiseinanareaitisdifficulttore-tool.Research,likemanythingsinlife,isheavilypath-dependent.Forthisreason,oneshouldanticipatelivingwithone’schoiceoftopicforaverylongtime.Adissertationwillnotonlyabsorbyourlifeoverthecourseofitsdurationbutalso,inalllikelihood,fordecadestocome–perhapsfortherestofyourlife.Indeed,manyscholarscontinuetobedefined,forbetterandforworse,bytheirfirstpublishedwork.So,thequestionofchoosingatopicisbynomeanstrivial.Agreatdealisatstake.Becausetheselectionofagoodtopicisdifficult,carefuldeliberationisinorder.Notethatthedifficultyoftopicselectionisaproductofthefactthateveryoneislookingforthesamething:fruitfultopicsforresearch,thenextbreakingwave.Thismeansthatthelow-hangingfruitisprobablyalreadypicked.Accordingly,oneshouldnotexpectagreatandheretoforeunexploredtopictofallintoone’slap.Evenifitsohappensthatone’sfirsthunchiscorrectitwilltakesometimebeforethepromiseofthistopicisfullyapparent.Manyinitialprobeswillhavetobefollowedthroughandanextensiveliteraturereviewmustbeundertakeninordertoconfirmthatthetopicistrulyinnovative.Inthisarduousprocess,adviceiswelcome–fromfriends,family,advisors,expertsinthefield.Solicitallthefeedbackyoucan.Butmakesurethat,attheendoftheday,youarecomfortablewiththechoiceyoumake.Itshouldrepresentyourconsideredjudgment.Thisislikelytorequiresometime.Howmuch,itisdifficulttosay.Findingatopicisaprocess,notanevent.Itdoesn’thappenallofasudden.Itstartsassoonasonetakesupscholarshipandtransposesgraduallyintotheresearchitself.Thereisnoclearbeginningorend-date.Althoughthewritermayberequiredtocomposeaformalgrantproposalorprospectusthisusuallyturnsout,inretrospect,tobeanarbitrarymarkerwithintheongoinglifeofaproject.Manyscholarsarenotpreparedfortheagonizingandtime-consumingtaskofhead-scratching(akachin-rubbing,forelock-tugging–chooseyourmeta-phor),whichseemstoruncountertotheinjunctiontopublish,publish,publish(quickly,quickly,quickly).Onceuponatime,lifeintheacademywasextolledasaviacontemplativa.Nowadays,oneisstruckbythefactthatthereisagreatdeal55BeginningsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
ofpublishingbutrelativelylittlesustainedcogitation.Mostofourtimeisspentintheimplementationofprojects.Wesecurefunding,overseestaff,constructsurveys,designexperiments,peruseevidence,writeupresults,allthewhilemaintainingafreneticemailcorrespondence.Onlyinbriefmomentsdoweallowourselvestheluxuryofthinkingdeeplyaboutasubject.Bythis,Imeanthinkingintrulyopen-endedways,waysthatmightleadtonewinsights.Atwhatpointshouldonemakeacommitmenttoaresearchquestionandaspecifichypothesis?Howdoesoneknowwhentoreachforclosure?Evidently,therearedangersassociatedwithprecipitousdecisionsandwithdecisionsthataretoolongdelayed.Considerthisfamiliarscenario,relatedbyKristinLuker.Astudent(“you”)entershisorheradvisor’sofficewithahazilyframedideaofwhatheorshewouldliketoworkon.Theadvisordemandstoknowwhatthehypothesisis.Ifyouflounderaroundtryingtoanswerthisquestion,heorshemayfollowupbyaskingwhatyourindependentanddependentvariablesare.Evenmorebasically,heorshewillaskwhatyourresearchquestionis.Youjustgoblank,feelinglikearabbittrappedontheroadwaywiththeheadlightsbearingdownonyou,asyoutrydesperatelytoexplainwhat’ssointerestingabout,say,privatizedwater,orrisingratesofimprisonmentinAmerica,oradolescentsexuality.Whenyouandyouradvisorpartattheendofthetimeallottedtoyou,morelikelythannot,youpartinmutualfrustration.51Inthissetting,thestudentisprobablynotreadytoidentifyaresearchquestion,muchlessaspecifichypothesis.Itisstillarelevantquestion,andtheadvisorisobligedtoraiseit.However,inthehastetoanswerthisquestioninasatisfactoryway–andescapefromthescenewithself-esteemintact–thestudentmaycommittoaquestionthatisnot,inthelongrun,veryfruitful.Thesamethinghappenswitharbitrarydeadlinesimposedbytheacademiccalendar–acon-ferencetowhichonehascommittedtopresent,aprospectusdefensedate,andsoforth.ThisistheScyllaofprematureclosure.Ontheotherextreme,oneencountersthedangerofbelatedclosure.Lukercontinues:Supposeontheotherhandthatyouhaveaneasygoingadvisor,andyouarepermittedtogooff“intothefield”…withoutansweringhisorherquestions.Anevenmoredreadedfatemaywellawaityou,worsethanbeingtorturedintoproducingindepen-dentanddependentvariablesondemandforyouradvisor,namely…theDamnationoftheTenThousandIndexCardsortheTenThousandEntriesintoyourcomputer-assistednote-takingsystem.TheDamnationoftheTenThousandWhatevershappens51Luker(2008:18).56PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
tounwittinggraduatestudentswhohavespentmanyyears…gatheringdatawithouthavingstumbleduponexactlywhatitwasthattheywerelookingforwhentheyfirstwentouttothatfabulousfieldsite(…orlibrary…).Theretheysit,doomedanddamned,infrontofthecomputerscreen,wonderinghowtomakeastoryoutofthetenthousandentries.Or,worseyet,theyfinallydostumbleontoastoryastheyporeyetagainoverthetenthousandentries,butthesinglepieceofinformation(orthebodyofdata)whichtheyneedtoreallynailthepointbeyondquibblesisbackinthefieldandtheydidn’tknowtheyneededit,orit’sdisappeared,ortheycan’taffordtogoback.Ortheydofindit,andrealizethateightypercentofthedatatheyhavegatheredisirrelevant…Anin-betweenoutcome…isthatyoumayactuallyfindtheresearchquestion,comeupwiththedatathatyouneedtomakethecase,andhaveacompellingand…well-writtenstorytotell.Theonlyproblemisthatyouhaveeighteenboxesofdataleftover,andtheentireenterprisetookyouatleastfouryearslongerthanitshouldhave.52Todescribethissortofdisaster,LukerquotesalinefromPaulineBart:“Data,dataeverywhereandnotathoughttothink.”53Inourownresearch–andregardlessofwhetherwearejuststartingoutasstudentsofsocialscienceorhavespentdecadesinthebusiness–wemustavoidtheScyllaofprematureclosureaswellastheCharybdisofbelatedclosure.Neitherwillservethecauseofscience,orourowncareers.Pushyourselftofindaresearchquestionasquicklyaspossible,butdon’tsettleonsomethingthatdoesn’tseemmeaningfultoyouortoyourintendedaudience.52Luker(2008:19).53Luker(2008:19).57BeginningsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.004Cambridge Books Online © Cambridge University Press, 2012
Cambridge Books Onlinehttp://ebooks.cambridge.org/Social Science MethodologyA Unified FrameworkJohn GerringBook DOI:http://dx.doi.org/10.1017/CBO9781139022224Online ISBN: 9781139022224Hardback ISBN: 9780521115049Paperback ISBN: 9780521132770Chapter3 – Arguments pp. 58-73Chapter DOI:http://dx.doi.org/10.1017/CBO9781139022224.005Cambridge University Press
3ArgumentsA:(Knock)B:Comein.A:Ah,Isthistherightroomforanargument?B:Itoldyouonce.A:Noyouhaven’t.B:YesIhave.A:When?B:Justnow.A:Noyoudidn’t.B:YesIdid.A:Youdidn’tB:Idid!A:Youdidn’t!B:I’mtellingyouIdid!A:Youdidnot!!B:Oh,I’msorry,justonemoment.Isthisafiveminuteargumentorthefullhalfhour?MontyPython,“TheArgumentClinic”Argumentationincontemporarysocialsciencedescendsfromtheancientartofrhetoricandtheequallyancientscienceoflogic.Acompleteargumentconsistsofasetofkeyconcepts,testablehypotheses(akapropositions),andperhapsaformalmodelorlargertheoreticalframework.Acausalargumentshouldalsocontainanexplicationofcausalmechanisms(Chapter8).Anargumentiswhatwespeculatemightbetrueabouttheworld;itengagestherealmoftheorizing.Sometimes,itisimportanttodistinguishamongargumentslyingatdifferentlevelsofabstraction.Themostabstractmaybereferredtoasmacro-leveltheories,theoreticalframeworks,orparadigms.Exampleswouldincludestruc-turalfunctionalism,modernizationtheory,exchangetheory,symbolicinterac-tionism,orconflicttheory.Ataslightlylessabstractlevelonefindsmeso-leveltheoriesormodels.Andatthemostspecificlevelonespeaksofhypotheses,58Downloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:27 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.005Cambridge Books Online © Cambridge University Press, 2012
inferences,micro-leveltheories,orpropositions,whichareassumedtobedirectlytestable.(Explanationsmayapplytoanylevel.)So,forexample,workonthetopicofschoolvouchersmightincludeageneraltheoryaboutwhyconsumerchoiceenhancestheeducationalprocess,aformalmodelincorporatingvariouselementsofthattheory,andaspecifichypothesisorsetofhypothesesregardingtheimpactofavoucher-basedinterventiononeducationalattainment.Granted,varyinglevelsofabstractionarenotalwayseasytodiscern.Oneperson’sabstractionisanotherperson’sspecificity.Historicalsociologistsanddemographerswouldunderstandabstractionquitedifferently.Moreover,thetermsdefinedabovearelooselyapplied.Inthevouchersexamplesketchedabove,forinstance,thegeneraltheorymightbereferredtoasamodel,thehypothesesmightbeaccompaniedbyadditionalmodels(explainingthework-ingsofeachone),andvirtuallyeveryelementmightbereferredtoasatheory.Notethatthewordtheorymayimplyahighlevelofabstractionormaysimplyindicatethatthereisadegreeofspeculationassociatedwithanargu-ment.Inthelattersense,theoryissynonymouswithpropositionorhypothesis.Theoriesarecharacteristicallyassociatedwithcausalinference–butnotalways.Evidently,thereisagreatdealofterminologicalfluidityinthissemanticfield.Consequently,Ishallnotinsistuponfinedistinctions,andtermssuchasexplanation,hypothesis,inference,model,proposition,theorywillbeemployedinterchangeablyinthetext.Allarearguments,thatis,assertionsaboutthenatureofsomeempiricalreality.Ourinterestinsocialscienceargumentsliesnotintheirsubstancebutratherintheirmethodologicalproperties.Whatmakesanargumentusefulforsocialscience?Whatisagoodargument?Andwhat,bycontrast,isabad(unhelpful)argument?Thesequestionsaretreatedfleetinglyinmostmethodologytexts.Often,theyaredismissedasamatterofphilosophy.1Andyettheyturnouttoplayacriticalroleinsocialscienceinquiry.Ishalltrytoshowherethatallsocialscienceargumentsstrivefortruth,precision,generality,boundedness,parsimony,coherence,commensurability,andrelevance,assummarizedinTable3.1.Naturally,thesedesideratameanslightlydifferentthingswhenappliedinthecontextofdescriptiveandcausalarguments.However,theyaresimilarenoughtobeintroducedtogether.Inlatersectionsofthebookweexplorethedistinguishingcharacteristicsofthesetwostylesofargumentation(seePartsIIandIII).1Brieflistsofscientificdesiderataappearoccasionallyintheliteratureonsocialsciencemethodology(e.g.,Laudan1996:132),butarerarelydeveloped.Authorsappeartoplaceratherlittleweightonthisapproachtothesubject.59ArgumentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:27 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.005Cambridge Books Online © Cambridge University Press, 2012
TruthArgumentsstrivetobetrue.Thisisthefirstandforemostvirtueofasocialscienceproposition,fortrueargumentsaregenerallymoreusefulthanfalsearguments.Granted,sometimesfalseinferencescanachievegoodthings,butwedonotthinkofsocialscienceashavinganyvalueexceptinsofarasitsinferencesarebyandlargetrue.Ifscienceisuntrue,thereislittlepointtotheenterprise.Yettheissueoftruthisnotsostraightforwardasitmayatfirstappear.First,onemustbearinmindthatthetruthofanargumentisusuallyunderstoodbyreferencetotheargumentitself:thatclaim,orsetofclaims,thataremadeabouttheworld.Scholarspickoutspecificissuestocontest.Theydonotclaimtorepresentthewholetruthaboutanysubject,muchlessaboutallsubjects.Thescholar’schosenargumentmaybeframedina“positive”or“negative”way.Theargument“TheoryAiswrong”isanegativeargument,whichmaybetrueorfalse.Itistheauthoroftheargumentwhodecidesthetermsbywhichheorshewillargue,thatis,whatthebaselineornullhypothesisis,againstwhichtheargumentwillbeframed.Therearealsoperipheraldimensionsofanargumentinvolvingthebound-ariesofaninference(itsscopeorpopulation),themechanismsofacausaltheory,andotherissuesassociatedwithparticularstylesofdescriptiveandcausalargumentation(asreviewedbelowandinsubsequentchapters).EachTable3.1Arguments:generalcriteria1.Truth(accuracy,validity,veracity)Isittrue?2.Precision(specificity)Isitprecise?3.Generality(breadth,domain,population,range,scope)Howbroadisthescope?Howmanyphenomenadoesatheorydescribe/explain?4.Boundedness(scope-conditions)Howwellboundedisit?5.Parsimony(concision,economy,Occam’srazor,reduction,simplicity)Howparsimoniousisit?Howmanyassumptionsarerequired?6.Coherence(clarity,consistency;antonym:ambiguity)Howcoherentisit?7.Commensurability(consilience,harmony,logicaleconomy,theoreticalutility;antonym:adhocery)Howwelldoesitcumulatewithotherinferences?Doesitadvancelogicaleconomyinafield?8.Relevance(everydayimportance,significance)Howrelevantisittoissuesofconcerntocitizensandpolicymakers?60PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:27 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.005Cambridge Books Online © Cambridge University Press, 2012
maybejudgedtrueorfalse.Itfollowsthatanargumentmaybetrueinsomerespectsanduntrueinothers.PrecisionAsecondquestionaboutanyargumentisitsdegreeofprecision.Themorepreciseaclaim,themoreusefulitis,inthesenseofprovidingmoreinformationaboutaputativephenomenon.Italso,notcoincidentally,makestheargumentmorefalsifiable.2Ifyouareunconvincedbythis,considertheobverse:aperfectlyimprecisestatementabouttheworld,forexample,“Municipalgovern-mentsinAfricaaredemocratic,autocratic,orsomewhereinbetween.”Thissortofstatementexcludesnologicalpossibility,sinceallpolitiesmustbeclassifiablesomewhereonthisspectrum.Atthelimit,astatementofnoprecisionwhatso-eversaysabsolutelynothingabouttheworld,andthereforeiscompletelyunfalsifiable.Ofcourse,theimpositionofgreaterprecisionmayimposecostsalongotherdimensions.Inparticular,precisiongenerallyvariesinverselywiththeprob-ableaccuracyofanargument.Thegreatertheprecision,thelesslikelyanargumentistobetrue.Thus,ifwemightmodifyourhypotheticalexampletoread“65percentofthemunicipalgovernmentsinAfricaaredemocratic”wearelesslikelytobeprovencorrect.Therearemanyopportunitiestofail.Therelativeprecisionorimprecisionofanargumentmaybeexpressedinavarietyofways.Qualifiersforprecision(e.g.,“exactly”)orimprecision(“usually,”“roughly,”“sortof,”“generally,”“may,”“approximately,”“tendsto”)maybeinsertedintheformulationofanargument.Alternatively,astatementmightbequantified(“65percent”),andthisnumbermayberepresentedwithanumberofdecimalplaces(“65.000percent”),correspondingtothedegreeofprecisionassociatedwiththeestimate.Anothertechniqueistheconfidenceinterval,theintervalaroundanestimatethatindicatestherangeofvaluesthatestimateislikelytotake(atagivenlevelofcertainty,say95percent).GeneralityIfthefundamentalpurposeofsocialscienceistotellusabouttheworld,thenitstandstoreasonthataninferenceinformingusaboutmanyphenomenais,byvirtueofthisfact,moreusefulthananinferencepursuanttoonlyafewphenom-ena.Iwillrefertothisdesideratumasgenerality(akabreadth,generalizability,or2Indeed,precisionandfalsifiabilityareinvokedasvirtualsynonymsbyPopper([1934]1968).61ArgumentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:27 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.005Cambridge Books Online © Cambridge University Press, 2012
scope).Onewishesforatheorytoencompassasmanyphenomenaaspossible.Themoreonecanexplainwithagivenargument(ceterisparibus)themorepowerfulthatargumentis.Theoriesofgreatbreadthtellusmoreabouttheworldbyexplaininglargerportionsofthatworld.Thus,atheoryofdemocracythatsatisfactorilydescribesorexplainsregimetypesacrossallnation-statesissuperiortoonethatappliestoonlyasingleregionoftheworldorasinglehistoricalepoch.Andatheoryortheoreticalframeworkdescribingorexplainingdifferenttypesofphenomenaismoreusefulthanonepertainingtoonlyasingleoutcome.NotethatthepowerofMarxismderivesfromitsapplicationtoawiderangeofsocialbehavior;itisnotjustatheoryofrevolutionoratheoryofeconomicbehavior.ThefactthatmembersofeverysocialsciencetribefindrecoursetosomeversionofMarxisttheorytestifiestotheextraordinarybreadthofthisframework.Bycontrast,Malinowskinotesinhisanthropologicalclassic,ArgonautsoftheWesternPacific:Thereisnovalueinisolatedfactsforscience,howeverstrikingandnoveltheymightseem.Genuinescientificresearchdiffersfrommerecurio-huntinginthatthelatterrunsafterthequaint,singularandfreakish–thecravingforthesensationalandthemaniaofcollectingprovidingitstwofoldstimulus.Scienceontheotherhandhastoanalyseandclassifyfactsinordertoplacetheminanorganicwhole,toincorporatetheminoneofthesystemsinwhichittriestogroupthevariousaspectsofreality.3Whileperhapstoostronglyargued(surely,sometimesweareinterestedinparticularoutcomes),itmaybeagreedthatbreadthissuperior,ceterisparibus,tonarrowness.Granted,researchersaredifferentlyenamoredofgeneralityasananalyticgoal.Someworkinthesocialsciences,usuallywithahistoricaltheme,focusesonsingleevents,forexample,thedeclineoftheEnglisharistocracy,theFrenchRevolution,theFirstWorldWar,orthefalloftheSovietUnion.Evenso,thissortofworkisnotentirelyheedlessofgenerality.First,eachoftheseeventsmayhavevastimplicationsforthesubsequentdevelopmentofindividualcountriesand/orfortheworldatlarge.InexplainingX,onemayalso,byextension,beexplainingotherphenomena.Second,eachoftheseeventsmayberegardedasacasestudyofamoregeneralphenomenon.Again,oneissheddinglightonamuchlargerpopulation.4Andfinally,macrosocialeventslikerevolutionsmayencompasshundreds,ifnotthousands,ofmicroevents.3Malinowski([1922]1984:509).SeealsoEaston(1953:55);Kincaid(1990);Lakatos(1978:33);Laudan(1977);Levey(1996);McIntyre(1996);PrzeworskiandTeune(1970:4);Scriven(1962);Skyrms(1980),andtheworkofothernaturalisticallyinclinedscholarslikeCarlHempelandErnestNagel.4Gerring(2007).62PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:27 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.005Cambridge Books Online © Cambridge University Press, 2012
Considerablereductionisrequiredinordertoreachanyconclusionaboutawar,revolution,orachangeinclassstructure.TogeneralizeabouttheFrenchRevolutionistogeneralizeacrossalargechunkofreality.Intheserespects,eventhemostidiographichistorianwouldprobablynotwishtoabsolvethemselvesofthedemandforgenerality.Themostimportantpointisthatwhilegeneralityisanacknowledgedgoalofscience,parochialismisnot.Indeed,whatidiographicallyinclinedwriterscringeatinworkofgreatbreadthisusuallynotbreadthpersebutratherthesacrificeofothervirtues,forexample,truthorprecision.Thus,myclaimforgenerality,ifcouchedinceterisparibusterms(asallcriteriaare),maybeviewedasaconsensualnormwithinthesocialsciencedisciplines(withthepossibleexceptionofanthropology,whichhasmovedtowardthehumanitiesinrecentdecades).Howeveroneviewsgeneralityasanormativeideal,itwillbeseenthatthecriterionisalwaysamatterofdegree.Noargumentisunlimitedinscope.(Boundaryconditionsareimplicitinthetermscontainedinanyhypothesis,descriptiveorcausal.)Likewise,fewargumentsinthesocialsciencespertaintoonlyasingleevent(howeverthatmightbedefined),asnotedabove.Considerthefollowingthreeresearchquestions:(1)whydidtheFrenchRevolutionoccur?;(2)whataccountsfortherevolutionsthathaveoccurredinthemodernera(1789–2000)?;and(3)whydorevolutionsoccur?Thefirstwayofframingthequestionisthemostspecific,thelastisthemostgeneral.Buteveninthelattercasewecanidentifyboundariestothepopulation.Forexample,arevolution(asthetermisusuallyunderstood)presumestheexistenceofapoliticalentitymorecomplexthanabandortribe,suchasanempireornation-state.Thislimitsthescopeofanyargumentthatmightbedevised.Forpurposesofthisbook,Iamconcernedprimarilywithargumentsthatspeaktopopulationsthatarelargerthanwhateversampleofobservationsisunderstudy.Ofcourse,externalvalidityiscontingentuponinternalvalidity(termsthatwillbedefinedinChapter4).IfoneisstudyingtheFrenchRevolutioninordertolearnsomethingaboutrevolutionsingeneral,thenonemustcometotermswiththecauseoftheFrenchRevolution.However,variousfeaturesoftheinferentialprocessaredifferentifone’sultimategoalistoexplainrevolutionsratherthansimplytheFrenchRevolution.Notethatevenwherepopulationsaresampledexhaustively–thatis,whenallcaseswithinapopulationarestudied(astheyoftenareinglobalstudiesofnation-states)–thereisadifferencebetweengeneralizingandparticularizinginferences.Ageneralizinginferenceregardsthechosensampleasasubsetofsomelarger(perhapsdifficulttodefine)populationthatextendsbackward63ArgumentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:27 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.005Cambridge Books Online © Cambridge University Press, 2012
andforwardintime–beyondthechosentime-frameoftheactualanalysis.Itmayalsoregardthesampleasonepossiblesamplefromasetof“alternateworlds”–acounterfactualthought-experiment.5Bycontrast,aparticularizinginferenceisfocusedonlyonexplainingwhathappenedwithinthechosencasesduringthetimeperiodofthestudy.Withrespecttocausalinference,particularizingargumentsareabout“causesinfact”(akaactualcauses)ratherthancausesingeneral.BoundednessWithrespecttothescopeofanargument,biggerisbetter–butevidentlyonlytoapoint.Indeed,asanargumentisextendeditsveracity,precision,orcoherenceoftendeclines.Thisisthepointatwhichthecriterionofgeneralitybeginstoconflictwithotherscientificcriteria(includingthoselikefecundityandimpactthataretakenupinsucceedingchaptersinconnectionwithdescriptiveandcausalinference,respectively).Inframinganargumenttheresearcher’sobjectiveistoidentifythosephe-nomenathatproperlybelongwithinthescopeofatheory,excludingthosethatdonot.Inferencesshouldbeclearlyandappropriatelybounded–neithertoobignortoosmall.Inempiricalcontexts,thepopulationofaninferenceisusuallyunderstoodasthepopulationfromwhichastudiedsampleischosen.IfasampleisdrawnrandomlyfromindividualslivingintheUnitedStatesthenthepresumedpopulationistheUnitedStates.Here,however,Iamreferringtothescopeofanargumentratherthantherepresentativenessofasample.Forcausalinferencesthishasaparticularmeaning.Thescopeshouldextendtoallcasesforwhichtheexpectedcausalrelationshipisthesame,givencertainbackgroundfactors.Informalnotation,forallN,E(Y|X).Thatis,forallprospectivecaseslyingwithinthescopeofanargumenttheexpectedvalueofanoutcome,givenacausalfactor(s)oftheoreticalinterest,shouldbethesame.Unfortunately,identifyingthisGoldilockspointisnotalwayseasy.Sometimes,thescope-conditionsofatheorysimplycannotbeprobedempiri-cally.Sometimes,theempiricalevidenceistestable,butresultsareambivalent:theargumentfadesoutslowlyasthescopeoftheinferenceexpands,withnodefinitivecut-offpoints.Andevenwhereevidenceistestableandregistersapparentlyconclusivecut-offpoints,onecanneversolvetheboundary5Lebow(2007);TetlockandBelkin(1996).64PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:27 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.005Cambridge Books Online © Cambridge University Press, 2012
problemsolelybyappealtoevidence.Thescope-conditionsofanargumentrestonunderlyingassumptionsaboutthelogicalpurviewofatheory.Considerthefollowingexample:voterscaremoreaboutdomesticpolicythanforeignpolicy.Ourquestionis,whatistheappropriatescopeofthisproposi-tion?Whatareitsboundaryconditions?Theempiricalapproachwouldbetotestallvoters,everywhere.Buttherearepracticallimitstothisapproach.Andhistoricalvoters,priortotheinitiationofsurveyresearch,cannotbepolled.So,oneiscompelledtoconsiderthelogicoftheargument.Underwhatcircum-stanceswouldthispropositionsensiblyapply?Thespecificationofaclearlyandproperlyboundedinferenceisessentialtoitsfalsifiability.Indeed,theentirelyunboundedpropositioncannotbetested,foritisnotapparentwherethetheoryapplies.Failingtospecifytheboundariesofatheoryisequivalenttosaying,ineffect,“Thescopeconsistsofthoseplaceswherethetheoryistrue,andtheareaoutsidethescopeconsistsofthoseplaceswherethetheoryisfalse.”Thisisaresearchquestion,butnotanargument.Ifthecaseswhereatheoryfailsareexcludedfromtheboundariesofaninferenceonehaseffectivelydefinedoutofconsiderationallcasesthatdonotfitthetheory.Studiesintherationalchoicegenrehavebeenaccusedofthissortofgerrymandering,whichDonGreenandIanShapirorefertoasan“arbitrarydomainrestriction.”6Whetherrational-choicersareguiltyofthissinisnotimportantforpresentpurposes.Thepointtorememberisthatthespecificationofscopeisonlythefirststepontheroadtoameaningfulargument.Onemustalsoensurethatthechosenboundariesmakesense.Anarbitrarilyboundedinference,onethatfollowsnoapparentlogic,isnotconvincing.Thereadercansurelythinkofexamplesoftheorieswhosescope-conditionsaretooambitious,notambitiousenough,orsimplyambiguous(perhapsleftimplicit).Anyinferencemaybestretchedintononsensebyanarbitrarilylargescope.Bythesametoken,aninferencemayberenderednonsensicalbyadoptinganarbitrarilysmallscope.What“makingsense”meansvariesbycontext,andwillbediscussedinchapterstofollow.Here,afewgeneralremarkswillsuffice.Considerthatsomescope-conditionsarepatentlyabsurd.Forexample,writersoccasionallyproclaimthattheirinferenceisintendedtoexplainthepastandthepresent,butnotthefuture.Whileitmaybegrantedthatatsomepointinthefuturesocialrealitieswillbesoalteredthatthescope-conditionsofcurrenttheoriesnolongerobtainitseemsunlikelythatthismomentwillbereachedonthedayabookorarticleispublished.Temporalboundariesarejustifiableonlyiftheycanbeconnectedtophenomenaintheworldthatmightimpacttheworkingsofagiven6SeeGreenandShapiro(1994:45).65ArgumentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:27 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.005Cambridge Books Online © Cambridge University Press, 2012
theory.Thesameappliestospatialboundaries,forexample,“LatinAmerica,”“NorthOswego,”“schoolswithviolentjuveniles.”Tosaythatanargumentisproperlyboundedistosaythatclearboundarieshavebeenspecified,and–moreimportantly–thattheseboundariesmakegoodtheoreticalsense.ParsimonyAseventhgeneralgoalofscienceisreduction,thatis,reducingtheinfiniteplenitudeofrealityintoacarefullyframedargumentfromwhichunnecessarydrossisremoved.Totheextentthatanargumentachievesthisgoalitisparsimo-nious.Likealever,itliftsheavyweightswithamoderateapplicationofforce.Itisefficient,anditsefficiencyderivesfromitscapacitytoexplainalotwithaminimalexpenditureofenergy.If,ontheotherhand,aninferenceisnotsummarizableinacompactform,itsimpactisdiminished.(Readerswillrecognizethatthegoalofparsimonyisimpliedbythegoalofgenerality,andviceversa.7)Thegoalofparsimonyisnotnecessarilyatwarwiththetotallengthofastudy.Indeed,lengthyanalysesmayberequiredtoprovideevidenceforapithyargument.OnemightconsidertheworkofCharlesDarwin,KarlMarx,AdamSmith,HerbertSpencer,andOswaldSpenglerinthisregard.Noneofthesemenwereknownfortheirshortnessofbreath.All,however,areknownfortheirparsimonioustheories.Parsimonydoesnotprecludelength,thoughitdoescallforasummarystatementofkeypropositions.8Notethatthecriterionofparsimony,sometimesexpressedasOccam’srazor,appliesinequalmeasuretoargumentssetforthinproseaswellasthoselaidoutinmathematicalsymbols.Intheformer,parsimonyisequivalenttoconcision.Inthelatter,parsimonyisreflectedinthenumberofparameterscontainedwithinamodelorthecomplexityandlengthofaproof.Parsimonyisvaluablenotbecauseweassumethatsimplicityconformstothenaturalorderofthings.Thisassumptionmayholdtruefornaturalphenomena;butitisofdoubtfulapplicationintherealmofhumanactionandhumanlycreatedinstitutions,wheretheassumptionofcomplexityusuallyclaimsgreaterfacevalidity.Itisforpragmatic,ratherthanontological,reasonsthatoneprefersaparsimoniousinferenceoveraprolixone.Weneedtobringknowledge7Forworkontheinterrelatedquestionsofreduction,simplicity,andparsimony,seeK.Friedman(1972);M.Friedman(1974);Glymour(1980);Hesse(1974);King,Keohane,andVerba(1994:20);Kitcher(1989);Popper([1934]1968);Quine(1966);Simon(2001);Sober(1975,1988).8ContemporaryexamplesofparsimonycoexistingwithlengthcanbefoundinCollierandCollier(1991);Fischer(1989).Somelongbooks,however,offervirtuallynoattemptatsynopsisatall,e.g.,Gay(1984–98);Kantorowicz(1957);Pocock(1975).66PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:27 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.005Cambridge Books Online © Cambridge University Press, 2012
togetherinreasonablycompactforminorderforthatknowledgetoserveausefulpurpose.9Moreover,aparsimoniousargumentrequiresfewerassumptionsabouttheworld.Notethatascientificpropositionisunderstandableonlyinlightofanexistinglanguage(technicalandordinary)andbodyofknowledge.Inferencesbuilduponwhatweknowalready–orthinkweknow–abouttheworld.Nothingstartsentirelyfromscratch.Agoodargumentrequiresfewerdepar-turesfromcommonsense,fewerleapsoffaith,fewerstipulations,feweraprioriassumptions.Itrests,intheserespects,uponmoresecurefoundations.Apoorlyconstructedtheory,bycontrast,asksthereadertoacceptagreatdealabouttheworldupontheauthorityoftheauthor.Thissortofinferencedoesnotbuildonsolidfoundations.Itisstipulative.Ofcourse,allargumentsrestuponassumptions,anditisadvisabletorendertheseassumptionsastransparentaspossiblesothattheargumentcanbeeasilyevaluated.(Thisisoneofthebenefitsofawell-constructedformalmodel.)Thatsaid,thefewerassumptionsnecessaryforapropositionthemoresecurethatpropositionis,andthelessempiricalworkisrequiredinordertoproveitsveracity.Onecanthinkofeachassumptioninanargumentasalinkinalogicalchain.Onecanevaluatetheprimafaciestrengthoftheoverallargumentbythenumberofassumptionsitrequiresandtheirrelativecertaintyoruncertainty.Theidealargumenthasonlyoneempiricalquestionatissue–themainhypothesis–allelseisregardedasfirmlygrounded,oralreadyestablished.ThereasoningbehindOccam’srazorpointstoalargertruthabouttheconductofscience,namely,innovationcannotoccuronallaspectsofaproblematonce.InOttoNeurath’smuch-citedanalogy,scientificreconstructionoccursasitwouldforashipwhileatsea.Eachbeammustbereplacedimmediatelybyanother,drawnfromthesamevessel,butperhapsservingsomedifferentfunctionsuchthattheeffect,overtime,istotransformtheoriginalpurposeofthevesselbeyondrecognition.10Thepointisthattheremovaloftoomanyplanksatoncewouldcausetheship(byextension,theargument)tosink.Wemustworkincrementally.Indeed,meaningbreaksdownentirelywhenlanguageisstretchedtoofar.Argumentsnolongermakesense.Thus,thefewerassump-tionsrequiredbyatheorythemorefalsifiableitis,andthemorebelievable.Itfitswithinwhatwetakeforgrantedabouttheworld.9King,Keohane,andVerba(1994:20,104)adoptthereality-is-simpleinterpretationofparsimonyandrejectthecriteriononthosegrounds.Ifinterpretedasapragmaticnorm,however,itmightnotberejectedbytheauthors.See,e.g.,theirdiscussionoftheimportanceofleverage(“explainingasmuchaspossiblewithaslittleaspossible”,p.29).10Neurath(1971:47).67ArgumentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:27 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.005Cambridge Books Online © Cambridge University Press, 2012
CoherenceInordertobemeaningfulanargumentmustdemonstratesomedegreeofcoherence(internalconsistency).Iftherearemanymovingparts,asinabroadandabstracttheory,theyoughttoholdtogether.Acomplextheoryshouldrevolvearoundasinglecoreandthepreceptsshouldbelogicallylinked.Onefacetshouldimplytheothers.Indeed,ifthepartsofanargumentareincon-sistent,theargumentitselfisvirtuallymeaningless,andsurelyuntestable.(Forsomewriters,parsimonyisequivalenttocoherence.Butsincethesetermshavesomewhatdifferentimplications,Ilistthemseparately.)CommensurabilityWehavesaidthatargumentsassumemeaningwithinafieldofpre-existingconceptsandtheories;indeed,theyarescarcelyunderstandablepurelyontheirownterms.(Whattermswouldthatbe?)Likefacts,theoriesdonotstandalone.Theyrelatetoabroadersetoftheories,typicallyafieldorsubfieldofstudy.Doesatheoryfitcomfortablywithin,above,orbesideothertheories?Doesitadvancelogicaleconomyinafield,perhapsbysubsumingneighboringtheories?Ifitdoesthesethings,thenwemaysaythatatheoryaidsinthecumulationofknowledgeaboutasubject.Itiscommensurable.Ifitdoesnot,ifitsitsbyitselfinacorneranddoesnotrelateproductivelytoothertheories.then,itislikelytobedismissedas“adhoc,”or“idiosyncratic.”Itdoesnotfitwithpresentunder-standingsoftheworld.Ithaslittletheoreticalorconceptualutility.Ofcourse,devianttheoriesandneologisms(novelconcepts)maybeextremelyusefulinthelongrun.Indeed,thefirstsignofbreakdowninabroadtheoryorparadigmistheexistenceoffindingsthatcannoteasilybemadesenseof.Yetuntilsuchtimeasanewtheoryorparadigmcanbeconstructed(onethatwouldgatherthenewfindingstogetherwiththeoldinasingleover-archingframe-work),thewaywardpropositionisadhoc,idiosyncratic,andapttobeignored.Commensurabilityrevisitsthedemandforparsimonyatabroaderscale.Ratherthanreferringtothequalitiesofindividualtheorieswearenowcon-cernedwithparsimonyinafield,adiscipline,acrossthesocialsciences,andperhapsacrossthesciencesatlarge.ErnstMachsawthefundamentalprojectofscienceintheefforttoproduce“thecompletestpossiblepresentmentoffactswiththeleastpossibleexpenditureofthought.”11Einstein,severaldecadeslater,endorsed“theefforttoreduceallconceptsandcorrelationstoasfewaspossible11Mach([1902]1953:450–451).68PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:27 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.005Cambridge Books Online © Cambridge University Press, 2012
logicallyindependentbasicconceptsandaxioms.”12Morerecently,EdwardO.Wilsonhasarguedthat“thereisonlyoneclassofexplanation.Ittraversesthescalesofspace,time,andcomplexitytounitethedisparatefactsofthedis-ciplinesbyconsilience,theperceptionofaseamlesswebofcauseandeffect.”13Argumentsfortheunityofsciencearemanyandvarious.14Granted,wearelikelytoexperienceconsiderablylesssuccessinthisendeavorinthesocialsciencesthanMach,Einstein,andWilsonenvisionedforthenaturalsciences.Commensurabilityisamatterofdegrees.Butthisshouldnotblindustotheneedforlogicaleconomy,andtheutilityofsucheconomyaswealreadyenjoy.Weareaccustomed,forexample,tocategorizingworksintovarioustraditions–Durkheimian,Weberian,Marxist,Freudian,rationalchoice,beha-vioralist–andintosmallernichesdefinedbyparticularsubfields.Thissortofgroupingmakestheacademicenterprisemanageable,totheextentthatitismanageableatall.(Imagineifwehadnosuchpigeon-holes.)Perhaps,overtime,weshalldobetter.Thisistheambitionofrationalchoiceandotherbroadtheoreticalframeworks.RelevanceSocialscienceisaspeciesofpracticalknowledge.“Anyproblemofscientificinquirythatdoesnotgrowoutofactual(or‘practical’)socialconditionsisfactitious,”asDeweywrites:Allthetechniquesofobservationemployedintheadvancedsciencesmaybecon-formedto,includingtheuseofthebeststatisticalmethodstocalculateprobableerrors,etc.,andyetthematerialascertainedbescientifically“dead,”i.e.,irrelevanttoagenuineissue,sothatconcernwithitishardlymorethanaformofintellectualbusywork.15Ifsocialscientistscannottellussomethingusefulabouttheworldthenthey(we)areservingverylittlepurposeatall(apointexploredfurtherinChapter14).Onecriterionofsocialutility–onemightevenregarditasanecessarycondition–isrelevance.1612Einstein([1940]1953:253).13Wilson(1998:291).14SeeHitchcock(2003);Homans(1967);King,Keohane,andVerba(1994:15–17);Kitcher(1981);Mill([1843]1872:143–144);Neurath,Carnap,andMorris(1971);PutnamandOppenheim(1958).WhatIrefertoascommensurabilityisalsosimilarto“coherence”approachestotruth,asthattermisemployedinepistemologyandphilosophyofscience(Kirkham1992;Laudan1996:79).15Dewey(1938:499).16Adcock(2009);Bloch([1941]1953);Bok(1982);Haanetal.(1983);LernerandLasswell(1951);LindblomandCohen(1979);McCallandWeber(1984);Mills(1959);Myrdal(1970:258);Popper([1936]1957:56);Rule(1997);Shapiro(2005);Simon(1982);Smith(2003);Wilensky(1997);Zald(1990).69ArgumentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:27 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.005Cambridge Books Online © Cambridge University Press, 2012
Byrelevance,Imeansignificancetothelaycitizensoftheworld.Unfortunately,inacademicworkonefindsthatwriterssometimesconfusethenotionofstatisticalsignificancewithreal-lifesignificance.Inawide-rangingreviewofeconomicsstudies,McCloskeyandZiliakrefertothisasthe“standarderrorofregressions.”17ThisiswhyIshallbelaborwhatmightseemtobeanobviouspoint.Therelevancecriteriondoesnotimplyasocialsciencecomposedofzealousadvocacy,wherewritersembraceparticularpoliciesordrawmoral/ethicalconclusionsabouthistoricalactorsandactions:wherethepastbecomes,inMichaelOakeshott’saptphrase,“afieldinwhichweexerciseourmoralandpoliticalopinions,likewhippetsinameadowonSundayafternoon.”18Bythesametoken,itseemsfruitlesstoinsistthatsocialscienceshouldentirelyeschewopinionizing,for“normative”concernsareoftendifficulttoavoid.ImaginewritingabouttheHolocaustorslaveryinawhollydispas-sionatemanner.Whatwouldaneven-handedtreatmentofthesesubjectslooklike?Everydaylanguageisnotmorallyneutral,andsocialsciencemustacceptthisaffectivelychargedvocabularyasaconditionofdoingbusiness.19Leavingasidesuchextremeexamples,itisdifficulttoconceiveofimportantstatementsabouthumanactionsandhumaninstitutionsthatdonotcarrysomenorma-tivefreight.Attheveryleast,one’schoiceofsubjectislikelytobeguidedbysomesenseofwhatisrightandwrong.“Intheory,”writesE.H.Carr,thedistinctionmay…bedrawnbetweentheroleoftheinvestigatorwhoestablishesthefactsandtheroleofthepractitionerwhoconsiderstherightcourseofaction.Inpractice,oneroleshadesimperceptiblyintotheother.Purposeandanalysisbecomepartandparcelofasingleprocess.20Icannotfathomwhyanyonewouldchoosetoinvestyears(typicallydecades)researchingasubjectifitdidnothavesomenormativeimportancetohimorher.Arguably,truth-claimsareenhancedwhenawriterfranklyproclaimshisorherpreferencesattheoutsetofthework.Thisway,possibleinaccuraciesinevidenceorpresentationareeasiertodetectandtoevaluate.Hiddenprejudicesprobablydomoreharmthanthosethatareopenlyavowed.Yetitmustbestressedagainthatthevalueofaworkofsocialsciencederivesfromitsvalue-added,notitsnormativepointofview.Tosay,“Yisgood”or“WeshoulddoY”istosayextraordinarilylittle.Fewarelikelytobepersuadedbysuchastatement,17McCloskeyandZiliak(1996);ZiliakandMcCloskey(2008).18QuotedinFischer(1970:78).19Collier(1998);Freeden(1996);Gallie(1956);HollisandLukes(1982);MacIntyre(1971);Pitkin(1972);Searle(1969);Strauss([1953]1963);Taylor([1967]1994).20Carr([1939]1964:4).70PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:27 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.005Cambridge Books Online © Cambridge University Press, 2012
unlessitissimplybyvirtueoftheauthorityofthewriter.Andwhatauthoritydomembersofthesocialsciencecastepossess,asidefromtheauthorityofsocialscience?Typically,socialscienceismostpowerfulwhenthenormativeangleofaworkishandleddelicately.Themostcompellingargumentsforsocialwelfarearethosethatdemonstratecausalrelationships,forexample,thatparticularprogramsaidinalleviatingconditionsofpovertyanddonothavenegativeexternalities.Suchstudiesdonotproclaimbaldly“Povertyisbad,”or“Weshouldincreasesocialwelfarespending,”althoughthereisnoquestionthattheseviewsundergirdmostresearchonpovertyandsocialpolicy.Aslongastheauthor’sresearchissound,oneisunconcernedwithhisorhernormativepositiononthematter.Toputitotherwise:thepersuasivenessofanynormativeargumentisitselfdependentonthepersuasivenessofwhateverdescriptiveandcausalproposi-tionscomprisethatargument.Descriptiveandcausalpropositionsserveasthemeatofanyprescriptivestatement.Similarly,whetherornottheresearcherismotivatedbysomevisionofabettersociety,oronlybypersonalormaterialinterests,isrightlyimmaterialtoourjudgmentofthequalityofhisorherwork.Thereareidiotsandgeniusesofeverypersuasion.Onewouldprefertoreadthegeniusesandleavetheidiotsalone,leavingasidetheirpersonalviewsandethicalcodes.Finally,itseemsappropriatetoobservethatthevastmajorityofsocialscienceanalysishaslittletodowithwhatisgoodorbad.Noone–orvirtuallynoone–arguesagainstthevirtuesofpeace,prosperity,democracy,andself-fulfillment.Whatisrelevant(inthelargersenseoftheword)isanyknowledgethatmighthelpustoachievethesedesiderata.21Hereiswheresocialsciencematters,oroughttomatter.Idonotwishtogivetheimpressionthatsocialscienceshouldbesolelypreoccupiedwithpolicyrelevance.Onewouldbehard-pressed,forexample,touncoverasinglepolicyprescriptioninworkbyDavidBrionDavis,EdmundMorgan,andOrlandoPattersonontheinstitutionofhumanslavery.22Yet,arguably,nooneignorantofthesewriters’workcanfullycomprehendanycontemporarysocialpolicydebateintheUnitedStates.Similarly,althoughworkontheAmericanRevolution,theConstitution,theCivilWar,andvariousotherhistoricaltopicsisundoubtedlyimportantforunderstandingwherewearetoday,itwouldbedifficulttoderivepolicyimplicationsfromeachoftheseevents.Thesamecouldbesaidformanysubjectsofstudyinthevariousfieldsof21Friedman([1953]1984).22Davis(1988);Morgan(1975);Patterson(1982).71ArgumentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:27 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.005Cambridge Books Online © Cambridge University Press, 2012
thesocialsciences.Thepoint,then,isnotthateverystudyshouldhaveapolicylesson,butthateverystudyshouldreflectuponsomethingthatcitizensandpolicymakerscareabout,ormightcareabout.Thetelosofrelevancethusembracesworkinhistory,anthropology,andotherinterpretivefieldswhoseimpactonpublicaffairsisboundtobemorediffuse.Indeed,oneofthestrongestargumentsagainstanaturalistmodelforthesocialsciencesisthatsuchamodelmightpreventusfromwritingaboutthingsthatmatter.Toopreoccupiedwithitsstatusasascience,BarringtonMoorethought,socialscienceoverlooksmoreimportantandpressingtasks.Themainstructuralfeaturesofwhatsocietycanbelikeinthenextgenerationarealreadygivenbytrendsatworknow.Humanity’sfreedomofmaneuverlieswithintheframeworkcreatedbyitshistory.Socialscientistsandalliedscholarscouldhelptowidentheareaofchoicebyanalyzingthehistoricaltrendsthatnowlimitit.Theycouldshow,impartially,honestly,andfreefromthespecialpleadingsofgovernmentsandvestedinterests,therangeofpossiblealternativesandthepotentialitiesforeffectiveaction.Suchhasbeen,afterall,theaimofinquiryintohumanaffairsinfreesocietiessincetheGreeks.23Nomatterhowvirtuousatheorymaybeonothercriteria,ifitcannotpassthesowhat?testitisnotworthverymuch.Inferences,largeorsmall,havevariouslevelsofrelevance.Therearesomethingsthat,howevermuchwemaysympathizewiththeauthor,wecannotbebotheredtoargueabout.Perceivedrelevancethusplaysavitalroleinidentifyingsocialscienceproblemsthatareworthyofstudy.Incausalanalysis,relevancealsoplaysaroleinidentifyingfactorsthatareworthyofanalysis.Considertheclassicquestionofwar,aselucidatedbyPatrickGardiner:Whenthecausesofwararebeinginvestigated,itmaybedecidedthatbotheconomicfactorsandhumanpsychologyarerelevanttoitsoutbreak;yetsincewedeemittobewithinourpowertoinfluenceoraltertheeconomicsystemofasociety,whereasthecontrolofhumanpsychologyseems,atleastatpresent,tobebeyondourcapacity,wearelikelytoregardtheeconomicratherthanthepsychologicalfactorsasthe“cause”ofwar.24Similarly,indiscussionsofsocialpolicy,causalargumentsthatrestupondeep-seatedpolitical-culturalfactorsareinsomerespectslessinterestingthanargu-mentsrestingonpolicydesign.Thelattercanberedesigned,whiletheformerarepresumablyoflongduration,andhencelessrelevanttocontemporary23Moore(1958:159).24Gardiner([1952]1961:12).72PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:27 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.005Cambridge Books Online © Cambridge University Press, 2012
policydiscussions(exceptasboundaryconditions).Relevantcausestendtobemanipulable.25Bywayofconclusion,itseemsfairtojudgethetheories(orcausalfactors)thatpossessastrongclaimtorelevanceassuperior(ceterisparibus)tothosethatdonot.Anditseemsfairtoaskwriterstojustifythereader’spotentialexpenditureoftime,effort,andmoneywithsomesortofpay-off.Thisistraditionallyhandledintheprefaceofabookorarticle,wheretheauthortriestofindahook(apointofgeneralinterest)onwhichtohanghisorherargument,orintheconclusion,wheretheauthorreflectsupontheramifica-tionsofastudy.Readersarenotlikelytobecarriedveryfaronthestrengthofawriter’smethodorproseiftheydonotfeelthatthereissomethingimportantatstakeintheinvestigation.Theymustcareabouttheoutcome.25ThisfollowsCollingwood’s(1940)analysis.Wegenerallyidentifyacausalfactorthatstates“itisinourpowertoproduceorprevent,andbyproducingorpreventingwhichwecanproduceorpreventthatwhosecauseitissaidtobe”(citedinGarfinkel1981:138).SeealsoGasking(1955);HarreandMadden(1975);Suppes(1970);vonWright(1971);Whitbeck(1977)–allcitedinCookandCampbell(1979:25).Notethatmanipulabilityalsoenhancesthetestabilityofacausalargument,asdiscussedinChapter9.73ArgumentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:27 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.005Cambridge Books Online © Cambridge University Press, 2012
Cambridge Books Onlinehttp://ebooks.cambridge.org/Social Science MethodologyA Unified FrameworkJohn GerringBook DOI:http://dx.doi.org/10.1017/CBO9781139022224Online ISBN: 9781139022224Hardback ISBN: 9780521115049Paperback ISBN: 9780521132770Chapter4 – Analyses pp. 74-104Chapter DOI:http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge University Press
4AnalysesButisittrue?AaronWildavsky1Havingdiscussedtheformal(super-empirical)criteriaofagoodargument,weturnnowtotheempiricalportionofsocialscienceresearch,thehoped-forencounterwithreality.2Thisstagemaybereferredtovariouslyasanalysis,assessment,corroboration,demonstration,empirics,evaluation,methods,proof,ortesting.(Whileacknowledgingthesubtledifferencesamongtheseterms,Ishalltreatthemaspartofthesameoverallenterprise.)Ofcourse,thedistinctionbetweentheoryformationandtheory-testingisneverclearandbright.Asisthecaseeverywhereinsocialscience,tasksinter-mingle.Onecannotformanargumentwithoutconsideringtheempiricalproblemofhowtoappraiseit,andviceversa.Moreover,thetaskof(dis)-confirmingtheoriesisintimatelyconjoinedwiththetaskofformingtheories.AsPaulSamuelsonnotes,“Ittakesatheorytokillatheory.”3Yetincomingtogripswiththecomplexprocessofsocialscienceitisessentialtodistinguishbetweentheformalpropertiesofanargumentandthemethodsbywhichthatargumentmightbeassessed.Whatareyouarguing?andIsittrue?arelogicallydistinctquestions,callingforthdifferentcriteriaofadequacy.4Moreover,therearegoodmethodologicalreasonstorespecttheseparationbetweentheoryandanalysis(see“Partition”below).Wenowproceedfromtheformertothelatter.Ofcourse,notallhypothesesrequireexplicitattentiontomethodsofapprai-sal.Manyhypothesesneednotbeformallytestedatall,fortheyarealreadyself-evident(e.g.,“civilwarisdislocating”),orareinsufficientlyimportanttojustifytheinvestmentoftimeandenergythataformalanalysiswouldrequire(e.g.,“lifeguardtrainingprogramshavepositiveeffectsontheprobabilityof1Wildavsky(1995).2Scientificrealistsrecognizeananalogousdistinctionbetweenthesuper-empiricalandempiricalelementsofatheory(Hitchcock2003:217).3QuotedinRosenbaum(2010:95).4Bhaskar([1975]1978:171);Bunge(1963:45);Hoover(2001:22).74Downloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
marriageandchild-bearingamongprogramparticipants”).Ourmotivationhereiscenteredonargumentsthatareimportantenoughtosubmittoaformaltestingprocedureandcomplexenough,intermsofpotentialthreatstovalidity,toworryaboutthenicetiesofresearchdesign.Methodologykicksinwherecommonsensefallsshort.DefinitionsAstandardempiricalanalysisinvolvesanumberofcomponents,whichmustbeclarifiedbeforewecontinue.Muchofthisvocabularyisborrowedfromsurveyresearch;nonetheless,theconceptsarehelpfulinallstylesofresearch,quantitativeorqualitative.Apopulationistheuniverseofphenomenathatahypothesisseekstodescribeorexplain.Itremainsunstudied,orisstudiedonlyinaveryinformalmanner,forexample,throughthesecondaryliterature.Sometimes,itisimpor-tanttodistinguishbetweenapopulationfromwhichasampleisdrawn(andwhichitpresumablyrepresents)andalarger,morehypotheticalpopulationthatthesamplemayormaynotrepresent,butwhichnonethelessdefinesthescope-conditionsoftheargument.Thesamplereferstotheevidencethatwillbesubjectedtodirectexamina-tion.Itiscomposedofunitsorcases:boundedentitiessuchasindividuals(subjects),organizations,communities,ornation-states,whichmaybeobservedspatiallyand/ortemporally(throughtime).(Thetermsunitandcasearemoreorlessequivalent.Theonlydifferenceisthatwhileaunitisboundedspatially,acasemayalsohaveimplicitorexplicittemporalboundaries.5)Typically,thesampleissmallerthanthepopulation;hence,thenotionofsamplingfromapopulation.(Note,however,thatmyuseofthetermsampledoesnotnecessarilymeanthatcasesunderstudy–thesample–havebeenrandomlychosenfromaknownpopulation.)Occasionally,oneisabletoincludetheentirepopulationinasample–acensus.Theobservationstakenfromunitsatparticularpoints(orperiods)intimecomposethepiecesofevidencepresumedtoberelevanttoadescriptiveorcausalproposition.Collectively,theobservationsinastudycompriseastudy’ssample.Eachobservationshouldrecordvaluesforallrelevantvariablesacrosseachunitataparticularpoint(orperiod)intime.Incausalanalysis,this5ForfurtherdiscussionseeGerring(2007).75AnalysesDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
includesX(thecausalfactoroftheoreticalinterest)andY(theoutcomeofinterest),alongwithanyothervariablesdeemedessentialfortheanalysis.Inmatrixformat,anobservationisusuallyrepresentedasarowandthetotalnumberofobservations(rows)inasampleas“N.”Confusingly,Nalsosometimesreferstothenumberofunitsorcases,whichmaybequitedifferentfromthenumberofobservations.Varyingusagesareusuallyclearfromthecontext.Afinalconcept,thedatacell,isusefulwhenonewishestorefertothedatapertainingtoaparticularunitatonepointintimealongonlyonedimension.Althoughthetermisnotcommonlyemployed,itissometimesessential.Considerthatanobservationconsistsofatleasttwocellsinanycausalanalysis:thecellrepresentingthevalueforXandthecellrepresentingthevalueforY.Sometimes,oneneedstodistinguishbetweenthem.TheseinterrelatedconceptsareillustratedinFigure4.1,wherewecanseeafairlytypicaltime-seriescross-sectionresearchdesigninarectangulardataset(matrix)format.Here,observationsarerepresentedasrows,variablesascolumns,andcellsastheirintersection.Notethatcellsarenestedwithinobservations,observationsarenestedwithinunits(akacases),unitsarenestedwithinthesample,andthesampleisnestedwithinthepopulation.Hypothetically,letusimaginethatthepopulationoftheinferenceincludesallUSschoolsandthesampleconsistsofeightschools,observedannuallyforfiveyears,yieldingasampleoffortyobservations(N=40).Theunitsofanalysis(thetypeofphenomenatreatedasobservationsinananalysis)inthishypo-theticalexampleareschool-years.Iftheresearchdesignhadbeenpurelycross-sectional,onlyoneobservationwouldbetakenfromeachunit,andtheunitsofanalysiswouldconsistofschoolsratherthanschool-years,andthetotalnumberofobservationswouldbeeight(N=8).Inthiscontext,thenumberofunitsisequaltothenumberofobservationsandthedistinctionbetweenunitandobservationislost.Iftheresearchdesignispurelytemporalthesamplewouldbecomposedofoneunit,observedthroughtime.Ifthesampleperiodisfiveyearsandobserva-tionsaretakenannually,thetotalnumberofobservationsisfive(N=5).Here,theunitsofanalysisareagainschool-years,asinthefirstexample.Allthesetermsareslipperyinsofarastheydependfortheirmeaningonaparticularpropositionandacorrespondingresearchdesign.Anychangesinthatpropositionmayaffectthesortofphenomenathatareclassifiedasobservationsandunits,nottomentionthecompositionofthesampleandthepopulation.Thus,aninvestigationofschoolvouchersmightbeginbyidentifyingschoolsastheprincipalunitofanalysis,butthenshifttoalowerlevelofanalysis76PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
(e.g.,students),orahigherlevelofanalysis(e.g.,schooldistricts)atdifferentpointsinthestudy.Sometimes,differentlevelsofanalysis(e.g.,students,schools,andschooldistricts)arecombined.Thisiscommonincasestudyworkandisthedefiningfeatureofhierarchical(multi-level)statisticalmodels.X1X2Y Obs 1.1 (T1)Obs 1.2 (T2)Obs 1.3 (T3)Obs 1.4 (T4)Obs 1.5 (T5)Obs 2.1 (T1)Obs 2.2 (T2)Obs 2.3 (T3)Obs 2.4 (T4)Obs 2.5 (T5)Obs 3.1 (T1)Obs 3.2 (T2)Obs 3.3 (T3)Obs 3.4 (T4)Obs 3.5 (T5)Obs 4.1 (T1)Obs 4.2 (T2)Obs 4.3 (T3)Obs 4.4 (T4)Obs 4.5 (T5)Obs 5.1 (T1)Obs 5.2 (T2)Obs 5.3 (T3)Obs 5.4 (T4)Obs 5.5 (T5)Obs 6.1 (T1)Obs 6.2 (T2)Obs 6.3 (T3)Obs 6.4 (T4)Obs 6.5 (T5)Obs 7.1 (T1)Obs 7.2 (T2)Obs 7.3 (T3)Obs 7.4 (T4)Obs 7.5 (T5)Obs 8.1 (T1)Obs 8.2 (T2)Obs 8.3 (T3)Obs 8.4 (T4)Obs 8.5 (T5)PopulationSampleCase 8Population = indeterminate; Cases/units = 8; Sample/observations = 40;Cells = 120; Time (T) = 1–5; Variables = 3.Case 7Case 6Case 5Case 4Case 3Case 2Case 1Figure4.1Time-seriescross-sectiondataset77AnalysesDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
Complicatingmattersfurther,thepreciseboundariesofaresearchdesignoftenremainambiguous.Thisisbecauseasubjectisusuallyinterrogatedinavarietyofwaysduringthecourseofastudy.Forexample,keyvariablesmaychange(perhapstocaptureadifferentdimensionoranalternativeoperationa-lizationofacomplexconcept),theunitsofanalysismaychange(movingupordowninlevelsofanalysis),thefocusmaychange(fromthemainhypothesistoadjuncthypothesesorcausalmechanisms),thesamplemaychange,anddiffer-entkindsofobservationsmaybeenlisted.Thesearejustafewofthevariationsinmethodthattypicallyco-habitinasinglestudy.Eachofthesealterationsmaybeconsideredasdistinctresearchdesignsorasvariationsonasingleresearchdesign.Likewise,theymaybedescribedasreplications,robustnesstests,ormultimethodresearch(asdiscussedinlaterchapters).Thus,itbecomesratherdifficulttosaywhatagivenstudy’sresearchdesignis,orhowmanythereare,withoutmakingsomeratherarbitrarydecisionsaboutwhatliesin,andoutof,thescopeofthisambientconcept.IshallleavethismatteropenbecauseIdonotthinkitcanbeeasilysettled.Perhapsitisnotessential.Theprovisoisthatwritersmustbeclearaboutwhattheymeanby“researchdesign”inagivencontext.ResearchdesignversusdataanalysisTraditionally,onedistinguishesbetweentwostagesofthetestingprocess.Researchdesignreferstotheselectionandarrangementofevidence.6Dataanalysisreferstotheanalysisofdataonceitiscollected.Inanexperiment,thesestagesareclearlyseparable:researchdesignpre-cedesdataanalysis.Oneisexante,theotherexpost.(Ofcourse,insuccessivecyclesofresearchthislinebecomesblurred.)Inobservationalresearch,thetwostagesareusuallyintermixed.Becausemuchofthisbookisfocusedonobservationaltechniques,thereadershouldbepreparedforsomeslippageacrossthesetwoconcepts.Still,thedistinctionisconsequential.Anoldertraditionofsocialsciencemethodologyfocusesonreachinginferencesaboutaphenomenonbasedonwhateverdataisathand.Themethodologist’sjobbeginsoncetheevidenceisin.Thisisthe“dataanalysis”approachtomethodologythatunderliesmosteconometricstexts.Textbooksinthisgenreincludediscussionsofstatisticalinferenceandofvariousclassesofestimators6Anexperimentallybasedunderstandingofdesignrefersto“allcontemplating,collecting,organizing,andanalyzingofdatathattakesplacepriortoseeinganyoutcomedata”(Rubin2008:810).Thisseemstonarrowforpresentpurposes,sinceinobservationalresearchtheselectionofaresearchsiteoftendependsonaninitialconsiderationof“outcome”data.Myunderstandingofdesignencompassesallfactorsthatmight(legitimately)impactthechoiceofobservationstobestudied.78PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
employedfordescriptiveandcausalinference(e.g.,correlation,differenceofmeans,regression,matching,randomizationinference,Bayesianversusfrequen-tistapproaches),alongwiththeassumptionseachmethodinvokes.7Usefulthoughsuchtechniquesare,itisimportanttorememberthatthecontributionofadvancedstatisticalprotocolsisfocusedlargelyonshortcom-ingsofdesign.Econometricsisthedeusexmachinahauledontothestagetorectifyproblemsofmeasurementerror,ambiguouscausalfactors,insufficientvariationalongkeyparameters,insufficientobservations,incomparabilitiesacrosscomparisoncases,biasedsamples,andotherissuesthatwewillshortlydiscuss.Fromthisperspective,itseemsappropriatetoconcludethatmattersofdesignareprimary,andmattersofdataanalysissecondary–bothsequen-tiallyandmethodologically.“Designtrumpsanalysis,”inthewordsofDonaldRubin.8Andfromthisperspectiveitfollowsthatthemethodologist’sjobbeginsatthefront-end–theresearchdesignphaseofaproject.Indeed,thereisoftennotmuchonecandotorectifyproblemsofdesignoncethedataisin.Forthosewhoarefondofmedicalanalogies,theresearchdesignapproachtomethodologymightbecomparedwiththepreventiveapproachtomedicine,thatis,howtoavoidcontractingillness,whilethedataanalysisapproachtomethodologyisakintoemergencycare,thatis,howtorestoreapatientwhoisalreadyfailing.Sometimes,ingeniousexpoststatisticaladjustmentsaresuccessful.Yetthereisincreasingskepticismaboutourcapacitytocorrectresearchdesignflawsatthepost-researchphase.Theoldadage,“garbagein,garbageout,”isstilltrue,despitemanyadvancesinthefieldofstatistics.RichardBerkcomments:Onecannotrepairaweakresearchdesignwithastrongdataanalysis.Almostinevitablywhatseemstoogoodtobetrueis,andoneissimplysubstitutinguntestableassump-tionsfortheinformationonedoesnothave.9Indeed,themostworryingpointofallisweusuallycannottellwhetherstatisticalcorrectionshaveachievedtheirintendedpurpose,forexample,whetheratwo-stageapproachtomodelingselectionbiashasactuallyprovidedacorrectandunbiasedestimateofX’seffectonY.AsBerkpointsout,thisisbecausetheassumptionsrequiredtoconductstatisticalprotocolsareoftennotdirectlytestable;theyhingeonapriori(“ontological”)assumptionsaboutthenatureofthedata-generatingprocess.Reviewingthefieldofregression-basedcausal7Forexample,Greene(2002).8Rubin(2008).SeealsoAngristandPischke(2010);BowersandPanagopoulos(2009);King,Keohane,andVerba(1995);Rosenbaum(1999,2010);Sekhon(2009);ShadishandCook(1999:294).9Berk(1991:316).79AnalysesDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
inference,DavidFreedmanstatesbaldly,“Iseenocasesinwhichregressionequations,letalonethemorecomplexmethods,havesucceededasenginesfordiscoveringcausalrelationships.”10Whilethisconclusionseemsatadextreme,oneisrightlycautionedtoregardstatisticallybasedcausalinferenceswithskepticism.Always,theyrestonassumptionsaboutthedata-generationprocess,thatis,onmattersofresearchdesign.Thus,althoughIdonotwishtodownplaytheimportanceofdataanalysis,Idowishtostakeaclaimfortheprimacyofdesign–especiallyincausalanalysisbutalsoindescriptiveanalysis.Thedesigncomponentsofresearcharegeneralinpurview;anyattempttodisentangleempiricalrelationshipsmustwrestlewiththem.Moreover,thisperspectiveonmethodologyisofteninsightful.Itclarifiestheobstaclesfacingthesocialsciencesandelucidatesarangeofpossiblesolutions.Finally,thedesignaspectsofsocialscienceresearchareunder-appreciated.Indeed,theonlyregionsofsocialsciencewhereissuesofdesignaregrantedprimacyarethosewhereexperimentalmethodsareemployed.Inlightofthis,itseemsarguablethatthewayforwardforsocialscienceistobefoundinwell-craftedresearchdesignsratherthaninthedevelopmentofnewestimators.BorrowingfromPaulRosenbaum,ourmottowillbe“choiceasanalternativeto[statistical]control.”11Accordingly,thefollowingchaptersincludelittlediscussionofstatisticsexceptasthelatterbearuponmattersofresearchdesign.Thismeansthatstatisticalmethodscloselyassociatedwithspecificresearchdesigns,suchasregressiondiscontinuityandinstrumentalvariables,willbediscussed(Chapter10),butnotstatisticalmethodsthataregeneralinemployment,suchasregressionormatching.CriteriaWiththesetermsandperspectivesclarified,wecannowproceedtothemainbusinessathand.Whatisitthatqualifiesaresearchdesign(and10Freedman(1997:114;emphasisadded).Ontheproblemsofstatisticalinferencebasedonobservationaldata,andthecorrespondingimportanceofresearchdesign,seeBerk(2004);BradyandCollier(2004);CloggandHaritou(1997);Freedman(1991,2008,2010);Gerber,Green,andKaplan(2004);Gigerenzer(2004);Heckman(2008:3);Kittel(2006);Longford(2005);Pearl(2009b:40,332);RobinsandWasserman(1999);Rodrik(2005);Rosenbaum(1999,2005);Seawright(2010);Summers(1991).Variousstudiescomparinganalysesofthesamephenomenonwithexperimentalandnonexperimentaldatashowsignificantdisparitiesinresults,offeringdirectevidencethatobservationalresearchisflawed(e.g.,BensonandHartz2000;FriedlanderandRobins1995;Glazerman,Levy,andMyers2003;LaLonde1986).Cook,Shaddish,andWong(2008)offeramoreoptimisticappraisal.11Rosenbaum(1999).80PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
correspondingdataanalysis)assatisfactory?Whatisagoodempiricalanalysis?Iwillarguethatcriteriaapplicabletosocialscienceanalysesmaybefruit-fullydividedintofourfundamentalareas:accuracy(validity,precision,anduncertainty);sampling(representativeness,samplesize,levelofanalysis);cumulation(standardization,replication,transparency);andtheoreticalfit(partition,constructvalidity,difficulty).Thesecriteria,summarizedinTable4.1,areregardedasgeneric,whichistosaytheyapplytoallapproaches.Nomethod–whetherdescriptiveorcausal,qualitativeorquantitative,experimentalorobservational–isexempt.Tobesure,eachstudyisapttoprioritizecertaincriteriaoverothers.Andoccasion-ally,criteriamaybelegitimatelyignorediftheyhavebeeneffectivelyestab-lishedbyotherstudies.Inthisrespect,itisdifficulttoevaluateagivenworkinisolationfromthefieldofstudiesinwhichitissituated.Butthelargerandmoreimportantclaimremains:thecriterialistedinTable4.1arebroadlyapplicablewhereverempiricalquestionsofsocialscienceareinplay.AccuracyTheoverallobjectiveofempiricalresearchistoaccuratelytestanargument.Accuracymaybeunderstoodashavingtwodimensions:validityandpreci-sion,eachwithanassociatedlevelofuncertainty.Table4.1Analysis:generalcriteria1.AccuracyAretheresults(a)valid,(b)precise(reliable),and(c)accompaniedbyanestimateofuncertainty(confidence,probability)withrespectto(d)thechosensample(internalvalidity)and(e)thepopulationofinterest(externalvalidity,akageneralizability)?2.SamplingArethechosenobservations(a)representativeoftheintendedpopulation,(b)sufficientlylargeinnumber,and(c)attheprincipallevelofanalysis?3.Cumulation(a)Istheresearchdesignstandardizedwithothersimilarresearchonthetopic?(b)Doesitreplicateextantfindingsandfacilitatefuturereplicationsbyotherscholars?(c)Areprocedurestransparent?4.Theoreticalfit(a)Doestheresearchdesignprovideanappropriatetestfortheinference(constructvalidity)?(b)Isthetesteasyorhard(severity)?(c)Isthetestsegregatedfromtheargumentunderinvestigation(partition)?81AnalysesDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
Thesenotionsaretypicallyappliedtotheestimatethatresultsfromanempiricalanalysis(i.e.,tothefinding).However,theymayalsobeappliedtotheresearchdesignandtechniqueofdataanalysisbywhichthatestimateisobtained.Indeed,thevariousphasesofresearchareallsubjecttodemandsforvalidityandprecision,andeachisassociatedwithalevelofuncertainty.Thus,whenspeakingofthesegoalsweshallspeakofthemapplyingacrossvarioustasksassociatedwiththegeneraltaskoftheoryappraisal.Othercriteria,discussedinsucceedingsectionsofthischapterandinsubsequentchapters,usuallyaiminonewayoranothertobolstertheaccuracyofananalysis,andinthisrespectmaybeviewedasancillarytothefunda-mentalgoalsofvalidityandprecision.Finally,adistinctionwillbeintroducedbetweenthechosensampleandalargerpopulationoftheoreticalinterest.Theformerisunderstoodasanissueofinternalvalidityandthelatterasanissueofexternalvalidity.Validity,precision,uncertaintyScholarsoftendistinguishbetweenthevalidityofatestanditsprecision(relia-bility).Ifaninferenceweretobetestedrepeatedly,theclosenessoftheseresults(onaverage)tothetruevaluewouldcapturethevalidityofthetest.Theclosenessofthesetestresultstoeachotherwouldcapturetheprecisionofthetest.Thiscontrastisbestilluminatedbyillustration.Letusrepresenttheobjectofinterest(initstrue,ontologicalreality)byadarkcircle,andvariousattemptstomeasurethatobjectbypoints.Withthisschema,threetestsarecomparedinFigure4.2.Thefirstisreliablebutnotvalid,asthepointsclustercloselytogetherbutaredistantfromthetruecenter.Thesecondisvalidbutnotreliable,asthepointsaredispersedbutareclusteredaroundthetruecenter.Thethirdisbothreliableandvalid.Reliable but not validValid but not reliableValid and reliableFigure4.2Reliability(precision)andvalidity82PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
Theseconceptsapplyequallytothedescriptivetaskofmeasurement(Chapter7),aswellastothetaskofestimatingcausaleffects(Chapters9,10,and11).Thereisaslightalterationofvocabulary,insofarastheprecisionofameasurementisusuallyreferredtoasaquestionofreliability(ratherthanprecision).Butthebasicideasarethesameacrosscontexts.Notethatprecisionisalsoacriterionofanargument(Chapter3).Here,however,weareconcernedwiththeprecisionofatest,nottheprecisionofthepropositionthatisbeingtested.Now,letusexploretheseissuesingreaterdetail.Aproblemofvaliditymaybeexpressedasaproblemofsystematicerrororbias.Ofcourse,itdependsuponassumptionsaboutthetruereality,whichmaynotbedirectlyapprehensible.Insomecircumstances,itispossibletogaugethevalidityofastatisticalmodelthroughMonteCarlosimulations.12Butusuallyissuesofvalidityareassessedinamorespeculativemanner.Ifthereisrecogniz-ablebias,orpotentialbias,insomeaspectoftheresearchdesignwesaythatthereisaproblemofvalidity–eventhoughwecannotknowforsure.Precision,wehavesaid,referstotheconsistencyofafindingacrossrepeatedtests,andisthusalarge-sampleproperty.Ifiteratedtestsdemonstratethesameresult(moreorless),theprocedureisdeemedtobeprecise.Thevarianceacrosstheseresultsprovidesanempiricalmeasureofthedegreeofprecisiontherebyattained.Ifthereisnoopportunitytocomparemultipleiterationsofasingleresearchdesign(iftheresearchisqualitativeinnature),thenthevarianceremainsatheoreticalproperty–thoughnolessimportantforbeingso.Manyfactorsmayaffecttherelativeprecisionofatest,includingmeasurementerror,thevariabilityofthephenomenaunderstudy,andthesizeofasample.Sinceprecisionisaboutvariance,notvalidity,allsucherrorsareregardedasstochas-tic(random),akanoise.Implicitinthenotionofvalidityistheconceptofuncertainty.Anyassertionabouttheworldisassociatedwithalevelofconfidence,orprobability;forallempiricalknowledgeistosomeextentuncertain.Thereisalwaysaproblemofinference,evenifthedegreeofuncertaintyisjudgedtobequitesmall.Thisuncertaintymaystemfromproblemsofconceptformation(Chapter5),mea-surement(Chapter7),sampling(discussedbelow),and/orvariousissuesasso-ciatedwithcausalinference(PartIII).Itdepends,obviously,ontheargumentinquestion.Ithasbeenallegedthat“perhapsthesinglemostseriousproblemwithqualitativeresearch…isthepervasivefailuretoprovidereasonableestimates12Mooney(1997).83AnalysesDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
oftheuncertaintyoftheinvestigator’sinferences.”13Ihavenodoubtthatthereissometruthtothisassertion,thoughqualitativescholarshaveworkedhardtoresolveit.Bycontrast,quantitativemethodsgenerateestimatesofuncertaintyasaroutineelementoftheanalysis.Certainaspectsofuncertaintycanbecapturedinastatisticsuchasaconfidenceintervalandassociatedpvalue,whichmeasurestheprobabilityofahypothesisrelativetosomenullhypothesis.Here,theconceptsofprecisionanduncertaintyaremergedinasinglestatistic.Tobesure,thesestatisticsarebasedonsamplingvariabilityandthustakenoaccountofotherthreatstoinference.Bayesianapproachesarebroaderinreach,incor-poratingsubjectiveknowledgeaboutasubject.ItisinthisspiritthatIproposeanencompassingapproachtotheestimationofuncertainty,onethatcombinesinformationdrawnfromlarge-samplemethodsofinference(whereversamplesarelargeenoughtopermitthis)withqualitativeknowledgeaboutadditionalthreatstoinference.Estimatingtheuncertaintyofaparticularfindingisnoteasy.Butitisessential.Internal/externalvalidityConventionally,oneanalyzesquestionsofvalidity,precision,anduncertaintyattwolevels.First,thereisthequestionofwhetherafindingistrueforthechosensample–anissueofinternalvalidity.Second,thereisthequestionofhowthisfindingmightbegeneralizedtoabroaderpopulationofcases–anissueofexternalvalidity.Notethatalthoughthisisphrasedintermsofvaliditythesamequestionsarisewithrespecttoprecision;Ishallthereforeassumethatbothareinferredwhenoneuttersthephrase“internalvalidity”or“externalvalidity.”Astudymaybevalidinternallybutnotexternally(beyondthechosensampleorresearchsite).Likewise,theinternalvalidityofastudymaybequestionable,whileitsclaimtoexternalvalidity–iftrueforthesample–isstrong.Ofcourse,theissueofexternalvalidityrestsinsomeimportantsenseonastudy’sinternalvalidity.ThegreaterourconfidenceaboutafindingincontextA(thechosenresearchsite),thegreaterourconfidenceaboutthatfindingincontextB(somewhereinthelargerpopulationofinterest).Bythesametoken,ifoneisnotconfidentaboutaresultwithinastudieddomainoneisevenlessconfidentaboutextendingthatresulttoalargerdomain.13King,Keohane,andVerba(1994:32).84PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
Theinternal/validitydistinctioniscrucialtovirtuallyeverymethodologicaldiscussion,eventhoughthedichotomyisnotalwayscrystalclear.Asanexample,considerahypotheticalstudyofaschooldistrictinthestateofNewYorkthatrestsonasampleofstudentsdrawnfromthatdistrict,butpurportstoelucidatefeaturesofallschoolswithinthestate.Thispresentsthreepotentiallevelsofvalidity:(1)thesampleofstudents;(2)theschooldistrict;and(3)schoolsthroughoutthestate(acrossmultipledistricts).Internalvaliditymayreferto(1)or(2),whileexternalvaliditymayreferto(2)or(3).Inthislight,theissueofinternal/externalvalidityisperhapsmorecorrectlyarticulatedasdegreesofgeneralizability.Justasargumentsaimtogeneralize,sodoresearchdesigns.Somedosomoresuccessfully,andmoreextensively(acrossabroaderpopulation)thanothers.Inthisvein,itissometimeshelpfultorecognizeconcentriccirclessurroundingthesamplethathasbeenstudied.Typically,theconfidencewithwhichoneextrapolatesresultsobtainedfromagivensampledecreasesasthesizeofthecircleexpands.Returningtotheexampleabove,letusconsidersixpossibletiersofvalidity:(1)thesampleofstudents;(2)theschooldistrict;(3)schoolsthroughoutthestate(acrossmultipledistricts);(4)schoolsinotherstates;(5)schoolsinothercountriesintheOECD;and(6)schoolselsewhereintheworld.Eachsucceedingclaimtovalidityseemslesslikely,butnoneiswhollyimplausible.Andfromthisperspectivethereisnocleardemarcationbetweeninternalandexternal.Orperhapsthereisafairlycleardemarcationbetweeninternalandexternal,buttherearemultiplespheresofexternalvalidity.Forheuristicpurposes,subsequentdiscussionwillassumethatthereisonecontextforastudythatisappropriatelylabeled“internal”andanotherthatisappropriatelylabeled“external.”Butreadersshouldbearinmindtheatten-dantcomplexities.Inrareinstances,thedistinctionbetweeninternalandexternalvaliditydisappearsbecausetheentirepopulationofaninferenceisdirectlystudied.Here,thesampleisthepopulation.Evenso,thereisroomforskepticismaboutexhaustivesamplingprocedures(acensus).Sincemostsocialsciencetheoriesarenotlimitedtothepast,thefutureprovidesapotentialsourceforout-of-sampletesting.Thismeansthatevenifallavailableexamplesthatfallintothedomainofasubjectarestudiedonemaystillbetheoreticallymotivatedtounderstandinamuchlarger–asyetunfathomable–population.Conceptually,onemayalsorecognizeadistinctionbetweencasesthatactuallyexistandthosethatcouldhaveexisted(inthepast).Thus,ifIamstudyingtherelationshipbetweeneconomicdevelopmentanddemocracyamongnation-statesinthemoderneraImightconsiderevenacomprehensive85AnalysesDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
sample–includingallnation-statessince1800–tobeasampleofallthenation-statesthatcouldhaveexistedduringthattimeperiod.Fromthisperspective,thereisalwaysalargerpopulationthatcannotbedirectlystudied.Notethatthedistinctionbetweeninternalandexternalvalidityisgroundedinadistinctionbetweenwhathasbeendirectlystudiedandwhathasnotbeendirectlystudied.Thismeansthattheissueofexternalvaliditycannotbetested,bydefinition.Itrestsatthelevelofassumption.(Ofcourse,itmaybetestedbysomefuturestudy.)Thequestionarises,onwhat(speculative)basisdoesonejudgeastudy’sexternalvalidity?Themostobviouscriterionistherepresentativenessofthesample,asdis-cussedbelow.Amoresubtleissue–relevantonlytocausalanalysis–isthescalabilityofthetreatment,asdiscussedinChapter9.SamplingTheselectionofunitsandobservationsforanalysisiscriticaltoanydescrip-tiveorcausalanalysis.Threeobjectivespertainbroadlytothistask:represen-tativeness,size,andlevelofanalysis.Inconstructingasampleoneshouldaimtoberepresentativeofabroaderpopulation,toincludesufficientobservationstoassureprecisionandleverageintheanalysis,andtousecasesthatlieatthesamelevelofanalysisastheprimaryinference.RepresentativenessThemostimportantgroundfordrawingconclusionsabouttheexternalvalidityofapropositionistherepresentativenessofachosensample.Isthesamplesimilartothepopulationwithrespecttothehypothesisthatisbeingtested?If,forexample,thehypothesisiscausal,thenthequestioniswhethertherelationshipofXtoYissimilarinthesampleandinthepopulation.Areweentitledtogeneralizefromagivensampletoalargeruniverseofcases?Inthecaseofvoucherresearch,onemustwonderwhetherthestudents,schools,andschooldistricts(alongwithwhateveradditionalfeaturesoftheresearchsitemayberelevanttotheinference)chosenforanalysisarerepre-sentativeofalargerpopulationofstudents,schools,anddistricts.And,ifso,whatisthatlargerpopulation?DoesitconsistofallstudentsandschoolsacrosstheUnitedStates,oracrosstheworld?Doesitconsistofasmallerpopulationofstudentswhoarewillingtovolunteerforsuchprograms?These86PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
arecriticalquestions.Unfortunately,theyareoftendifficulttoanswerinadefinitivefashionforthereasonsalreadydiscussed.Thebestwaytoobtainarepresentativesampleistosamplerandomlyfromalargerpopulation.Therearemanytechniquesfordoingso(muchdependsuponthecharacterofthatlargerpopulation,themethodsatone’sdisposalforsamplingfromit,andtheinferenceonewishestoestimate).Butthebasicideaisthateachunitorobservationwithinthepopulationshouldhaveanequalchanceofbeingchosenforthesample.Anadvantageofthisapproachisthatonecanestimatesamplingvariability(fromsampletosample),thusprovidingestimatesofprecisiontoaccompanywhateverinferencesonewishestodraw.14Unfortunately,itisnotpossibletoapplymethodsofrandomsamplingtomanyresearchproblems.Voucherstudies,forexample,dependuponthewill-ingnessofschooldistrictstoimplementtheirprotocols–arareoccurrence.Assuch,thesampleofschooldistrictsstudiedbyresearchersisnotlikelytobedrawnrandomlyfromthegeneralpopulation.Evenwhererandomsamplingproceduresarefeasible,theyarenotalwaysmethodologicallydefensible.Ifthesampleunderstudyisverysmall–say,asinglecaseorahandfulofcases–itdoesnotmakesensetodrawrandomlyfromalargepopulation.Whilethechosensamplewillberepresentativeofthepopulationonaverage,anygivensample(ofoneorseveral)isquitelikelytoliefarfromthemean(alongwhateverdimensionsarerelevanttothequestionunderstudy).Consequently,case-studyresearchgenerallyreliesonpurposive(non-probability)case-selectionstrategies,reviewedelsewhere.15Whereverrandomsamplingtechniquesareinapplicable,researchersmuststruggletodefinetherepresentativenessofasample,andhencetheplausiblegeneralizabilityofresultsbasedonthatsample.Thisistrueregardlessofwhetherthesampleisverysmall(i.e.,acase-studyformat)orverylarge.Beforeconcludingthissectionitisimportanttoremindourselvesthatthegoaldrivingtheselectionofasetofcasesisnotsimplytoassurerepresenta-tiveness(and,hence,externalvalidity).Itisalso,andperhapsmoreimpor-tantly,toachieveinternalvalidity.Frequently,thesetwoobjectivesconflict.Forexample,researchersoftenfindthemselvesinsituationswheretheycancraftanexperimentwithanonrandomsampleorconductanonexperimentalstudywitharandomsample.Usually,theyoptfortheformerapproach,signifyingthattheyplacegreaterpriorityoninternalvaliditythanonexternalvalidity.Butinsomesituationsonecanimaginemakingtheoppositechoice.14Weisberg(2005).15SeeGerring(2007:ch.5).87AnalysesDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
Size(N)Moreobservationsarebetterthanfewer;hence,alarger“N”(samplesize)issuperiortoasmallerN,allotherthingsbeingequal.(Nmaybeunderstoodasstandardized“dataset”observationsorasirregular“causal-process”observa-tions,adistinctionintroducedinChapter11.)Thisisfairlycommonsensical.Alloneissaying,ineffect,isthatthemoreevidenceonecanmusterforagivenproposition,thestrongertheinferencewillbe.Indeed,thesamelogicthatcompelsustoprovideempiricalsupportforourbeliefsalsomotivatesustoaccumulatemultipleobservations.Thepluralof“anecdote”is“data,”astheoldsawgoes.Supposeoneistryingtofigureouttheeffectofvouchersonschoolperfor-mance,butonehasavailableinformationforonlyonestudentoroneschool.Underthecircumstances,itwillprobablybedifficulttoreachanyfirmconclu-sionsaboutthecausalinferenceatissue.Ofcourse,oneobservationisalotbetterthannone.Indeed,itisaquantumleap,sincetheabsenceofobservationsmeansthatthereisnoempiricalsupportwhatsoeverforaproposition.Yetempiricalresearchwithonlyoneobservationisalsohighlyindeterminate,andapttobeconsistentwithawidevarietyofcompetinghypotheses.Considerascatter-plotgraphofXandYwithonlyonedatapoint.Throughthispoint,HarryEcksteinobserves,“aninfinitenumberofcurvesorlinescanbedrawn.”16Inotherwords,onecannotknowfromthisinformationalonewhatthetrueslopeoftherelationshipbetweenXandYmightbe,andwhethertherelation-shipisinfactcausal(aslopedifferentfrom0).Themoreobservationsonehas,thelessindeterminacythereis,andthemoreprecision,withrespecttoX’sprobablerelationshiptoY.Notethatwithasmallsample,resultsarenecessarilycontingentuponthe(perhapspeculiar)characteristicsoftheseveralchosenobservations.Conclusionsaboutabroaderpopulationarehazardouswhenoneconsidersthemanyopportunitiesforerrorandthehighlystochasticnatureofmostsocialphenomena.Alargesampleofobservationsalsohelpswithothertasksinvolvedincausalassessment.Itmayassistinformulatingahypothesis–clarifyingapositiveandnegativeoutcome,asetofcaseswhichthepropositionisintendedtoexplain(thepopulation),andoperationaldefinitionsoftheforegoing.Alltheseissuesbecomeapparentintheprocessofcodingobservations,wherevertherearemultipleobservations.Butifthereisonlyoneobservation,ormultipleobserva-tionsdrawnfromasingleunit,thesetasksoftenremainambiguous.The16Eckstein(1975:113).88PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
problemisthatwithanarrowempiricalambittheresearcherisfacedwithanover-abundanceofwaystooperationalizeagivenhypothesis.Schoolperfor-mance–themainoutcomeatissueinourvouchersexample–couldbemeasuredbyanyobservablefeatureinagivenschool.Bycontrast,wheremultipleschoolsarebeingobservedtherangeofpossibleoutcomemeasuresisinevitablynarrowed(byvirtueofthepaucityofdataorcostlinessoftrackingmyriadindicators).Likewise,itwillbenecessarytostipulateinmorecertaintermshow“success”willbedefined–forthecomparisonsacrossschoolsmustbeexplicit.Theprocessofmeasurementacrossmultipleobservationsforcesonetocometotermswithissuesthatmightotherwiseremainlatent,andambiguous.Oneexceptiontothelarge-Ncriterionconcernsanempiricalstudywhosepurposeistodisproveacausalordescriptivelaw(aninvariant,“determinis-tic”proposition).Aslongastheobservedpatterncontradictsthehypothesis,alawmaybedisprovenwithasingleobservation.17Inallothersettings,alargersampleisadvisable–withtheusualceterisparibuscaveat.Thus,ifincreasingthesizeofasampledecreasestherepre-sentativenessofthesampleonemightdecidethatitisnotworththesacrifice:asmaller,morerepresentativesampleissuperior.Ifoneislimitedbytimeorlogisticalconstraintstostudyeitheralargesampleofcross-caseobservationsorasmallersampleofwithin-caseobservationsonemightdecidethatthelatterofferstrongergroundsforcausalinference(foranyofthereasonstobediscussedinPartIII).Inshort,therearemanysituationsinwhichasmallersampleispreferredoveralargerone.However,thereasonsforthispreferencelieinothercriteria.Thatiswhyitisstillcorrecttoviewthesizeofasampleasafundamental(ceterisparibus)criterionofsocialscience.BeforeconcludingthissectionImustbrieflymentiontheproblemofmissingdata,asitintersectsbothsamplerepresentativenessandsamplesize.Usually,whatismeantbymissingdataisthatasamplelacksobservationsforsomeunitsthatshould(bysomeprincipleofselection,randomorotherwise)beincluded.Ifthepatternofmissing-nessissystematic,thenthesamplewillbebiased.If,ontheotherhand,itcanbedeterminedthatthepatternofmissingdataisrandom,thenthesamplewillbesmallerthanitshould,butstillperhapsrepresentative(oratleastasrepresentativeasitwouldhavebeenwithoutthemissingdata).Apotentialsolution,ifpatternsofmissing-nessarefairlypredictable(usingknowndatapoints)andthenumberofmissingdatapoints(relativetothetotalsample)isnottoolarge,istoimputemissingdata.18Inothersituations,itmaybefeasibletogenerateasimpledecisionruleforestablishinga“bestguess”for17Dion(1998).18Allison(2002).89AnalysesDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
missingdatapoints,withoutaformalstatisticalmodel.Inanycase,patternsofmissing-nessmustbereckonedwith.Asampleof1,000withmissingdataisnotthesameasasampleof1,000withnomissingdata.Whenoneconsiderstheproblemofsamplesizeonemustwrestlewiththecompletenessoftheobserva-tionscomprisingthesample.LevelofanalysisObservationsaremosthelpfulinelucidatingrelationshipswhensituatedatthesamelevelofanalysisasthemainhypothesis.19Ifthecentralhypothesisconcernsthebehaviorofschools,thenschoolsshould,ideally,comprisetheprincipalunitofanalysisintheresearchdesign.Ifthehypothesisiscenteredonthebehaviorofindividuals,thenindividualsshouldbetheprincipalunitofanalysis.Andsoforth.Oneoftenfacesdifficultiesifoneattemptstoexplaintheactivityofaparticularkindofunitbyexaminingunitsatahigher,orlower,levelofanalysis.Suppose,forexample,thatoneisinterestedinexplainingthebehaviorofschoolsbuthasdataonlyatthedistrictlevel(anaggregationofschools).Thisisacommonsituation,butnotanenviableone,foronemustinferthebehaviorofschoolsfromthebehaviorofschooldistricts(raisingaproblemofestimationknownasecologicalinference).20If,conversely,onehasdataatalowerlevelofanalysis(forexample,forstudents)thenonefacesasimilarprobleminthereversedirection:onemustinferupward,asitwere,fromstudentstoschools.Thisspeciesofinferenceisalsoproblematic.Sometimes,macro-levelphenomenadonotreflectobservablephenomenaatthemicro-level,introducingaproblemofreductionism(akathefallacyofnonequivalence).Granted,knowingsomethingabouttheresponseofstudentstoastimulusmaybeextremelyhelpfulinunderstandingtheresponseofschools.Indeed,itmaybecrucialtodemonstratingthecausalmechanism(s)atwork.Thisiswhycase-studyresearch,whichtypicallyinvokesdatalyingatalowerlevelofanalysis,isoftenemployed.However,inprovingtheexistenceofacausaleffectitisimportantalsotomusterevidenceattheprincipalunitofanalysis(asdefinedbytheproposition).Inthiscontext,student-leveldatawillbemostusefulifitcanbeaggregatedacrossschools.Andforpurposesofestimatingthesizeofacausaleffect,alongwithsomelevelofprecision/uncer-tainty,observationsdrawnfromtheprincipallevelofanalysisareessential.19Lieberson(1985:ch.5).20AchenandShively(1995).90PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
Whilethelevel-of-analysisproblemisusuallyunderstoodwithreferencetocausalinference,itisequallyproblematicwhentheobjectiveoftheresearchisdescriptive.Forexample,inaddressingthequestionofglobalinequalitytheissueoftheoreticalandsubstantiveimportconcernsindividuals.Yetdataforindividualspriortothe1980sisscarcethroughoutthedevelopingworld.Thus,analystsareinthepositionoftryingtoinfertheincomestatusofindividualsfromaggregate,national-leveldata(GDP)–theproblemofecologicalinferencenotedabove.CumulationScienceisnotasolitaryventure;itisbetterconceptualizedasacollaborativeprojectamongresearchersworkingonaparticularsubjectarea.Thismeansthataresearchdesign’sutilityispartlyaproductofitsmethodologicalfitwithextantwork.Threeelementsfacilitatecumulation:thestandardizationofproceduresacrossstudies;thereplicationofresults;andthetransparencyofprocedures.StandardizationOneofthechiefavenuestocollaborationisthestandardizationofproceduresacrossresearchdesigns.Ifthereisausualwayofinvestigatingaparticularissuethisshouldbeslavishlyimitated,atleastasapointofdeparture,forthestandardizationofapproachesprovidesabenchmarkagainstwhichnewfindingscanbejudged.Thismaysoundlikearecommendationfortheoreticallymodestexercisesthatmerelyre-testoldideas.Itisnot.Recallthatinthissectionwearediscussingcriteriarelevanttotheoryappraisal,nottheoryconstruction.Weassumethatatheory(andamorespecifichypothesisorsetofhypotheses)isalreadyathand.Giventhistheory–beitboldandoriginal,ortamelyderivative–itisadvisabletostandardizetheresearchdesignasmuchaspossible,atleastattheoutset.Thestandardizationofresearchdesignsallowsfindingsfromdiversestudiestocumulate.Considerthatifeachnewpieceofresearchonvouchersutilizesidiosyncraticinputandoutputmeasures,backgroundcontrols,andotherresearchdesignfeatures,ourknowledgeofthistopicisunlikelytomoveforward.Athousandstudiesofthesamesubject–nomatterhowimpeccabletheirinternalvalidity–willmakeonlyasmallcontributiontothegrowthofknowledgeaboutvouchersiftheyaredesignedinadhoc(andhenceincommensurable)ways.91AnalysesDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
Noveltiesmustbedistinguishablefromoriginalcontributions,andthequestionisassessableonlyinsofarasastudycanbemeasuredbytheyardsticksprovidedbyextantworkonasubject.Thecallforstandardizationisacallforamoreorganizedapproachtoknowledge-gathering.RichardBerknotesthegreatpotentialgainsthatmightberealizedfrom“suitesofstudiescarefullydesignedsothatvariantsintheinterventions[can]betestedwithdifferentmixesofsubjects,indifferentsettings,andwithrelatedoutcomes,allselectedtodocumentusefulgeneral-izationtargets.”21Soconstructed,thepossibilitiesformeta-analysisarevastlyenhanced,andwithittheprospectoftheoreticaladvance.Unfortunately,inthecurrenthighlyindividualizedworldofsocialresearchitisvirtuallyimpossibletoaggregateresultsemanatingfromseparatestudiesofthesamegeneralsubject,foreachstudytendstoadoptanidiosyncraticsetofprocedures.22Incontrasttothenaturalsciences,thereappearstobeverylittlepremiumonstandardizationinthesocialsciences.Yetthecaseforstandardiza-tionseemsstrong.Justastheoriesshouldfitwithinabroadertheoreticalframe-work–thecriterionofcommensurability,discussedinChapter3–researchdesignsshouldfitwithinthebroaderframeworkwithinwhichaparticularissuehasbeenaddressed.ReplicationAnotherwaythatscientificactivityrelatestoacommunityofscholarsisthroughthereplicationofresults.Thisprojectofreplicationtakesplaceattwostages:(a)atthebeginningofastudy,asawaytoverifyextantfindingsinanewvenue;and(b)afterastudyhasbeencompleted,asawayoftestingthatstudy’sinternalandexternalvalidity.(Ifreplicationisconductedduringastudyitislikelytobereferredtoasrobustnesstesting,discussedinChapter10.23)Researchonatopictypicallybeginsbyreplicatingkeyfindingsrelatedtothatresearch.Tobesure,notallsubjectshave“findings”inthenatural-sciencesense.Yetmostfieldsrecognizeasetofpropositionsthatarewidelybelievedtobetrue;weshallcallthemfindingseveniftheyareclosertocommon-sensebeliefs.Whatevertheterminology,itishelpfulifnewresearchonatopic21Berk(2005:16).SeealsoBerketal.(1992);Bloom,Hill,andRiccio(2002).22Briggs(2005);Petitti(1993);Wachter(1988).Onepossibleexceptiontothispessimisticconclusionmaybefoundinthefieldofexperimentalstudiesthathavebeenconductedoverthepastfewdecadesonsubjectssuchasvoterturnout(seetheGOTVwebsitemaintainedbyDonGreenatYale:http://research.yale.edu/GOTV)oremploymentdiscrimination(Pager2007).23Firebaugh(2008:ch.4).92PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
beginsbyexploringthesewell-knownhypotheses.Aretheytruehere(inthissetting)?Thiswillhelpclarifythevalidityofthechosenresearchdesign,nottomentionthevalidityofthepreviousfinding.Thisistheinitialreplication.Otherreplicationsoccurafterastudyhasbeencompleted,eitherpriortoorafterpublication.(Thisisthemoreusualemploymentoftheterm.24)Inordertofacilitatereplication,aresearchdesignmustbeconductedinsuchawaythatfuturescholarscanreproduceitsresults.Considerthatfindingsarelikelytoremainsuspectuntiltheycanbereplicated–perhapsmultipletimes.Wearecognizantthatanynumberoffactorsmighthaveinterferedwiththevalidityofanyparticularstudy,including(amongotherthings)measurementerrorandthewillfulmis-reportingofdata.Verificationinvolvesrepetition;claimstotruth,therefore,involveassurancesofreplicability.Ifafindingisobtainedundercircumstancesthatareessentiallyun-repeatable,thenwerightfullyentertaindoubtsaboutitsveracity.Thisconformstothenarrowunderstandingofreplication–theabilityoffutureresearcherstoreplicateastudy’sfindingsbycarefullyfollowingthemethodsofprocedureandsourcesofdatathatwereoriginallyemployed.Butreplicationdoesnotreferonlytothenarrowlycircumscribedreiterationofastudy,innear-identicalcircumstances.Italsoreferstothevariationsthatmaybe–andoughttobe–introducedtotheoriginalstudy.PaulRosenbaumcomments:Themerereappearanceofanassociationbetweentreatmentandresponsedoesnotconvinceusthattheassociationiscausal–whateverproducedtheassociationbeforehasproduceditagain.Itisthetenacityoftheassociation–itsabilitytoresistdeterminedchallenges–thatisultimatelyconvincing.25Afindingthatpersistsinthefaceofdramaticalterationsinsetting(back-groundconditions),measurementinstruments,specification,andtreatmentstrengthisafindingthatisstronglycorroborated.Itismuchmorelikelytobetruethanafindingthathasbeenreplicatedinonlyminorrespects.Inthisvein,itisimportanttonotethatreplicationsoffernotonlyawaytocheckastudy’sinternalvaliditybutalsoameansoftesting–andwherenecessary,re-evaluating–astudy’sexternalvalidity.Whataretheboundariesofatheory?Granted,somestylesofresearchareeasiertoreplicatethanothers.Experimentsandlarge-Nobservationalstudiesarereplicabletoadegreethatqualitativeworkisgenerallynot.However,inthecaseoflarge-Nobservationalstudiesthemeaningof“replication”isusuallyunderstoodinafairlyrestrictivefashion,thatis,taking24Freese(2007);King(1995);King,Keohane,andVerba(1994:23,26,51).25Rosenbaum(2010:103).93AnalysesDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
theauthor’sdataset(orasimilardataset)andreplicatingtheauthor’sresults.Thisisafairlymechanicalprocedure.Forexample,inreplicatingacross-nationalstatisticalstudyofeconomicdevelopmentanddemocracyascholarmighttrytoreplicateextantfindingsandthenproceedtomakesmallalterations–addingcountries(withimputeddata),addingyears,orusingdifferentmeasuresofdemocracy.Bycontrast,thereplicationofqualitativeworkisusuallyunderstoodtoinvolvethedata-collectionphaseofresearch,whichmaybearchival,ethnographic,ordiscursive.Forexample,aseriousattempttoreplicateJamesMahoney’shistor-icalworkondemocratizationinCentralAmericawouldpresumablyinvolveareviewoftheauthor’sextensivelistofprimaryandsecondarysources,andperhapsadditionalsourcesaswell.26Thisrepresentsmonthsofresearch,andisnotatallmechanical.27Theequivalentdata-gatheringreplicationinalarge-Nsettingwouldbetore-codeallthedataforakeyvariable.Inourpreviousexamplethismightmeanre-codingthedemocracyvariableforallcountriesandallyears.Thisisnotwhatisusuallyintendedbyreplicationinaquantitativecontext.Butthereisnoreasonnottoapplytheconceptofreplicationtothiscommendablecross-checkingoffindings.Whateverthedifficultiesandambiguities,replicabilityisanidealforwhichallresearchoughttostrive.Arguably,itisevenmoreimportantforqualitativeworkthanforquantitativework,giventhedegreeofauthorialinterventionthatisusuallyinvolvedinthelatter(andhencethegreaterpossibilityofinvestigatorbias).Historicalresearchersshouldincludescrupulousanddetailedfootnotesoftheirsourcessothatfuturescholarscanre-tracetheirsteps.Interview-basedworkshouldincludenotationsaboutinformantssothatfutureresearcherscanlocatethesepeople.Theymayalsoputonfiletheirsetofnotes,transcripts(orrecordings)ofinterviews–whatevermightbeusefulforpurposesofreplication(withoutcompromisingtheidentitiesofsourceswhosesecrecyhasbeenpromised).28TransparencyEvidently,standardizationandreplicationarepossibleonlyinsofarasproce-duresemployedinempiricalanalysesaretransparenttoscholars.Onecannot26Mahoney(2002).27AnexampleofthissortofreplicationcanbefoundinLieshout,Segers,andvanderVleuten(2004),anattempttoreplicatethearchivalworkofMoravcsik(1998).28SeeHammersley(1997);Mauthner,Parry,andBackett-Milburn(1998),andthearticlesinCorti,Witzel,andBishop(2005).94PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
standardizeorreplicatewhatisambiguous.Thus,implicitinthecallforcumulationisthecallfortransparency.“Thepathwaybetweenthedataandtheconclusionsshouldbe…clear.”29For,withouttransparency,nofindingcanbefullyevaluated.Itiscommoninnaturalsciencesforresearcherstomaintainalaboratorynotebookinwhichacloserecordiskeptofhowanempiricalanalysisunfolds.Whileitmaynotbenecessarytorecordeveryspecificationtest,itshouldatleastbepossibleforfuturescholarstoseewhichtestswereconducted,inwhatorder,andwithwhatimplicationsforthetheory.Bycontrast,ifscholarsseeonlythefinalproductofapieceofresearch(whichmayhaveunfoldedovermanyyears)itismoredifficulttorenderjudgmentonitstruth-value.Onefears,inparticular,thatthefinaldatatablesmaycontaintheonesetofteststhatculminatedin“positive”(i.e.,theoreticallysignificant)results,ignoringhundredsofpriortestsinwhichthenullhypothesiscouldnotberejected.Granted,theachievementoffulltransparencyimposescostsonresearchers,mostlyintheformoftimeandeffort(sincethepostingofnotebooksisessentiallycost-less).Anditdoesnotentirelysolveproblemsofaccountability.Someonemustreadtheprotocols,aninvestmentoftime.Eventhen,weshallneverknowifallproceduresandresultswerefaithfullyrecorded.However,theinstitutionofatransparencyregimeisapreconditionofgreateraccount-ability,andmayintimeenhancethevalidityandprecisionofempiricalanalysisinthesocialsciences.TheoreticalfitRecallthatthepurposeofanempiricalanalysisistoshedlightonanargumentortheory.Therelationshipofthetesttotheargumentis,therefore,aparticu-larlysensitiveissue.Threeissuesbearonthetheoreticalfitofaresearchdesign:constructvalidity,severity,andpartition.Allmaybeconsideredaspectsofageneralscientificidealknownasthecrucial(orcritical)test.30ConstructvalidityConstructvalidityreferstothefaithfulnessofaresearchdesigntothetheorythatisunderinvestigation.31Thisincludesconceptvalidity:theoperationalizationof29Cox(2007:2),quotedinRosenbaum(2010:147).30Eckstein(1975);Forsyth(1976);Popper(1965:112).Platt(1964)suggeststhatthenotionmaybetracedbacktoFrancisBacon.31Shadish,Cook,andCampbell(2002).95AnalysesDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
akeyconceptwithasetofindicators.Butitalsoincludesbasicassumptionsorinterpretationsofthetheory.Considerthatifaresearchdesigndeviatessignificantlyfromthetheory–involving,letussay,questionableassumptionsaboutthetheoryorbuildingonperipheralelementsofthetheory–thenthetheorycanscarcelybeprovenordisproven,fortheresearchdesigndoesnotbearcentrallyuponit.Bythesametoken,ifaresearcherchoosesahypothesisthatliesatthecoreofatheory,theresearchdesignhasgreaterrelevance.Inthiscontext,onemightcontemplatethevastrangeofworkoneducationpolicythatbearsinsomewayoranotheronvouchers.32Agooddealofthisresearchliesattheperipheryofthecorehypothesisaboutschoolvouchersandschoolperformance;itissomewhatrelevant,butnotprimary.Forexample,ifastudyshowsthatvouchershavenoeffectonracialharmonyinschoolsthisfinding,whileinteresting,isnotlikelytobeconsideredcentraltothetheory.Assuch,thetheoryisrelativelyunaffectedbythefinding.If,bycontrast,astudyshowsthatvouchershavenoeffectoneducationalperformancethisisdevastatingtothetheory,preciselybecausetheresearchdesignandthetheoryaresocloselyaligned.Granted,manygrandtheoriesdonotrestonasinglecentralhypothesis(suchasvouchersandeducationalperformance).Considerthelargertheoryoffreemarketcompetitionthatinformsthevoucheridea.Thistheory,asframedbyMiltonFriedman,FriedrichvonHayek,orAdamSmith,isnotamenabletoanyknock-downtestsofwhichIamaware.Capitalism,likesocialism,resistsfalsification.Evidently,themoreabstractthetheory,theharderitistotranslatethattheoryintoaviableempiricaltest.33Evenso,researchersmustworkhardtoensurethatempiricaltestsarenottheoreticallytrivial.Ahighlevelofinternalandexternalvaliditywillnotrescueatheoreticallyirrelevantstudy,forwhichwereservetheepithet“straw-man.”SeveritySomeempiricaltestsareeasy,requiringlittleofatheorytoclearthehurdle(whichmayormaynotbeformalizedinastatisticaltestsuchasat-test).Otherempiricaltestsarehard,requiringagreatdealofatheory.Ceterisparibus,wearemorelikelytobelievethatatheoryistruewhenithaspassedasevereempiricaltest(aslongasthetesthassomedegreeofconstructvalidity).“Confirmationsshouldcount,”insistsPopper,32Daniels(2005).33Gorski(2004);GreenandShapiro(1994);Lieberson(1992).96PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
onlyiftheyaretheresultofriskypredictions;thatis,if,unenlightenedbythetheoryinquestion,weshouldhaveexpectedaneventwhichwasincompatiblewiththetheory–aneventwhichwouldhaverefutedthetheory.34Thesamefactorsworkinreverseifoneisattemptingtodisprove(falsify)atheory.Ifthetheoryfailsaveryhardtest,onemaynotbeinclinedtoconcludethatitiswrong.If,ontheotherhand,itfailsaneasytest–onethat,accordingtothepremisesofthetheoryitoughttohavepassed–thenone’sattitudetowardthetheoryisapttobemoreskeptical.Ananalogydrawnfromtrack-and-fieldmayhelptoillustratethepoint.Suppose,forexample,wewishtotesttherelativeabilityofvariousathletesinthehighjump,aneventthattracesitslineagetoancientGreece.Inthefirsttest,wesetthebarat10ft(3m)–aridiculousgoal,giventhatthehighestrecordedfreejumpisjustover8ft(2.5m).Predictably,alltheathletesfailtoclearthismost-difficulttest.Inthesecondtest,weapproachthematterdifferently,settingthebarat3ft(1m).Predictably,alltheathletesclearthisleast-difficulttest.Evidently,wehavelearnednothingwhatsoeveroftherelativeabilitiesofthisgroupofathletesattheendofthesetwotests.Tobesure,hadanyoftheseathletespassedthehardtest(orfailedtheeasytest)wewouldhavelearned,beyondashadowofadoubt,thatthatparticularathletewasanextraordinarilygood(bad)highjumper.Thisistheironyofthecriterionofseverity:itdependsontheoutcomeofthetest.Otherwisestated,onewishestosetthebarjusthighenoughthatitcanbeclearedbysomepeople(butnohigher),orjustlowenoughthatitcannotbeclearedbysomepeople(butnolower).Oneapparentresolutionofthisproblemistoavoidsettingarbitrarythresh-olds.Instead,askathletestojumpashighastheycanandsimplymeasuretheirrelativeperformance–acontinuousmetric.Or,ifcircumstancesdemand(e.g.,ifitisnecessarytoestablishabarinordertomeasuretheheightofajump),setupnumeroustestswithvaryingthresholds.Thesetwoapproachesamounttothesamething,exceptthatthelatterrequiresmultipleiterationsandisinthissenselessefficient.Aflexibleapproachtotestingisjustifiedinmanycontexts.However,thesacrificeonemakesinadoptingaflexiblestandardshouldbeclear.Whereverthecriteriaforsuccessandfailurearenotspelledoutclearlyinadvancetheresultingresearchislessfalsifiable,thatis,moreliabletovaryinginterpreta-tionsofsuccessandfailure.34Popper(1965:36).SeealsoPopper([1934]1968);HowsonandUrbach(1989:86);Mayo(1996:ch.6);MayoandSpanos(2006).97AnalysesDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
Moreover,evenifoneeliminatesanapriorithresholdforsuccess/failure,manyfactorsarelikelytoremainthatservetostructurethedegreeofdifficultyofatest.Returningtoourtrack-and-fieldexample,itwillbeseenthatathletes’performanceisaffectedbyagreatmany“contextual”factors–altitude,whethertheeventisheldindoorsoroutdoors,thequalityofthesurface,theaudienceinattendance,andsoforth.Relativeperformancevarieswithallofthesefactors(andperhapsmanymore).Insocialsciencesettings,thelistofcontextualfactorsisalsoquitelarge.Hereonemightconsidervariousresearchdesignfactorsthat“loadthedice”for,oragainst,aschoolvouchersstudy.Suppose,forexample,thatastudyofvouchersisconductedinacommunitywhereteachersandadministrators,aswellasmanyoftheparticipantsintheprogram,areskepticalabout–andevendownrighthostileto–thereform.Orsupposethatteachersworkinginvouchersschools(schoolsattendedbychildrenwithvouchers)arelessexperiencedorlesseducatedthanteachersworkinginpublicschools.Suppose,finally,thatthemonetaryvalueofthevoucherthatstudentsreceivedwasminimal–lessthanpriorworkandtheorysuggestswouldbenecessarytoachievesignificantchangesinstudentachieve-ment.Theseareallfactorsthatwouldseemtoloadthediceagainstapositivefinding.If,underthecircumstances,thatstudyfindsthatvouchersinduceapositive(andstatisticallysignificant)effectonstudentperformance,wearelikelytobeespeciallyimpressedbythefinding.Ontheotherhand,iftheforegoingfactorsarereversed,andthebiasofastudyappearstofavorthevouchershypothesis,apositivefindingwillhavelittlecredibility.Indeed,itisquitelikelyspurious.Assumptionsaboutthedirectionofprobablebiasmayplayanimportantroleinevaluatingtheempiricalfindingsofastudy(expost),aswellasindesigningastudy(exante).Rosenbaumnotesthatasometimescompellingstudydesignexploitsaclaimtoknowthatthemostplausiblebiasrunscountertotheclaimedeffectsofthetreatment.Inthisdesign,twogroupsarecomparedthatareknowntobeincomparable,butincomparableinadirectionthatwouldtendtomaskanactualeffectratherthancreateaspuriousone.Thelogicbehindthisdesignisvalid:ifthebiasrunscountertotheanticipatedeffect,andthebiasisignored,inferencesabouttheeffectwillbeconservative,sothebiaswillnotleadtospuriousrejectionofnoeffectinfavoroftheanticipatedeffect.35Inshort,thedegreeofdifficultyimposedbyaresearchdesignwithrespecttoaparticularhypothesisisanintrinsicpartofanystudy.Whetherthepurposeof35Rosenbaum(2010:123).98PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
theresearchispositive(toproveacausalproposition)ornegative(todisproveacausalproposition),thevalueofaresearchdesignderivespartlyfromitsrelative“crucial-ness.”Thefollowingquestionthusariseswithrespecttoanystudy:howlikelyisitthattheoryAistrue(false),giventheevidence?Theharder(easier)thetest,themoreinclinedwearetoaccepttheconclusion–ifthetestispassed(failed).Evenifonedispenseswitharbitrarythresholdsforjudgingsuccessandfailure,itwillstillbethecasethatbackgroundfactorsbuiltintoaresearchdesignqualifythattestas“easy”or“difficult”withrespecttoaparticularhypothesis.Thesefactors,whichmovewellbeyondthenarrowissuesaddressedbyquantitativemeasuresofstatisticalsignificanceorstatisticalpower,mustbetakenintoaccountifwearetoarriveatajudgmentoftheoveralltruth-valueofafinding.Suchissuesbegconsiderationexante,duringthedesignofastudy,andexpost,asresearchersassessastudy’scontribution.Whetheroneoptsforaresearchdesignthatleanstowardgreaterorlesserdifficultydependsuponmanyfactors.Easytestsareoftenappropriateatearlyphasesofhypothesistesting,whenaprojectisstilllargelyexploratoryandwhenfewextantstudiesofasubjectexist.Hardtestsbecomeappropriateasahypothesisbecomeswellestablishedandasthenumberofextantstudiesmultiplies.Ofcourse,hardtestsarebetteriftheycanbedevisedinawaythatisfairtothetheoryunderinvestigation–iftheymaintainconstructvalidity,inotherwords.Agooddealofresearchinthenaturalsciencesseemstofollowthismodel.Considerthislistofriskypredictionsthatservedtoconfirmorrefuteimportanttheoriesinphysics:Newton’spredictionofellipticalorbitsoftheplanetsfromtheinversesquarelawofgravitation;variousexperimentsconfirmingthewavetheoryoflight;Maxwell’spre-dictionofelectromagneticwavesfromamathematicalmodel;theMichelson–Morleyexperimentthatdisprovedtheexistenceoftheetherandconfirmedtheconstantvelocityoflight;Kelvin’spredictionofabsolutezerotemperature;derivationsfromPoisson’sandFourier’smathematicaltheoryofheat;inferencesbasedonthekinetictheoryofgasesandstatisticalmechanics;thepredictionofvarioussubatomicparticles;Gamow’spredictionthattheBigBanghadleftitsmarkinradiationattheedgeoftheuniverse;and,mostfamously,Einstein’spredictionsthatledtotheconfirmationofhisspecialandgeneraltheoriesofrelativity,suchasthe“bending”ofastar’slightbygravitationalattraction.3636Coleman(2007:129–130).99AnalysesDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
Theauthorofthiscompendium,StephenColeman,alsohelpfullyidentifiesseveralfeaturesofthesetheoreticalpredictionsthatprovedusefulinestablish-ingacrucialtest.Theseinclude:•Predictionofaconstantorinvariant(likethespeedoflightorafreezingpoint)•Predictionofaspecificnumber•Predictionofasymmetry,oftenderivedfromamathematicalmodel•Predictionofatopologicalfixedpoint•Predictionofalimitorconstant,ordynamiclimitcycle•Predictionofaspecificorunusualdynamicbehaviorpattern•Predictionofaspecificspatial(geographic)pattern•Predictionofastatisticaldistribution,possiblyanunusualdistribution•Predictionthatdatawillhavea“signature”–auniquemathematicalshape(asusedfordetectingheartarrhythmias,nucleartests,tsunamis,orsubmarines).37Theseareusefulexemplarsandsuggestions.Itisespeciallyimportanttoappreci-atethatthereareamultitudeofwaystoconstructatestforagivenhypothesis,onlyoneofwhichtakestheformofaclassiclinearandadditivemodel.Acommonapproachistospecify(orexamineforclues,expost)adose–responserelationship,thatis,thewayinwhichYrespondstoachangeinX.38Manyofthesealternativesofferahigherdegreeoffalsifiabilitybecausetheyofferhighlyspecificpredictions,drawndirectlyfromthetheory–predictionsthatareunlikelytobetrueunlessthetheoryistrue–asopposedtotherun-of-the-millsocialsciencepredictionthat“anincreaseinXwillleadtoanincreaseinY.”Ofcourse,onemaybeskepticalaboutthepracticalityofthisadvice.39Howmanysocialphenomenaareamenabletopreciseaprioripredictions?Howmanyareamenabletomathematicalmodelsofthesortthatwouldyieldprecise,aprioripredictions?Thepresentstateofformalmodelinginmostsocialsciencedisciplines,whileaimingtoachievethecrucialtestsofphysics,isstillalongwayfromthatgoal.Wedonotneedtoresolvethisquestion.Forpresentpurposes,itissufficienttoobservethattheprecisionofatheoryisessentialtotheseverityofatest.Bothareamatterofdegrees,andbothareakeycomponentofthattheory’sfalsifiability.PartitionFalsifiabilityisalsoenhancedinsofarasanargumentcanbeeffectivelyisolated,orpartitioned,fromtheempiricalanalysis.Thisreducesthepossibi-litythatatheorymightbeadjusted,posthoc,soastoaccommodatenegative37Coleman(2007:130).SeealsoTaagepera(2008).38Rosenbaum(2010:124–125).39Grofman(2007).100PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
findings.Italsoreducesthetemptationtoconstructargumentscloselymod-eledonaparticularempiricalsetting(“curve-fitting”),orresearchdesignswhosepurposeistoprove(ratherthantest)agivenargument.Ideally–atleastforpurposesofappraisal–theconstructionofanargumentshouldbeconsideredaseparatestepfromthetestingofthatsameargument.40Anothersortofpartitioncansometimesbeerectedbetweentheresearchdesignphaseofastudyandthedataanalysisphaseofastudy.Thisdistinction–betweenprospectivedesignandretrospectiveanalysis–isahallmarkoftheexperimentalmethod,andoneofthereasonswhyexperimentsarerightlyregardedasenhancingthefalsifiabilityofastudy.41Thereislessopportunityforexpostfactoadjustmentsofdesigntorectifyinconvenientempiricalresults.Granted,thegoalofpartitioningisalwaysamatterofdegree.Itisnotclearhowtheadvanceofknowledgecouldoccurifpartitionsweretobecompleteandfinal.(Whatdoes“final”mean?)Notethatanyfailedtest(nottomentionsuccessfultests)mustbefollowedupwithfurthertests,andthesefurthertestsmusttakethefailures(andsuccesses)ofthepastintoaccount.Inthissense,allresearchisaniterativeprocess,movingbackandforthbetweentheoryandevidence.Thecriterionofpartitionmaybeunderstood,first,asreferringtothelengthoftimethatensuesbetweeninitialtestingandsubsequentreformulationandre-testing.Ifthedurationisminute–forexample,statisticalspecificationtestsconductedatintervalsofseveralsecondsthroughanautomatedroutine–thenweareapttolabeltheprocedurecurve-fitting.Oneisnotreallytestingamodel;oneisfindingthebestfitbetweenasetofvariables(representingasetofveryloosehypotheses)andasampleofdata.If,ontheotherhand,thedurationislengthy–say,ayearormore–thenwewouldbemoreinclinedtofeelthatthegoalofpartitionhasbeenachieved.Theoryformationhasbeensegregatedfromtheory-testing.Second,partitionreferstodataemployedfortesting.Ideally,argumentsshouldbetestedwithasampleofobservationsdifferentfromthoseemployedtogeneratethetheory.Thisprovidesout-of-sampletests.Tobesure,ifsamplesarelargeandrepresentativethisshouldnotmakemuchdifference;thesameresultsshouldobtain.Andifsamplesaresmalland/ornon-representative,40King,Keohane,andVerba(1994)advise:“Adhocadjustmentsinatheorythatdoesnotfitexistingdatamustbeusedrarely”(p.21).“Always…avoidusingthesamedatatoevaluatethetheory[you]usedtodevelopit”(p.46).Originaldatacanbereused“aslongastheimplicationdoesnot‘comeoutof’thedatabutisahypothesisindependentlysuggestedbythetheoryoradifferentdataset”(p.30).SeealsoEckstein(1992:266);Friedman([1953]1984:213);Goldthorpe(1997:15).41Rubin(2008:816).101AnalysesDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
astrongargumentcanbemadeforcombiningallavailabledataintoasinglesample–therebymaximizingsamplesizeandrepresentativeness.So,onemaybeskepticalofhowpracticaltheout-of-sampletestisinpractice.Nonetheless,wherepracticable,itiscertainlydesirable.Finally,andmostimportantlyIthink,partitionreferstoastateofmind.Insofarastheorizingandtestingareseparable,themostimportantfeatureofthisseparationisnotthelengthoftimethatoneissegregatedfromtheotherorthedifferenceinsamples,butrathertheattitudeoftheresearcher.Mentalpartitionrequiresmultiplepersonalities.Atthestageoftheory-generation,theresearchermustbenurturing–aboosterofthetheorythatisbeingcreated.Alleffortsarefocusedsingle-mindedlyonthecreationandsustenanceofthatnewandstillfragileidea.Apriorispeculationsabouttheworldarederigueur,foronemustpositagreatdealinordertoestablishthefoundationforatheory.Argumentsareargumentative.Atthestageoftheory-testing,bycontrast,asecondpersonalitymustbeadopted.Thispersonalityisnon-partisan,orperhapsevenopenlyskepticalwithrespecttothemainhypothesisunderexamination.Thebabyhasbeenborn,ithassuckled,itisnowstrongenoughtofacetherigorsoftheworld(i.e.,empiricaltesting).Tocontinuethemetaphor,goodresearchrequireskillingone’sownchildrenfromtimetotime.Thisisthesortofmentalpartitionthatresearchrequires.Arguably,itisonlyfullyachievablewhenthetwostagesofresearch–theory-formationandtheory-testing–arecarriedoutbydifferentpersons,thatis,wherethetesterhasnoincentivetodisprovethenullhypothesis.Butintherealworldofresearch,especiallysocialscienceresearch(wherefundingandpersonnelarelimitedrelativetothenumberofresearchquestionsunderconsideration),thisisrarelypossible.So,wemustappealtotheresearcher’sgoodsenseandtohisorhercapacitytotransitionfromthementalityoftheorizingandnurturingtothementalityofanalysisandseveretests,thatis,fromdiscoverytoappraisal(Chapter2).Itisvitalthattheaudienceforapieceofresearchfeelconfidentintheimpartialityoftheresearcherthroughoutthetestingphase.Therearemanywaysinwhichresearcherbiascancreepin,andthereisnowayforaudiencestomonitorthesituationifresearchersareinchargeoftestingtheirownhypoth-eses.Principal–agencycomplicationsaretoogreat.Thismeansthattrustisrequired,andtheresearchermustworkhardtoearntheaudience’strust.Onetechniqueistodeclareone’sbiasesattheoutset,sothatitiscleartothereaderofareportwheretheresearcher’spointofdepartureis(andsothatthedistinctionbetweentheorizingandtestingispreserved,atleast102PartIGeneralDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
rhetorically).Ifithappensthataresearchfindingrunscountertotheoriginalhypothesis,audiencesmaybemoreinclinedtobelievethatresult,ontheassumptionthatithasclearedanespeciallyhighhurdle(or,attheveryleast,thatinvestigatorbiashasnotinfectedtheresult).Insituationsofpooroversight,themind-setoftheresearcherishighlyrelevanttoanexpostanalysisoffindings.103AnalysesDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
Downloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:36:36 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.006Cambridge Books Online © Cambridge University Press, 2012
PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:02 BST 2012.http://ebooks.cambridge.org/ebook.jsf?bid=CBO9781139022224Cambridge Books Online © Cambridge University Press, 2012
Downloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:02 BST 2012.http://ebooks.cambridge.org/ebook.jsf?bid=CBO9781139022224Cambridge Books Online © Cambridge University Press, 2012
Cambridge Books Onlinehttp://ebooks.cambridge.org/Social Science MethodologyA Unified FrameworkJohn GerringBook DOI:http://dx.doi.org/10.1017/CBO9781139022224Online ISBN: 9781139022224Hardback ISBN: 9780521115049Paperback ISBN: 9780521132770Chapter5 – Concepts pp. 107-140Chapter DOI:http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge University Press
5ConceptsThehistoryofthesocialsciencesisandremainsacontinuousprocesspassingfromtheattempttoorderrealityanalyticallythroughtheconstructionofconcepts–thedissolutionoftheanalyticalconstructssoconstructedthroughtheexpansionandshiftofthescientifichorizon–andthereformulationanewofconceptsonthefoundationsthustransformed…Thegreatestadvancesinthesphereofthesocialsciencesaresubstantivelytiedupwiththeshiftinpracticalculturalproblemsandtaketheguiseofacritiqueofconcept-construction.MaxWeber1Asweare…prisonersofthewordswepick,wehadbetterpickthemwell.GiovanniSartori2Descriptionwillbeunderstoodinthisbookasanyempiricalargument(hypothesis,theory,etc.)abouttheworldthatclaimstoanswerawhatquestion(e.g.,how,when,whom,orinwhatmanner).Bycontrast,whereverthereisanimplicitorexplicitclaimthatafactorgeneratesvariationinanoutcometheargumentwillberegardedascausal.Thedistinctionbetweenthesetwokeyconceptsthushingesonthenatureofthetruth-claim–notonthequalityoftheevidenceathand,whichmaybestrongorweak.3Description1Weber([1905]1949:105–106).2Sartori(1984:60).3Thisissomewhatatvariancewithcurrentlinguisticpractices,wherethesetermsarefrequentlyemployedasasignalofthequalityoftheevidenceathand:with“causal”reservedforexperimentalorquasi-experimentalevidenceand“descriptive”reservedforevidencethatis(forwhateverreason)weak.AndrewGelmanadvises:“Whendescribingcomparisonsandregressions,trytoavoid‘effect’andothercausalterms(exceptinclearlycausalscenarios)andinsteadwriteorspeakindescriptiveterms”:www.stat.columbia.edu/~cook/movabletype/archives/2009/03/describing_desc.html.Inthisvein,someresearchersprefertoregardallevidenceasdescriptive,soastoemphasizetheinterpretiveleapthatcausalinferencerequires(Achen1982:77–78).Theevidentproblemwiththisdefinitionalmoveisthatitdeprivesusofawayofdistinguishingbetweenargumentsthatembracedifferentgoals.Notethatanyattempttoappraisethetruth-valueofanempiricalpropositionmustbeginbyresolvingwhatthegoalsofthatpropositionare,i.e.,descriptive,causal,orsomeother.Ifthetruth-claimisunclearthenitisimpossibletofalsify.Fromthisperspective,preservingthetraditionaldistinctionbetweenwhatquestionsandwhyquestionsoughttobeahighpriorityforthediscipline.107Downloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
isthetopicofPartII,whilecausationisthetopicofPartIII.Descriptionrightlycomesfirst;onemustdescribeinordertoexplain(causally).However,thereaderwillfindmanycomparisonsandcontrastsacrossthetwotopicsinterwoventhroughoutthebook.Becausethisbookisfocusedongeneralizingstatementsabouttheworld(Chapter1),Iamnotconcernedwithdescriptionsthatreflectonlyonindi-vidualcasesorevents(withoutanyattempttoexemplifylargerpatterns).4Consequently,inthisbookdescriptionisalwaysaninferentialact.Togeneralizeistoinferfromwhatweknow(orthinkweknow)towhatwedonotknow.5Onesortofinferentialleapisfromobservationswithinasamplethataredeemedsecuretothosethatareuncertainormissing(problemsof“measure-menterror”or“missingdata”)andtodimensionsthatareinherentlyunobser-vable(“latentcharacteristics”).Anothersortofinferentialleapisfromastudiedcaseorsampletoalarger(unstudied)population.Inbothrespects,descriptivemodelsoffera“theory”abouttheworld,6“a‘formula’throughwhichthedatacanbereproduced.”7Inrecentyears,thequestforscientificunderstandinghascometobeequatedwiththequestforacausalunderstandingoftheworldacrossthesocialsciences.Bycontrast,thetaskofdescriptionisidentifiedwithidiographicstorytelling–impressionisticnarrativesrelatingdetailsaboutparticulartimesandplaces–orwithissuesofmeasurement.Thetermitselfhascometobeemployedasaeuphemismforafailed,ornotyetproven,causalinference.Studiesthatdonotengagecausalorpredictivequestionsarejudged“merely”descriptive.8Likewise,evidenceforacausalpropositionthatisjudgedespeciallyweakislikelytobecharacterizedas“descriptive.”Moregenerally,theviewofdescrip-tionthatobtainsinthesocialsciences(andespeciallyineconomicsandpoliticalscience)isofamundanetask–necessary,tobesure,butoflittleintrinsicscientificvalue.Thesubordinationofdescriptiontocausationisproblematicfromanum-berofperspectives.Firstandforemost,alargeclassofdescriptivetopicsis4Toreiterate:thisdoesnotprecludethediscussionofparticulareventsandoutcomes,butitdoesmeanthatthegoalofthesecasesistoreflectuponthecharacteristicsofalargerpopulation.5Onsomefundamentallevel,allempiricalknowledgemaybeconsideredinferential.However,itishelpfultodistinguishbetweenreadilyapprehensiblefactsabouttheworld(“observables”)andthosewhichmustbespeculatedupon(“unobservables”).Ireservetheconceptofinferenceforthelatter.6Jacoby(1999).7Berk(2004:207).8Itisnotclearwhen,precisely,thispejorativeconnotationarose.Itwasinvoked,orcommentedon,inthesocialscienceliteratureatvariouspointsinthemid-tolatetwentiethcentury(e.g.,Klimm1959;Sen1980;Singer1961).However,itprobablystretchesbackfurtherintimewithinthetraditionofAnglo-Americaneconomicsandpoliticalscience(e.g.,ClarkandBanks1793:157).108PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
intrinsicallyimportant.Intothisclassfallsubjectslikedemocracy,humanrights,war,revolution,standardsofliving,mortality,ethnicconflict,happiness/utility,andinequality.Thesetopics(andmanyothers)deservetobeexploreddescriptively.Weneedtoknowhowmuchdemocracythereisintheworld,howthisquantity–orbundleofattributes–variesfromcountrytocountry,regiontoregion,andthroughtime.Thisisimportantregardlessofwhatcausesdemocracyorwhatcausaleffectsdemocracyhas.9Theconcernisthatifconceptualizationandmeasurementofdemocracyoccursonlyinthequestforcausalinferencewemaynotachievethesamelevelofaccuracy,precision,andcomprehensivenesswithrespecttothetopic.Aresearchagendamotivatedsolelybyacausalhypothesisisapttotakeshort-cutswhenitcomestodescribingtheleft-andright-handvariables.Moreover,thatwhichonechoosestodescribemaybeinfluencedbythegeneralX/Yrelationshiponeexpectstofind,andthismayintroducebiasesintohowwedescribethephenomenon.Tobesure,thereisnothingwrongwithcausallyorienteddescrip-tion.Butitmayposeaproblemifthisistheprincipalmeansofapproachingatopicwithinafieldovermanyyears.10Asecondreasonforliberatingdescriptionfromspecificcausalhypothesesispracticalinnature.Often,itismoreefficienttocollectevidencewhentheobjectiveoftheinvestigationisdescriptiveratherthancausal.Considerthat9ForexamplesofnaturalscienceresearchthatisdescriptiveratherthancausalseeBunge(1979).10Naturally,ifthesocialsciencesweregroundedinasinglecausal-theoreticalframeworkontheorderofevolutionwithinthebiologicalsciencesthenwewouldpossessacausalmodelaroundwhichacoherentdescriptionoftheworldmightbereliablyconstructed.However,welacksuchaunifyingparadigm,andinitsabsenceitisdifficulttosayhowacausallyordereddescriptionofthepoliticalworldmightbeorganizedorwhatitwouldlooklike(inconcreteterms).Onemightcounterthatinamultiparadigmaticuniverseoneshouldlooktosmaller-scalecausalhypothesestoorganizetheworkofthediscipline,alongthe“behavioralist”model.Buthereonestumblesuponanotherproblemofindeterminacy.BecausecausalattributionisdifficulttoestablishformostnontrivialquestionsinsocialscienceitisproblematictoassertthatXmattersasasubjectofinvestigationonlyinsofarasitcausesY(orYmattersonlyinsofarasitiscausedbyX).AmbiguityaboutwhetherXreallycausesYmeansthatitmaybesafertoapproachXandYfirstasdescriptivephenomena–importantintheirownright–ratherthanaspotentialindependentanddependentvariables.Asanexample,letusreconsiderthequestionof“democracy.”Presumably,thisfeaturehasmanycausalproperties.However,wedonotknowforsurewhattheseare;andcertainly,wedonotknowpreciselywhattheyare.Consequently,thesubjectisperhapsbetterapproached,atleastinitially,asadescriptiveissue.Ofcourse,Idonotmeantosuggestthatdescriptiveinferencebecarriedoutinignoranceofallcausalpotentialities.Imean,rather,thatincircumstanceswherecausalframeworksareopen-ended–presumablythevastmajorityofcasesinsocialscience–descriptiveinferenceoughttobecarriedoutindependentofanyparticularcausalhypothesis.Thishelpstoavoidahighlyprejudiced(i.e.,particularistic,idiosyncratic)definitionofasubjectmatter.Allplausiblecausalhypothesesarerelevant–thoseinwhichasubjectservesasanindependentvariable,thoseinwhichitservesasadependentvariable,andthoseinwhichitservesasacausalpathwayinsomelargersubject.Whenconsideredinthisopen-endedfashionthesubjectofinterest(e.g.,democracy)isrightlyapproacheddescriptivelyratherthansimplyasanadjuncttosubsequentcausalanalysis.109ConceptsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
dataiscollectedfrompersons,governments,archives,andotherorganiza-tions.Collectingevidencefromthesesourcesinasystematicfashionrequiresconsiderableenergyandresources,sustainedovermanyyears.Whenadata-collectioneffortisconstructedaroundasinglecausalhypothesisortheorythescholar’spurviewisnaturallyquitelimited;onlythosefactorshavingdirectbearingonthehypothesiswillbecollected.Thismaybeefficientintheshortrun,butitisnotlikelytobeefficientinthelongrun.Narrowlyfocuseddataexpeditionsentailscalinghighcliffsandreturningtobasecampwithonlyasmallsampleofwhatonefindsatthepeak.Laterexpeditions,focusedondifferenthypotheses,willrequirere-scalingthesamepeak,atime-consumingandwaste-fulenterprise.Bycontrast,ifanevidence-gatheringmissionisconceptualizedasdescriptiveratherthancausal(whichistosay,nosinglecausaltheoryguidestheresearch),itismorelikelytoproduceabroadrangeofevidencethatwillbeapplicabletoabroadrangeofquestions,bothdescriptiveandcausal.11Insum,therearegoodreasonstoapproachdescriptionasadistinctive–andessential–taskofsocialscience.ThisisthemotivationofPartIIofthebook.Thischapterfocusesonsocialscienceconcepts,thelinguisticcontainersweusetocarveuptheempiricalworld.Chapter6offersatypologyofdescriptivearguments,andChapter7focusesonthetaskofmeasurement,the“analysis”ofdescriptivepropositions.ThequandaryofdescriptionConventionalwisdompresumesthatcausalinferenceisharder,methodologi-callyspeaking.“Whatquestionsaregenerallyeasiertoanswerthanwhyques-tions”statesGlennFirebaugh.12“Empiricaldatacantelluswhatishappeningfarmorereadilythantheycantelluswhyitishappening,”affirmsStanleyLieberson.13Readingthemethodologicalliterature,onemightinferthatdescrip-tionisarelativelysimpleandintuitiveactofapperception.Andyet,manydescriptivequestionscirculatingthroughthedisciplinesofsocialsciencearerecalcitrant.Considerthefollowing:(1)Dovotersconceptualizepoliticsideologically14ornonideologically?15(2)Isglobalinequalityincreasing16orremainingaboutthesame?1711Schedler(forthcoming).12Firebaugh(2008:3).13Lieberson(1985:219).SeealsoGelman(2010).14Nie,Verba,andPetrocik(1976).15Converse(1964).16Milanovic(2005).17BourguignonandMorrisson(2002);Dollar(2005);Firebaugh(2003).110PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
(3)IsAmericanpoliticalcultureliberal/egalitarian,18republican,19oramix-tureofboth,alongwithvariousascriptiveidentities?20Theseareallessentiallydescriptivequestionsaboutthesocialworld(though,tobesure,theycontaincausalimplications).Theyhavealsoproventobehotlycontested.Andtheyarenotunusualinthisregard.Arandomsampleof(nontrivial)descriptiveargumentswouldlikelyrevealahighlevelofuncertainty.Indeed,thereisgreatconsternationoverthepoorqualityandmeaslyquantityofevidencebywhichweattempttomakesenseofthesocialworld.21Descriptiveaccountsofmid-levelphenomenalikecorruption,campaignfinance,civilserviceprotection,judicialindependence,andpartystrengthareoftenhighlyproblematic,orarerestrictedinpurviewtoveryspecificcontexts(andhenceresistgeneralization).Andthebigconceptsofsocialscience–suchasdemocracyandgovernance–havenostandardandprecisemeaningormea-surement.22Meanwhile,wholetractsofsocialandpoliticalactivityremainvirtuallyterraincognita.23Asaresult,empiricalphenomenaontheleftandrightsidesofthetypicalcausalmodelarehighlyuncertain.ToparaphraseGiovanniSartori,themoreweadvanceincausalmodeling,themoreweleaveavast,unchartedterritoryatourbacks.24Togetaglimpseofthemethodologicalproblemswefaceinreachingdescriptiveinferencesletuscontrastthefollowingtwoquestions:(1)Whatisdemocracy,andhowmightitbeoperationalized?(2)Doesdemocracyenhancetheprospectofpeacefulcoexistence?Notethatthecausalquestion(2)presumesananswertothedescriptivequestion(1).Inordertoestimatedemocracy’scausaleffectonemustfirstestablishthedefinitionandmeasurementofthisvexingconcept.LogicsuggeststhatifProposition2buildsonProposition1itmustbeatleastasdifficulttoproveasProposition1.Andyet,byallappearances,thereisgreaterscholarlyconsensusontheanswertoquestion(2)thanontheanswertoquestion(1).Scholarsof18Hartz(1955);Tocqueville(1945).19Pocock(1975).20Smith(1993).21HeathandMartin(1997);HerreraandKapur(2007);KurtzandSchrank(2007);Munck(2009);Rokkanetal.(1970:169–180).22Ondemocracy,seeBowman,Lehoucq,andMahoney(2005);Coppedge(forthcoming);HadeniusandTeorell(2005);Munck(2009);MunckandVerkuilen(2002).Ongovernance,seeKurtzandSchrank(2007);MarchandOlson(1995);Pagden(1998);Pierre(2000).Awide-rangingcompendiumofindicatorsfordemocracyandgovernancecanbefoundinUSAID(1998).23Asoneexampleonemightconsiderlocalgovernmentinthedevelopingworld,atopicthathaselicitedlittlesystematicempiricalattention,despiteitsevidentimportance.ForarecentreviewofthisneglectedfieldofstudyseeUNHabitat(2004).24Sartori(1970:1033).111ConceptsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
internationalrelationsgenerallyagreethatregimestatushasacausaleffectonpeaceandwarsuchthatdemocraciesarelesslikelytofightwarswithoneanother,allotherthingsbeingequal.Whetherornotdemocracyisasufficientconditionforpeacemayneverbedetermined,andscholarscontinuetodebatethecausalmechanismsatworkinthisrelationship.However,thereisstillalargemeasureofagreementonthedemocraticpeaceas–attheveryleast–aprobabilisticcausalregularity.25Allthingsbeingequal,twodemocraticcoun-triesarelesslikelytogotowarwithoneanotherthantwocountries,oneorbothofwhicharenondemocratic.Bycontrast,nosuchconsensusexistsonhowtoconceptualizeandmeasuredemocracy.Thecausalpropositionisfairlycertain,whilethedescriptivepropositionthatunderliesitishighlyuncertain.Thisistheparadoxicalpatternformanydescriptiveinferences.Despitethefactthatcausalinferencesbuildondescriptiveinferencestheformerareoftenmorecertainandmorefalsifiable.Thereasonsforthisarepartlyintrinsictotheenterprise.Forexample,descriptionsoftencenteronmattersofdefinition,andthereforearenotasamenabletoappealstoevidence.Descriptionsarealsooftenexploratoryinnature,andthereforeconstructedinclosecontactwiththeevidence(aproblemofinsufficientpartition[Chapter4]).Thatsaid,someofthemethodologicalproblemsencounteredbydescriptiveinferenceareremediable.Arguably,theyareaproductofthegenerallackofmethodologicalself-consciousnessthatpermeatesthisenterprise.Myhopeisthatbyclarifyingthecommoncriteriapertainingtodescriptivearguments,andbyclassifyingtheimmensevarietyofdescriptivearguments,wemayimprovethequalityofdescriptiveinference–and,perhaps,overtime,enhanceitsstandinginthesocialsciences.ConceptsConceptformationliesattheheartofallsocialscienceendeavors.26Itisimpossibletoconductworkwithoutusingconcepts.Itisimpossibleeventoconceptualizeatopic,asthetermsuggests,withoutputtingalabelonit.Conceptsareintegraltoeveryargumentfortheyaddressthemostbasicquestionofsocialscienceresearch:whatarewetalkingabout?Ifconceptsallowustoconceptualize,itfollowsthatcreativeworkonasubjectinvolvessomereconceptualizingofthatsubject.Astudyofdemocracy,ifper-suasive,islikelytoalterourunderstandingof“democracy,”atleasttosome25Brown,Lynn-Jones,andMiller(1996);Elman(1997).26Sartori(1970:1038).112PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
degree.27Nouseoflanguageissemanticallyneutral.Authorsmakelexicalandsemanticchoicesastheywriteandthusparticipate,wittinglyorunwittingly,inanongoinginterpretivebattle.Thisissobecauselanguageisthetoolkitwithwhichweconductourwork,aswellasthesubstanceonwhichwework.Progressinthesocialsciencesoccursthroughchangingtermsanddefinitions.Thisishowwemapthechangingterrain(orourchangingperceptionsoftheterrain).Unfortunately,allisnotwellinthelandofconcepts.Ithasbecomeastandardcomplaintthattheterminologyofsocialsciencelackstheclarityandconstancyofnaturalsciencelexicons.Conceptsarevariouslyemployedindifferentfieldsandsubfields,withindifferentintellectualtraditions,amongdifferentwriters,andsometimes–mostalarmingly–withinasinglework.Conceptsareroutinelystretchedtocoverinstancesthatliewelloutsidetheirnormalrangeofuse.28Ortheyarescrunchedtocoveronlyafewinstances–ignoringothersthatmightprofitablybehousedunderthesamerubric.Olderconceptsareredefined,leavingetymologicaltrailsthatconfusetheunwittingreader.Newwordsarecreatedtorefertothingsthatwereperhapspoorlyarticulatedthroughexistingconcepts,creatingahighlycomplexlexicalterrain(giventhattheoldconceptscontinuetocirculate).Wordswithsimilarmeaningscrowdaroundeachother,vyingforattentionandstealingeachother’sattributes.Thus,weplaymusicalchairswithwords,inGiovanniSartori’smemorablephrase.29Aresultofthesepathologiesisthatstudiesofthesamesubjectappeartobetalkingaboutdifferentthings,andstudiesofdifferentsubjectsappeartobetalkingaboutthesamething.Cumulationisimpededandmethodologicalfragmentationencouraged.Conceptsseemtogetinthewayofclearunderstanding.Onesolutiontoourseeminglyendlessconceptualmuddleistobypassconceptualdisputesaltogether,focusingonthephenomenathemselvesratherthanthelabelsanddefinitionsweattachtothem.If,asGalileoobserved,alldefinitionsarearbitrary,thenwemightaswellbeginbyrecognizingthisfact.30Itiscommonlysaid,forexample,thatonecanprovepracticallyany-thingsimplybydefiningtermsinaconvenientway.Thisiswhatpromptssomecommentatorstosaythatweoughttopaylessattentiontothetermsweuse,andmoretothethingsouttherethatwearetalkingabout.“Neverletyourselfbegoadedintotakingseriouslyproblemsaboutwordsandtheirmeanings,”KarlPopperwarns.“Whatmustbetakenseriouslyarequestions27DiscussionoftheconceptofdemocracyinthischapterandthenextdrawsonCoppedge(forthcoming);CoppedgeandGerring(2011);Munck(2009).28CollierandMahon(1993);Sartori(1970).29Sartori(1975:9;seealso1984:38,52–53).30Robinson(1954:63).113ConceptsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
offact,andassertionsaboutfacts,theories,andhypotheses;theproblemstheysolve;andtheproblemstheyraise.”31Theempiricistperspectiveseemsreasonableonthefaceofthings.Andyetweareunabletotalkaboutquestionsoffactwithoutgettingcaughtupinthelanguagethatweusetodescribethesefacts.Tobesure,thingsexistintheworldseparatefromthelanguagethatweusetodescribethem.However,wecannottalkaboutthemunlessanduntilweintroducelinguisticsymbols.Anycumula-tionofknowledgedependsuponreachinganunderstandingaboutwhattocallathingandhowtodefineit.Thismilitatesagainstablithenominalism(“callitwhateveryouwant”).Asecondapproachtoresolvingconceptualdifficultyinthesocialsciencessuggeststhatconceptformationisirreduciblyamatterofcontext.Thereislittleonecansayingeneralaboutconceptformationbecausedifferentconceptswillbeappropriatefordifferentresearchtasksandresearchvenues.Thishoarybitofwisdomisabsolutelytrue–butalsohighlyambiguous.Whatdoescontextmean,andhowmightithelptoguidetheprocessofconceptformation?Isuspectthateveryauthorhastheirownpreferredcontext,whichmeansthatconceptualdisputesaresimplydisplacedfrom“concept”to“context.”Ofcourse,Iamnotarguingthatthechoiceoftermsanddefinitionsshouldbeinsensitivetoresearchcontexts.Iam,rather,raisingthequestionofpreciselyhowcontextswouldorshouldguideconceptformation.Athirdapproachtoconceptualdis-ambiguationadvisesustoavoidhigh-orderconceptsinpreferenceforlessabstract(more“concrete”)concepts.Becausemostoftheconceptualambiguitiesofsocialscienceinvolvelargeconceptualcontainers,suchasculture,democracy,ideology,legitimacy,power,publicgoods,rationality,andthestate,perhapsweoughttoparedownourconceptualambitionsinfavorofmanageableunitssuchasdeaths,votes,andpurchasingpower.Thisalsoseemsreasonable.However,thereareimportanttradeoffstosuchastrategy(knowntophilosophersasphysicalism).Mostobviously,wewouldbelimitedinwhatwecouldtalkabout.Wecoulddiscussvotesbutnotdemocracy.Andalthoughthisconcretizedlexiconmightleadtogreateragreementamongsocialscientistsonewouldhavetowonderabouttheoverallutilityofasocialsciencereconstructedalongsuchlines.Doestheactofvotingmatteroutsideaframeworkofdemocracy?Isitmeaningfulatall?Arguably,asocialsciencelimitedtodirectlyobservableentitieswouldhaveverylittleofimportancetosay.Moreover,itwouldhavenowayofputtingthesesmall-orderideastogetherintoacoherentwhole.Large-orderconceptscomprise31Popper(1976:19;quotedinCollier1998).114PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
thescaffoldingonwhichwehangobservables.Withoutgeneralconcepts,sciencecannotgeneralize,andwithouttheabilitytogeneralizesciencecannottheorize.32Asocialsciencecomposedpurelyofconcreteconceptswouldbeaseriesofdisconnectedfactsandmicromechanisms.Afinalapproachtoconceptdis-ambiguationseeksataxonomicreconstruc-tionofscientificconcepts,anapproachsometimesdesignatedas“Classical”aftertheworkofAristotleandlatter-daylogiciansintheAristoteliantradition.33Thisisanattractiveideal,asthetaxonomypossessesmanydesirablequalities(reviewedinthepreviouschapter).Yetwhileitmaybepracticableinsomeareasofnaturalsciencesuchasbiology,thetaxonomicapproachdoesnotseemtoapplyacrosstheboardinsocialscience.Taxonomieshavetheiruses,buttheseusestendtoberestrictedtospecializedsettings:individualstudiesorveryspecificterrains.Itisaspecializedtool,notageneral-purposetool.Thegeneralemploymentofsocialscienceconceptscannotbesuccessfullycontainedwithinasetoftaxonomies–muchless,withinasingleall-embracingtaxonomy.Meaningsoverflowtheneatandtidybordersofsocialsciencetaxonomies;rarelyareconceptsreducibletonecessaryandsufficientattributes.Andevenifsocialscientistsweretoacceptsuchareconstruction,onemightwonderabouttheutilityofarigidlytaxonomiclexicon.Notethattheworldofdecisionalbehaviorthatthesocialsciencesseektodescribeandexplainischaracterizedbyagreatdealofmessinessandin-discreteness.Phenomenaofthisnaturedonotreadilygrouptogetherinbundleswithclearbordersandhierarchicalinterrelationships.Thus,whileitistruethatasimplifiedtaxonomiclanguagewouldreducesemanticconfusionitmightalsoreduceourcapacitytocorrectlyunderstandthesocialworld.Wecouldagreeonalot(ifweallagreedtousesymbolsinthesameway),butwecouldnotsayverymuch.InthischapterIofferasomewhatnewapproachtothetaskofconceptua-lization.Thechapterbeginswithadiscussionofseveralkeycriteriapertaining32By“theorize,”Imeanthesearchfordescriptiveorcausalinferencesthataregeneralinscope–notthedevelopmentofatheoryaboutasingleeventorcontext.Forfurtherdiscussion,seeChapter4.33TheclassicalapproachtoconceptformationisusuallytracedbacktoAristotleandthescholasticphilosophersoftheMiddleAges.Nineteenth-centuryexponentsincludeMill([1843]1872:73)andJevons(seediscussioninKaplan1964:68).Inthetwentiethcentury,seeChapin(1939);CohenandNagel(1934);DiRenzo(1966);DumontandWilson(1967);Hempel(1952,1963,1965,1966);Landau(1972);LasswellandKaplan(1950);Lazarsfeld(1966);Meehan(1971);Stinchcombe(1968,1978);Zannoni(1978);and,mostimportantly,Sartori(1970,1984).ForasomewhatdifferentreconstructiveapproachbasedontheanalyticphilosophictraditionseeOppenheim(1961,1975,1981).ForfurtherdiscussionoftheclassicalconceptanditslimitationsseeAdcock(2005);CollierandLevitsky(1997);CollierandGerring(2009);CollierandMahon(1993);Goertz(2006);Kaplan(1964:68);Lakoff(1987);Taylor(1995).115ConceptsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
toallempiricalconcepts.Itcontinuesbyofferingasetofstrategiesthatmayhelptostructurethetaskofconceptformationinsocialsciencesettings.CriteriaofconceptualizationFourelementsofanempiricalconceptareconventionallydistinguished:(a)theterm(alinguisticlabelcomprisingoneorafewwords);(b)attributesthatdefinethosephenomena(thedefinition,intension,connotation,orprop-ertiesofaconcept);(c)indicatorsthathelptolocatetheconceptinempiricalspace(themeasurementoroperationalizationofaconcept);and(d)phenom-enatobedefined(thereferents,extension,ordenotationofaconcept).Asanexample,letusconsidertheconceptofdemocracy.Thetermis“democracy.”Acommonlycitedattributeis“contestedelections.”Anindicatormightbe“acountrythathasrecentlyheldacontestedelection.”Andthephenomenaofinterestare,ofcourse,theentitiesoutthereintheworldthatcorrespondtotheconcept,sodefined.Whenaconceptisformulated(orreformulated)itmeansthatoneorallofthefeaturesisadjusted.Notethattheyaresointerwoventhatitwouldbedifficulttochangeonefeaturewithoutchanginganother.Theprocessofconceptformationisthereforeoneofmutualadjustment.Toachieveahigherdegreeofconceptualadequacyonemay(a)chooseadifferentterm,(b)alterthedefiningattributescontainedintheintension,(c)adjusttheindicatorsbywhichtheconceptisoperationalized,or(d)redrawthephenomenalboundariesoftheextension.Itfollowsthatachangeinanyoneaspectofaconceptislikelytoaffecttheotherthree.34Andforthisreason,ourtopicmustbeviewedholistically.Itisdifficulttoseparateouttasksthatpertainonlytothephenomenalrealmfromthosethatpertaintothelinguistic/semanticortheoreticalrealms.Socialscience,fromthisperspective,isanattempttomediatebetweentheworldoflanguage(thetermanditsattributes)andtheworldofthings(beyondlanguage).Neitheristemporallyorcausallyprior;botharealreadypresentinaconcept.Withthisunderstandingofourtask,sevencriteriamaybedeemedcriticaltotheformationofempiricalconceptsinthesocialsciences:(1)resonance,(2)domain,(3)consistency,(4)fecundity,(5)differentiation,(6)causalutility,and(7)operationalization(i.e.,measurement).ThelastcriterionformsthetopicofChapter7,sothischapterwillcoveronlythefirstsixcriteria.Forconvenience,allsevendesiderataaresummarizedinTable5.1.34Hoy(1982).116PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
ResonanceThedegreetowhichatermordefinitionmakessense,orisintuitivelyclear,dependscruciallyonthedegreetowhichitconformsorclasheswithestablishedusage.Atermdefinedinahighlyidiosyncraticwayisunlikelytobeunderstood.Atthelimit–thatis,withnonsensewords–itisnotunderstoodatall.Theachievementofcommunicationthereforeinvolvesasearchforresonancewithestablishedusage.35Anyoneinclinedtodiscounttheimportanceofresonanceinconceptformationmightcontemplatethefollowingdefinitionofdemocracy:afurryanimalwithfourlegs.Thisisnonsense,ofcourse.Theimportantpoint,forpresentpurposes,isthatthenon-senseofthisdefinitionliesinitsutterlackofresonance.Itviolatesnormsofusagetodefine“democracy”withtheattributescommonlyassociatedwith“dog.”Thisistheproblemencounteredbydefini-tionsthatarepurelystipulative(ontheauthorityoftheauthor).ConceptsTable5.1Criteriaofconceptualization1.Resonance(familiarity,normalusage;antonyms:idiosyncrasy,neologism,stipulation)Howfaithfulistheconcepttoextantdefinitionsandestablishedusage?2.Domain(scope)Howclearandlogicalis(a)thelanguagecommunity(ies)and(b)theempiricalterrainthataconceptembraces?3.Consistency(antonym:slippage)Isthemeaningofaconceptconsistentthroughoutawork?4.Fecundity(coherence,depth,essence,fruitfulness,naturalkinds,power,real,richness,thickness)Howmanyattributesdoreferentsofaconceptshare?5.Differentiation(context,contrast-space,perspective,referencepoint,semanticfield)Howdifferentiatedisaconceptfromneighboringconcepts?Whatisthecontrast-spaceagainstwhichaconceptdefinesitself?6.Causalutility(empiricalutility,theoreticalutility)Whatutilitydoesaconcepthavewithinacausaltheoryandresearchdesign?7.Operationalization(measurement)Howdoweknowit(theconcept)whenweseeit?Canaconceptbemeasuredeasilyandunproblematically,i.e.,withoutbias?(Chapter7)35Resonanceisthecriterialembodimentofordinary-languagephilosophy.Themeaningofaword,declaresWittgenstein(1953:43),“isitsuseinthelanguage.”Pitkin(1972:173)expatiates:“Themeaningofaword…iswhatonefindsinagooddictionary–awordorphrasethatcanbesubstitutedforit.Themeaningof‘justice’hastodowithwhatpeopleintendtoconveyinsayingit,notwiththefeaturesofthephenomenatheysayitabout.”SeealsoAustin(1961);Caton(1963);Chappell(1964);Ryle(1949);Ziff(1960),aswellasthevariouswritingsofG.E.M.Anscombe,StanleyCavell,JerryFodor,JerroldKatz,NormanMalcolm,andJohnWisdom.117ConceptsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
seemarbitraryiftheydonotfitwithestablishedunderstandingsofatermoraphenomenon.Resonanceinthedefinitionofagiventermisachievedbyincorporatingstandardmeaningsandavoidingnon-standardones.Resonanceinthechoiceofatermisachievedbyfindingthatwordwithintheexistinglexiconthat(ascurrentlyunderstood)mostaccuratelydescribesthephenomenonofinterest.Whereseveralexistingtermscapturethephenomenoninquestionwithequalfacility–as,forexample,thenear-synonyms“worldview”and“Weltanschauung”–achievingresonancebecomesamatteroffindingthetermwiththegreatestcommoncurrency.Simple,everydayEnglishtermsaremorefamiliarthantermsdrawnfromlanguagesthataredead,foreign,orhighlyspecialized.Wherenotermwithintheexistinglexiconadequatelydescribesthephenom-enainquestionthewriterisevidentlyforcedtoinventanewterm.Sometimes,neologismisunavoidable,andthereforedesirable.Indeed,allwordswereonceneologisms,sowecannotcomplaintooloudlyabouttheforcesofinnovation.Traditionmustoccasionallybeoverturned.Thatsaid,onemustcarefullyjustifyeveryneologism,everydeparturefromordinaryusage.“Thesupremeruleofstipulation,”writesRichardRobinson,“issurelytostipulateaslittleaspossible.Donotchangereceiveddefinitionswhenyouhavenothingtocomplainofinthem.”36AnexampleofratherpointlessneologismmaybedrawnfromRobertDahl’sworkon(asIwouldsay)democracy.Notingthesemanticdifficultiesofthisterm,andwishingtoavoidits“largefreightofambiguityandsurplusmeaning,”Dahlproposedadistinctionbetweendemocracy,understoodasanunattainableideal,and“polyarchy”(derivedfromtheGreek:ruleofmany),whichwastobeunderstoodasexistingstatesthatexhibitsomeofthequalitiesofdemocracyandarecommonlyreferredtoasdemocracies.This,Dahlthought,wouldresolvetherecurrenttensionbetween“is”and“ought”thatembroilsthetermdemocracyinscholarlyandpopulardiscourse.37Dahl’smotivesarelaudable,butonecannotsaythattheattemptedneologismhasbeensuccessful,despitehisprominenceinthefield.Theproblemisthatthemeaningsofthetwotermsaresoclosethatwehavetroublehearingpolyarchywithoutthinkingofdemocracy.Onemightalsoobservethattheattempttoweansocial-scientificwordsfromtheirnormativefreightisapttobeunavailing,forsocialscienceis36Robinson(1954:80).SeealsoLinnaeus,Aphorisms243–244(reproducedinLinsleyandUsinger1959:40);Connolly([1974]1983);Durkheim([1895]1964:37);Mahon(1998);Mill([1843]1872:24);Oppenheim(1975);Pitkin(1972).37Dahl(1971:9).118PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
generallyconcernedwiththingsthatpeoplehavestrongfeelingsabout,andthesefeelingsareembeddedinordinarylanguage.Moreover,evenifthisdescriptive–normativedivisionwereultimatelysuccessfulitwouldhavetheunfortunateeffectofdeprivingacademicworkofpopularrelevance(Chapter3).Inanycase,thekeypointisthatanystrikingdeparturefromnormalusageimposesacostonthereaderofatext.Moreoftenthannot,thiscostistoohighandthetermisdiscarded.Likewise,eventheinventionofnewtermsisneverentirelyremovedfromtheextantlexicon.Neologisms,whilerejectingordinaryusage,strivetore-entertheuniverseofintelligibility.Theyarerarelynonsensewords;theyare,instead,newcombinationsofexistingwords(e.g.,bureaucratic-authoritarianism)orroots(e.g.,polyarchy,heresthetic),ortermsborrowedfromothertimeperiods(e.g.,corporatism),otherlanguageregions(e.g.,equilibrium),orotherlanguages(e.g.,laissezfaire).38ByfarthemostfertilegroundsforneologismhavebeenClassical(e.g.,Id,communitas,polis,hermeneutics)andeponymous(e.g.,Marxism,Reaganism).Inallthesecaseswords,orwordroots,areimportedfromtheirnormalcontextstoadifferentcontextwheretheytakeonnewmeaningoradditionalsenses.Howeverseverethesemanticstretch,someoriginalpropertiesremainintact.39Tosumup:termsanddefinitionschosenforuseinthesocialsciencesoughttoresonateasmuchaspossiblewithestablishedusage.Inconsistencieswithordinaryusageusuallyintroduceambiguityintoaworkorafield,despiteanauthor’sbestintentions.Thoseconceptsthatresonateleastwithordinaryusagemaybereferredtoasneologismsorstipulativedefinitions;theyareexcusableonlyifamoreresonantconceptisunavailable.DomainGranted,allofthisdependsuponthelinguisticterrainwithinwhichaconceptisexpectedtoresonate.Aconcept,likeanargument,canbeevaluatedonlyinsofarasitsdomainofusageisunderstood.Greaterbreadthofcomprehensionandusageisalwaysdesirable,allotherthingsbeingequal.Evenso,nosocialscienceconceptcanhopetobetrulyuniversal.“Democracy”isunderstood38Onpolyarchy,seeDahl(1971);onheresthetic,seeRiker(1986);oncorporatism,seeCollier(1995)andSchmitter(1974).39Robinson(1954:55)notes:“Menwillalwaysbefindingthemselveswithanewthingtoexpressandnowordforit,andusuallytheywillmeettheproblembyapplyingwhicheveroldwordseemsnearest,andthustheoldwordwillacquireanothermeaningorastretchedmeaning.VeryrarelywilltheydowhatA.E.Housmanbadethemdo,inventanewnoisetomeanthenewthing.”Forasurveyofcontemporaryneologisms,seeAlgeo(1991).119ConceptsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
somewhatdifferentlyindifferentpartsoftheworld.40Otherterms,suchas“vouchers,”mayhavelittleornoresonanceforlaycitizensanywhere.Evenwithinthesocialsciencesthereareimportantterminologicaldifferencesacrossfieldsandsubfields,andthroughtime.Economistsspeakasomewhatdifferentlanguagethananthropologists.Consequently,wemustbeconcernednotonlywithhowresonantaconceptis,butalsowithhowmanylanguagecommunitiesitwillembrace.Therewillalwaysbesomeone,somewhere,whounderstandsatermdifferently,forwhomaproposeddefinitiondoesnotresonate.Thus,itisimportantthatauthorsspecify–wheneverthematterisambig-uous–whichlanguageregionsagivenconceptisexpectedtoencompass.Offoremostconcernisthedistinctionbetweenlayandacademicaudiences.Ashasbeensaid,itisdesirableforsocialscientiststoavoidspecializedterms(“jargon”)infavorofnaturallanguagesothatabroaderaudiencecanbecultivatedfortheirwork.Andyet,itmustbeacknowledgedthatsocialscience,likealllanguageregions(e.g.,medicine,law,streetgangs,baseball),requiresaspecia-lizedvocabulary.41Socialsciencecannotacceptwordssimplyastheypresentthemselvesinordinaryspeech.Somefiddlingwithwordsanddefinitionsisincumbentontheresearcher,ifonlybecauseordinaryusageisunsettled.Socialscienceconcepts,Durkheimpointsout,donotalways,orevengenerally,tallywiththatofthelayman.Itisnotouraimsimplytodiscoveramethodforidentifyingwithsufficientaccuracythefactstowhichthewordsofordinarylanguagereferandtheideastheyconvey.Weneed,rather,toformulateentirelynewconcepts,appropriatetotherequirementsofscienceandexpressedinanappropriateterminology.42Thelimitsofordinarylanguageasafoundationforsocialsciencedefinitionareapparentinthefactthatmostcomplexterms–forexample,democracy,justice,publicgoods–carrymultiplemeanings.Insofarassocialscientistsneedtocraftspecializedconceptswithgreatercoherenceandoperationaliz-ability,theyarecompelledtodepartfromordinaryusage.Establishingthedomainofaconceptdependsuponthegoalsofapieceofresearch.Sometimes,ageneraldefinition–onethattravelswidelyacrossacademicandnonacademicvenues–isrequired.Ifoneisattemptingtoappealtopolicymakersand/orthegeneralpublicthenonemustpaycloseattentiontohowagivenconceptwillresonatewithordinaryusage.Ifoneisattemptingtoreachbeyondaparticularcultureorlanguage,thenusagesinotherculturesandlanguagesmustalsobeconsidered.Onotheroccasions,itmaynotbenecessary40Schaffer(1998).41Robinson(1954:73);Sartori(1984).42Durkheim([1895]1964:36–37).120PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
totravelwidelyortogarneruniversalconsensus.Thisgoesformanysocialsciencesettings,whereconceptsarecraftedforuseinaspecificproject.Here,amorespecializedapproachtoconceptformationiswarranted–alsoknownasastipulativedefinition,definition-in-use,contextualdefinition,orsystematizedconcept.43Toillustratethenotionofaconceptualdomainletusconsidertheconceptofdemocracy.Thedomainofthisconceptmaybesaidtorangefromasinglesubfield(e.g.,thedemocratizationsubfieldofpoliticalscience),toanentirediscipline(e.g.,politicalscience),toasetofdisciplines(e.g.,socialscience),tonaturallanguage(e.g.,English),ortoallnaturallanguages.Eachrequiresabroadeningoflanguagecommunities,andhence(probably)abroaderrangeofdefinitionsandusagesthatmustbeencompassed.Inorderfortheconcepttofunctionadequatelywithinitsdomainitmustbeunderstood(i.e.,resonate)withinthatdomain.Thisistrueregardlessofhowlarge,orsmall,thedomainmightbe.Justaseveryconcepthasalinguisticdomain(i.e.,thelanguageregionwhereitisintendedtoresonate)italsohasanempirical(phenomenal)domain.Considerfourcontextsinwhichtheconceptofdemocracyiscurrentlyemployed:(1)localcommunities;(2)nation-states;(3)trans-nationaladvocacycoalitions;and(4)modesofdressandcomportment.Evidently,someattributesaremorevalidinsomeofthesecontextsthaninothers.Forexample,“contestation”seemstoapplymostclearlyto(2),andnotatallto(4).Inthislight,themanydefinitionsofdemocracythathavebeenpropoundedinrecentyearsarenotwrong,butratherpartial.Theyexplorethemeaningofdemocracyinsomecontextswhileignoringordownplayingothercontexts.Theyare,inthissense,stipulative,arbitrary–butonlyifunderstoodasall-purposedefinitions.If,instead,welookuponthesedefinitionsaslimitedindomainitbecomespossibletorestoreamodicumofclaritytothevexedenterpriseofconceptformation.ConsistencyThecriterionofdomainimpliestheassociatedcriterionofconsistency.Aconceptoughttocarrythesamemeaning(moreorless)ineachempiricalcontexttowhichitisapplied.Therangeofcontextslyingwithinaconcept’spopulationshouldnotelicitdifferentconnotations.4443AdcockandCollier(2001);Bierwisch(1981);BierwischandSchreuder(1992);Robinson(1954);Taylor(1995:ch.14).44Goertz(2008:109)callsthis“homogeneity.”121ConceptsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
Aviolationofconsistency–whereatermmeanssomethingdifferentindifferentcontexts–createsaproblemofconceptual“stretching.”45Thus,ifcorporatismisdefinedasaninstitutionofpeakbargainingamongrelativelyautonomousunitswithincivilsocietyitmightbeconsideredaconceptualstretchtoextendthisconcepttoincludeLatinAmericancases,whereunionsandotheractorsincivilsocietywere(andinsomecasesstillare)oftenmanipulatedbythestate.Ofcourse,ifcorporatismisdefinedmorebroadly–as,say,includinganyformalbargainingamongorganizedsectorsofcivilsociety(withorwithoutstatecontrol)–thenitdoesnotcompromisetheconcept’sintegritytoapplyittotheLatinAmericancontext.Theusualwaytoadjustthescopeofaconceptistoaddtoorsubtractfromitsdefiningattributes.Usually,onefindsaninversecorrelationbetweentheintensionandextensionofaconcept.Specifically,whenattributesareunderstoodasnecessary,necessary-and-sufficient,oradditive-and-continuous,addingattributestoadefinitiondiminishesthenumberofphenomenathatsatisfythedefinition.Morefocuseddefinitionsencompassfewerphenomena.Inthismanner,aninverserelationshipexistsbetweenintensionandexten-sion,illustratedbythesolidlineinFigure5.1.46Asanexample,letussupposethatwestartoutwithadefinitionofdemocracythatincludesonlythecriterion“freeandfairelections.”Nowsupposethatwedecidetoaddasecondattribute,“civilliberties.”Iftheseattributesareunder-stoodasnecessaryornecessary-and-sufficienttheadditionofeachdefiningtraitislikelytonarrowthenumberofpolitiesthatqualifyasdemocratic,limitingtheextensionoftheconcept.Ifthesequalitiesareunderstoodasadditiveandmattersofdegree(electionsaremoreorlessfree,civillibertiesaremoreorlessrespected),theadditionofattributeswillattenuatetheempiricalfitbetweentheintensionanditsextension,inthismannernarrowingtheempiricalbound-ariesoftheconcept.(Thesamesetofentitieswillbeviewedaslessdemocratic.)Ineithersituation,theadditionofattributescannotincreasetheextensionofaconcept,foroneisaddingdefinitionalrequirements.45CollierandMahon(1993);Sartori(1970).46Thisrelationshipissometimesreferredtoasa“ladderofabstraction.”However,thiswayofviewingthingsissomewhatmisleading.Ifdemocracyisdefinedbythreeattributesratherthanfouritisnotmoreabstract;itsimplyhasanarrowerscope(withthecaveatnotedinthetext).Inanycase,thetradeoffbetweenintensionandextensionhasalonglineageintheliteratureonlogicandconcepts.Overacenturyago,StanleyJevons([1877]1958:26)pointedoutthatwhenthedefinitionalattributesofawordareexpanded–e.g.,when“war”becomes“foreignwar”–itsempiricalbreadthisnarrowed.Weber(quotedinBurger1976:72)alsonoticedthat“conceptswitheverwiderscope[have]eversmallercontent.”Inrecentyears,thisideahascometobeassociatedwiththeworkofGiovanniSartori(1970:1041,1984;CollierandGerring2009).SeealsoAngeles(1981:141);CohenandNagel(1934:33);CollierandMahon(1993);Frege(quotedinPassmore[1961]1967:184).122PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
Theutilityofthisschemaisthatitallowstheconceptualizertoadjustthescopeofaconcepttofittheneedsofananalysissothatviolationsofconsistencyareavoided.Aconceptshouldbedefinedsoasto“travel”asfarasneeded,butnofurther.IfonewishestoformaconceptofdemocracythatappliestobothAncientAthensandtothecontemporaryera,onewillneedabroaderconceptthanifoneisseekingtodescribeonlyoneortheother.Broadeningtheconceptmeanschoosingadefinitionthathasfewerattributes,andthereforeawiderambitofapplicability.Ofcourse,thistradeoffworksdifferentlywhendefiningattributesareunder-stoodassufficientconditions.Here,anyadditionofattributesincreasesthepotentialentityspace,foreachattributeissubstitutableforanyotherattribute.47If“contestation”isindividuallysufficientforapolitytoqualifyasdemocratic,thentheadditionofasecondsufficientcondition(e.g.,“participation”)canonlyincreasethepopulationofdemocracies.48Here,wefindadirectcorrelationbetweenintensionandextension,illustratedbythedottedlineinFigure5.1.Necessary-conditionattributes Sufficient-conditionattributes LowHighLowHighExtension(Number of referents)Intension(Number of attributes) Figure5.1Intensionandextension:tradeoffs47Goertz(2006).48Ifthereaderfeelsthatthisexampleisforced,onemightconsiderthefollowing.Democracymaybedefinedgenerallyasrulebythepeople,withspecificdimensionsoftheconceptincluding:(a)directpopularrule(throughreferendaandmassassemblies);(b)indirectpopularrule(throughelectedrepresentatives);and(c)deliberativepopularrule(throughconsultativebodies).Arguably,eachoftheforegoingelementsservesasafunctionalsubstitutefortheothers.Assuch,theymayberegardedassufficient-conditionattributes.123ConceptsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
Itshouldberecognized,however,thatconceptualattributesarerarelyunderstoodassufficient.Moretypically,theyareregardedasnecessary-and-sufficient,necessary,orcontinuous(mattersofdegree).Thismeansthatthetradeoffexemplifiedbythesolidlineismorecommonlyencounteredintheworkofsocialsciencethanthetradeoffexemplifiedbythedottedline.(FurtherdiscussionofconceptstructureispostponeduntilChapter6.)FecunditySocialscientistsgenerallyassociateexplanationwithcausalargumentsandunderstandingwithdescriptivearguments.However,thereisasenseinwhichdescriptiveconceptsalsoexplain.Theydosobyreducingtheinfinitecomplex-ityofrealityintoparsimoniousconceptsthatcapturesomethingimportant–something“real”–aboutthatreality.Ishallcallthiscriterionfecundity,thoughitmightalsobereferredtoascoherence,depth,fruitfulness,illumination,informative-ness,insight,naturalkinds,power,productivity,richness,orthick-ness.Whatevertheterminology,itseemsclearthatabidforconceptsisabidtotellusasmuchaspossibleaboutsomeportionoftheempiricalworld.Conceptsdevelopedbyresearchersworkingwithintheinterpretivisttradi-tionoftengiveprioritytofecundity.Interpretivistsinsistthatsocialsciencecannotevadethecallforrich,evocativeanalysis.Thickdescriptionoffersadvantagesoverthindescription,andthicktheoriesoverthintheories:theytellusmoreaboutasetofcases.Onemustappreciate,however,thatnarrativeanalysisinandofitselfdoesnotensurefecundity,justasstatisticalworkdoesnotleadinexorablytothin,orreductive,analysis.Onecanthinkofmanyproseartistswhoseforteisthesweepinggeneralization,whichisneitherinformativenorevocative.Onecanthinkofanequalnumberofstatisticalstudiesthatdescribeorexplainagreatdealabouttheirsubject.49Indeed,qualitativeandquantitativemethodsofconceptformationseekthesamegoal,thoughbydifferentmeans.Thus,whensystemsofbiologicalclassifi-cationshiftedtocomputer-generatedmodelsinthe1960s,resultingclassifi-cationswerestrikinglysimilartotheexistingcategories(largelyinheritedfromLinnaeus).50Likewise,quantitativeexplorationsofpoliticalculturehavetendedtofollowtheoutlineofargumentslaiddowndecadesbeforebyTocqueville,Hartz,andotherswritingatatimewhenquantitativeanalysiswasnotroutinelyappliedtosocialquestions.51Notethatthepurposeofalldescriptivestatisticalroutines49Forexample,Campbelletal.(1960);Verba,Schlozman,andBrady(1995).50Yoon(2009:202).51AlmondandVerba([1963]1969).124PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
(e.g.,Pearson’sr,factoranalysis,principalcomponentanalysis,clusteranalysis,andQ-sortanalysis)istoelucidatesimilaritiesanddifferencesamongentities,withtheusualaimofsortingthemintomost-similarandmost-differentpiles.(Thesameobjectiveapplieswhetherthesortingfocusesoncasesorontraits.)Abovethelevelofmeasurement,theoverallgoalofaconceptmightbespecifiedasfollows:tofocusourattentiononsomeaspectofreality–topluckitoutfromtheubiquityofextantdata.Whatmakestheconceptconvincingorunconvincingisthedegreetowhichit“carvesnatureatthejoints”(tousethePlatonicmetaphor)oridentifies“naturalkinds”(inAristotelianlanguage).Conceptsstrivetoidentifythosethingsthatarealike,groupingthemtogether,andcontrastingthemtothingsthataredifferent.Appleswithapples,andorangeswithoranges.Tobesure,allconceptsareonsomeelementallevelconventional.(Peoplearebornwiththecapacityforlanguage,buttheyarenotbornwithknowledgeofaspecificlanguage.)However,goodconceptsmovebeyondwhatismerelycon-ventional.Theyrevealastructurewithintherealitiestheyattempttodescribe.Totheextentthataconceptmanagestoidentifyrealsimilaritiesanddifferencesithassucceededinidentifyingnaturalkinds.Itisontologicallytrue.Considerthreeconceptualizationsofregimetype.Onedifferentiatesbetweendemocraciesandautocracies;52anotherdistinguishespuredemocracies,com-petitiveauthoritarianstates,andpureautocracies;53andathirdestablishesatwenty-one-pointindexthatisintendedtofunctionasanintervalscale.54Whichoftheseismostsatisfactory?Evidently,eachmaybesatisfactoryfordifferentcausalpurposes(seebelow).However,fordescriptivepurposestheutilityofaschemahingeslargelyuponitsfecundity.Inthepresentinstance,thismeans:whichschemabestdescribesthesubjectmatter?Morespecifically,whichschemamostsuccessfullybundlesregimecharacteristicstogether,differ-entiatingthemfromotherbundles?Isthenaturalbreak-pointamongregimestobefoundbetweenautocraciesanddemocracies(atwo-partclassification);amongpuredemocracies,competitiveautocracies,andpureautocracies;oristhereinsteadacontinuumofcharacteristicswithnoclear“bundles,”justifyingacontinuousdimensionalspace?Naturally,manyotheroptionsmightalsobeconsidered.Somemightarguethatregimetypesaremultidimensional,andthereforeinappropriateforanordinalorintervalscale.55Butallsuchargumentsappealtotheidealoffecundity.5652Alvarezetal.(1996).53LevitskyandWay(2002).54MarshallandJaggers(2007).55CoppedgeandGerring(2011).56Arecentquantitativeattempt,employingfactoranalysis,canbefoundinCoppedge,Alvarez,andMaldonado(2008).125ConceptsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
Becauseofitscentralitytoconceptformation–andtodescriptiveinferencemoregenerally–itisimportantthatwepursuethenotionoffecundityinmoredetail.Conceptsdonotmakesenseunlesstheattributesthatdefinetheconceptbelongtooneanotherinsomelogicalorfunctionalmanner.Theymustbecoherent.WithintheUnitedStates,forexample,theconceptof“theWest”isvulnerabletothechargethatwesternstatesdonotsharemanyfeaturesincommon(asidefromcontiguity).Thus,althoughonecanstipulateaprecisesetofborders(e.g.,thesevenwestern-moststates)onecannothelpbutfeelthatthesebordersareatrifleartificial.Thisdoesnotmaketheconceptwrong,butitcertainlymakesitlessmeaningful–lessfecund–andhencepresumablylessusefulinmanycontexts.Thedeeperorricheraconceptthemoreconvincingisitsclaimtodefineaclassofentitiesdeservingofbeingcalledbyasinglename.Acoherenttermcarriesmoreofapunch:itis,descriptivelyspeaking,morepowerful,allowingustoinfermanythings(thecommoncharacteristicsoftheconcept)withonething(theconcept’slabel).Theconceptof“theSouth,”followingtheopinionofmosthistorians,wouldbeconsideredmorecoherentthan“theWest,”sinceamuchlongerlistofaccompanyingattributescouldbeconstructedanddifferencesvis-à-visotherregionsaremoreapparent.Themostcoherentdefinitionsarethosethatidentifyacore,or“essential,”meaning.57RobertDahl,inhisinfluentialworkonpower,setsouttodiscover“thecentralintuitivelyunderstoodmeaningoftheword,”“theprimitivenotion[ofpower]thatseemstoliebehindall[previous]concepts.”58Thisessentializingapproachtodefinitioniscommon(and,indeed,oftenjustified).Theessentialmeaningofdemocracy,forexample,isoftenthoughttoberulebythepeople.Thismaybeviewedasthesingleprinciplebehindallotherdefinitionalcharacteristics,associatedcharacteristics,andusagesoftheterm.Whenonesaysdemocracy,whatoneisreallytalkingaboutisrulebythepeople.Totheextentthatthisreductionisteffortissuccessful–totheextent,thatis,thatasingleprincipleisabletosubsumevarioususesandinstancesoftheconcept–thehighestlevelofcoherencehasbeenachievedinthatconcept.(Notethatessentializingdefinitionsoftentaketheformofminimaldefini-tions,discussedbelow.)57An“essential,”“real,”or“ontological”definitionisdefinedas:“Givingtheessenceofathing.Fromamongthecharacteristicspossessedbyathing,oneisuniqueandhierarchicallysuperiorinthatitstates(a)themostimportantcharacteristicofthething,and/or(b)thatcharacteristicuponwhichtheothersdependfortheirexistence”(Angeles1981:57).SeealsoMill([1843]1872:71);Goertz(2006).58Dahl([1957]1969:79–80).126PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
DifferentiationAconceptcannotbeinternallycoherentunlessitisdistinguishablefromotherconcepts.Externaldifferentiationisthusimpliedbythenotionoffecundity.Fecundityreferstohowsimilarasetofphenomenaaretoeachother,whiledifferentiationreferstohowdifferenttheyarefromsurroundingphenomena.Theyareflipsidesofthesamecoin.Ifapplesareindistinguishablefromoranges,thecoherenceof“apple”iscalledintoquestion.59Theimportanceofdifferentiationisembeddedinthewordsdefinitionandterm.Definitionis“theactorproductofmarkingout,ordelimiting,theoutlinesorcharacteristicsofanyconceptionorthing.”60Termhassimilarconnotations,JohnDeweypointsout.Itis“derivedfromtheLatinterminusmeaningbothboundaryandterminallimit.”61HannaPitkinexplains,“themeaningofanexpressionisdelimitedbywhatmighthavebeensaidinstead,butwasn’t.Greenleavesoffwhereyellowandbluebegin,sothemeaningof‘green’isdelimitedbythemeaningsof‘yellow’and‘blue.’”62Agoodconceptis,therefore,onewithclearlydemarcatedboundaries.How,then,doesaconceptestablishclearlydemarcatedborders?Akeyelementistospecifycarefullyhowaconceptfitswithinalargersemanticfieldcomposedofneighboringconceptsandreferents.Weshallrefertothisasthebackgroundcontextorcontrast-spaceofaconcept.Wehavenotedthatconceptsaredefinedintermsofotherconcepts–boysintermsofgirls,nation-statesintermsofempires,partiesintermsofinterestgroups.Theseneighboringterms(synonyms,near-synonyms,antonyms,andsuperordinate–subordinateconcepts)givemeaningtoaconcept.Preciselybecauseoftheinterconnectednessoflanguage,theredefinitionofaterm59Thetwindesiderataofcoherenceanddifferentiationcorrespondto“lumpingandsplitting”operationsinsocialclassification(Zerubavel1996)andto“similarityanddifference”judgmentsincognitivelinguistics(TverskyandGati1978).ThetwindesideratamayalsoberecognizedinRosch’sworkonbasic-levelcategories,which“(a)maximizethenumberofattributessharedbymembersofthecategory;and(b)minimizethenumberofattributessharedwithmembersofothercategories”(Rosch,quotedinTaylor1995:50–51).60ReprintedinChapin(1939:153).Angeles(1981:56)tracestheLatinoriginsofthetermintheverb“definire,”whichistranslatedas“tolimit,”“toend,”“tobeconcernedwiththeboundariesofsomething.”61Dewey(1938:349).62Pitkin(1972:11).“Wecallasubstancesilver,”writesNormanCampbell([1919]1957:49),“solongasitisdistinguishedfromothersubstancesandwecallallsubstancessilverwhichareindistinguishablefromeachother.Thetestwhetherapropertyisadefiningoranon-definingpropertyrestssimplyonthedistinctionbetweenthosepropertieswhichservetodistinguishthesubstancefromothersandthosewhichitpossessesincommonwithothers.Anysetofpropertieswhichservetodistinguishsilverfromallothersubstanceswillservetodefineit.”127ConceptsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
necessarilyinvolvessomeresettlingofitssemanticfield.Itisimpossibletoredefineonetermwithoutalso,atleastbyimplication,redefiningothers.Anyredefinitionofcorporatismchangesourunderstandingofpluralism,justasaredefinitionofdemocracychangesourunderstandingofauthoritarianism.Itfollows–ifthemeaningofalanguageistobesustained–thatanewconceptshouldunsettlethesemanticfieldaslittleaspossible,leavingotherconceptsastheywere(moreorless).63Indeed,anewtermorredefinitionthatpoachesattributesfromneighboringconceptsislayingthegroundforfutureconceptualanarchy.Itmayresonateonfirstreading,butislikelytofosterconfusioninthatfieldorsubfieldoverthelongerterm.“Crowded”semanticfieldsareanexampleofthis.Considerthemanytermsthathavebeendevelopedoverthepastseveraldecadestorefertocitizen-basedgroups,includingcivicassociation,voluntaryassociation,civilsocietyorganization(CSO),citizensectororganization,non-governmentalorganization(NGO),interestgroup,andgrassrootsorganization.Whilesubtledifferencesmaybeestablishedamongthesetermsitisdifficulttoaccepttheendlesspropagationoftermsasproductiveforthefield.Often,neologismsareasignofconceptualdisarrayratherthanoftheoreticalfecundity.Inanycase,itisincumbentuponwriterstoclarifyhowtheirchosenconcept(s)differfromneighboringconceptssharingthesamesemanticandphenomenalspace.Thisrequiresestablishingclearcontrastswithwhatliesoutsidetheboundariesofaconcept.ConsiderrivalconceptsseekingtoexplainAmericanpoliticalculture,whichmaybesummarizedasliberalism(LouisHartz,AlexisdeTocqueville),64republicanism(J.G.A.Pocock,GordonWood),65andacombinationofliberal-ism,republicanism,andascriptiveidentities(RogersSmith).66Whatisofinter-esthereisthatthesedivergentperspectivesareofteninformedbydifferenttemporaland/orspatialcontrasts.PartisansoftheliberalthesisinvokeanimplicitcomparisonbetweentheUnitedStatesandEurope.Partisansoftherepublicanthesisinvokecomparisonsbetweentheeighteenthandnineteenthcenturies–theformerbeingmorerepublicanandthelattermoreliberal.Partisansoftheascriptivethesisinvokecomparisonswithcontemporaryidealsandpractices–deemedmoreegalitarian.Eachschoolofthoughtisprobablycorrect.However,theyarecorrectwithrespecttodifferentcomparisons.Americanpoliticalculturelooksdifferentwhendifferenttemporalandspatialcontrastsareinvoked.63Sartori(1984).64Hartz(1955).65Pocock(1975);Wood(1969).SeealsoShalhope(1972).66Smith(1993).128PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
Thesameproblemofcompetingcontrast-spacescanbeobservedinmanyotherconceptualdebates.Forexample,writersarguevehementlyoverthebasisofpoliticalconflictincontemporaryAmericanpolitics,withsomeemphasizingthepre-eminenceofstatus,race,andmorality67andothersemphasizingthepre-eminenceofsocialclass.68(Atpresent,theseargumentswillberegardedasprimarilydescriptiveratherthancausal.)Again,therearemanyfinepointstothisdebate.Thatsaid,itappearsthatsomeportionofthedisagreementcanbeexplainedbycontendingframesofcomparison.Thosewhoholdtothestatus/valuesargumentmayplausiblyenlist(a)aspatialcomparisonwithEurope(asdidthepartisansoftheliberalthesis),(b)atemporalcomparisonwiththeNewDealera,and(c)afocusonelite-levelbehavior.Thosewhoholdtothesocioeconomicinterpretationgenerallyhaveinmind(a)atemporalcompar-isonthatembracesthepasthalf-century(butnotEuropeoralongerchunkofhistoricaltime),(b)mass-levelpoliticalbehavior,and(c)contemporaneouscomparisonsbetweentherelativestrengthofstatus/valuesissuesandclassissuesinstructuringthevote.Again,bothschoolshaveplentyofgroundtostandon.Butitisnotthesameground.Thingsaresimilarwithrespecttorecentargumentsaboutglobalinequality.Thosewhoemphasizethewideninggapinglobaldistributionofincometendtobasetheirargumentsonevidencedrawnfromthepastseveraldecades,aperiodwhenindividual-leveldataisavailable.69Thosewhoemphasizetherelativeconstancyofinequalitygenerallyencompassalongertimeperiod–extendingbacktothemid-twentiethcentury,andperhapsfurther.70Again,one’sconclusionsdependcriticallyuponthehistoricalcontextonechoosestoinvoke.Ofcourse,causalargumentsalsounfoldagainstacontrast-spaceandthistoomaycreateproblems,asdiscussedinChapter8.71However,itislesslikelytoengenderconfusionbecausethecounterfactualisusuallymoreexplicit.Tosaythat“XcausesY”istosay,implicitly,thatwhenXchangesvalue,sowillY(atleastprobabilistically).Thisisfairlywellunderstood,andisformalizedinthenullhypothesis.Buttosaythat“YisX”(i.e.,X,anadjective,describesY),istoinvokeamuchmoreambiguouscontrast-space.“NotY”canrefertoanytemporalorspatialcontrastortothe(nonempirical)meaningoftheterm“X”(asinRogersSmith’sargumentaboutAmericanpoliticalculture).Weareat67Frank(2004);LaddandHanley(1975);Morone(2004);Rogin(1987).68Bartels(2006);Fiorina(2005);McCarty,Poole,andRosenthal(2008).69Milanovic(2005).70BourguignonandMorrisson(2002);Dollar(2005);Firebaugh(2003).71Achinstein(1983);Garfinkel(1981);Hitchcock(1996);vanFraassen(1980).Allworkinthe“counterfactual”traditionemphasizesthispoint.129ConceptsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
sea,forthenullhypothesis–againstwhichthehypothesismightbejudged–isnotapparent.Nonetheless,theproblemofcontextbecomestractableinsofaraswritersareabletoaddressavarietyofcompetingreferencepoints,explicitlyandempirically.Ofthese,therearethreepossibledimensions:spatial,temporal,andconceptual.Thelatter,ofcourse,refertothedefiningattributesofaconcept,andofneighboringconcepts.Bybringingthesecomparisonstothefore,virulentarguments,evenoverhighlyabstractmatterssuchaspoliticalcultureandequality,maybejoined,andperhapsovertimeresolved.Thisisthevirtueofexplicitcomparison,whichplaysanevenmorevitalroleindescriptiveinferencethanincausalinference.CausalutilityConceptsfunctioncausally,aswellasdescriptively.Thatis,theyserveascomponentsofalargercausalargument.Inthislattercapacity,theyfacedesideratathatsometimesshapethewaytheyareformed.Forexample,supposeoneisexaminingtheroleofelectoralsystemsinstructuringpoliticalconflict.Here,onewouldprobablywanttolimittheambitofstudytopolitiesthatarereasonably,oratleastminimally,democratic.Consequently,oneneedsaconceptofdemocracythatachievesthisobjective.Anideal-typedefinition(seebelow)willnotsuffice;clearbordersbetweendemocraticandnondemocraticregimesarerequired.Hence,causalconcernsrightlydriveconceptformation.Intheforegoingexample,conceptsofdemocracydemarcatetheboundariesofacausalinference.Likewise,conceptsalsoidentifycausalfactors(indepen-dentvariables)oroutcomes(dependentvariables).Avariableinacausalargumentmustalsofunctionasaconcept;thereisnosuchthingasaconcept-lessvariable(iftherewas,itwouldlackmeaning).Typically,conceptsdesignedforuseasdependentvariablesgrouptogethermanyattributes.Here,anideal-typedefinitionmaybefruitful.Bycontrast,conceptsdesignedforuseasindependentvariablesaregenerallysmaller,moreparsimonious.Thisfitswiththegoalofcausalargumentation:toexplainalotwithalittle.Italsofitswiththegoalofcausalargumentationtohaveaclearlydefined,discrete“treatment,”onethatisspecificenoughtobemanipulated(atleastinprinciple)andthatcanbeclearlydifferentiatedfrombackgroundfactors(potentialconfounders).Additionally,conceptformationinthecontextofcausalmodelsmustbecarefultoemploycon-ceptsthatdifferentiateacausefromitseffect,sothatcircularityintheargumentisavoided.130PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
Ofcourse,conceptsdefinedforuseinaspecificcausalanalysisarespecializedconcepts,notonesthatareintendedtocoverallcircumstancesandallsettings.Theyarenotgeneralinpurview.Sometimes,thissortofspecializeddefinitionbreakswithestablishedusageandthusincursacostintheresonanceofaconcept.Thiscostmustbereckonedwith.Causalmodelsareconfusing,andimpossibletogeneralizefrom,ifkeyconceptsaredefinedinidiosyncraticways.Insum,causalityisonlyonefactor,amongmany,thatrightlyaffectstheformationofconcepts(seeTable5.1).Evenwheretheneedsofacausalmodelarepre-eminent,aconceptneverentirelylosesitsdescriptivepurpose.Ifitdid,thecausalargumentwithinwhichitisembeddedwouldloseconnectionwithreality.Thisis,ofcourse,theverythingofwhichhighlyabstractcausalmodelsareoftenaccused.72StrategiesofconceptualizationHavingsurveyedgeneralcriteriapertainingtoconceptformation,weturnnowtostrategiesthatmayhelptoachievethesegoals.Conceptformationgenerallybeginswithaformalorinformalsurveyofpotentialconcepts.Itproceedsbyclassifyingtheattributesofeachconceptsothatanoverviewofeach(relevant)conceptcanbeattained.Fromthence,threegeneralstrategiesofdefinitionarerecommended:minimal,maximal,andcumulative.ThesesequentialstrategiesaresummarizedinTable5.2.Thechapterconcludeswithabriefdiscussionofthepotentialutilityofthisapproachforbringinggreaterorderandclaritytothesocialsciencelexicon.Table5.2Strategiesofconceptualization1.Surveyofplausibleconcepts2.Classificationofattributes3.Definition(a)MinimalNecessary(andperhapssufficient)conditionsofmembership,understoodasestablishingaminimalthresholdofmembership.(b)MaximalAll(nonidiosyncratic)characteristicsthatdefineaconceptinitspurest,most“ideal”form.(c)CumulativeAseriesofbinaryattributes(0/1)arrangedinanordinalfashion.72Bewley(1999);Hausman(1994);Hedstrom(2005:3);Maki(2002);Piore(1979);SpieglerandMilberg(2009).131ConceptsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
SurveyofplausibleconceptsManyinvestigationsbegininafranklyinductivemode.Thereisanempiricalterrainofinterest–perhapsacommunity,aninstitution,orapolicy–thatbecomesthesubjectofinvestigation,butwithoutaclearresearchquestionorhypothesis.Here,theresearcherarrivesslowlyataconcept,orasetofconcepts,toencompassthesubject.Thisisconceptualizationinitsbroadestsense.Inthissituation,theresearchermustcanvaswidelybeforesettlingonakeyterm(s).Prematureclosuremaycutshortthedeliberativeprocessbywhichasubjectisprocessedandunderstood.Granted,preliminaryconceptswillalwaysberequired;withoutthem,onecannotdeliberateatall.However,thecanvassingofpotentialterms–eachonetreatedgingerly,asahypothesis–iswhatallowsaresearchertotestalternativewaysofthinkingaboutatopic.Whatstoriesarecontainedintheresearchsite(thearchive,thedataset,theethnographicsetting)?Whichisthemostinterestingofthesestories?Everystorysuggestsadifferentlabelfortheproject.ThisistheexploratoryprocessdiscussedinChapter2.Oncetheresearcherhassettledonapreliminaryconceptheorsheoughttobrieflyreviewthepossiblealternatives–thatis,thefamilyofnear-synonymsthatmostcloselyfitsthecircumstance–resortingtoneologismonlywhereabsolutelynecessary(asdiscussedabove).Sinceeachextanttermbringswithitacertainamountofsemanticluggage,thechoiceamongterms–aswellasthechoiceofhowtodefinethechosenterm–rightlyinvolvesacanvassingofpotentialattributes.Thisstepfindsprecedentinvirtuallyalltraditionsofconceptualanalysis.Itistheconceptualequivalentofa“literaturereview.”Ofcourse,sometopicsaresimpleenoughtoprecludeanextensivecanvas.Here,recoursetoanaturallanguagedictionaryoraspecializedtechnicaldictionaryissufficient.Alternatively,theauthormaybeabletorelyonarticlesorbooksthatprovideamoreexpandeddiscussionofaterm’smeaningandusagepatterns,andperhapsitsetymology.However,wheretheseshort-cutsareunavailingtheauthorwillbeforcedtoundertakehisorherowncon-ceptualresearch.Aconscientioussemanticcanvassingbeginswitharepresentativesampleofformaldefinitionsandusagepatternsforachosenterm,asdrawnfromrelevantscientificfields,fromnaturallanguage,andfromhistory(etymology).Notethatusagepatternsmaybringtolightmeaningsthatarenotcontainedinformaldefinitions(perhapsbecausetheyaresoobvious),andmayhelptoclarifymeaningwhenformaldefinitionsarevague.Usagealsoentailsaconsiderationofthereferentsofaconcept(thephenomenaouttheretowhichtheconceptrefers–itsextension).132PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
Insituationswherethedifferentsensesofawordareradicallydisparate–forexample,“pen”(writinginstrument)and“pen”(enclosure)–onemustnarrowtheconceptualanalysistoonlyonemeaningofaterm.Ofcourse,homonymy(ofwhichthetworadicallydifferentmeaningsof“pen”areanexample)andpolysemy(whereawordinvokesanumberofcloselyrelatedmeanings)isoftenamatterofdegrees.Inborderlinecases,theanalystwillhavetojudgewhichsenseshouldbehivedoff(tobeconsideredasanindependentconcept),andwhichshouldberetained,soastocreatearelativelycoherentconcept.Representativenessinthesamplingprocessisachievedbysearchingforwhatevervariationinusageandformaldefinitionmightexistwithinalan-guageregionandkeepingtrackoftheapproximatefrequencyofthesevarioususagesanddefinitions.Infuture,wemaybeabletorelyondigitizedlibrariesthatcanbesampledrandomly,enablingonetoattainamorepreciseestimateofthefrequencyofusageanddefinitionalvariations.Evenso,mechanizedsamplingwillprobablynotalterourunderstandingofkeytermssignificantly,forusagepatternswithinalanguageregiontendtoexhibitgreatregularity.Moreover,ourintentistodiscardonlyveryidiosyncraticusagesanddefini-tions.Thus,aslongasthesampleissufficientlybroadoneislikelytopickupallcommon(nonidiosyncratic)usages.Theprincipleofredundancymayserveasanindicatorofsufficiency:whenonereachesapointwheredefini-tionalattributesandusagesbegintorepeat,onemayjustifiablyterminatetheexpedition.Onehassampledenough.Theissueoflinguisticdomain–howmanylanguageregionstosurvey–isalsocrucial.Asamplingisbetterifitcoversmorelanguageregions.Yetifthisbroadsearchrevealssignificantdifferencesinmeaningthentheanalystmayrestrictthescopeoftheinvestigationinordertopreserveconsistencyandcoherence.Anysamplingislikelytohaveahometurf–perhapsaparticularfieldofsocialscience–thatisextensivelycanvassed,andotherareasthataresurveyedmoresuperficially.Inanycase,thedomainofthesurveywillhelptoestablishthedomainoftheresultingdefinition.ClassificationofattributesThenexttaskistoreducetheplenitudeofmeaningsimpliedbyatermintoasingletable.Theconstructionofsuchatablerestsontheassumptionthat,althoughdefinitionsforagiventermare,inprinciple,infinite(sinceevenasmallnumberofattributescanbecombinedinmanyways,andsincetherearealwaysmultiplewaystoconveyasimilarmeaning),mostdefinitionsand133ConceptsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
usagesjugglethesamebasicsetofattributes.Bycombiningnear-synonymsandbyorganizingthemalongdifferentdimensionsoneoughttobeabletoreducethedefinitionalprofusionofeventhemostcomplexconceptintoarelativelyparsimonioustableofattributes.Weregardthistableasthelexicaldefinitionofatermbecauseitreportsthemanymeaningsofthattermextantacrossagivenlinguisticdomain.Asanexample,letusexplorethedefinitionalattributesof“democracy.”Oursurveyofdefinitionsandusagesrestsonanumberofrecentstudiesthatattempttodelineatethemeaningofthiskeyterm,focusingprimarilyontheWesterntradition(historicalandcontemporary).73Thisisthereforeregardedastheprincipaldomainoftheconcept.Empirically,Ichoosetofocusonapplicationsofthisconceptwithinpoliticalcontexts,andespeciallyinlargepolitiessuchasthenation-state(ratherthanwithinsmall,localbodies).Thiswillbetheempiricaldomainoftheconcept.Fromthiscom-pendiumofdefinitionsandusages,onemaydistillalistofcommonattri-butes,depictedinTable5.3.Obviously,thislistrestsatafairlyabstractlevel;onecouldextendittoincludemuchmorespecificfeaturesofthepoliticallandscape.Butthiswouldrequireamuchlargertableandisunnecessaryforpresentpurposes.Withacomplexsubjectlikedemocracyitishelpfuliftheattributescanbearrangedinataxonomicfashion(Chapter6).Ofcourse,thisisnotalwayspossible,andonecanglimpsemorethanafewviolationsoftaxonomicprinciples(e.g.,componentsthattraverseseveralcategories).Still,thisexerciseinsemanticreductionisusefulwhereverpracticable.Definition:concepttypesWiththecaveatsnotedabove,itseemsfairtoregardTable5.3asafairlyencompassinglexicaldefinition,includingmostoftheattributescommonlyassociatedwiththetermintheWesterntradition.Evenso,becauseofthenumberanddiversityoftheseattributes,Table5.3doesnottakeusveryfartowardafinaldefinition.Inordertocreateamoretractableempiricalconcept,onemustgofurther.Thisnextstep–fromlexicaldefinitiontospecializeddefinition–iscrucial.Toachieveit,threeapproacheswillbereviewed:minimal,maximal,andcumulative.73Beetham(1994,1999);CollierandLevitsky(1997);Held(2006);Lively(1975);Sartori(1962);Saward(2003);Weale(2007).134PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
MinimalOnelong-standingdefinitionalstrategyseekstoidentifythebareessentialsofaconcept,sufficienttodifferentiateitextensionallywithoutexcludinganyofthephenomenagenerallyunderstoodaspartoftheextension.Theresultingdefinitionshouldbecapableofsubstitutingforall(nonidiosyncratic)usesofthetermwithouttoomuchlossofmeaning.Thismeans,ofcourse,thatitshouldnotconflictwithany(nonidiosyncratic)usages.Eachattributethatdefinesaconceptminimallyisregardedasanecessarycondition:allentitiesmustpossessthisattributeinordertobeconsideredamemberoftheset.Collectively,theseattributesarejointlysufficienttoboundtheconceptexten-sionally.Minimaldefinitionsthusaimforcrispborders,allowingfortheTable5.3Aclassificationoffundamentalattributes:“Democracy”Coreprinciple:rulebythepeopleIElectoral(akaelite,minimal,realist,Schumpeterian)Principles:contestation,competition.Question:aregovernmentofficesfilledbyfreeandfairmultipartyelections?Institutions:elections,politicalparties,competitiveness,andturnover.IILiberal(akaconsensus,pluralist)Principles:limitedgovernment,multiplevetopoints,horizontalaccountability,individualrights,civilliberties,transparency.Question:ispoliticalpowerdecentralizedandconstrained?Institutions:multiple,independent,anddecentralized,withspecialfocusontheroleofthemedia,interestgroups,thejudiciary,andawrittenconstitutionwithexplicitguarantees.IIIMajoritarian(akaresponsiblepartygovernment)Principles:majorityrule,centralization,verticalaccountability.Question:doesthemajority(orplurality)rule?Institutions:consolidatedandcentralized,withspecialfocusontheroleofpoliticalparties.IVParticipatoryPrinciple:governmentbythepeople.Question:doordinarycitizensparticipateinpolitics?Institutions:electionlaw,civilsociety,localgovernment,directdemocracy.VDeliberativePrinciple:governmentbyreason.Question:arepoliticaldecisionstheproductofpublicdeliberation?Institutions:media,hearings,panels,otherdeliberativebodies.VIEgalitarianPrinciple:politicalequality.Question:areallcitizensequallyempowered?Institutions:designedtoensureequalparticipation,representation,protection,andpoliticallyrelevantresources.Institutions:bothgovernmentalandnongovernmental(e.g.,interestgroups,parties,civicassociations).Source:CoppedgeandGerring(2011).135ConceptsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
classificationofentitiesas“in”or“out.”Ofcourse,theymaynotalwaysachievethisgoal,butthisistheiraim.74Sometimes,minimalconceptsarecraftedaroundanabstractcoreprinciplesuchas“rulebythepeople.”Inthisinstance,thecoremeaningsatisfiesthecriterionofresonance,forallinvocationsofdemocracyrevolveinsomewayaroundthisidea.However,suchanabstractdefinitiondoesnotachievecrispbordersfortheconcept;indeed,itscarcelyidentifiesborders.Inthisrespect,itisproblematic.Amorecommonapproachistoidentifyaspecificcomponentofthetermthateveryone(ornearlyeveryone)agreesupon.Ifwearelimitingourselvestorepresentativepolities(excludingdirectdemocracies)onemightarguethatfreeandfairelectionsconstitutesanecessaryconditionofdemocracy.Thisattributesufficesasaminimaldefinition,foritissufficienttoboundtheentityempirically.Thatis,havingfreeandfairelectionsmakesapolityademocracy;nootherattributesarenecessary.Atleast,soitmightbeargued.Thecaveat,ofcourse,isthatwearedefiningdemocracyinaveryminimalfashion,leavingotherattributesoftenassociatedwiththeconceptinabeyance.Thisimposessomecostsinresonance.Thestrippeddownmeaningofthetermsoundsstrangetothoseattunedtodemocracy’smanynuances.MaximalMaximaldefinitions,incontrasttominimaldefinitions,aimfortheinclusionofall(nonidiosyncratic)attributes,therebydefiningaconceptinitspurest,most“ideal”form.Thiswould,ofcourse,includetheattribute(s)thatdefinestheconceptminimally:itsnecessarycondition(s).AsWeberdescribesit,“anideal-typeisformed…bythesynthesisofagreatmanydiffuse,discrete,moreorlesspresentandoccasionallyabsentconcreteindividualphenomena,whicharearrangedaccordingtothoseone-sidedlyemphasizedviewpointsintoaunifiedanalyticalconstruct.”75Followingthisrecipe,onemightcreateanideal-typedefinitionofdemoc-racythatincludesmost,orall,ofthedimensionslistedinTable5.3.Ofcourse,74Definitionalstrategiessimilartothe“minimal”strategyhavebeenemployedbyvariouswriters,althoughnotusuallybythisname.See,e.g.,Debnam(1984)on“power”;Freeden(1994:146)on“ineliminable”attributes;Hamilton(1987)on“ideology”;Pitkin(1967:10–11)on“basicmeaning”;Murphey(1994:23–24).Sartoriendorsesminimaldefinitioninearlywork(1975:34–35,1976:61),butdropsthematterinhisclassicworkonconceptformation(1984).Itshouldbenotedthatminimaldefinitionissimilar,thoughnotidentical,toa“proceduralminimum”definition(CollierandLevitsky,1997).Inthelatter,thesearchisforanoperationalizationthatsatisfiesalldefinitionalrequirementsofaconcept.75Weber([1905]1949:90).SeealsoBurger(1976).IncitingWeber,Idonotclaimtobeusingtheconceptofanideal-typeinpreciselythewaythatWeberenvisioned.136PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
somemightbeexcludedifitcouldbearguedthattheydetractsignificantlyfromthecoherenceoftheoverallconcept.Blatantlycontradictoryelementsshouldbeavoided.Ideal-types,asthetermsuggests,neednothaveaspecificreal-lifeempiricalreferent.Perhapsnoextantpolityachievesperfectdemocracy.However,inordertobeofserviceanideal-typemustapproximatereal,existingentities,whicharethenscoredaccordingtohowcloselytheyresembletheattributesoftheideal-type.Ideal-typesarealwaysmattersofdegree,andhencegenerallyoperationalizedbyintervalscales(discussedinChapter6).CumulativeAthirdstrategyofconceptformationisanattempttoreconcileminimalandmaximalapproachesbyrankingthe(binary)attributescommonlyassociatedwithaconceptinacumulativefashion,thatis,asmoreorlessessentialtoaconcept.76Thisresultsinanordinalscale(discussedinChapter6).Followingtheseprinciples,onecanenvisionacumulativescaleindicatorofdemocracythatbeginswithfreeandfairelections–theminimaldefinition–andproceedsthrougheightadditionalcriteria,listedinorderofcentralitytotheconceptofinterest,asdepictedinTable5.4.Ifthisorderingofattributesisaccepted–if,thatis,itisagreedthat1ismoreessentialthan2and2ismoreessentialthan3–thenitmaybepossibletoarriveatanacceptabledefinitionofdemocracythatincorporatesmanyoftheattributescommonlyassociatedwiththeterm,whilealsorecognizingtherelativeimportanceofeachoftheseattributes.Ithastheadditionaladvantageofallowingustoorderallextantpolitiesempiricallyaccordingtotheirdegreeofdemocracy:themoreattri-butesapolitypossesses,themoredemocraticitis.77(Thissolvestheaggrega-tionproblem,anissueofmeasurementdiscussedinChapter6.)Ofcourse,wewillnotbeabletodeterminehowmuchmoredemocraticonepolityisthananother,forwecannotpresumethateachlevelisequidistantfromthenext(thedistinctionbetweenanordinalandintervalscale).Asecondshortcomingofthisparticularcumulativedefinitionisthattheordinalscaleofattributesmaynotbefullycomprehensive;someattributesmaybedifficulttorankintermsoftheircentralitytotheconcept.Indeed,onecanseethatnotallofdemocracy’slexicalattributes(seeTable5.3)arecontainedinthecumula-tiveconceptinTable5.4.76ThisisverysimilarinspirittotheconstructionofaGuttmanscale,exceptthatwearedealingwithattributesratherthanindicators,andwiththetheoretical(ratherthanempirical)propertiesoftheseattributes.77ForanotherexampleoftheordinaltechniqueseeCoppedgeandReinicke(1990).137ConceptsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
DiscussionHavingoutlinedthreestrategiesofconceptdefinition–minimal,maximal,andcumulative–thereadermaywonderwhetherthisexhauststhefield.Naturally,itdoesnot.Conceptsservemanytheoreticalandempiricalfunc-tions,andthesefunctionsrightlyconditionhowtheyareformedwithinthepurviewofagivenwork.However,generaldefinitionsofaconcept–thoseintendedtotravelwidely–tendtoadoptminimalormaximalapproachestodefinition.(Occasionally,theymayemployacumulativeapproach.)Thisisbecausetheseapproachestendtobemostsuccessfulinestablishingresonance,consistency,andcoherenceacrossabroaddomain.(IssuesofmeasurementTable5.4Cumulativedefinition:“Democracy”OrdinalscaleAttributes123456789(a)Freeandfairelectionsxxxxxxxxx(b)Self-government(domestic)xxxxxxxx(c)Self-government(complete)xxxxxxx(d)Executiveelectedandparamountxxxxxx(e)Universalmalesuffragexxxxx(f)Universalsuffragexxxx(g)Executiveconstitutionalityxxx(h)Executiveconstraintsxx(i)Civillibertyx(a)Freeandfairelections:nationalelectionsareregularlyheld,areopentoallmajorpartiesandcandidates(includingalloppositionpartiesandfigureswhomightposeasignificantchallengetotherulinggroup),andappearonbalancetoreflectthewilloftheelectorate(whateverirregularitiesmightexist).(b)Self-government(domestic):sovereigntyoverdomesticpolicy.(c)Self-government(complete):sovereigntyoverdomesticandforeignpolicy.(d)Executiveelectedandparamount:executiveiselectedandisparamount(i.e.,superior,defacto,tootherleadersandinstitutions).(e)Universalmalesuffrage:alladultmalecitizensareallowedtovoteandnogroupofcitizensisselectivelydiscouragedfromvoting.Presumption:citizenshipincludesamajorityofpermanentresidentsinaterritory.(f)Universalsuffrage:alladultcitizensareallowedtovoteandnogroupofcitizensisselectivelydiscouragedfromvoting.Presumption:citizenshipincludesamajorityofpermanentresidentsinaterritory.(g)Executiveconstitutionality:executiveactsinaconstitutionalmanner,anddoesnotchangetheconstitutiontosuititspoliticalneeds(thoughitmaytry).(h)Executiveconstraints:executive,althoughparamount,iseffectivelyconstrainedbyotherpoliticalinstitutions,actingintheirconstitutionalrole(e.g.,judiciary,legislature,monarch,independentagencies).(i)Civilliberty:citizensenjoyfreedomofspeechandfreedomfrompoliticallymotivatedpersecutionbygovernment.138PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
aregenerallysecondarywhenaconceptmusttravelwidely.)Inotherwords,minimalandmaximaldefinitionsofferabetterresolutionofthecriterialdemandsthatallconceptsface(seeTable5.1).Tobesure,someconceptsresistthiseffortatsemanticreduction.Itisallegedthatsomeconceptsembody“family-resemblance”attributes,wheredifferentusagessharenosinglecharacteristicincommonandthereforehavenocoremeaning.Anoft-discussedexampleis“mother,”whichmaybedefinedas(a)abiologicalfact,(b)thepersonwhoplaysaprincipalroleinnurturingachild,or(c)accordingtorulesandnormswithinspecializeddomains(e.g.,MotherSuperiorwithintheCatholichierarchy).Thesedefinitionssharenosingleelementincommon.Theyaredisparate.78Insocialsciencecontext,however,wearelesslikelytowitnessfamily-resemblanceconcepts.Democracyisanessentiallycontestedconcept.Evenso,allcommentatorsseemtoagreethat,asappliedtopoliticalcontexts,thisconceptrevolvesaroundasinglecoreattribute–rulebythepeople.“Justice,”anotherboneofcontention,alsohasacoremeaning:toeachhisorherdue.(Asithappens,bothofthesecoremeaningscanbetracedbacktoAncientGreece.)Moretothepoint,eveninsituationswherefamilyresemblancesmightbesaidtoexistthereislittleprofitintrumpetingthedisparatenatureofaterm’sdefinitions.Thus,while“corporatism”hasbeenregardedasafamily-resemblanceconcept79itcouldalsobesubjectedtoaminimalormaximaldefinition.Iwouldarguethatwearebetterservedbythelatterthanbytheformerpreciselybecauseminimalandmaximaldefinitionscreatemorecoherentconcepts,andonesthatareeasiertolocateinempiricalspace(i.e.,tomeasure),albeitwithsomelossofresonance.Betteraminimal,maximal,orcumulativedefinitionthatisflawed–asinsomesense,allsocialsciencedefinitionsare–thanafamily-resemblancedefinitionthatresultsinanincoherentconcept.Beforeconcludingitisworthtakingnoteofthefactthatwehavefocusedthusfaron“hard”cases–democracy,justice,andthelike.Otherconceptsinthesocialsciencelexiconarerarelyastroublesome.Fromthisperspective,theproblemofconceptualizationisperhapssomewhatlessseverethanitmayseemfromacursoryreadingofthischapter.Bywayofcontrast,letusquicklyexamineaneasier,moreconcreteconcept.“Politicalparty”maybedefinedminimallyasanorganizationthatnominatesindividualsforoffice.Thisdefinitionimposescrispbordersandissubsti-tutableforallextantusagesofwhichIamaware.Amaximaldefinitionwould,78Wittgenstein(1953).SeealsoCollierandMahon(1993);Goertz(2006);Taylor(1995:ch.3).79CollierandMahon(1993:847).139ConceptsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
ofcourse,encompassotherattributescommonlyassociatedwiththeworkofpoliticalparties,suchasasharedideology,anorganizationalapparatus,well-definedmembership,andenduranceovertime.Theseattributesdescribepartiesintheirstrongest,mostidealsense,andaremattersofdegree.Acumulativedefinitionwouldarrangethesesameattributes(orsomesubsetofthem)accord-ingtotheircentralitytotheconcept.80Whicheverstrategyonechoosestoemploy,defining“politicalparty”isconsiderablyeasierthandefining“democ-racy.”Andsoitmaybeforotherconceptsthatlieclosertotheempiricalbone.Evenwiththemostcomplexconcepts,carefullycrafteddefinitionsintheminimal,maximal,orcumulativemoldshouldprovideacommonscaffoldinguponwhichtheworkofsocialsciencecanrestinareasonablystableandconsistentmanner.Tobesure,meaningschangeovertime;butsuchchangeoccursslowly.Newterms,ornewmeaningsforoldterms,appearidiosyncraticatfirst.Overtime,ifneologismsgainadherents,theybecomeestablished.However,atanygivenpointintimereasonablyauthoritativedefinitionsshouldbefeasible–withthecaveatthatmultipleapproachestothesameconcept(minimal,maximal,andcumulative)canoftenbejustified.81Thus,itisincum-bentuponauthorstoclarifywhatstyleofdefinitiontheyareadopting.Notealsothattheconstructionofminimalandmaximaldefinitionsestab-lishessemanticboundariesaroundaconcept.Itspecifiestheminimalandmaximalattributes,andthecorrespondingminimalandmaximalextensions.Thissortofexercise–equivalenttoan“extremebounds”analysis–isespeciallyusefulwhendealingwithfar-flungconceptssuchasdemocracy.80ForfurtherdiscussionofthisconceptseeGuntherandDiamond(2003:172).81Forfurtherdiscussionandadditionalexamples,seeGerring(1997);GerringandBarresi(2003).140PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:01 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.008Cambridge Books Online © Cambridge University Press, 2012
Cambridge Books Onlinehttp://ebooks.cambridge.org/Social Science MethodologyA Unified FrameworkJohn GerringBook DOI:http://dx.doi.org/10.1017/CBO9781139022224Online ISBN: 9781139022224Hardback ISBN: 9780521115049Paperback ISBN: 9780521132770Chapter6 – Descriptive arguments pp. 141-154Chapter DOI:http://dx.doi.org/10.1017/CBO9781139022224.009Cambridge University Press
6DescriptiveargumentsObviouslythereisnoclassificationoftheUniversenotbeingarbitraryandfullofconjectures.Thereasonisquitesimple:wedonotknowwhattheuniverseis.JorgeLuisBorges1Whatthedevilisgoingonaroundhere?AbrahamKaplan2Howdosocialscientistsdescribeasocialreality?Whatargumentsdoweemployinourattemptstobringordertothegreatblooming,buzzingconfu-sionoftheworld?3Onemightsupposethattheshapeofadescriptiveinferenceislimitedonlybythesocialphenomenonthatweseektodescribe,themodels(cognitive,linguistic,mathematical,andvisual)thatwehaveatourdisposal,andourimagination.Inpractice,however,descriptiveinferencesdrawfromastandarditineraryoftropes.Ishallarguethatmostdescriptiveclaimscanbeclassifiedasindicators,syntheses,typologies,orassociations,alongwiththeirvarioussubtypesasillustratedinTable6.1.Thisishowsocialscientistscarveupnatureatthedescriptivelevel.Thesearethepatternsthatwelookforwhenattemptingtodescribeclassesofeventsinthesocialworld.Eachofthesewaysofdescribingtheworldhasalonghistory.Indeed,theyarealmostsecond-nature.Yettheyrarelyreceivetheattentionthattheydeserve.Theirveryfamiliarityseemstohavefosteredadegreeofnonchalance.Inrenderingaformaltreatmentoftheseinformalsubjectsmygoalistobringgreaterself-consciousnesstotheactofdescriptionand,atthesametime,toestablishavaluedplacefordescriptiveanalysisinthesocialsciencedisciplines.Naturally,thesegenresofdescriptionmayalsobeenlistedincausalinfer-ence.Indeed,itisoftenthecasethatthesamepatternofdatacanbeinter-pretedaseitherdescriptiveorcausal.Thisdependsupontheresearcher’s1Borges([1942]1999:231).2Kaplan(1964:85).3James(1981:462).141Downloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:08 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.009Cambridge Books Online © Cambridge University Press, 2012
understandingofthedata-generationprocess;itisaninferencenotaself-evidentfactabouttheworld.4IndicatorsAnindicatoraimstodescribeonefeatureofapopulation,andmayalsobereferredtoasanattribute,dimension,factor,measure,parameter,property,variable,orunidimensionaldescription.Itmaybedirectlyobservable(e.g.,testscores)ormaybeobservableonlythroughproxies(e.g.,intelligence).Itmaybecomposedofasinglephenomenon(e.g.,theanswertoaparticularquestiononasurvey)orofmultiplephenomena(e.g.,theanswertoseveralquestionsonasurvey).However,ifmultiplecomponentscontributetoanindicatortheymustbereducible,withouttoomuchlossofinformation,toasingledimension,thatis,anindex.Thisiswhatqualifiesitasaspeciesofindicator.(Whetherornotthisreductionofpropertyspaceissuccessfulisanempiricalmatter.Forthemoment,ourfocusisontheaprioriqualityoftheargument.)Likewise,indicatorsmaybecalibratedaccordingtoanytypeofscaleexceptanominalscalewithmorethantwocategories,whichwould,ofcourse,bemultidimensional(Chapter7).Table6.1DescriptiveargumentsIndicators▪Unidimensional(akaattributes,dimensions,factors,measures,parameters,properties,scales,variables).Syntheses▪Asinglemultidimensionalcategoryinwhichdiverseattributesrevolvearoundacentraltheme.Typologies▪Multidimensionalcategoriesthataremutuallyexclusive,exhaustive,anddefinedbyuniformprinciples.Simple▪(Noadditionalcriteria.)Temporal▪Categoriescorrespondtodiscretetimeperiods.Matrix▪Categoriesderivedfromtheintersectionofseveralfactors.Taxonomic▪Categoriesarrangedinagenusetdifferentiumhierarchy.Configurational▪Categoriesdefinedbysubtractingattributesfromacoreconcept.Sequential▪Categoriesarrangedinatemporalsequence,withconsequencesforeachcategory.Associations▪Multidimensionalwithastrongprobabilisticcomponent.Trend▪Correlationbetweenaphenomenonandtime.Network▪Interrelationsamongunits–spatial,temporal,orfunctional.Correlational▪Correlationamongindicatorsand/orsets.4Achen(1982:77–78).142PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:08 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.009Cambridge Books Online © Cambridge University Press, 2012
Someindicatorslieclosetotheempiricalbone,requiringlittleornointerpretation.Ameasureofinfantmortalitybasedonthenumberofdeathspriortoageoneper1,000livebirthsisfairlyself-evident.Here,theindicatoristheinfantmortalityrate(IMR).If,ontheotherhand,thisvariableisemployedasanindicatorofsomemoreabstractconceptsuchashumanwelfareitbecomesmorecontroversial:questionsofconceptualvaliditycomeintoplay,asdiscussedinChapter7.Forpresentpurposes,therealmofindicatorsisunderstoodinaninclusivefashion,includingbothself-evident“facts”andlargerclaims.Note,foranindicatortohavemeaningitmustbeassociatedwitha(linguis-tic)concept.Well-knownindicatorsarefamousbyvirtueoftheconceptualfreighttheyareaskedtopull.Theseincludeindicatorsofdemocracy(e.g.,PolityIV5andFreedomHouse6),interstateconflict(e.g.,CorrelatesofWar7),goodgovernance(e.g.,theWorldBankGovernanceindicators8),electoralmalappor-tionment,9andpartyideology.10Likewise,anyempiricalstudydrawsonatleastoneindicator.Thereareno“nonindicator”studies,becauseindicatorsaretheprimitiveempiricalpropo-sitionsunderlyingallotherpropositions,descriptiveorcausal.Thatistosay,morecomplex,multidimensionalargumentsarecomposedofindicators.Ofcourse,notallindicatorsareexplicitlymeasuredacrossalargenumberofcases.Qualitativeresearchoftenrestsonunmeasuredfactorsorfactorsthataremeasuredforonlyafewcases.Yetthisdoesnotdispeltheirimportance,ortheirdifficulty.IfacasestudyofAngolaassertsthatthiscountryhasa“high”mortalityrate,thisisaqualitativejudgmentthatrestsonabroadersetofcomparativereferencepoints(presumablyothercountriesintheregionandtheworld).Forourpurposes,thisisalsoanindicator,despitethefactthatitisnotassociatedwithprecisemeasurement.SynthesesAsynthesisisamultidimensionalcategoryinwhichdiverseattributesaresaidtorevolvearoundacentraltheme.Thetheme,usuallyexpressedinasingleconcept,unifiestheattributes,thuslendingcoherencetoanotherwisedis-paratesetofphenomena.Asyntheticargumenttherebyoffersanexplanation5MarshallandJaggers(2007).6SeeFreedomHouseat:www.freedomhouse.org.7SingerandDiehl(1990).8Kaufmann,Kraay,andMastruzzi(2007).9SamuelsandSnyder(2001).10Forexample,Budge,Robertson,andHearl(1987);Laver,Benoit,andGarry(2003);PooleandRosenthal(1985).143DescriptiveargumentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:08 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.009Cambridge Books Online © Cambridge University Press, 2012
forthephenomena,thoughnotofacausalnature.Thereisnoexplicitattempttodistinguishacauseandaneffect.Instead,thesynthesisembracesevery-thing(oratleastmanythings)withinitsdomain.Thesynthesisisthereforeaholisticendeavor,emphasizingsimilaritiesratherthandifferencesamongthechosensampleofcases.Typically,thisconceptualumbrellaisabstractenoughtorequireextensiveeffortsatdefinitionandoperationalization.Afewexampleswillsufficetoillustratethegenre.Consider,first,thevarietyofcompetingargumentsaboutAmericanpoliticalculture,introducedabove:egalitarian-liberal-individualist;11republican;12oracombinationofmultipletraditions,includingthatwhichSmithdescribesasascriptive.13Consider,second,theroleoftheAmericanPresident,whichRichardNeustadtlikenedtotheofficeof“clerk,”sincehispowerresideslargelyinpersuasionratherthancommand.14Consider,third,thetopicofnationalism,whichaccordingtoBenedictAndersondrawsonimaginedcommunities.15Consider,fourth,theidea(creditedtoJamesScott)thatpeasantsinresource-threatenedenviron-mentsareimbuedwithamoral,ratherthanstrictlyinstrumental,viewofmarketbehavior.16Consider,finally,OrlandoPatterson’sargumentthatslav-eryisaformofsocialdeath.17Thesearealldescriptivesyntheses.Theyaresyntheticinsofarastheyaimtosummarizemanyattributesandmanyphenomenainasingleconceptorphrase.Ofcourse,theattempttosynthesizeisalso,atthesametime,anattempttodifferentiate.Forexample,theliberalismofAmericanculture(accordingtoTocquevilleandHartz)iscontrastedtothenonliberalculturesofEurope.Insofarasthesedistinctionsareexplicitandinsofarastheyprovidethegristforextensiveempiricalanalysisasynthesisbeginstolookmorelikeatypology–ournexttopic.(Indeed,inalaterstudyHartzapplieshis“frag-ment”thesistosettlersocietiesintheUnitedStates,Canada,Australia,andSouthAfrica.18Here,thecomparisonsacrosscasesareexplicit,andtheresultingstudyisrightlyclassifiedastypological.)TypologiesTypologiesresolvecasesintodiscretecategoriesthataremutuallyexclusiveandexhaustiveonthebasisofauniformcategorizationprinciple(s).19These11Hartz(1955);Tocqueville(1945).12Pocock(1975);Shalhope(1972);Wood(1969).13Smith(1993).14Neustadt(1960).15Anderson(1991);Gellner(1983).16Scott(1976).17Patterson(1982).18Hartz(1964).19Confusingly,threewordsareoftenusedsemi-synonymously:typology,classification,andtaxonomy.Inmyadoptedusage,“taxonomy”referstoaspecifickindoftypology.Forworkontheseinterrelated144PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:08 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.009Cambridge Books Online © Cambridge University Press, 2012
comeinseveralcommonvarieties:(a)simple,(b)temporal(periodization),(c)matrix,(d)taxonomy,(e)configurational,and(f)sequential.SimpletypologyAsimpletypologyfollowsonlythegeneralrulesforatypology,asexplainedabove.Letusexploreafewexamples.PolitiesmaybeclassifiedinAristotelianfashionasmonarchies(ruleofone),oligarchies(ruleofafew),anddemoc-racies(ruleofmany).Historicalpolities,arguesSamuelFiner,maybeclassi-fiedaccordingtotheirrulersaspalace,church,nobility,orforum.20AlbertHirschmanarguesthattheinfluenceofconstituentsonorganizationsmaybefeltthroughexitand/orvoice.21MaxWeberarguesthatpoliticalauthoritydrawsuponthreeformsoflegitimacy:traditional,charismatic,andrational-legal.22GostaEsping-Andersendividestheworldofwelfareregimesintothreesorts:liberal,corporatist,orsocialdemocratic.23TheodoreLowifindsthatthepoliticsofpublicpolicyfollowsoneoffourlogics:distributive,constituent,regulative,orredistributive.24Notethatwhilemosttypologiesassumetheformofanominalscale,somerevealanimplicitorexplicitrankingamongcategories,qualifyingthetypol-ogyasanordinalscale(Chapter7).Forexample,Aristotle’sclassificationofpolitiesmightbeviewedasestablishinganordinalscaleofgreaterorlesserpopularinvolvementinpolitics.TemporaltypologyTemporaltypologies(akaperiodizations)aresimpletypologiesthataretem-porallyordered.Forexample,itisarguedthatseveralwavesofdemocratiza-tionhavebrokenovertheworldinthecourseofthepasttwocenturies,eachwithdistinctivefeatures.25AnevenbroaderattemptathistoricalperiodizationisTocqueville’sproclamationofademocraticage,beginningsometimeinthelateeighteenthcentury,whichmaybecomparedwiththepreviousfeudaloraristocraticages.Alongtheselines,Marxproposedtotypologizerecordedhumanhistoryintofeudal,capitalist,andcommuniststages.Otherperiodizationschemesfocusonasinglecountry.Forexample,manystudentsofAmericanpoliticalhistoryareconvincedthatfundamentalsubjectsseeBailey(1972);Capecchi(1968);Collier,LaPorte,andSeawright(2008);Elman(2005);GeorgeandBennett(2005:ch.11);LangeandMeadwell(1991);Lenski(1994);Lijphart(1968);McKinney(1950,1957,1969);Nowotny(1971);Smith(2002);Whittaker,Caulkins,andKamp(1998);Wiseman(1966).20Finer(1997).21Hirschman(1970).22Weber([1918]1958).23Esping-Andersen(1990).24Lowi(1972).25Doorenspleet(2000);Huntington(1991).145DescriptiveargumentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:08 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.009Cambridge Books Online © Cambridge University Press, 2012
politicalchangeshaveoccurredonlyepisodically,during“realignment”per-iods.26Othersdefendanoldertradition,dividingAmericanpoliticalhistoryinto“eras”(Revolutionary,Jacksonian,CivilWar,Reconstruction,etc.).Stillothersarguethatthetopicisbestapproachedthroughanevenmorediffer-entiatedperiodizationdefinedbypresidencies.27Eachattempttoestablishatemporaltypologyappealstothesamegeneraldesiderata,thatis,toidentifykeypointsofchangewithinahistoricaltopicsuchthattheresultingperiodsaremutuallyexclusiveandexhaustive(alongwhateverdimensionsareofinteresttothestudy).MatrixtypologyMatrixtypologiesaresubjecttosimilarcriteria,butareformedinamorecomplicatedfashion.Here,thecategoriesofatypologyaretheproductofanintersectionofseveralcategoricalvariables.Suppose,forexample,onebeginswithtwocomponentsofdemocracy,contestationandparticipation,whichweassumevaryindependentlyandcanbecodeddichotomouslywithouttoomuchlossofinformation.Theintersectionofthesetwofactorsproducesfourtypes,whichRobertDahlhaslabeled(a)closedhegemony,(b)inclusivehegemony,(c)competitiveoligarchy,and(d)polyarchy,asillustratedinTable6.2.28Notethatmatrixtypologies,likesimpletypologies,oftenproduceorderedcategories,andthusareordinalscales.Inthisexample,polyarchyisthemostdemocraticandclosedhegemonytheleastdemocratic.However,thematrix,byitself,doesnotrevealanorderingamongtheothertwocells.29Table6.2Amatrixtypology:regimetypesParticipationLowHighContestationLowClosedhegemonyInclusivehegemonyHighCompetitiveoligarchyPolyarchy26Sundquist(1983).27Fordiscussion,seeCochran(1948),Zelizer(2002).28Dahl(1971:7).AnotherexampleofamatrixtypologyisAristotle’sancienttypologyofregimetypes(Lehnert2007:65).Here,thenumberofrulers(one,afew,ormany)iscross-tabulatedwiththerulers’goals(self-interestorthegreatergood)toproducesixcategories:tyranny,oligarchy,democracy,monarchy,aristocracy,andpolity.AdditionalexamplesofmatrixtypologiesrelatedtotheconceptofdemocracycanbefoundinAlmondandVerba([1963]1989:16);Weyland(1995).29Collier,LaPorte,andSeawright(2008:157).146PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:08 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.009Cambridge Books Online © Cambridge University Press, 2012
Notealsothatmatrixtypologiesmaycontainanynumberoffactors,resultinginanynumberofcompoundtypes(cells).However,thetwo-by-twomatrixisstillthemostcommon–presumablybecauseaddingathird(orfourth)dimensiondoesnotusuallycreatediscreteandrecognizabletypes.TaxonomyTaxonomiesaretypologiesthatstretchinahierarchicalfashionacrossseverallevelsofanalysis.Accordingly,onemightstipulatethattherearetwobasicpolitytypes:autocracyanddemocracy.Amongdemocracies,somearedirectandothersrepresentative.Amongrepresentativedemocracies,onefindselectoral,liberal,majoritarian,participatory,andegalitarianvarieties.Thenestedqualityofthisfamilyoftermsmaybeillustratedintabularformat(seeTable6.3)orinatreediagram(seeFigure6.1).30Notethateachsubordinatelevelofthetaxonomypossessesalltheattri-butesofthesuperordinatecategory,plusone(orseveral).Eachconceptwithinataxonomymaythereforebedefinedbyspecifyingitssuperordinatecategoryplusitsdifferentiatingattributeorattributes–itsgenusetdiffer-entium.(Conceptssodefinedaresometimesdescribedas“classical”inrefer-encetotheirAristotelianlineageandtheirvenerableplacewithinthefieldoflogic.)ConfigurationaltypologyConfigurationaltypologies,liketaxonomies,formsubtypesoutofasinglesuperordinatecategory.However,subtypesarecreatedfromasuperordinatecategorybysubtracting,ratherthanadding,attributes.Thisgeneratesdimin-ishedsubtypes–sometimescalledradialcategories–ratherthanaugmentedsubtypes(asinthetaxonomy).Thesesubtypesradiateoutwardfromthesuperordinatecategory,whichtakestheformofanideal-type(Chapter5).31Inthisfashion,itissometimesarguedthatdemocracyisbestunderstoodasasetofrelativelydistinctmodels–electoral,liberal,majoritarian,participa-tory,deliberative,andegalitarian(orsocial)–eachemphasizingadifferentaspectofthekeyterm.32Asanideal-type,thesuperordinatecategorycontainsalltheattributesofthesubtypes.Thesubtypes,however,possessonlyone(orsome)oftheattributesoftheideal-type,asillustratedinTable6.4.30Asasecondexample,onemightconsiderReynolds’andReilly’s(2005:28)taxonomyofelectoralsystems.ThestillclassicexampleofataxonomyistheLinnaeansystemofbiologicalclassification(LinsleyandUsinger1959).31CollierandMahon(1993);Lakoff(1987).32CoppedgeandGerring(2011).SeealsoHeld(2006).147DescriptiveargumentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:08 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.009Cambridge Books Online © Cambridge University Press, 2012
Table6.3AtaxonomyintabularformatATTRIBUTESCONCEPTSFormofgovernmentRulebyfewRulebythepeopleDirectIndirectElectionsRuleoflawMajorityrulePopularparticipationConsultativebodiesEqualityTotal(I)PolityX1(A)AutocracyXX2(B)DemocracyXX2(1)DirectXXX3(2)RepresentativeXXX3(i)ElectoralXXXX4(ii)LiberalXXXX4(iii)MajoritarianXXXX4(iv)ParticipatoryXXXX4(v)DeliberativeXXXX4(vi)EgalitarianXXXX4Downloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:08 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.009Cambridge Books Online © Cambridge University Press, 2012
PolityAutocracyDemocracyDirectRepresentativeElectoralLiberalMajoritarianParticipatoryEgalitarianDeliberativeFigure6.1Ataxonomyintree-diagramformatDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:08 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.009Cambridge Books Online © Cambridge University Press, 2012
SequentialtypologySequential(akaprocessual33)typologies,likesimpleperiodizations(temporaltypologies),areorganizedalongatemporalaxis:timematters.However,whiletheperiodizationsimplyassertsthattimeperiodsaredifferent(inwaysdefinedbythetypology),asequentialtypologyassertsthatthesequenceofeventsmattersforsubsequentevents.Liketaxonomies,thesequentialtypologymaybediagramedinatreefashion.Buthere,thebranchesrepresenttemporalsequences(whichmayormaynotembodytaxonomicfeatures).Theclassicexampleisthe“cladistics”methodofbiologicalclassification,whichsupposesthateachbranch(“clade”)inthetreeoflifeisdistinctivesuchthatspeciessharingthesamepointoforiginalsosharesignificantbiologicalcharacteristics(aclaimthathassubse-quentlybeendisputed).34Inthesocialsciences,claimsassociatedwithsequentialtypologiesaregenerallymoremodestandthetypologieslesselaborate.Still,manywritersassertthatthesequenceofeventsmatters,andthatthesesequenceeffectscanbetheorizedinageneralfashion.35Forexample,T.H.Marshallarguesthatdemocraticdevelopmentischaracterizedbythreephases:civil,political,andsocial.36W.W.Rostowconceivesofmodernizationasafive-stageprocess:“thetraditionalsociety,thepreconditionsfortake-off,thetake-off,thedrivetomaturity,andtheageofhighmass-consumption.”37Here,theauthors’meth-odsarequalitative.Wherelargesamplesareavailable,sequentialtypologicalTable6.4Aconfigurationaltypology:ideal-typeandradialcategoriesATTRIBUTESCONCEPTSCompetitiveelectionsRuleoflawMajorityrulePopularparticipationConsultativebodiesEqualityTotalDemocracyXXXXXX6ElectoralX1LiberalX1MajoritarianX1ParticipatoryX1DeliberativeX1EgalitarianX133Nowotny(1971:24–29).34Gould(1983:ch.28).35Falleti(2010:chs.1–2);Pierson(2004);Rueschemeyer,Huber,andStephens(1992);Shefter(1994).36Marshall(1964).37Rostow(1960:4).150PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:08 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.009Cambridge Books Online © Cambridge University Press, 2012
featuresmaybediscernedwithvariousstatisticaltechniques,asdevelopedbyAndrewAbbottandothers.38AssociationsAssociationaldescriptionsareprobabilisticmultidimensionalcomparisonsacrossindicatorsorunits.Thereis,inotherwords,noattempttoachievethecrispfeaturesofatypology.Threemajorsubtypesmaybedistinguished.TrendAtrendisanassociationbetweenaphenomenonandtime.Ifthedimensionofinterestiscorrelatedinsomemannerwithtime(e.g.,linearlyornon-linearly,monotonicallyornonmonotonically,cyclicallyornoncyclically),wesaythatthedataistrended.Reachingadeterminationonthesemattersmaybeassistedbyawidevarietyoftime-seriesmodelingtechniquesaswellasbyvisualgraphs.39Leavingasidetheirobviousutilitytocausalinference,trendsprovideinsightintothenatureofprocessesandsometimesallowustoextrapolateintothefuture.Itisnotsurprisingthatagooddealofinvestigatoryresearchhasbeendevotedtothediscoveryoftrendsinthesocialsciences.Wehavealreadynotedthedebateoverglobalinequality.40SomeseesignsofaseculardeclineinsocialcapitalwithintheUnitedStates,41whileothersviewthisapparenttrendasanartifactofmeasurementerrorand/oranunusualpointofinitialcomparison.42Somearguethatpolicy-makingtrendsalignwithamodelofpunctuatedequilibrium,withlongperiodsofstasisorincrementalchangefollowedbyshortburstsofcomparatively“revolutionary”change.43Othersviewpolicy-makingasincremental44orstochastic,andthereforenotcharacterizedbyanyclearandpersistenttrends.45NetworkAnetworksignifiesanassociationinwhichtheinterrelationshipamongmultipleunitsformsthetopicofinterest.Thismaybeunderstoodinspatial,temporal,orfunctionalways.Anetworkanalysismightfocusonthedistancebetweenvariousunits(understoodasmeandistance,totaldistance,shortest38Abbott(1995,2004);AbbottandForrest(1986);AbbottandTsay(2000);Everitt,Landau,andLeese(2001:ch.4).39Hamilton(1994).40ContrastMilanovic(2005)andDollar(2005).41Putnam(2001).42Paxton(1999).43BaumgartnerandJones(1993).44Lindblom(1979).45Cohen,March,andOlsen(1972);Kingdon(1984).151DescriptiveargumentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:08 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.009Cambridge Books Online © Cambridge University Press, 2012
distance,andsoforth).Itmighttrytoestimatethetimeelapsedbetweenseparateevents(understoodasmean,total,shortest,longest,andsoforth).Oritmightfocusonfunctionalinterrelationships,forexample,trade,diffusionofideasandpractices,conflict,andsoforth.AmoredifferentiatedtypologyisprovidedbyDavidKnokeandSongYang,whocategorizenetworkrelationshipsastransactions(“actorsexchangecon-troloverphysicalorsymbolicmedia,forexample,ingiftgivingoreconomicsalesandpurchases”);communications(“linkagesbetweenactorsarechannelsthroughwhichmessagesmaybetransmitted”);boundarypenetration(“tiesconsistofmembershipintwoormoresocialformations,forexample,corporationboardsofdirectorswithoverlappingmembers”);instrumental(“actorscontactoneanotherineffortstosecurevaluablegoods,services,orinformation,suchasajob,abortion,politicaladvice,orrecruitmenttoasocialmovement”);sentimental(“actorsexpresstheirfeelingsofaffection,admira-tion,deference,loathing,orhostilitytowardoneanother”);orkinship(“bondsofbloodandmarriage”).46Oneprominenttypeofnetworkisgeographicinnature,focusedonthespatialrelationshipsamongunits.Historiansoftenfinditusefultomapchangesthroughtime,providingaspatialrepresentationofhistory.47Socialscientistsarekeentomapspatialinequalitiesinincome,wealth,innovation,technology,andhealthacrosscountriesandregions.48Especiallycloseatten-tionispaidtotradenetworks.49Politicalscientistsandsociologistsareofteninterestedinspatialpatternsofglobalhegemony.50Sociologistshaveexaminedthespreadofreligionthroughtimeandspace.51Spatiallyorderedprocessesofchangeareoftendescribedasdiffusion(amechanismthatsitsastridethedescriptive–causaldivide).52Thediffusionofdemocracyhasinspiredagooddealofwork.53Wherepreciselocationsareimportant,theempiricalcomponentofaspatialnetworkmaybeplottedinaGeographicInformationSystem(GIS)format.Thisprovidesastandardizedmethodforrecordingthelocationofunitsandevents,andisincreasinglyprominentintheworkofsocialscientists.54Networks,likeotherempiricalpatterns,maybeprobedqualitativelyorquantitatively.Statisticalmodelsareadvisablewhereverdataissufficiently46KnokeandYang(2008:12).SeealsoWassermanandFaust(1994).47Knowles(2008).48Clark,Gertler,andFeldman(2000);GoeslingandFirebaugh(2000);KanburandVenables(2005).49KimandShin(2002).50Wallerstein(1974).51Montgomery(1996).52Henisz,Melner,andGuillén(2005).53BrinksandCoppedge(2006);GleditschandWard(2006).54GregoryandEll(2007).152PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:08 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.009Cambridge Books Online © Cambridge University Press, 2012
numerousandsufficientlycomplexthatpatternsofinteractionsurpassthatwhichcanbeanalyzedthroughinformalmethods.55Asanexample,letusconsiderthequestionofpolicynetworks.Weknowthatagreatmanypeopleareconsultedinthecourseofpolicydeliberations,especiallyinmassdemoc-racies.Yetweknowverylittleabouttheprecisenatureandshapeofthesenetworks(despiteagooddealoftheorizing).AstudybyHeinzetal.attemptstoshedlightonthisimportantquestion.Onthebasisofarelativelycompre-hensivenetworkanalysisoffourissueareasinthecontemporaryUnitedStatestheauthorsconcludethatpolicynetworkshavenoconsistentcenter.Rather,theyarehighlydispersedandmuchmorerandomthanwritersonthesubjecthadpreviouslysurmised:a“hollowcore”ratherthanan“irontriangle.”56CorrelationAcorrelationalstyleofargumentreferstoanymultidimensionalassociationthatisnotfoundedontrendsornetworks.Admittedly,thisisalargeresidualcategory.However,itcorrectlydescribesagooddealofthedescriptiveworkinthesocialsciences,andthereseemsnobetterwaytodescribeit.Acorrelationalargumentmight,forexample,focusonthequestionofwhetherdemocraciesarelesslikelytopersecuteminoritiesthanautocracies.Oritmightfocusonfeaturesofanindividualpolity.Issupportfordemocracycorrelatedwithincome?Isthestructureoforganizedintereststilted–inmembership,staffing,andmission–towardthemiddleandupperclasses?Inotherwords,doespoliticalpositionco-varywithsocialclass?57Areelitesmore“ideological”intheirviewofpoliticsthanrank-and-filevoters?58Theseareallcorrelationalargumentsinsofarastheeffortistodemonstrateamulti-dimensionalassociationalpatternwithoutcausalassumptions.Thatis,theassociationmaybetheproductofsomeunderlyingcausalfactor(s),buttheauthorisnotstakinganyclaimsofthissortbecausethecorrelationisinter-estingandimportantinitsownright(andthecausalforcesatworkmaybedifficulttouncover).ConclusionsThereareaninfinitenumberofwaystodescribeagivenreality.However,ifthewriter’spurposeistogeneralizeacrossapopulation,adescriptionislikely55KnokeandYang(2008);WassermanandFaust(1994).56Heinzetal.(1993).57Schattschneider(1960);Verba,Schlozman,andBrady(1995).58McClosky,Hoffmann,andO’Hara(1960).153DescriptiveargumentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:08 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.009Cambridge Books Online © Cambridge University Press, 2012
totaketheformofanindicator,asynthesis,atypology,oranassociation–eachpresupposingadifferentsetofcriteria.Descriptionisthusunifiedbyseveralcommoncriteria(capturedinTable6.1)anddifferentiatedbycriteriapertainingtoeachdescriptivestrategy(assummarizedinTable6.1).Additionalcriteriapertaintoeachelementofadescriptiveargument–theconceptsemployed(Chapter5)andthemeasure-mentstrategiesenlisted(Chapter7).154PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:08 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.009Cambridge Books Online © Cambridge University Press, 2012
Cambridge Books Onlinehttp://ebooks.cambridge.org/Social Science MethodologyA Unified FrameworkJohn GerringBook DOI:http://dx.doi.org/10.1017/CBO9781139022224Online ISBN: 9781139022224Hardback ISBN: 9780521115049Paperback ISBN: 9780521132770Chapter7 – Measurements pp. 155-194Chapter DOI:http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge University Press
7MeasurementsGrown-upslovefigures.Whenyoutellthemthatyouhavemadeanewfriend,theyneveraskyouanyquestionsaboutessentialmatters.Theyneversaytoyou,“Whatdoeshisvoicesoundlike?Whatgamesdoeshelovebest?Doeshecollectbutterflies?”Instead,theydemand:“Howoldishe?Howmanybrothershashe?Howmuchdoesheweigh?Howmuchmoneydoeshisfathermake?”Onlyfromthesefiguresdotheythinktheyhavelearnedanythingabouthim.Ifyouweretosaytothegrown-ups:“Isawabeautifulhousemadeofrosybrick,withgeraniumsinthewindowsanddovesontheroof,”theywouldnotbeabletogetanyideaofthathouseatall.Youwouldhavetosaytothem:“Isawahousethatcost$20,000.”Thentheywouldexclaim:“Oh,whataprettyhousethatis!”Justso,youmightsaytothem:“Theproofthatthelittleprinceexistedisthathewascharming,thathelaughed,andthathewaslookingforasheep.Ifanybodywantsasheep,thatisaproofthatheexists.”Andwhatgoodwoulditdototellthemthat?Theywouldshrugtheirshoulders,andtreatyoulikeachild.Butifyousaidtothem:“TheplanethecamefromisAsteroidB-612,”thentheywouldbeconvinced,andleaveyouinpeacefromtheirquestions.Theyarelikethat.Onemustnotholditagainstthem.Childrenshouldalwaysshowgreatforbearancetowardgrown-uppeople.Butcertainly,foruswhounderstandlife,figuresareamatterofindifference.Ishouldhavelikedtobeginthisstoryinthefashionofthefairy-tales.Ishouldhavelikedtosay:“Onceuponatimetherewasalittleprincewholivedonaplanetthatwasscarcelyanybiggerthanhimself,andwhohadneedofasheep…”Tothosewhounderstandlife,thatwouldhavegivenamuchgreaterairoftruthtomystory.AntoinedeSaint-Exupéry1TheLittlePrincearticulatestheinvidious,de-humanizingelementinherentinanyattempttomeasure,andtherebycompare,humanbeings.“Treatingthemlikestatistics,”asthephrasegoes.Abhorrentthoughitmayseem(andsurely,themeasurementofintimatematerialandemotionalstatesisanactofextreme1deSaint-Exupéry([1943]1971:16–17).IwasledtothispassagebyFreedmanetal.(1991:29).155Downloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
hubris),theremayalsobegoodreasonsformeasuring,say,theincomesoffamiliesinacommunity.Measurementmightberegardedastheanalysisphaseofdescription.Itisherewheretheresearchermakescontactwithempiricalreality(onehopes).Itisherethatconcepts(Chapter5),andthelargerdescriptiveargumentstheysitwithin(Chapter6)areoperationalized.Ofcourse,arguing,conceptualizing,andmeasuringblendintooneanother,andthismeansthatthereisconsiderableoverlapamongtopicsdiscussedinPartIIofthebook.Thischaptershould,therefore,beunderstoodasacon-tinuationoftopicsbroachedinpreviouschapters.Chapters5and6haveamoredeductiveflavor,whilethischapterhasamoreinductiveflavor.Butthemoreimportantpointisthatnoneofthesetopicscanbeneatlyseparatedfromtheothers.Investigationsalwayscontainamixofdeductiveandinductivecompo-nents–usually,thereisacontinualbackandforth.Itisimpossibletodefineaconceptwithoutsomesenseoftheempiricalterrain,anditisimpossibletooperationalizeaconceptwithoutsomesenseofhowtheconceptisdefined.Conceptsandperceptsareinseparable;onecanhardlybeconsideredwithouttheother.Thetaskofmeasurementmaybedefinednarrowlyas“theassignmentofnumberstoobjectsoreventsaccordingtorules”2or,morebroadly,as“theprocessoflinkingabstractconceptstoempiricalindicants.”3Forourpurposes,abroaderdefinitionisappropriate.Indeed,itisnotclearwhatassigningnumberstoobjectswouldmean,unlessconnectedtoidentifiableconcepts(numbersofwhat?).Thecriticalquestion,inanycase,ishowwerecognizeaconceptwhenweseeit.Candemocracybedistinguishedfromautocracy?Canpowerbedistinguishedfrompowerlessness?Whatdotheseconceptsmeanempirically?Intimatelyrelatedtothetopicofmeasurementistheadjoiningtopicofdatacollection.Whenoneiscollectingdatainasystematicfashion–thatis,dataorganizedaroundselectedconceptsoftheoreticalinterest–onefacesthechallengeofmeasurement.Measurementandsystematicdatacollectionarethereforevirtuallysynonymous.(Theyare,ofcourse,quitedifferentfromanadhoccollectionofdata.)Sodefined,thetopicofmeasurementisvastandunbounded,extendingintoallterrainsofsocialscience.Eachfieldandsubfieldoffersitsownchallenges,2Stevens(1951:22).3CarminesandMeller(1979:10).Inthisvein,seeSeawrightandCollier(2004:295).AdditionalworkonmeasurementinthesocialsciencesincludesAdcockandCollier(2001);Bartholomew(2007);Blalock(1982);Boumans(2007);Duncan(1984);Goertz(2006);Jackman(2008);Kempf-Leonard(2004);Krantzetal.(1971,1989,1990);Reiss(2007).156PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
andeachhasgenerateditsownliterature.Arguably,thisisthemostcontext-specificofallthesocialsciencetasksdiscussedinthisbook.Thatsaid,therearesomecommonchallenges.Theproblemofmeasure-mentstemsfromthefactthatmost(andperhapsall)importantsocialscienceconceptsarenotdirectlyobservable.Theyarelatent.Allabstractconceptsfallintothiscategory.Wecannot“see”justice,democracy,governance,orpower.Withrespecttothelatter,RobertDahlwrites,“Thegapbetweentheconceptandoperationaldefinitionisgenerallyverygreat,sogreat,indeed,thatitisnotalwayspossibletoseewhatrelationthereisbetweentheoperationsandtheabstractdefinition.”4Evensomethingasconcreteasavouchersprogramcannotbedirectlyobserved.Manytermsinthesocialsciencelexiconsufferfromthisproblem.Alienation,anomie,charisma,civilsociety,collectiveconscience,crisis,culture,democracy,dogmatism,equality,falseconscious-ness,hegemony,ideology,legitimacy,masssociety,nationalcharacter,patternvariable,pettybourgeois,rationalization,sovereignty,state,andstatusanxietyareall“fuzzy”concepts.Wemaybeabletodefinetheminageneralway,butwehaveimmensedifficultylocatingtheirreferentsinempiricalspace.5Thesearethesortsofmeasurementproblemsthatsocialscienceisatpainstoresolve.6Thedifficultyofmeasurementinthesocialsciencesalsostemsfromtherecalcitrantnatureofoursubjectmatter.Recallthatwehavedefinedsocialscienceasthestudyofhumanaction:behaviorthatisinsomedegreedecisional(Chapter1).Humanactionisthereforeinfusedwithactor-definedmeaningsandmotivations,andthisraisesasetofmeasurementchallengesthataredistinctivetothesocialsciences.Inspecificterms,whatwearedealingwithisasetofphenomenathataresensitiveand/ordifficulttointerpret.Thetwineffectistoobscureagooddealoftheideasandactionsofinteresttosocialsciencefromthepryingeyes,andabstractclassificatorycategories,of4Dahl(1968:414),quotedinDebnam(1984:2).5Geddes(1996:5)notesthat“stateautonomy”isgenerally“inferredfromitseffectsratherthandirectlyobserved.Noone,itseems,isquitesurewhat“it”actuallyconsistsof.Stateautonomyseemsattimestorefertotheindependenceofthestateitself,theregime,aparticulargovernment,somesegmentsoragenciesofthegovernment,orevenspecificleaders.Itseemsthephrasecanrefertoanyindependentforcebasedinthecentralgovernment.6Recallthatallsocialscienceconceptsaspiretocapturesomethingrealintheworldaroundus.Thereferentmaybehighlyattenuated,butitisnonethelessalwayspresent.Themoreeasilythesereferentscanbelocatedanddifferentiatedfromothersimilarreferentsthemoreusefulthatconceptwillbe,ceterisparibus.Aconceptofdemocracythatcannottelluswhichphenomenaaredemocraticandwhicharenotislessusefulonthataccount.Conceptsofjustice,capitalism,socialism,ideology,oranythingelseinthesocialscienceuniversearesubjecttothesamedemand.“Conceptswithoutperceptsareempty;perceptswithoutconceptsareblind,”notesKant(quotedinHollis1994:71).157MeasurementsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
researchers.Ishallrefertothismeasurementchallengeashermeneuticorinterpretive.7Forexample,ininvestigatingthetopicofcorruptionwefacetheproblemthatthosewhoengageincorruptactivitiestryhardtoconcealtheseactionsand,perhapsequallyimportant,areofteninformedbydifferentdefinitionsofcorrup-tion.Thelatterissueisimportantbecauseitbearscentrallyonhowwemightinterpret“corrupt”behavior.Itisquitedifferentifanactofpatronageisseenasamoralobligation(e.g.,tohelpkithorkin)ratherthananactofself-advancement.Becausequestionsofmeaningandintentionalityareoftencentraltoourunder-standingofaphenomenontheyarealsocentraltotheproblemofmeasurement.8(Idonotmeantoimplythattheunderstandingsofactorsarealwayscriticaltoproblemsofmeasurement.Sometimes,itissufficienttoknowwhetherornotanactionhastakenplacewithoutworryingaboutthemeaningsascribedtoit.)Thesamedifficultiesareencounteredwithmanyothersocialsciencesub-jects,forexample,clientelism,crime,democracy,discrimination,economicoutput,happiness,humanrights,identity,ideology,intelligence,nationalism,prejudice,publicopinion,utility,andwellbeing.9Wehavetroublemeasuringthesethingsbecauseactorshavestrongincentivestomis-representthemselvesandbecausetheseactorsoftenhavedifferingunderstandingsoftheirownactionsorexperiences.Addingtoourperplexity,issuesofduplicityandperspectiveareoftendifficulttodisentangle.Itishardtotellwhensomeoneis(a)lyingor(b)tellingthetruthfromadifferentangle.Ibeginbyreviewinggeneralcriteriathatallmeasurementsseektoachieve.Next,Idiscussvariousstrategiesofmeasurement.Finally,Iofferabriefreviewofexpostvaliditytests(waysofjudgingthevalidityofindicatorsoncetheyarearrivedat).CriteriaInpursuingthetaskofmeasurement,twooverallgoalsareubiquitousandparamount:reliability(akaprecision)andvalidity.Thesecriteriawereintroduced7ThedifficultiesofobtainingsensitiveinformationareaddressedinLee(1993).ThehermeneutictaskisaddressedfromphilosophicalandempiricalanglesbyGadamer(1975);Geertz(1973);RabinowandSullivan(1979);Taylor(1985);vonWright(1971);Winch(1958);YanowandSchwartz-Shea(2006).8Issuesofintentionalityare,ofcourse,centraltotheinterpretivisttradition.However,theyarebynomeansincidentaltothepositivisttradition.Indeed,theyarecentraltothepracticeofsurveyresearch(Chong1993;Kritzer1996;Schwartz1984;Stoker2003).9Detailedexpositionsofproblemsofmeasurementconnectedtoparticularconceptsareworthconsulting,e.g.,onwellbeing(GoughandMcGregor2007),identity(Abdelal,Herrera,andJohnston2009),happiness(BertrandandMullainathan2001),andcorruption(reviewedinthefinalsectionofthischapter).158PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
initiallyinChapter4.Here,weareconcernedsolelywiththeirapplicationtoproblemsofmeasurement.Precision–usuallyunderstoodasreliabilityinmeasurementcontexts–referstolevelofstochastic(random)error,ornoise,encounteredintheattempttooperationalizeaconcept.Thisisoftenassessablethroughreliabilitytests.Ifmultipleapplicationsofameasurementinstrumentrevealahighlevelofconsistencyonemayregardthechosenmeasureasreliable(precise).Levelsofreliabilityaretypicallycalculatedastheinverseofthevariance(i.e.,dispersionaroundthemean)acrossmeasurements.Greatervariancemeanslessreliability.Testsdepend,ofcourse,onthespecificinstrumentthoughwhichthemea-surementisobtained.If,forexample,theinstrumentinvolvescodingthentrialsmayconsistofinter-coderreliabilitytestsconductedonthesamematerial.Surprisingly,suchtestsarenotcommonlyadministeredamongthosewhodevelopandusecross-nationalindicatorsofdemocracy,despitethefactthattheseindicesresttoaconsiderableextentuponcodingdecisions.10Iftheopportunitytotestmultipleiterationsofanindicatorisnotpresentthentheissueofreliabilityremainsatthelevelofanassumption.Butitisnonethelesscrucial.Ahighprobabilityofrandomerrormaydoomeventhesimplestgeneralizationabouttheworld.Moreover,iftheconceptformsthebasisforsubsequentcausalanalysisthenerrorsassociatedwithacausalfactor(X)arepronetointroducebiasintotheresultinganalysis,generallyattenuat-ingthetruecausaleffectofXonY.11Validityreferstosystematicmeasurementerror,errorthat–bydefinition–introducesbiasintotheresultingconcept(andpresumablyintoanycausalanalysisthatbuildsonthatconcept).Oneoftenfinds,forexample,thatthelevelofaccuracywithwhichanindicatorismeasuredvariesdirectlywithsomefactoroftheoreticalinterest.Forexample,itcouldbethatbetterqualityschoolsarealsomoreconscientiousatrecord-keeping,meaningthatwewillhavemoredata,andmorereliabledata,fromcertainschoolsandthischar-acteristicofthedatawillbecorrelatedwiththeoutcomeofinterest(schoolperformance).Itcouldalsobethatbadmeasurementtoolsofferopportunitiesformis-reportingthatbiasresultsforschoolsacrossthesample,suchthatbadschools(withsloppyaccountingprocedures)reportinflatedschool10FreedomHousedoesnotconductsuchtests,oratleastdoesnotmakethempublic.Politydoesso,butitappearstorequireagooddealofhands-ontrainingbeforecodersreachanacceptablelevelofcodingaccuracy.11ForafulldiscussionseeCoppedge(forthcoming).159MeasurementsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
performance.Thisisthesortofsystematicmeasurementerrorthatresearch-ersmustbeonguardagainst,andforwhichthereisusuallynoeasyfix.Inclarifyingtheseideasitmaybehelpfultoconsidertheproblemofmeasurementindiagrammaticform.Figure7.1representstheunderlyingcon-ceptofinterestasL(i.e.,theterm,itsdefiningattributes,andthephenomenaitisintendedtodescribe),whichisbracketedtoindicateitslatentstatus.TheobservabletraceofLisI,thechosenindicator(s).SourcesofbackgroundnoisenotcorrelatedwithLarerepresentedbyB,anorthogonalcovariate(i.e.,randommeasurementerror).Potentialconfounders,thatis,factorsthatarecorrelatedwithLandwithI,thereforeintroducingsystematicbiasintomeasurement,arerepresentedbyC.Inthissimplifiedschema,thetaskofameasurementinstru-mentistoidentifyanindicator(I)thatiscorrelatedwithL,butnotwithconfounders(C),andminimizesnoise(B).Notethatthepotentialthreatstoinferencearevirtuallylimitless–extending,astheydo,toanythingthatmighthaveacausaleffectonIthatisalsocorrelatedwithL(theunobservableconceptofinterest).Thisincludesinvestigatorbias,afailureininstrumentation(thetechnicalpartofameasurement),andallmannerofcontextualfeatures,includingsimilarphenomenathataredifficulttodistin-guishfromL.Becausethefactoroftheoreticalinterest,L,is(bydefinition)notmeasurableitcannotbedirectlyverified.Thisiswhyitissohardtowriteageneraltreatiseonmeasurement;somuchrestsonmattersofassumption.Butletusbrieflyconsidersomeoftheobstacles.Measurementisaninherentlycomparativeventure.Itpresumesascale:thatis,astandardmetricbywhichheterogeneousthingscanbesystematicallyandpreciselycompared.Hence,allattemptsatmeasurementfaceaproblem[L]ICB?[L] = Latent concept of interestI = IndicatorB = Covariate (source of noise, i.e., unreliability)C = Confounder (source of bias, i.e., invalidity)General featuresCausal relationshipCovariation (possibly causal)Figure7.1Ameasurementdiagram160PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
ofequivalenceorconsistencyacrosscontexts.Achosenmeasuremustmeanthesamething,andmustadequatelyrepresenttheconceptoftheoreticalinterest,acrossallcontextstowhichitisbeingapplied.Thechallengeisthere-foretofindawaytocomparethingsacrossdiversecontextswithouttoomuchlossofmeaningordistortion.Considertheconceptofcorruption.Thisabstractconceptiseasiertooper-ationalizeifonefocusesinonparticulartypesofcorruption,forexample,bribesgivenbybusinessmentogovernmentofficialsforthepurposeofobtainingabusinesslicense.Evenso,thereareconsiderabledifficultiesifonewishestocomparethisindicatoracrosscountriesonaglobalscale.First,reportsofcorruptactionsareprobablynotgoingtobedirectlyobservable,andwillthereforerelyonsurveysofbusinessmen.This,inturn,introducesmultiplepotentialsourcesofbias(systematicerror).Businessmeninonecountrymaybemorelikelytoanswerquestionsaboutbribesinamoreforthrightfashionthanbusinessmeninanothercountry,eveniftheactuallevelofbribe-givingisidentical.(Forexample,greaterfranknessmaybemanifestedindemocraticcountriesthaninauthor-itariancountries.)Second,themeaningandpurposeofthesebribesmaybequitedifferentacrosscontexts,ashasalreadybeenalludedto.Forexample,inonecountryabribemaysignalamoreorlessvoluntarygift,asignofrespectingift-givingcultures.Inanothercountry,abribemaybecoercedbytheofficial.Theseareevidentlyquitedifferentevents,evenifthemonetarytransactionisidentical.Finally,itmaybeproblematictogeneralizefromonespecificindicatorofcorruptiontothelargerconceptoftheoreticalinterest.Thatis,bribesontheoccasionofobtainingabusinesslicensemaynotbeindicativeoftheoverallscaleofcorruptioninacountry:corruptionmayberifeindifferentsectors.(Later,weshallexploresomepossiblesolutionstotheseproblems.)Notethatthequestionofvalidityinmeasurementreferstothecorrespon-dencebetweenaconcept’sdefinition(itsattributes)andthechosenindicators(representedbyI).Assuch,itisneverapurelyempiricalproblem.Considerthatanyconceptcanbeoperationalizedbythesimpleactofstipulatingameasurableindicator.Weproclaim,“LshallbeoperationalizedbyI.”AslongasIismeasurable,onecanclaimthatLhasbeenoperationalized.AndifmultiplemeasuresofLareconsistent,themeasurecanclaimhighreliability.However,itisanothermattertoprovethatIisavalidmeasureofL.PerhapsIcapturesonlyonedimensionofL(amultifacetedconcept),excludingothers.PerhapsImeasuressomethingelseentirely(someconceptotherthanL).Inthesecircumstances,thereisaproblemofconceptualvalidity.Issuesofconceptualvaliditybedevilmostlargesocialscienceconcepts.Letusconsiderourusualexemplar,democracy.Aninfluentialteamofresearchers161MeasurementsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
ledbyAdamPrzeworski(includingMichaelAlvarez,JoseAntonioCheibub,JenniferGandhi,andFernandoLimongi)adoptsaminimaldefinitionofdemocracycenteredontheexistenceofcompetitiveelections.12Toopera-tionalizethisdefinitionthreenecessaryconditionsareposited:(1)thechiefexecutiveandlegislaturemustbeelected;(2)theremustbemorethanasinglepartycompetingintheseelections;and(3)atleastonealternationinpowerunderidenticalelectoralrulesmusttakeplace.13Notethatthissetofdecisionrulesmakesiteasytomeasure.Weanticipatelittledisagreementoverthecodingofcases.Thus,themeasurecanclaimhighreliability.However,twoproblemsofconceptualvaliditycanberaised.Thefirstcon-cernsthefitbetweentheoperationalizationandthechosendefinition.Inparticular,thissetofcodingrulescannotdistinguishbetween(a)apolitywhereasinglepartywinselectionafterelectionunderfreeandfairelectoralrulesandwouldbepreparedtocedepowertoacompetitor(ifitlost)and(b)apolitywhereasinglepartywinselectionafterelectionunderfreeandfairelectoralrulesandwouldnotbepreparedtocedepower.Thishascometobeknownasthe“Botswana”problem.14Thereasonthisposesaproblemofvalidityisthatdemocracyisgenerallyunderstoodtorestuponfreeandfairelections(anunderstandingtowhichPrzeworskietal.alsosubscribe),whichmeansthatthepartywinningtheelectionshouldbeallowedtoassumeoffice.Butwedonothaveempiricalinformationaboutthis,andthecodingrulestreatsituations(a)and(b)asidentical,evenwhenitseemsfairlyobviousthattheyarenot.(Therearesomecountrieswherethecontinualvictoryofapartyisasignalofauthoritarianruleandothersinwhichitprobablyisnot.)Asecondproblemofconceptualvalidityconcernsthewayinwhichthechosenindicatorfitswithintheoverallconceptofdemocracy.Manyunder-standingsofdemocracyextendbeyondtheelectoralfeaturesofapolity,orhaveabroaderviewofwhatelectoraldemocracyisthanthatadoptedbyPrzeworskiandcolleagues,asdiscussedinthepreviouschapter.Inthisrespect,theproblemofvalidityislargelyconceptualinnature.Ithingesuponhowwechoosetodefineakeyterm.Thus,onemightobjectthatalthoughPrzeworskiandcolleagueshaveeffectivelyoperationalizedoneaspectofdemocracytheyhaveneglectedothers,andthereforehaveadoptedaninvalid,oronlypartiallyvalid,measure.Przeworskietal.mightdefendthemselvesbyclaimingtorepresent12Morespecifically,thePrzeworskietal.definitionentails:“(1)Exanteuncertainty:theoutcomeoftheelectionisnotknownbeforeittakesplace,(2)Expostirreversibility:thewinneroftheelectoralcontestactuallytakesoffice,and(3)Repeatability:electionsthatmeetthefirsttwocriteriaoccuratregularandknownintervals”(CheibubandGandhi2004).13CheibubandGandhi(2004:3).14CheibubandGandhi(2004).162PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
onlyonefacetofdemocracy,capturedintheradialconceptofelectoraldemoc-racy.Ortheymightclaimtoadoptaminimaldefinitionofthesubject.Inanycase,debatesovermeasurementvalidityoftenhingeuponhowtolabelanddefinetheconceptofinterest(L);theyareconceptualratherthanpurelyempirical.Here,weencounterwhatmightbecalledthefundamentalproblemofmeasurement:issuesofconceptualvalidity(unlikeissuesofreliability)cannotusuallybetestedempirically,atleastnotformostkeyconceptsofsocialscience,whicharelatentratherthandirectlyobservable.Indeed,ifvaliditycouldbemeasuredtherewouldbenoproblemofmeasurement,forwewouldknowwhatitisthatwewanttoknow.TheproblemofmeasurementliesinthefactthatthecorrelationbetweenLandIinFigure7.1remains–andmustremain–tosomeextenthypothetical.StrategiesWehavesaidthattheproblemofmeasurementinsocialsciencestemsfromthefactthatmostconceptsoftheoreticalinterestarenotdirectlyobservable.Conceptslikeclientelism,crime,democracy,discrimination,economicout-put,andhappinesscannotbecountedlikebicycles.Ofcourse,thereareobservablefeaturesthatwepresumetobeconnectedtotheseconcepts;otherwise,theywouldnotbeamenabletoempiricalinquiryofanysort.Itisfromtheseobservablefeaturesthatweconstructindicators.Inthissection,wereviewvariousstrategiesfordoingso,alongwiththechallengesthateachstrategyentails.Thesestrategiesinvolve:(a)levelsofabstraction;(b)structure;(c)aggregationtechniques;(d)scales;(e)basicobjectives;(f)deductionversusinduction;(g)ethnography;(h)surveysandexperiments;(i)cross-referencing;and(j)causalrelations,assummarizedinTable7.1.Thereadershouldbearinmindthatthesearevastsubjectsandmytreatmentherewillbefocusedontheimplicationsofthesestrategiesforthetaskofmeasurement.Thereadershouldalsobearinmindthatbecausemeasurementisahighlycontextualart,strategiesforoperationalizingcon-ceptsdifferfromfieldtofieldandtopictotopic.Thischapterrestsatafairlygenerallevel(despitethefactthatweentertainquiteanumberofconcreteexamples),andleavesasidespecificstatisticaltechniquesofmeasurement,ofwhichtherearequiteafew(e.g.,contentanalysis,clusteranalysis,discrimi-nantanalysis,andsoforth).163MeasurementsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
LevelsofabstractionIngrapplingwiththeproblemofmeasurementitishelpfultoacknowledgethatallempiricalconceptsofinteresttosocialscienceencompassmultiplelevelsofabstraction.Attheveryleast,onegenerallydistinguishesbetweentheattributesthatdefineaconceptandtheindicatorsthatoperationalizeit,generatingtwotiers:(1)conceptualizationand(2)measurement.Thisisprobablysufficientforasmall-orderconceptlikevouchers.Formoreabstractconceptslikedemocracymultipletiersmayberequiredinordertoadequatelyrepresentallthelevelsofanalysisimplicitintheconcept,andinordertofullyoperationalizeit–tobringitdowntoearth,sotospeak.Considerthefollowinghierarchy:Democracy(thelatentconceptoftheoreticalinterest)Electoral(aconceptionofdemocracy)Freeandfairelections(akeycomponentofelectoraldemocracy)Validationofanelectionbyinternationalelectionobservers(anindicatoroffreeandfairelections)Here,fourtiersofaconceptareillustrated.Ofcourse,onemightaddfurtherlevels,forexample,amorespecificandoperationaldefinitionofhowthefree-nessandfairnessofelectionsshouldbevalidated.Evidently,onefacesapoten-tiallyinfiniteregress.Asoriginallydevised(ina1927physicstextbookbyP.W.Bridgman),operationalizationreferredtotheactualphysicaloperationsapersonwouldemploytolocateaphenomenon.Inconstructingareasonablestandardonemustfallbackonwhatisnecessaryinordertoachieveahighdegreeofreliability(precision).Oncethishasbeenachieved,thereisnofurtherTable7.1MeasurementstrategiesLevelsofabstractionHigh;Medium;LowStructureSet-theoretic(necessary,sufficient,necessary-and-sufficient);Additive;FuzzysetsAggregationBooleanlogic;WeightingsScalesCategorical(nominal,ordinal);Numeric(interval,ratio)ObjectivesDiscrimination;GroupingApproachesDeductive;InductiveEthnographyParticipant-observationSurveys/experimentsStandardizedsurveysandrandomizedtreatmentsCross-referencingEstablishingequivalenceacrossdiversecontextsCausalrelationsCausesandeffectsofthephenomenonofinterest164PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
needforspecification.Inanycase,thelowestlevelofabstractionisusuallyreferredtoasanindicator,regardlessofhowprecisethelatteris.Problemsofreliability(precision)canoftenberesolved,oratleastmiti-gated,bymovingdownthisladder.Inthecaseofdemocracy,conceptionsareeasiertomeasurethanthecoremeaning,componentsareeasiertomeasurethanconceptions,andindicatorsaretheeasiestofall.Small,concretethingsareusuallyeasiertomeasurethanlarge,abstractthings.Naturally,atacertainpointmicro-levelphenomenabecomelessobserva-ble,andmoredifficulttomeasure.Thisisthesituationfacedinfieldslikebiologyandphysics,wherecells,molecules,andsubatomicparticlesarethefrontiersofmeasurement.Inthesocialsciences,however,theindividual(i.e.,thewholehumanbeing)isusuallyregardedasthemostdisaggregatedunitofanalysis.Here,problemsofmeasurementaregenerallytheproductofabstraction,notspecificity.Itshouldalsoberememberedthattheproblemofmeasurementinvolvesbothreliabilityandconceptualvalidity.Asonescopesdownfrom“democ-racy”tolow-levelindicatorsonemayfindthattheconnectionbetweentheconceptofinterestandthephenomenabeingmeasuredbecomeshighlyattenuated.Achosenindicatormaybehighlyprecise,butofquestionablevaliditywithrespecttoahigh-orderconceptoftheoreticalinterest.Thisisthetradeoffencounteredwhenmovingalongaladderofabstraction:precisionisusuallyenhancedasonemovesdown,whileconceptualvalidityisenhancedasonemovesup.StructureConceptsandtheirindicatorsaredifferentlystructured.Thisisbecausemembershipintheconceptisdeterminednotjustbythechoiceofattributesandindicators,butalsobytheroledefinedforeachattributeorindicator.Whilewecannotaffordalengthydisquisitiononthissubject,itisimportantthatwereviewsomeofthechoicesthatdeterminehowconceptsandindica-torsarestructured.15Attributesandindicatorsmaybeunderstoodasset-theoretic(necessary/suffi-cient)oradditive.Considerthe“freeandfairelections”componentofdemocracy.Ifunderstoodasanecessary-and-sufficientconditionofdemocracy,thisistheonlycharacteristicthatmatters.Apolitywithfreeandfairelectionsisa15Goertz(2006)discussestheseissuesingreatdetail,thoughhisterminologyissomewhatdifferenttomine.165MeasurementsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
democracy;onewithoutisanautocracy.Notethatanecessary-and-sufficientconditionmaybetheproductofseveralnecessaryconditions,deemedjointlysufficient(e.g.,freeandfairelections,civilliberties,andturnoverofpower).Itneednotbelimitedtoasinglefactor.Iffreeandfairelectionsisunderstoodasanecessaryconditionthenapolitymustembodythisattribute,thoughtheremaybeothermembershipcondi-tionsaswell.Minimaldefinitions–thosethatdefineaconceptbyitsbareessentials–relyonnecessary-conditionattributes(Chapter5).Iffreeandfairelectionsisunderstoodasasufficientconditionthenitissufficientbyitselftoqualifyapolityasdemocratic,thoughtheremaybeotherconditionsthatwouldalsoqualifyapolityasdemocratic.Eachissubstitutablefortheother.Thisis,tobesure,anunusualwaytodefineakeyconcept,thoughitismorecommonattheindicatorlevel.16If,ontheotherhand,freeandfairelectionsisunderstoodinanadditivefashionthenapolityisconsideredmoredemocraticinsofarasitpossessesthisattribute.Thisishowattributesarehandledinmaximaldefinitions(Chapter5).Thechoicebetweenset-theoreticandadditivestructuresisthushighlycon-sequential.Indeed,thesamesetofattributesorindicatorswillyieldverydifferentconceptswhendifferentchoicesaremadewithrespecttoconceptstructure.Anotherapproachtotheroleofanattributeorindicatoristhroughfuzzysets,whichmayberegardedasamidwaypositionbetweenset-theoreticandadditivestructures.17Notethatintherealworldphenomenaoftenclusterincategories,buttheirmembershipinthesecategoriesisnotperfect.Fuzzysetsallowtheconceptualizertoassignascoretoeachentitythatreflectsitspartial(orcomplete)membership,basedonwhatevermembershipconditionsaredefined.Theboundariesare0(entirelyabsent)and1(fullypresent).Ofcourse,theuseoffuzzysetsmaycomplicatetheinterpretationanduseofacategoriza-tionschemesinceitnolongercarriesitsusual“crisp”or“continuous”mean-ings.Yetforsomepurposesitmaybeusefultoknowwhichentitiesare30percentor40percentmembersofagivenset.Indeed,forsomepurposes,itmaybeimportanttobeabletodefinecategoryboundariesinnonexclusiveterms(e.g.,as51percentmembershipratherthanas100percentmembership).Forotherpurposes,theintegrationofsuchcomplexitieswouldserveonlytodistractandconfuse.Likemanythings,choicesinconceptualizationandmeasurementrestuponthepurposethataconceptisexpectedtoserve.16Goertz(2006).17Ragin(2000,2008);Smithson(1987);SmithsonandVerkuilen(2006).TheapplicationoffuzzysetstodemocracyisexploredbyBowman,Lehoucq,andMahoney(2005)andSchneider(2011).166PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
AggregationTheresearchermustalsoconsiderhowtoaggregate(puttogether)alltheattributesandindicatorsattachedtoaconcept.Set-theoreticconditionsareeasy.Theyaggregateinanexplicitandclear-cutfashion,followingthedictatesofBooleanlogic.Anynumberofnecessaryorsufficientconditionscanbeaccommodatedinasingledefinition,andanynumberofconditionsmayberegardedascollectivelynecessaryandsufficient.18However,manysocialscienceconceptsregardattributesandindicatorsinanadditivefashion.Thismeansthatthetaskofaggregationisnotself-evident,andthereforeanexplicitaggregationprinciple(s)mustbeadopted.Forexam-ple,attributes(orindicators)maybeequallyweighted(theapproachtakenbymanydemocracyindices).Alternatively,adifferentialweightingschememaybeappliedaccordingtoaprioriassumptionsabouttheimportanceofdifferentcomponentstotheoverallconcept.Ifonebelievesthatsomedimensionsofdemocracy(say,electoralandparticipatory)aremoreimportantthanothers(say,deliberationorequality),thentheformermightbegrantedgreaterweightintheaggregatedconcept.Weightingsmayalsobearrivedatinductively,asdescribedbelow.Whateversolutiontostructureandaggregationischosen,itshouldbeclear,explicit,andreplicable.Thatis,itshouldbepossibleforanotherresearchertofollowthechoicesmadebytheoriginalresearcherandtomakedifferentchoiceswiththesameunderlyingdata,thatis,toreconstructtheconcept.Thisallowsforsensitivitytests(howrobustisananalysisinthefaceofdifferentaggregationchoices?)aswellasfordisaggregation,whichmayserveusefulpurposesindifferentcontexts.Unfortunately,althoughmostextantindicatorsofdemocracyhavefairlyexplicitaggregationrulestheyaresometimesdifficulttocomprehendandconsequentlytoapply(e.g.,Polity).Theymayalsoinclude“wildcard”ele-ments,allowingthecoderfreereintoassignafinalscoreinaccordancewithhisorheroverallimpressionofacountry(e.g.,FreedomHouse).Thisviolatestheidealofasystematicapproachtoaggregation.ScalesInordertooperationalizeaconceptonemustchooseascale,orsetofscales(iftheconceptismultidimensional),toemploy.19Somescalesarecategorical18Goertz(2006);Ragin(1987).19Stevens(1946).167MeasurementsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
(aka“qualitative”),byvirtueofthefactthatthedistancebetweencategoriesisundefined.Otherscalesarenumeric(aka“quantitative”)byvirtueofthefactthatthedistancebetweencategoriesisdefinedandmeasuredalonganumericscale.Withinthistwo-partclassificationothersubtypesfall,asindicatedinTable7.2.Notethatthisclassificationhasramificationsforthesortofstatisticalanalysisthatcanbeperformedontheresultingindicator.Inprin-ciple,parametrictestsshouldbeusedonlyfordatathatisproperlynumerical(thoughexceptionsmaybetoleratedincertaininstances).20Amongcategoricalscales,thosethatarenominaldefinemembersofthesameclass(theyareexamplesofsomething)butareun-ranked.Forexample,apples,oranges,andgrapesarenotmoreorlessofanythingrelativetoeachother,thoughtheyareallfruit.Ordinalscalesaremembersofthesameclassandalsoranked:verysweetissweeterthansweet.Butonedoesnotknowthetruedistanceseparatingeachlevelinthescale.Itisunclear,forexample,howmuchsweeter“verysweet”isrelativeto“sweet.”Amongnumericscales,thosethatareintervalarecharacterizedbyaconsistentmeasureofdistancebetweencategories.Forexample,thedistancebetween3and4onatemperaturescale(CelsiusorFahrenheit)isthesameasthedistancebetween25and26,andisdefinedbyaformalrule,consistentlyappliedacrossthescale.Ratioscalesareintervalscaleswithatruezero,indicatingtheabsenceofwhateverquantityisbeingmeasured(anullset).Inthecaseofmoney,0signalsnomoney.InthecaseoftemperatureontheKelvinscale,0indicatestheabsenceofallthermalenergy.Frequently,intervalandratioscalesfulfilltherequirementsofanumericscaleonlywithincertainbounds.Forexample,lifespanisboundedonthelowerTable7.2TypologyofscalesDifferentcategoriesRankedcategoriesDistancebetweencategoriesmeasuredTruezeroCategorical{NominalxOrdinalxxNumeric{IntervalxxxRatioxxxx20Stevens(1946,1951).168PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
endat0(arguably,itisalsoboundedattheupperend,thoughthisboundaryismoredifficulttodefine).Amorecomplicatedsituationoccurswhenanumericscalepossessesthecharacteristicof“equidistance”onlywithincertainbound-aries:thatis,distancesbetweenadjacentpointsonascaleareequidistant,butonlyaboveacertainthresholdand/orbelowacertainthreshold.Anexampleofthisisdiscussedbelowwithrespecttotheconceptofdemocracy,wherescalesseemtobreakdownattheextremes.Becausescalesaredefinedforspecificpurposesthesamephenomenamaybedifferentlyclassifiedaccordingtotheresearcher’spurpose.Forsomepurposes,itmaybesensibletoconsidervarietiesoffruitasnominalcategories.Forotherpurposes,itmaybesensibletoconsiderthemaspartofanominalscale(moreorlessacidic)oraratioscale(usingaratiomeasureofacidity).Formanytopics,itiscorrecttoregardhigher-levelscalesasmoreinformative.Thus,wewouldordinarilyinterpretanordinalscalefortemperature(“hot,”“medium,”“cold”)aslessprecise(andthereforelessinformative)thananintervalorratioscale.However,thisistrueonlywithreferencetothatparticularphenomenon.Itwouldnotbetrueforsex,forexample,sincethisdimensionadmitsofonlytwocategories.Here,anintervalscalereducestoanominalscale.Notealsothatwhilemorepreciseindicatorspromisemore,theyalsodemandmore.Specifically,theyrequireagreaternumberofassumptionsaboutthenatureoftheunderlyingdata.Ifanyoftheseassumptionsarefalse,oronlypartiallytrue,anyinferencebuildinguponthatindicatorwillbecastintodoubt.Anadditionalconsiderationrestsontheutilityofeachsortofscaleforsubsequentanalysis.Inbivariateandmultivariateanalyses,whereoneisinter-estedintherelationshipbetweentwoormorefactors,itmaybeimportanttochangethescalebywhichsomefactororfactorsaremeasured.Often,ordinalscalesaretreatedasisiftheywereinterval.Atothertimes,anintervalorscalevariableisre-codedasnominalorordinal.Thepointtokeepinmindisthatsuchre-scalingefforts,whileanalyticallyconvenient,ofteninvolveeitheralossofinformationand/ortheintroductionofbiasinthevariableofinterest.Thereareno“natural”scales;however,someinterpretationsofrealityaremoreplausiblethanothers.Formanypurposes,itisessentialtodistinguishpolitiesinabinaryfashion,asdemocraticorauthoritarian(autocratic).21Thisproducesanominalscalewithtwocategories,orperhapsmoreaccurately,anordinalscalewithtwocategories(sincetheyareordered).Whateverdisagreementsmayexistoverhowtooperationalizethisconcept,mostbinaryapproachestodemocracy21Przeworskietal.(2000).169MeasurementsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
featurethekeycomponent,freeandfairelections,whichwehavealreadydiscussedinthecontextoftheminimaldefinition(Chapter5).Becauseofthetendencyforminimaldefinitionstoimposecrisp(operational)bordersonaconceptthereisanaturalaffinitybetweenthisstrategyofdefinitionandthetwo-categorynominal(orordinal)scale.Forotherpurposes,onemayrequireamorefinelygradedindicatorofdemocracy.Acumulativeconceptisconstructedofcategoriesthatcanbeorderedinaunidimensionalfashion,forexample,asdegreesofcentralitytotheconceptofdemocracy.Limitinghimselftotheconceptofelectoralcon-testation(adimensionofthelargerconceptofdemocracy),GerardoMunckdefinesafour-partnominalscaleincludingcategoriesforauthoritarianism,semi-authoritarianism,semi-democracy,anddemocracy.22Here,eachcate-goryisdistinguishableandclearlyrankedrelativetotheconceptoftheoreticalinterest.DefiningattributesforeachcategoryareelaboratedinTable7.3.Theadvantageofthisapproachisthatitallowsonetoincorporateawiderarrayofattributesandoneisnotconstrainedtoseparateeachattributeintoadifferentcategory.Theunconstrainednominal-scaleindicatorisalsomorelikelytoapproximatethevirtuesofanintervalscale,whereneighboringcategoriesareequidistantfromeachother.Indeed,asthenumberofcategoriesincreasesscholarsmaybeinclinedtotreatnominalscalesasintervalscales.TheFreedomHouseindexofPoliticalRights,aswellasthePolityindex,arebothcommonlytreatedasintervalscales,eventhoughitseemsunlikelythatthecriterionofequidistancebetweencategoriesisfullysatisfied.ConsiderthePolityscaleofdemocracy,whichrunsfrom–10to+10inintegerintervals,thuscreatingatwenty-one-pointindex.23AlthoughcommonlyTable7.3Asinglescalewithmultipleinterpretations:“Electoralcontestation”0Authoritarianism:Noelectionsorelectionswithonlyonepartyorcandidate.1Semi-authoritarianism:Electionsinwhichmorethanonepartyorcandidateruns,butnotallpartiesandcandidatesfacethepossibilityoflosing.2Semi-democracy:Electionsinwhichmorethanonepartyorcandidaterunsandallpartiesandcandidatesfacethepossibilityoflosing,butnotallpartiesorcandidatesareallowedtoparticipate.3Democracy:Electionsinwhichonlyanti-systemextremistgroupsarebannedandallpartiesandcandidatesfacethepossibilityoflosing.Source:drawnfromMunck(2009:45).22Munck(2009:45).23MarshallandJaggers(2007).170PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
treatedasanintervalscale,theempiricalpropertiesofthedatabeliethisinter-pretation.Figure7.2displaysahistogramofPolityscoresacrossallcountriesandallyears(1800–2006).Itwillbeobservedthatthedata“bunches”attwopoints,at–7and+10.Thiscouldbeanempiricalpropertyoftheworld.However,itseemsmorelikelytobeanartifactofthescaleitself.AlookatPolity’scomplexcodebooksuggeststhattherearemultiplewaysacountrymayachievea–7score.Andthefactthat+10definesanend-pointofthescale(perfectdemocracy)suggeststhatthemembersofthislargecategorymayberelativelyheterogeneous(somemaybemoredemocraticthanothers,despitehavingthesamescore).24Constrainedscales(wherethereisanimposedmaximumorminimum)oftenencounterthisproblem.ForthesereasonsitmaybemoreappropriatetoviewthePolityscaleasordinalratherthaninterval.Butagain,itdependsuponone’spurposes.Sometimes,itisimperativetoreducethedimensionsofaconceptinordertoachieveempiricaltractability.Afinaloptionforthefour-pointscaleinTable7.3mayalsobeenvisioned.Insofarasthefirstcategorycomprisesatruezero–nocontestationwhatso-ever–thekeydimensionofelectoralcontestationmayberedefinedasaratioscale(anoptionthatMunckendorses).Thisexamplenicelyillustratesthefactthatthesamesetofcategoriesmaybedifferentlyinterpreted,accordingtodifferentassumptionsabouttheunderlying–1000.10.2Density0.3–50510Figure7.2Histogramof“Polity”scaleofdemocracy24Thisinterpretationisborneoutbyseveralre-aggregationsoftheunderlyingdatausingBayesiantechniques(Pemstein,Meserve,andMelton2010;TreierandJackman2008).171MeasurementsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
empiricalphenomenaanddifferentusesforwhichthesameindicatormaybeenlisted.ObjectivesTheconstructionofanindicatormayaimtoachievemaximumdiscrimina-tionamongentitiesoroptimalgroupingamongentities.(Thefirstwillutilizenumericscalesandthesecondwillutilizecategoricalscales,asdiscussedabove.)Oneortheotherofthesefundamentalobjectivesseemstogovernallmeasurementinstruments.Needlesstosay,asingleinstrumentisunlikelytoservebothgoalsatonce.Discriminationreferstotheabilityofaninstrumenttorevealfinelygradeddifferencesofdegree–usuallyunidimensionalbutoccasionallymultidimen-sional–insomelatenttraitpossessedbyasampleofpeople,objects,orevents.Thisisthetraditionalgoalofmeasurementinpsychometrics,andespeciallyinitem-responsetheory(IRT).25Accordingly,atestofeducationalachievementshouldprovidethebasisformaximallysensitivescale(measuringdifferencesinknowledgeorabilityinasubjectamongtest-takers)withaminimalnumberofstandardizedquestions.Thisrequiresthateachquestiononthetestbeindependentofallothersandthateachreflectdifferentlevelsofthelatenttraitofinterest(knowledge/abilityinsomesubjectarea),thusaddingtotheinforma-tionprovidedbytheotherquestions.Iftwoindividualswithdifferentlevelsofknowledge/abilitygivethesameanswertothesamequestionthatquestionisnothelpingtodiscriminatebetweenthem;itisredundant.Likewise,iftwoquestionsareinterdependent–suchthatananswertoquestion2depends(insomelogicalfashion)upontheanswergiventoquestion1–thennonewinformationislearnedfromquestion2.Theresultofawell-craftedmeasurementtool(con-structedforthepurposeofmaximumdiscrimination)isafinelygradedscalewithnobunching,thatis,scoresareevenlydistributedacrossthesampleofrespondents.Grouping,ontheotherhand,referstotheabilityofaninstrumenttosortitemsintodiscretecategoriesonthebasisofsimilaritiesanddifferencesinsomelatenttrait(s).Commontechniquesincludefactoranalysis,principalcompo-nentanalysis,clusteranalysis,andQ-sortanalysis.Notethatthegoalofcrispcategoriesmaynotalwaysbefullyachievable.Nonetheless,itistheguidingobjective.Thesuccessofatechniqueisitsabilitytosortitemsintodiscretecategories,appleswithapples,orangeswithoranges.Ifphenomenaarenot25Hambleton,Swaminathan,andRogers(1991).172PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
groupednaturallyintocategories(“apples,”“oranges,”etc.),thenthejobofthemeasurementinstrumentistodiscernbreak-pointsinnumericscales.Thismaybeaccomplishedinaninductiveordeductivemanner–whichbringsustoournexttopic.Approaches:deductiveandinductiveBroadlyspeaking,therearetwowaysofgainingpurchaseonaconceptthatisnotdirectlyobservable(andhencemeasurable).Deductiveapproachestomeasurementconstructindicatorsaccordingtoaprioridecisionrules.Thesemaybefairlysimple,asinthesetofnecessaryconditionsadoptedbyAdamPrzeworskiandcollaborators(seeabove).Ortheymaybehighlycomplex,asinthemultipleprobabilisticcomponentsthatcom-prisethePolityandFreedomHouseindices.Ineithercase,adeductiveapproachtomeasurementderivesindicatorsfromthedefiningattributesofaconcept(Chapter5)oralargerdescriptiveargument(Chapter6).Thetaskofmeasure-mentistotesthowsomeaspectoftheempiricalworldfitstheconcept,sodefined.26Aninductiveapproachbeginswithasetofempiricalmeasuresthatarethoughttocaptureelementsofaconcept(thispartoftheprocessisdeductive),andthenarrivesatanultimateindicator(orindicators)bylookingatobservedpatternsinthedata.Researchersgenerallywishtodiscoverwhethermultiplemeasuresofaconceptcorrelatewithoneanother,andifsowhetherthesecorrelationsareone-dimensionalormultidimensional.Havingresolvedthisquestion,theywillwanttoknowwhethertherevealedpatternsareinterpretable,thatis,whethertheyconformtorecognizablecomponentsoftheconcept.Techniquesforinterrogatingempiricalpatternsinasampleofdata–andrestructuringtoformnewindicators–includefactoranalysis,principalcom-ponentsanalysis,structuralequationmodels,regression,maximumlikelihoodmodels,andIRT.2726Here,onemightcontrasttheRaschmodelofmeasurement(whichimplementsanapriorimodelofthetraittobemeasured)withtheIRTapproachtomeasurement(whichconstructsameasurementtoolindialoguewiththephenomenabeingmeasured).27Choicesamongtheseoptionsdependlargelyupontheenvisagedsourcesoferrorandthescaleofthevariablesoneisdealingwith.Jackman(2008)offersanoverview.Bollen(1989);BollenandLennox(1991)addressstructuralequationmodeling.Hambleton,Swaminathan,andRogers(1991)isatextbookonitem-responsetheory.ExamplesofsomeofthesetechniquesappliedtopoliticalsciencequestionscanbefoundinPoliticalAnalysis17(3),Summer2009.173MeasurementsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
Inthisfashion,MichaelCoppedge,AngelAlvarez,andClaudiaMaldonadoexploreempiricalpatternsacrossfifteenmeasuresofdemocracy,includingthewell-knownFreedomHouseandPolityindices.28Theydiscoverthatabout75percentofthevarianceinthesemeasuresisreducibletotworelativelydistinctcomponents:contestation(competitiveelectionsandassociatedinsti-tutions)andinclusiveness(e.g.,broadsuffrageandhighturnout).Sincethesecomponentshavestronggroundingindemocratictheory(especiallyintheworkofRobertDahl),29therearegoodreasonstoregardthemasmorethanempiricalartifacts.Theysatisfytheinductiveaswellasthedeductivelogicsofconceptmeasurement.Ofcourse,itisimportanttobearinmindthataninductiveapproachtomeasurementisnotimmunetoerrorscontainedinthedataemployedfortheconstructionoftheindicator.Whilerandomerrorsinmeasurementwillbereducedwhenmultiplemeasuresarecombinedintoasmallernumberofindicators,systematicerrorswillbereproduced.Thus,ifproxymeasurescaptureonlycertaincomponentsofanunderlyingconceptitisthesecomponentsthatwillbereflectedinthenewindicator.Asithappens,extantmeasuresofdemoc-racy(suchasFreedomHouseandPolity)probablyemphasizetheelectoralandparticipatorydimensionsofthisphenomenon,excludingotherdimensionssuchasdeliberation,responsiveness,accountability,andsocialequality.Accordingly,inductiveapproachestothemeasurementofdemocracy(includingCoppedge,Alvarez,andMaldonado)reflectthisbias.30Theproblematicaspectofaninductivemeasureofanythingisusuallynotthetechnicalissueofwhichstatisticalmethodtoemploytoanalyzethechosenproxies.Itis,rather,theidentificationofsuitableproxyvariables,aswellasthequestionofwhatinterpretationtogranttheresultingdimension(s).Onemay,forexample,resolveproblemsofconceptualvaliditybyredefiningaconceptsothatitalignsproperlywithitsindicators–inthisinstance,bycallingtheresultingindicatorelectoraldemocracyratherthandemocracy.Butthisdoesnotresolvetheproblematissueifourobjectiveistomeasuredemocracy(toutcourt).Inductivetechniquescannotperformalchemy.Moregenerally,itisworthreiteratingacentraltheme:alldeductiveapproachestomeasurementcontainaninductivecomponent,andallinductiveapproachestomeasurementcontainadeductivecomponent.Conceptsandperceptsare28WhileCoppedge,Alvarez,andMaldonado(2008)employprincipalcomponentsanalysis,otherrecentstudieshaveenlistedBayesiantechniques(Pemstein,Meserve,andMelton2010;TreierandJackman2008).29Dahl(1971).30Coppedge,Alvarez,andMaldonado(2008:645)acknowledgethisasalimitationoftheirapproach.174PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
inseparable.Nonetheless,itissometimeshelpfultodistinguishapproachestomeasurementthatleantooneortheothersideofthespectrum:theyareeitherpredominantlydeductiveorpredominantlyinductive,asspecifiedabove.EthnographyProblemsofmeasurementaresometimesbestapproachedinanethnographicfashion,especiallywhereanelementofmysteryisinvolved(i.e.,whenproblemsofduplicityandambiguityaresuspected).Ifyouwishtoknowwhatisgoingon,observethepeopleclosesttotheaction.Tounderstandcrimetalktothepolice,tocriminals,tothefamiliesofperpetrators,andtothemembersoftheaffectedcommunity.31(Idonotmeantoimplythatethnographyisequivalenttomeasurement;evidently,therearemanyotherimportantusesforthistech-nique,someofthemrelatedtocausalinference,asdiscussedinPartIII.)Naturally,informantsareoftencagey.Andgettingthetruthoutofthem–oranythingatall–mayrequireagooddealofsoakingandpoking.Itmayalsorequireadegreeoftrustandfamiliarity.Informantswithmuchatstakeinanissuearenotlikelytodivulgesecretstoanoutsider.Moreover,anoutsidermaymisunderstandthesubtlesignalsofaninformant,therebyintroducingmeasurementerror.Thepracticeofethnographyisfocusedongaininglocalknowledgeandthisknowledgemayrequire“goinglocal.”Sometimes,investigatorsarebornintothespherethattheystudy,orarealreadymembersofthatspherewhentheybegintheirresearch.Someofourbestinsightsintosocialactionareprovidedbynativesofthatcultureorclass.32Likewise,someofourbestinsightsintothebehaviorofthemediacomefromcurrentandformercorrespondents,33andsomeofourbestinsightsintopoliticscomefromcurrentorformerpublicservants.34Alternatively,researchersmayassumepositionswithinacultureoranorga-nizationasatemporaryparticipant:forexample,joiningacluborengaginginsomeactivityofinterest.MartinSanchezJankowskiwasobligedtoparticipatein31Onopen-endedinterviewingseeChong(1993);Dexter(2008);HammerandWildavsky(1989);Kritzer(1996);Leechetal.(2002);Peabodyetal.(1990);RubinandRubin(1995).ThecontributionsofethnographicapproachestosocialsciencearediscussedinAdlerandAdler(2003);BayarddeVoloandSchatz(2004);Lieberman,Howard,andLynch(2004);Schatz(2009);Vidich(1955);YanowandSchwartz-Shea(2006).Examplesofworkinthisvein(inadditiontostudiescitedabove)includeAllina-Pisano(2004);Burawoy,Gamson,andBurton(1991);EdinandLein(1997);Francis(1991);Laitin(1986);Liebow(1967);Luker(1984);Scott(1985).32Ortner(2005).33Crouse(2003);Epstein(2000);McGinniss(1988).34Crossman(1976);Reedy(1970).175MeasurementsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
initiationriteswhenresearchinggangsinLosAngeles,Chicago,andNewYorkinordertogainaccesstoinformants.35Arguably,thebestwaytounderstandanactivity,andtheideasandincentivesthatmotivateit,isfortheresearchertoengageinithim-orherself.Ofcourse,thisisnotalwayspossible,oradvisable.Evenso,onemaygaininsightintoanactivitybycloseobservation,intensiveopen-endedinterviews,andlongacquaintanceship.InordertounderstandthemeaningascribedtoinfantdeathinapoorcommunityNancyScheper-HughesspentseveralyearsinashantytowncommunityinnortheasternBrazil.Whatsheuncoveredduringthecourseofherresidence(firstasaPeaceCorpsvolunteerandthenasaprofessionalanthropologist)israthersurprising–andprobablywouldnothaveoccurredtodistantobserversporingoverspreadsheetsorsecondaryaccounts.36WhenstudyingthebehaviorofmembersofCongress,RichardFenno(oneofthemostinfluentialinterpretersofthisstoriedbutsecretiveinstitution)gotasclosetotheactionaspossible–ridingalongwithmembersastheyheldconstituencymeetings,addressedpublicfunctions,officiatedatfundraisers,andmadedeals.37ThereisalongtraditionofAmericanpoliticalscientistswhoapprenticethemselvesascongressionalaidesorstaffersontheHillpriortoenteringtheacademy.Likewise,academicsfrommanyfieldsoftenfindthattheirpersonalbackgroundandconnectionsserveasanentréeintotheactivitytheyarestudying.Allformsoflocalknowledgearewelcome,includingthosethatareentirelyfortuitous.ScottPalmerhappenedtobestationedasaPeaceCorpsvolunteerinthePeruvianAyacuchocommunitythatspawnedtheShiningPath(SenderoLuminoso)inthe1960s.Indeed,heknewAbimaelGuzmán,thefounderofSendero,personally.Later,havingjoinedtheacademy,PalmerwasabletobringhispersonalacquaintancewiththecultureandthepersonalitiesofSenderotobear,providingrareinsightintoareclusiveandviolentpoliticalmovement.38Sometimes,serendipityisthebestresearchstrategy.Verysensitivesubjectscansometimesbebroachedifaresearcherisabletoenterintoacommunityasatrustedobserver.Indeed,researchsubjectsaresometimesdelightedtounburdenthemselves,andmaybecomequitechattywhenquestionedbyaninterestedandseeminglysympatheticobserver,oncetheyareassuredofanonymity.WhenKathrynKirschenmanandJoleenNeckermaninterviewedwhiteemployersintheChicagoareaabouttheir35SanchezJankowski(1991).AsimilarapproachistakenbyBillBufordinhisstudyofsoccerhooliganism(Buford1991).36Scheper-Hughes(1992).37Fenno(1978,1986,1990).SeealsoGlaser(1996).38Palmer(1992).176PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
hiringpracticestheyweresurprisedattheforthrightdiscussionthatensued.Employersspokeopenlyaboutwhytheypreferredmembersofoneracialgroupoveranother,givingexamplesoftheirowndecisionsandexperiences.Theyalsofreelydiscussedtheirunderstandingsofracismanddiscrimination,thetwoconceptsoftheoreticalinterest.39Gainingarich,contextual“feel”ofanactivitywillassisttheresearcherinjudgingthesagacityandforthrightnessofhisorherinformants.Whileethnographicapproachesarecommontomanyfields,theyarenotextensivelypracticedineconomics.Andyetitseemslikelythattheyholdgreatpromise,eveninthis“dismal”science.Onerecentstudyoftherelationshipbetweenrecessionandwagesreliescentrallyuponopen-endedinterviewswiththosewhomostdirectlyaffectwagepolicy:personnelmanagers,unionrepre-sentatives,laborlawyers,andjobservicecounselors.TrumanBewley’smethodisqualitative,buthisresultsarehighlyinformative–and,insofarasonemayjudge,asaccurateastraditional,large-Napproachestomeasurement.40(Myhunchisthateconomistsaremoreinfluencedbytheirpersonalexperiencesintheprivateandpublicsectorthantheyareinthehabitofrevealing.)Inanycase,thereisclearlymuchtobesaidforthevalueofethnographicmethods.Granted,suchapproachesdonotalwaysgettothebottomofthings,thatis,renderanauthoritativeinterpretationofanactivity.Theyarepronetoproblemsofunrepresentativenessandobserverbias,andareoftendifficulttoreplicate.Moreover,subjectsmaywithholdinformation.However,itisdifficulttoimagineanontrivialtopicthatwouldnotbenefitfromcloseobservation.Wherevermeasurementerrorinsomerealmofhumanactionissuspected,ethnographyiswelladvised.SurveysandexperimentsInthecontextofsurveyresearch–wherethereisordinarilynoopportunitytogainthetrustofrespondentsortojudgetheirresponsesinacontextualfashion–therearenonethelesswaysofaccessingsensitivesubjects.41Inordertopreserveanonymity,onemayomittheindividual’snamefromthesurvey.Onecanalsoadoptananonymoussettingforthesurvey,whichmaybeadministeredbymailoron-line.Anotherapproachistoconstructasurveyinstrumentinwhichsensitivesubjectsarecouchedasquestionsabout39KirschenmanandNeckerman(1991).40Bewley(1999).SeealsoHelper(2000).41TheseissuesareaddressedinDryzek(1988);Fowler(2008);Kingetal.(2004);Lee(1993);SchaefferandPresser(2003);Schwartz(1984);Stoker(2003);TourangeauandSmith(1996);Weisberg(2005);ZallerandFeldman(1992).177MeasurementsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
otherpeople,forexample,“Doyouthinkthatotheremployersuseraceasacriterionformakinghiringdecisions?”Theassumptionhereisthatthoseengagedinactivitiesthataredenigratedbysociety(e.g.,discriminationorcorruption)willbeinclinedtoseetheseactivitiesaswidespread,asthismayassuagefeelingsofguiltorshame.(Ofcourse,itcouldalsobethatthosewhorefrainfromsuchactivitiesseeothersasespeciallyactive,bywayofexplainingtheirlackofsuccessorpopularity.)Onemayalsoenlistanexperimentalsurveydesigninordertomaskindi-vidualidentities.42Thelistexperimentbeginsbysortingrespondentsrandomlyintotwogroups,eachofwhichisgivenasmallsetofquestionstoponder.Thequestionnairesareidenticalexceptthatthetreatmentgroupisgivenoneaddi-tionalquestionofasensitivenature(e.g.,pertainingtoracismorcorruption).Respondentsarethenaskedtoreportthetotalnumberofquestionsthattheyagree(ordisagree)with,butnottheiranswerstoanyspecificquestion.Sincethetreatmentandcontrolgroupsareassumedtobecomparableinallrespectsexcepttheoneadditionalquestionaskedofthetreatmentgroup,anydifferencesinresponses(i.e.,inpercentageof“agree”answers)maybeattributedtothisquestion.Theinnovationofthemethodistoallowforaccurateaggregate-levelresults,whileavoidinganypossibilityoflinkinganindividualwithaspecificanswer.43Anotherexperimentalsurveyresearchtechniquevariesthequestionsonaquestionnaireinsmallwayssoastogaugetheeffectofacarefullychosentreatment.Forexample,inordertoprobehiddenracismPaulSnidermanandcolleaguesconstructsurveysthatinquireaboutrespondentviewsofgovern-ment’sresponsibilitytoassistthoseinneed.Inoneversionofthesplit-samplesurveythescenarioinvolvesanunemployedblackworker,whileinanotherversionitinvolvesanunemployedwhiteworker.Thescenarios(i.e.,thequestions)areidentical,withtheexceptionoftheraceoftheworker,andsoarethetwogroups(whichhavebeenrandomlychosen).Thus,anydifferencesinresponseacrossthetwogroupsmaybeinterpretedasaproductofthetreatment.44Onemayalsoadoptafieldexperimentinordertodeterminevaluesandbeliefsonsensitivesubjects.Forexample,inordertogaugetheextentofracismamongemployersonemightdesignexperimentsinwhichjobapplicants–identicalinallrespectsexceptforrace–applytothesameposition.Therateof42Peeters,Lensvelt-Mulders,andLashuizen(2010);Warner(1965).ExperimentsemployedtomeasuretheconceptoftrustarereviewedinNannestad(2008).43Kane,Craig,andWald(2004);SnidermanandCarmines(1997).44Snidermanetal.(1991).178PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
successacrossthematchedapplicantsmaythenbeinterpretedasagaugeofracismamongemployers.45Ofcourse,inshiftingfromarticulationstoactionswemaylosesightofthemotivationsoftheparticipants.Weknowfromanexperimentsuchasthatdescribedabovewhichemployerschoosethewhiteapplicantandwhichchoosetheblackapplicant,butitmaybedifficulttoinferfromthiswhytheymadethosechoices.Forthisreason,experimentswhosepurposeistogaugeissuesofmeaningandmotivationareoftenaccompaniedbyethnographicinvestigation.Alternatively,theset-upoftheexperimentmaybealteredinsubtleways,forexample,toadjustforthesocialbackgroundoftheapplicants,theireducation,theirplaceofresidence,andtheirmannerisms.Ifanyofthesealterationsaffectsthevariableoftheoreticalinterest–race–thenwemaybeabletoreachtentativeconclusionsaboutthemotivationsbehindtheactionsofemployers.Inshort,experimentscanshedlightonmotivations,butitgenerallyrequiresmultipleiterations.Itmayseemstrangetoemployanexperimentalframeworkinordertosolveproblemsofmeasurement,fortheexperimentseemstopresupposeacausalquestion(embodiedinthetreatment).However,theexperimentaltechniquehasmanyuses,andtheusestowhichitisputdependlargelyonthepurposeoftheinvestigation.Ifone’sinvestigationiscausal,oneisinterestedingaugingthecausalimpactofatreatmentlikeraceonemployerdecisions.Ifone’sinvestigationisdescriptivethesameset-upmaybeemployedtoshedlightonmeasurementquestions,forexample,thepervasivenessofracismamongemployers.Here,thetreatmentismerelyastimulusthataffordsanoccasiontoobserveresponses.Inthetermsofourmeasurementdiagram(Figure7.1),itisawaytocontrolconfounders(C)sothatthelatentconceptofinterest(L)canbepreciselyandaccuratelyobserved.Cross-referencingWhenfacedwithrecalcitrantproblemsofmeasurement,sometimesitispos-sibletogaintractionbylookingatanadjacentcontextinwhichtheconceptoftheoreticalinterestismeasuredinaconvincingfashion.Ishallrefertothisasacross-referencingstrategy.Theproblemof“mediabias”isacaseinpoint.Thereisageneralsensethatmediaoutletsofferdifferingideologicallyinformedperspectivesonthenews.45Forexample,KenneyandWissoker(1994);Neumark,withBankandVanNort(1996).ForanoverviewofthisgenreoffieldexperimentseePager(2007).179MeasurementsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
Someareacknowledgedtobemoreliberalorconservativethanothers.Butwedonotseemtobeabletomeasurethisbiaswithanydegreeofprecision,anditoccasionsenormouscontroversy(mostmediaoutletsresisttheideathattheirreportingisanythingbut“fairandunbiased”,anduntaintedbypartisanship).TimGrosecloseandJeffreyMilyoaddressthisquestionbyreferencingacrossthreecontexts.Asabaseline,theyadoptADA(AmericansforDemocraticAction,aliberalpolicygroup)scoresformembersofCongressasmeasureofliberalism/conservativism.Second,theycountthefrequencywithwhichdiffer-entmembersofCongresscitevariousthinktanksintheirspeechesfromtheflooroftheHouseandSenate.Thisallowsforacodingofallthinktanksalongasingleliberal/conservativespectrum.Third,theycountthefrequencywithwhichvariousmediaoutletscitethesesamethinktanks.Thisallowsforajudgmentofwhereeachoutletstandsontheideologicalspectrum.Bythisaccounting,mostAmericanmediaoutletsarejudgedtobetotheleftofthecongressionalmean.Intheauthors’words:Ourresultsshowastrongliberalbias:allofthenewsoutletsweexamine,exceptforFoxNews’SpecialReportandtheWashingtonTimes,receivedscorestotheleftoftheaveragememberofCongress.Consistentwithclaimsmadebyconservativecritics,CBSEveningNewsandtheNewYorkTimesreceivedscoresfartotheleftofcenter.ThemostcentristmediaoutletswerePBSNewsHour,CNN’sNewsnight,andABC’sGoodMorningAmerica;amongprintoutlets,USATodaywasclosesttothecenter.46Thefindingsarecontroversial,andnotbeyondreproach.Itcouldbe,forexample,thatthinktanksontheliberalendofthespectrumoffermorecompre-hensiveandscholarlyanalysisofpolicyproblemsthanthoseontherightendofthespectrum.Ifso,thenthepredominanceofliberalthinktanksinthepressmightbeareflectionoftheirsuperiorinformationratherthanasignofideo-logicalbiasinthenewsmedia:thatis,membersofCongressaswellasreportersfortheNewYorkTimesmightbemoreinclinedtocitetheBrookingsInstitution(onthecenter-left)thantheLibertyFund(ontheright)becausetheformeroffersmoredetailedandreliableanalysisofaproblemofcurrentinterest.Ifso,theproposedmeasurementinstrumentisflawed.(Otherobjectionsmightbefoundwiththestudy.47)ThepointremainsthatGrosecloseandMilyohaveofferedanintriguingsolutiontoanintransigentmeasurementproblem,andonewithimportanttheoreticalandpracticalrepercussions.Moreover,itisagoodexampleofa46GrosecloseandMilyo(2005:1191).47BrendanNyhan’sblogposting(accessed:August2009)addressesseveralofthecritiquesanddefenses,availableat:www.brendan-nyhan.com/blog/2005/12/the_problems_wi.html.180PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
techniquethatcanbeappliedinmanyrealms.Anywhereabaselinemeasureofaquantityofinterestcanbelocatedinonerealmandtransportedtoanother–viasomemeasureofequivalence–initialproblemsofmeasurementmaybeovercome,aslongaspotentialconfounders(suchasthosediscussedabove)arenottoosevere.CausalrelationsAfinalangleontheproblemofunobservablesistoconsidercausalrelationshipsimplicitintheconceptofinterest.WhatcausesL(theconceptofinterest),andwhatdoesLcause?Occasionally,factorsthatarepresumedtohaveacausaleffectonaphe-nomenonareeasiertomeasurethanthephenomenonitself.If,forexample,onecanmeasurethedegreeofeducationreceivedbyanindividualwithgreaterfacilitythanthatindividual’slevelofactualintelligence,andifwecanassumethateducationistheprincipalcausalfactorbehindintelligence,thenitmaymakesensetooperationalizeintelligence(L)withaninputindicatorcomposedofeducationalattributes(I).Here,thepresumedcausalarrowrunsfromItoL.Alternatively,onemightconsiderthecausaleffectsofanunobservablecon-cept.Suppose,forexample,thatwearestudyinganorganizationalspherewhereintelligenceistheprincipalcriterionofadvancement.Inthismeritocraticsettingitmaybeplausibletoregardanindividual’spositionwithinthathierarchyasanoutcomeindicator(I)ofhisorherintelligence(L),undertheassumptionthathisorherlevelofintelligencecausedhimorhertoreachthatposition.Here,thepresumedcausalarrowrunsfromLtoI.Inmostrealmsofsocialscience,outcome-basedstrategiesofmeasurementaremorepromisingthaninput-basedstrategies.Indeed,manyofthefore-goingexamplesdiscussedinthischaptermightberegardedasoutput-based.Asanadditionalexample,letusconsidertheproblemofideology.Specifically,whatisthetrue(sincere,authentic)“ideal-point”ofacitizen,legislator,orparty?Thequestionhasagonizedpoliticalscientistsforacenturyormore.Itisaclassichermeneuticproblem,forsubjects–especiallythoseholdingelitepositions–oftenhavestrongreasonstocamouflagetheirtruepolicypreferences.Moreover,thequestionitselfmaybedifficulttoarticulateand,hence,beopentomultipleinterpretations.Underthecircumstances,itisnotsurprisingthatscholarshaveresortedtobehavioralmeasuressuchasvoting.Whenattemptingtoanalyzetheideal-pointsofmembersoftheUSCongress,forexample,researchersexaminepatternsofcorrespondenceamongvotingrecords,undertheassumptionthat181MeasurementsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
thosewhovotetogethersharethesameideology.Thisisthebasisforthewidelyused“NOMINATE”scoredevelopedbyKeithPooleandHowardRosenthal.48Notonlyisthisoutcomemeasureobjective,butitisalsoconsequential–andthereforearguablyabettermeasureofanindividual’struepreferences,allthingsconsidered.Outcomemeasuresfollowtheoldadageaboutpoliticians:“Watchtheirfeet,nottheirmouth.”Ofcourse,theresearchermuststillinterpretthemeaningoftheseactions,whicharenotalwaysself-evident.“Objective”mea-suresmayrequire“subjective”judgmentsinordertobeuseful.Thisisthecase,forexample,whenattemptingtodeterminethemeaningofdifferentdimen-sionsinthevotingdataprovidedbytheNOMINATEproject.Morefundamen-tally,onemightwonderaboutthepresenceofconfounders–factorsotherthanpersonalideologythatinfluencealegislator’svotingrecord.Presumably,legis-latorsarealsoaffectedbypressuresfromconstituents,lobbyists,andpartyleaders.Thesepressuresstraintheinterpretationofalegislator’sNOMINATEscoreasanexpressionofpersonalideology.Closelyrelatedisthequestionofhowtomeasurethequalityofgovernanceincountriesaroundtheworld.Wehavelittleinformationabouttheinputsofgovernancethatmightallowustojudgetheperformanceofgovernmentsaroundtheworld.Ofcourse,weknowhowmuchgovernmentstaxandspend,andweknowsomethingaboutthecategoriesofexpenditure(howmuchisspentondifferentprograms).Butwecaninferlittleaboutthequalityofgovernancefromhowmuchgovernmentsarespendingandwheretheyareputtingtheirmoney.Biggovernmentmaynotbebetterorworsethansmallgovernment.Anoutcome-basedapproachtogovernancemightbeginwithoutcomessuchasgrowth,inflation,unemployment,health,mortality,education,orinfrastructure.Weassumethattheseoutcomesareinfluencedbygovernmentactions–eveniftheyarealsoinfluencedbymanyotherfactors,whichmaybeclassifiedasrandom(B)ornonrandom(C).Thus,iflevelsofinfantmortalityarehigherinCountry1thaninCountry2wemightinferthatCountry2enjoysbettergovernancethanCountry1,allotherthingsbeingequal.49Thestickingpointistheceterisparibus(otherthingsequal)clause.Sometimes,itmaybepossibletoadjustanoutcome-basedindicatorbycontrollingforpotentialconfoundersandpotentialsourcesofnoisesothattheresultingindexcapturesmoreaccuratelywhatitisintendedtocapture:thatis,thequalityofgovernanceinaparticularpolicyarea.Thus,ifwewishtomeasurethequalityofgovernanceinpublichealthwiththeinfantmortalityratewemightwishtocontrolfortheimpactofeconomicfactorsbyincluding48PooleandRosenthal(1991).49GerringandThacker(2008:ch.6);Gerringetal.(2008).182PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
ameasureofeconomicdevelopment(e.g.,GDPpercapita)andwemightwishtocontrolfortheimpactofgeographicfactorsbyincludingaseriesofclimateandgeographiccontrolsthatarethoughttoaffectthehealthofpopulations.Wherevertheoutcomeofconcernandpotentialconfoundersaremeasurable,amodel-basedapproachtooperationalizationisinformative.50Asimplerapproachistoidentifysuitablebaselinecomparisons.Thus,inattemptingtomeasurethenumberofwomenwhosedeathsareaconsequenceofgenderdiscrimination(asopposedtopovertyandotherfactors),AmartyaSenidentifiessub-SaharanAfrica(SSA)asabaseline.CountriesbelowtheSaharaconstitutethepoorestportionoftheworld,butitisalsoanareaofthedevelopingworldwherethefemale/maleratioisrelativelyfavorable–roughly1.05inthe1980s.Senreasonsthatratiosbelowthatlevelincountriesthatareatanequalorhigherlevelofeconomicdevelopmentmustbeaproductofdiscriminatorypoliciesandpractices.Bythisaccounting,hereckonsthatthereweremorethan100million“missingwomen”intheworld(anumberthathasprobablygrownsubstantiallyinsubsequentdecades).51Again,theapproachiscausal,eventhoughthereisnoexplicitcausalmodel.Fromacertainperspective,itmightbearguedthatallmeasurementtech-niques(exceptwherethephenomenonofinterestcanbedirectlyobserved)arecausal.Weknowthatsomethinglatentexistsinsofarasitcausessomethingelse,orinsofarasitcanbepresumedtobecausedbysomethingelse.Thus,giventhatcorruptionisanabstraction(andthereforeinherentlyunobservable),allindi-catorsoftheconceptmightberegardedaseithercausesoreffects.Thisistheperspectiveadoptedbymanystatisticaltechniques(e.g.,regression-basedorstructural-equationmodels),whichdistinguishbetween“independent”and“dependent”variables.Inotherrespects,causalmodelsemployedforpurposesofmeasurementdonotmeetthedesiderataofcausalanalysis.Thosethatareoutcome-basedpositcausalinterventions(L)thatcannotbeobserved,muchlessmanipulated.Thosethatareinput-basedpositcausalconnections(fromItoL)thatcannotbetested.Thisisnotverysatisfactoryifonewishestoestablishcausalrelation-ships,forreasonsdiscussedatlengthinPartIII.Indeed,theinabilitytodirectlymeasureL,theconceptofinterest,meansthatanypotentialcausalrelationshipsinvolvingLmustremainnotional.Empiricalevidenceisuseful–attheveryleast,fordiscardingfalsetheoriesaboutL.Butsuchevidenceisnevercon-clusive,exceptinsofaraswecandevisewaystoobserveLinamoredirectfashion.50Gerringetal.(2008).51Sen(1990).183MeasurementsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
Considerthematterofemotion,animportantelementofanytheorythatdoesnotrestsolelyonrationalmotivations.Emotionslikehatred,love,andgriefpresumablymotivateagooddealofhumanbehavior,andmayhavevastramificationsforoutcomesofinteresttosocialscience.52Wecanmeasurethe“outputs”ofemotions,forexample,violence,tears,self-reportsofemotionalstatus(I),andregardthemasevidenceoftheinnerlivesofoursubjects(L).Ofcourse,wedon’treallyknowwhetherourhunchiscorrect,thatis,whethersomeinneremotionaldriveiscausingsubjectstomanifestparticularbeha-vioralpatterns,ortotestifythattheyareangry,inlove,orsad.However,ifwecanmeasurechemicalprocessesthatareassociatedwithemotions(asrevealedbyself-testimony),thenwearearguablyclosertoadirectmeasurementofthephenomenonofinterest.Thus,whenpsychologistsmeasureemotionstheycommonlyemployphysiologicalindicatorssuchasbloodpressure,galvanicskinresponse(GSR),heartrate,pupillarydilation,andeyeblink(startle)responseinadditiontoself-reports.53Thesemeasurementstrategiesmayberegardedascausalonlyifthechemicalprocessesactuallyhelptogeneratethelivedexperiencethatweknowashatred,love,orgrief,orareaby-productofthatexperience.Theyarecorrelativeinsofarastheyaresymptomaticoftheseemotions.However,forpurposesofmeasure-mentthedistinctionisimmaterial.WedonotcarewhetherIcausesL,isacauseofL,orismerelyatraceofL.Allthatcountsisthealignment(i.e.,correlation)betweenconceptandindicator.Thisiswhatoccupiesourattentionwhenthepurposeofananalysisistovalidateameasurementinstrument.54Corruption:adetailedexampleThreatstomeasurementreliabilityandvaliditymaybedescribedinageneralway,aswehaveattemptedtodo.However,manyoftheproblemsandsolutionscanbeexploredonlybyexample,fortheyareheavilycontextual.Thereisnogeneralsolutiontoproblemsofmeasurementontheorderoftheexperimentalresearchdesignforcausalquestions.Thereareonlyparticularsolutions.Thus,weclosethischapterwithdiscussionofatopicofcentralimportancetothesocialsciences,andonethatalsoposesrecalcitrantproblemsof52Petersen(2002).53RoseMcDermott(personalcommunication,November2009).SeealsoOxleyetal.(2008).54Ofcourse,Irecognizethatifourassumptionsaboutalignmentareinformedbyassumptionsofcausalitythenwemustwrestlewiththelatter.Evenso,thepurposeofthisapproachtomeasurementistodealwithsituationsinwhichtheconceptitselfappearstobeimpossibletomeasureinadirectfashion.Thisimpliesthatitwillalsobeimpossibletotestthecausalassumptionsunderlyingthe(causal)measurementtechnique.184PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
measurement:politicalcorruption,understoodastheuseofpublicofficeforprivategain.55Howdoweknowwhencorruptactivitiesareoccurring,andhowextensivetheyare?Andhowmightwecomparetheseoccurrencessystematicallyovertimeandacrosspolitiessothatthemeasurementinstru-mentcanbegeneralized?Inrecentyears,cross-nationalsurvey-basedindicatorsofcorruptionhavegainedprominence,bothamongacademicsandamonglaypublics.PrincipalamongthesearetheCorruptionPerceptionsIndex(CPI)developedbyTransparencyInternational(TI)andtheCorruptionControlIndexdevelopedbyDanielKaufmannandcollaboratorsattheWorldBank(WB).56Foreachindex,avarietyofquestionsarecompiledthatseektogaugeaninformant’ssenseofhowcommoncorruptpracticesareinacountry,withspecialfocusoncorruptpracticeslikebribesthatdirectlyimpactinvestors.Dataisnowavailableformostcountriesintheworld,allowingforcomparisonsacrosscountrieswithveryhigh(e.g.,Nigeria)andverylow(e.g.,Norway)levelsofcorruption.Thechiefadvantageofasurvey-basedmeasure,basedongeneralquestionsaboutperceivedcorruption,isthatonecancraftageneralizedmeasureofthisambientconcept.However,numerouscriticismshavebeenleveledatthesemeasures.57TheTIandWBindicatorsaggregatesurveystakenbymanyoutfits,mostofwhicharecommercialconsultancieswithawiderangeofobjectives.Usually,thatpurposeistailoredtotheneedsofinvestors.Arguably,theTIandWBindexesarebetterunderstoodasindexesofbribe-payingratherthanofcorruptionatlarge.Combiningmultiplesurveysintoasingleindexprovidesmultiplemeasurements,whichshouldincreaseprecisionandallowsforanestimateofaconfidenceinterval.However,italsocreatesambiguitywithrespecttointerpretation,sinceeachquestion,andeachsurvey,isdifferent.Moreover,itisuncleartowhatextentresponsestomultiplesurveysareactuallyindepen-dentofoneanother,sincetheymaysimplyreflectcommonassumptionsabouthowcorruptorcleanacountryis.Questionsusuallyprobeperceptionsofcorruption,orofbribes,andsomayincorrectlyreflectrealitiesontheground.58Respondentsmaynotanswertruthfully;worse,thedegreeoffranknessmayvarybycountryorregion,impedingsystematiccomparisons.Becausesamplesvaryfromyeartoyear,andarenormalizedtozeroeachyear,55Thereare,ofcourse,manywaysofdefiningthiskeyconcept(JohnstonandHeidenheimer2002;Sampfordetal.2006).Ileavetheseissuesasidesoastofocusontheempiricalaspectsofthemeasurementproblem.IwanttoacknowledgeMichaelJohnston’sinputtothefollowingsection.56Kaufmann,Kraay,andMastruzzi(2007);Lambsdorff(2003).57Knack(2006);Sampfordetal.(2006).58Abramo(2007);Kenny(2006);Seligson(2006).185MeasurementsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
comparisonsarenotpossiblethroughtime;itisdifficulttosay,forexample,whetheracountry’sgovernanceisimprovingordeteriorating.Respondentsaregenerallyconcentratedamongurbandwellersandbusinesspeople,oftennotnativetothecountry,raisingquestionsofrepresentativeness.Somequestionsaskrespondentstocompareacountrytoaselectedlistofothers;someaskforaglobalcomparison.Someinvitetheconclusionthatwhereeconomicproblemsaresevere,corruptionmustbeextensive.Despitetheseflaws,theseindicescontinuetobeemployedinawidearrayofsettings,suggestingthattheyareperforminganimportantfunction.Someoftheflawsareinherenttothemeasurementinstrument(masssurveys);othersmighteasilyberectified.Certainly,itispossibletoconstructmorerepresen-tativesamples,tostandardizesurveyquestionsandformats,tocarefullydistinguishbetweendifferenttypesofrespondents(e.g.,in-countryandout-of-country,eliteandmass,businessandnonbusiness,urbanandrural),todistinguishdifferenttypesofcorruption(e.g.,bribes,fraud,vote-buying,etc.),toemploypollingtechniquesthatprovidesomeguaranteeofanonymitytotherespondent(asdiscussedabove),andtofocusonactualexperiencesofcorrup-tionratherthansimplyongeneralperceptions.Theprincipalobstacletothesesortsofimprovementsisnotmethod-ologicalbutratherorganizational.Onemustbearinmindthatthecon-sultancieswhocommissionmostofthepollscomprisingtheTIandWBindexeshavespecificandlimitedgoalsfocusedontheirbusinessclienteles.Moreover,theseoutfitsarenotinapositiontopooltheirresourcessoastoconstructregular,standardizedsurveysofcorruptionthroughouttheworld.Thissortofpublicgoodisunlikelytobecreatedbymarketforces.Thus,citizens,policymakers,andacademicswhoyearnforamorepreciseandsystematicsurveyinstrumentareunlikelytogetitbycontinuingtofree-rideofftheprivatesector.Doingsowillrequireinternationalorgani-zationandcommensuratefunding–somethingontheorderoftheWorldValuesSurvey(thoughwithbetterexecutionandanannualorsemi-annualsurvey).Goodmeasurementofdifficultsubjectsofteninvolvesconsiderableexpense.Inrecentyears,onecanperceiveaturnawayfrom“macro”surveyssuchasthoseincorporatedintheTIandWBindexes,whichattempttomeasuretheoverallqualityofgovernanceinacountry.Instead,researchersaredeveloping“micro”surveys,whichfocuson(a)anarrowandcarefullyidentifiedsetofrespondentswhoareinasituationtoknowaboutaparticularformofcorrup-tion;(b)aparticularindustry,sector,orregion;and(c)highlyspecificquestions,mostlyaboutobjectiveeventsorfactsthattherespondentcanevaluatefrom186PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
personalexperience.59Forexample,theWorldBusinessEnvironmentSurvey,financedjointlybyseveralinternationalfinancialinstitutions,asksrespondentsabouttheexperiencesof“firmslikeyours”withbribestogovernmentofficials.60Occasionally,itispossibletomeasuretheextentofcorruptioninamoreorlessdirectand“objective”fashion(i.e.,withouttheuseofsubjectiveinter-pretationsonthepartofrespondents).InastudyofcorruptioninIndonesiaBenOlkenmanagestomeasuretheinputsandoutputsofaroadsprojectbuiltunderpubliccontract.Olkenexplains:Iassembledateamofengineersandsurveyorswho,aftertheroadsbuiltbytheprojectwerecompleted,dugcoresamplesineachroadtoestimatethequantityofmaterialsused,surveyedlocalsupplierstoestimateprices,andinterviewedvillagerstodeterminethewagespaidontheproject.Fromthesedata,Iconstructanindependentestimateoftheamounteachroadactuallycosttobuild,andthencomparethisestimatetowhatthevillagereporteditspentontheprojectonaline-itembyline-itembasis.Thedifferencebetweenwhatthevillageclaimedtheroadcosttobuildandwhattheengineersestimateditactuallycosttobuildformsmyobjectivemeasureofcorruption,whichIlabel“missingexpenditures.”61Thismeasurementstrategyfollowstheoutlinesofanormalgovernmentauditprocedure(withsomeextrabellsandwhistles).62Forexample,publicexpen-dituretrackingsurveys(PETS)followthepapertrailofgovernmentprojectsinordertodeterminewhethertherecordprovidedbyagenciesandsubcontractorsmatchesthemoneysspent.63Wherevergovernmentexpendituresculminateinadiscreteserviceorproduct,thiselementmaybedirectlyobserved(isthetrashpickedupregularly?).Truancyamongworkersisalsofairlyeasytoobserve.Ifclassroomsareemptyduringofficialschoolhours,ordoctorsarenotpresentatmedicalfacilities,thisisasignthatsomethingisamiss.64Naturally,itisdifficulttodifferentiatebetweenintentionalcorruptionandunintentionalinefficiencies.(Forsomepurposesthisdistinctionmaybecon-sequential;forothers,itmaynotbe.)Moreover,extensiveoversightproce-duresaresometimesquiteexpensive–anindirectcostthatmayrivalthecostofthecorruptionitisintendedtodeter.Finally,onemustreckonwiththepossibilitythathighlyspecificmeasurementstrategiesmightleadtomoresophisticatedstrategiesofcorruption(manymeasurementinstrumentscanbe59ReinikkaandSvensson(2006).60Galtung(2006:103).61Olken(2009:950).62InanotherstudyinIndonesia,Olken(2006)examinestheefficiencyofananti-povertyprogramthatdistributesricetothepoor.Corruptionisestimatedbycomparing“administrativedataontheamountofricedistributedwithsurveydataontheamountactuallyreceivedbyhouseholds”(Olken2006:853).SeealsoGoldenandPicci(2005).63Duncan(2006:149–150);ReinikkaandSvensson(2006).64ChaudhuryandHammer(2003).187MeasurementsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
gamed),therebyinvalidatingthemeasurementinstrument.Forexample,inresponsetoOlken’smeasurementofoutputsfromaroadproject,villageleadersmightinfuturereportinflatedfiguresofwhattheyspentonroadconstruction.Highlytargetedmeasurementinstrumentsaredifficulttogeneralizeforthesimplereasonthatactorshavestrongincentivestoevadethemonitoringregime.Bycontrast,generalsurveyquestionsabouttheextentofcorruptioninasectoraremorerobustforchangingpracticesofcorruptionandareinthissenseamenabletocross-temporalandcross-contextualcomparisons(thoughtheysufferfromproblemsofambiguity,asdiscussedabove).Fraudmayalsobeinferred(thoughnotdirectlyobserved)fromunusualpatternsofactivity.SuchatechniquehasbeendevelopedbyMalcolmSparrowtoestimatefraudintheUSgovernment-providedmedicalprogramknownasMedicaid.Sparrowfocusesonnetworksamongproviders.Typicalnetworksarefairlybroad,andbyvirtueofthatfactarelesspronetocorruptionsincemoreprovidersmustbecomplicit.Smallernetworks,bycontrast,areeasiertogame.Bylookingatthestructureofmedicalnetworks,whilecontrollingforahostofpotentialconfounders,SparrowisabletoprovideanestimateofthetotalfraudintheMedicaidsystem,aswellasaclueastoitspreciselocation.65Anotherinferentialapproachtocorruptionexaminestherelationshipbetweenpoliticallyconnectedfirmsandshareprices.Itiswidelysuspectedthatfirmsincorruptcountriesreceivepreferentialtreatmentbyvirtueoftheirpoliticalconnections.Yet,aswithmanycorruptpractices,cronycapitalismisdifficulttoprove–muchlesstomeasurewithanyprecision.RaymondFismanappliesaquasi-experimentaldesigntothispersistentissueofmeasurement.Specifically,hecompareschangesinsharepricesoffirmsthatarepoliticallyconnectedwiththosethatarenotwhenrumorsofaleader’sill-healtharecirculating.Theassumptionisthatthevalueofapoliticalconnectionisthreatenedwhenthekeypoliticalplayerisinjeopardy,andthisinsecurityshouldberegisteredinstockmarketbehavior.FismanappliesthismeasurementtechniquetoIndonesia(connectionstoSuharto)66andtotheUnitedStates(connectionstoVicePresidentDickCheney),67findingthatconnectionsmatteredagooddealtothesharepriceofcertainpoliticallyconnectedfirmsintheformercasebutnotinthelatter.68Ofcourse,therearelimitationstoameasurementinstrumentofthissort.First,itrestsonthesubjectiveperceptionsofinvestors.Iftheybelievethat65Duncan(2006:139);Sparrow(2000).66Fisman(2001).67Fismanetal.(2006).68TheOpacityIndex,developedbytheMilkenInstitute,incorporatesasimilarmeasurementstrategy,availableat:www.milkeninstitute.org/publications/publications.taf?function=detail&ID=38801146&cat=ResRep.188PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
connectionsmatter,a“connected”firmwillrespondtochangesintheirpatron’shealthstatusorpoliticalstatus,regardlessofwhetherornotthefirmactuallyreceivespreferentialtreatmentfromthegovernment.Second,itsupposesthatconnectionstoaparticularindividual,ratherthantoagrouporinstitution,arewhataffectsthefortunesofafirm.Finally,itisdifficulttoconstructatemporallyandcross-nationallycomparablemeasurementinstru-mentfromthistechniquesinceeveryhealthrumorisdifferent(somebeingmoreseriousthanothers,someoccurringinamoredramaticandunexpectedfashionthanothers),andsincethesetofcompaniestowhichaninfluentialpoliticianisattachedarediverse(introducingasetofpotentialconfounders).Anotherquasi-experimentalsettingisexploitedbyRaymondFismanandTedMigueltomeasurenationalpropensitiestoengageincorruptbehavior.TheoccasionisprovidedbyNewYorkCity’sextremescarcityofparkingspaces,thelocationoftheUnitedNationsheadquartersinthatcrowdedcity,andthediplomaticimmunitythatallowedmissionpersonnelandtheirfamiliestoavoidpayingparkingfines(priorto2002).Notethatthesecircum-stancesplacediplomatsfromallcountriesoftheworldinasituationwheretheyenjoyidenticalincentivestobreakthelaw,thatis,toparkillegally.Itfollowsthatvariationinparkingticketsmayprovideanindicatorofnormsagainstcorruptpracticesaroundtheworld.Independentverificationofthevalidityofthismeasureisprovidedbysurvey-basedcross-nationalindexessuchasthoseconstructedbyTIandWB,whicharehighlycorrelatedwithFismanandMiguel’smeasurementinstrument.Ofcourse,onemightchallengethegeneralizabilityofthesefindings:dodiplomatsbehavesimilarlyinNewYorkasintheirhomecountries?Arediplomatsrepresentativeofthepoliticalclass?Evenso,themeasurementinstrumentishighlysuggestive,foritholdsconstantmanyoftheconfoundersandthesourcesofnoisethatusuallyobscureourestimatesofthislatentconcept.69Thetopicofcorruptionhasalsobeenapproachedwithinanexperimentalframework.Cameron,Chaudhuri,Erkal,andGangadharanrecentlyconductedasetofparallellaboratoryexperimentsinAustralia,India,Indonesia,andSingaporeinordertodetermineiftheincidenceofcorruptionandanti-corruptionpracticesissimilarordis-similaracrossthesediversecultures.Theset-upbuildsonpublicgoodsexperiments,exceptthatherethevariousactionsareexplicitlylabeledwithchargedwordssuchas“bribing.”Participants(students69ItshouldbenotedthatFisman’sandMiguel’s(2007)primarypurposeinthisstudyistoassessacausalquestion–whethernormsorsanctionsaremoreimportantininfluencingcorruptbehavior.However,thestrengthofthatcausalassessmentrestslargelyonthestrengthofthemeasurementinstrument.189MeasurementsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
atuniversities)areassignedtooneofthreeroles:asamemberofafirm,agovernmentofficial,oracitizen.Thefirmisfirstgivenanopportunitytoofferabribetothegovernmentofficial,whomayacceptorreject.Bothwillbenefitifthebribeisofferedandaccepted.Thecitizenisthengivenanopportunitytosanctionthefirmandthegovernment,butchoosingtodosoinvolvesasacrifice(forthecitizen)equaltothecostofthebribe(monetaryincentivesareadjustedforpurchasingpowerparityinthefourcountries).Theresearchersfindthatthereisgreatercross-nationalvariationinwillingnesstopunishthaninwillingnesstoparticipateinbribing.70Whilethemainpurposeoftheresearchistoassesstheeffectofcultureoncorruptpractices,weshallleaveasidethequestionofcausalattributionsoastofocusontheviabilityofthemeasurementinstrument.Isthisagoodwaytomeasurecorruptpracticesacrossdiversesettings?Becauseanimportantpartofthetreatmentisverbalonemustworryaboutthetranslatabilityofthesekey-words(e.g.,“bribe”).Ifsubjectsintheexperimentarerespondingtospecificverbalcuesratherthantocommonsituationstheresultsmaynotbegeneral-izable.Onemustalsoworryaboutgeneralizingfromthebehaviorofstudents–aparticularconcerninpoorsocietieswherefewattainacollegeeducation.Eveniftheseconcernsareassuaged,itisnotclearthatbehaviorregisteredintheseexperimentswouldnecessarilytranslateintodifferencesofbehaviorintherealworld–ortheymaymapontoreal-worldbehaviorindifferentwaysacrossthesefoursettings.Thatsaid,thereismuchwecanlearnbyexaminingcorruptionincarefullycontrolledsettings.Afinalapproachtothemeasurementofcorruptionisethnographicinnature,relyingoncloseobservationandintimateacquaintancewithapar-ticularsetting.AclassicexampleisRobertWade’sstudyofirrigationsystemsinseveraldozensouthIndianvillagesinthelate1970s.71Wadereports:Onlygradually,fromconversationswithdozensofengineers,governmentofficialsfromotherdepartmentsandfarmersdiditbecomeapparentthata“system”wasawork,whichprobablyhadanimportantimpactonhowcanalsareoperatedandmaintained.Inparticular,oncesomedegreeoftrustwasestablished,farmersoftenvolunteeredinformationabouthowmuchtheyhadtopaytheIrrigationDepartment;andwhileonewoulddiscounttheirfiguresinone,twoorthreeinstances,theregularityinfarmers’statementsacrossmanyvillagesdidsuggestthatsomethingmorethanwildexaggerationorgeneralisationwasinvolved…Thisledtocautious,alwaysinformalenquiriesofofficersinotherdepartmentsandofirrigationstaffthemselves,aspartofwidercon-versationsaboutthesortsofdifficultiestheysawthemselvesfacingindoingtheirjobs70Cameronetal.(2009).71Wade(1982).SeealsoSmith(2007).190PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
well.Theseconversations,itshouldbenoted,werewithirrigationstafffromoutsidetheareaofdetailedfieldworkaswellaswithmanyservingwithinit,andcoveredtheway“thedepartment”and“thegovernment”workedinthestateasawhole,aswellasinthespecificdistrict.Someoftheengineerswerethoroughlydisgruntledatthesituationtheywerecaughtin,andsincedisgruntledpeopletendtoexaggeratethereasonsfortheirdiscontent,onehadtobecautiousaboutacceptingdetailsfromanyonepersonatfacevalue.Again,aswithfarmers,itistheregularitiesintheindividualcommentsandincidents,andtheconsistencyintheordersofmagnitude(asbetween,forexample,whatadistrictCollectortoldmeaSuperintendingEngineerhadtoldhimhehadhadtopaytogetaone-yearextension,andwhatanAssistantEngineerinoneDivision–inanotherdistrictfromthefirstcase–saidinstrictestconfidencehisExecutiveEngineerhadhadtopaytogetthetransfer)thatgivesconfidenceinthecorrectnessofthebroadpicture.Wade’sdetailedresearchnarrativeprovidesacleardescriptionofthisapproachtomeasuringcorruption,itspromisesanditspitfalls.Evidently,ethnographicinvestigationsarepossibleonlywheninformantscanbeassuredofanonymity,asispossible(sometimes)withworkofanacademicnature.Iftheinvestigatingbodyisgovernmental,mouthsarelikelytoshut.Andifanyrepercussionsfollowfromareport,nofurtheraccesstotheresearchsiteislikelytobeforthcoming.Moreimportantly,thissortofintensivesoakingandpokingdoesnotprovidefodderforsystematiccomparisonsacrosstimeandspace,oracrosssectors.Thoughthemeasurementsmaybequiteprecise–Wadeoffersestimatesofhowlargethetypicalbribeisinavarietyofspecificsettingsconnectedtoirrigationinthestudiedvillages–onecannotderiveageneralizablemeasure-mentinstrument.Evenso,onecanlearnalotfromiteratedconversationsonceonehasgainedthetrustofinformantsandacertainamountofsavvyaboutthesubjectunderinvestigation.Moreover,themeaningsembeddedintheactionsunderstudyaremorelikelytobeinterpretablewhengainedthroughanethnographicstyleofinquirythanwhengarneredfromtheotherapproacheswehavereviewed.ExpostvaliditytestsHavingdiscussedvariousstrategiesofmeasurement,weconcludethischapterwithabriefdiscussionofexpostteststhatmayhelptoshedlightonthevalidityofachosenindicator.Facevalidityisnotreallyatestatall.Itreferstoanobviousorintuitiveappeal–anindicatorthatseemsrelatedtoaconceptinawaythatobviatessystematicempiricaltesting.Elections,onemightargue,areanindicator(thoughnotnecessarilytheindicator)ofdemocracywithhighfacevalidity.191MeasurementsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
Convergentstrategiesattempttovalidateanindicator,I,bycomparingitwithothermeasuresthataredeemedtobevalidmeasuresofthesameconcept,I2.Ahighcorrelationdemonstratesconvergentvalidity.Convergentvaliditystu-dieshaveshownthattheleadingindicatorssuchasFreedomHouseandPolityarehighlyintercorrelated,renderingaPearson’srcorrelationof0.88acrossallcountriesinrecentdecades,andthishasbeeninterpretedasevidencethatallsuchindicatorsarevalid.72Ofcourse,theoperatingassumptionisthataddi-tionalindicatorsofaconceptarethemselvesvalid.Ifthemeasuresthatcomposeaconvergentvaliditytestaresubjecttobiasthetechniqueholdslittlepromise.Discriminantstrategiesattempttodistinguishtheentitiesbelongingtotheconceptofinterest,asmeasuredbyI,fromthosepresumedtobelongtoneigh-boringconcepts,C.AlowcorrelationbetweenIandCdemonstratesdivergentvalidity.Thisstrategyislesscommon,atleastwithrespecttodemocracy(perhapsbecausethebordersofthisambientconceptaresohardtoidentify).Causalstrategiesattempttovalidateameasurebylookingatits(presum-ablycausal)relationshiptoaninputoroutputtowhichitispresumedtobestronglyrelated.Astrongrelationship(inthepredictedrelationship)mayberegardedasprovidingconfirmationforameasure.Writershaveattemptedtogaugethevalidityofcross-nationaldemocracyindicatorsbyexaminingwhethertheyareresponsivetofactorspresumedtobeunrelatedtodemoc-racy,suchasshiftsinUSforeignpolicy(functioninghereasaconfounder).IfUSforeignpolicyappearstopredictcodingchangesonFreedomHouse’sPoliticalRightsIndexonemightpresumethattheindexismeasuringsome-thingotherthanwhatitpurportstomeasure.Case-basedstrategiesexaminekeycasestoseeifthecodingforthesecasescorrespondstotheexpectedpattern.73Forexample,scholarsofCentralAmericahaveshownthatscoresforthesecasesareoftenpatentlyerroneous,andcannotbeaccountedforbychanceerror.Thissortofinvestigationrestsonascouringofprimaryandsecondarysourcesforthecountriesinquestion,includinglocalnewspapers,governmentdocuments,andUSdiplomaticcorrespondence,as72However,oncloserexamination,itappearsthatconsensusacrossthetwodominantindicesislargelytheproductofcountrieslyingatthedemocraticextreme–Sweden,Canada,theUnitedStates,etc.Whencountrieswiththehighestdemocracyscoresareexcludedfromthesampletheintercorrelationbetweenthesetwoindicesdropsto0.78.AndwhencountrieswiththetoptwoscoresontheFreedomHousescale(1–2outof7)areeliminated,Pearson’srdropsagain–to0.63.Thisisnotanimpressivelevelofagreement,especiallywhenoneconsidersthatscholarsandpolicymakersareusuallyinterestedinpreciselythosecountrieslyinginthemiddleandbottomofthedistribution–countriesthatareundemocraticorimperfectlydemocratic.CoppedgeandGerring(2011).SeealsoGoertz(2008);HadeniusandTeorell(2005).73Bowman,Lehoucq,andMahoney(2005).192PartIIDescriptionDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
wellasinterviewswithlocalinformants–afarmoreextensivereviewthaniscommoninmostcross-nationalcodingoperations.74Ofcourse,theexanteconstructionofameasureandtheexposttestingofthatvariablearenotrigidlysegregatedfromoneanother.Mostmethodsofvalidationcanalsobeemployedasmeasuresofvariableconstruction.Indeed,validationtestsoftheconceptofdemocracyareoftenperformedasapreludetotheconstructionofanewindex.Thisbringsusfullcircle.74Bowman,Lehoucq,andMahoney(2005).193MeasurementsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
Downloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:06 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.010Cambridge Books Online © Cambridge University Press, 2012
PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:45 BST 2012.http://ebooks.cambridge.org/ebook.jsf?bid=CBO9781139022224Cambridge Books Online © Cambridge University Press, 2012
Downloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:45 BST 2012.http://ebooks.cambridge.org/ebook.jsf?bid=CBO9781139022224Cambridge Books Online © Cambridge University Press, 2012
Cambridge Books Onlinehttp://ebooks.cambridge.org/Social Science MethodologyA Unified FrameworkJohn GerringBook DOI:http://dx.doi.org/10.1017/CBO9781139022224Online ISBN: 9781139022224Hardback ISBN: 9780521115049Paperback ISBN: 9780521132770Chapter8 – Causal arguments pp. 197-217Chapter DOI:http://dx.doi.org/10.1017/CBO9781139022224.012Cambridge University Press
8CausalargumentsSurely,iftherebeanyrelationamongobjectswhichitimportstoustoknowperfectly,itisthatofcauseandeffect.Onthisarefoundedallourreasoningsconcerningmatteroffactorexistence.Bymeansofitaloneweattainanyassuranceconcerningobjectswhichareremovedfromthepresenttestimonyofourmemoryandsenses.Theonlyimmediateutilityofallsciencesistoteachushowtocontrolandregulatefutureeventsbytheircauses.Ourthoughtsandenquiriesare,therefore,everymoment,employedaboutthisrelation:Yetsoimperfectaretheideaswhichweformconcern-ingit,thatitisimpossibletogiveanyjustdefinitionofcause.DavidHume1IarguedinPartIIforaresuscitationofdescriptiveinferencewithinthesocialsciences,bothasatopicofmethodologyandasatopicofsubstantiveresearch.However,Idonotsupposethatdescriptionwilldisplacecausationasthereigningmotifofsocialscience.Wewishtoknownotonlywhathappenedbutalso,perhapsmorecritically,whythesethingshappened.Causationisthecentralexplanatorytropebywhichrelationshipsamongpersonsandthingsareestablished–thecementoftheuniverse,inHume’smuchquotedwords.2Withoutsomeunderstandingofwhoisdoingwhattowhomwecannotmakesenseoftheworldthatwelivein,wecannotholdpeopleandinstitutionsaccountablefortheiractions,andwecannotactefficaciouslyintheworld.Withoutacausalunderstandingoftheworlditisunlikelythatwecouldnavigateeventhemostmundanedetailsofourlives,muchlessmattersoflong-termpolicy.Thisisobviousinthepolicyworld,wherecausalunderstandingundergirdsanyrationalintervention.Anditisobviousinotherareasofpolitics,forexample,insocialmovements,lobbying,voting,andrevolutionarychange.Anyonewhoengagesintheseactivitiesmustbeconsciousofoneselfasacausalactorintheworld,andaccordingly,mustmakeassumptions(implicitorexplicit)aboutwhatone’sactionsmightachieve–whetheronesupportsthestatusquoorwishestoundermineit.1Hume(1960:220).2Hume(1888).197Downloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:44 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.012Cambridge Books Online © Cambridge University Press, 2012
Lenin,likeMetternich,wasvitallyconcernedwiththecausesofrevolution.Evenwherecausalunderstandingdoesnotleadtosocialchange(fornotallcausalanalysisisdirectlyrelevanttopublicpolicy,andmoretothepoint,notallpolicyproposalsareimplemented),wearelikelytobereassuredwhenwecanordereventsaroundusintocause-and-effectrelationships.“Whenwehavesuchunderstanding,”notesJudeaPearl,“wefeel‘incontrol’evenifwehavenopracticalwayofcontrollingthings.”3Oneimportantpurposeofcausalinferenceistoofferinsightintowhatmayhappeninthefuture.Althoughthereisanunfortunatetendencytodichot-omizecausalandpredictiveknowledge,thedistinctionisrarelyhardandfast.Considerthatfewcausalargumentsareentirelyrestrictedtothepasttense.TosaythatXcausesYistoimply(usually)thatitwillcontinuetodoso–perhapsnotindefinitely,butatleasttomorrowandnextyear.Thus,althoughpredic-tionisbynomeansidenticaltocausation,forecasting(onetypeofprediction)isimpliedbymostcausalargumentsinthesocialsciences,whichtendtofocusoncontemporaryphenomena,orphenomenainthepastthathavecontem-poraryrelevance.Notethateverypolicyintervention–everyratechangebyacentralbank,everysocialprogram,andeveryreformofthetaxcode–impliesapredictionaboutthecausaleffectsofthatintervention.Manyoftheseforecaststurnouttobesubstantiallycorrect.Allarebasedprimarilyoncausalmodels,formalorinformal.Inanycase,thosewhocomplainabouttheinabilityofsocialsciencemodelstoofferreliableforecastsofthefuturearenotusuallyarguingfortheabandonmentofcausalmodels.Thereis,asfarasIcansee,noviablealternative.Problemsofcausalitywillbedividedintofourareas.Thischapterdefinescausalityandlaysoutcriteriapertainingtoallcausalarguments.Chapter9discussesgeneralcriteriaofcausalinference(i.e.,analysis).Chapters10and11exploreavarietyofapproachestocausalanalysis.Chapter12servesasacodatoPartIIIofthebook,incorporatingseveralapproachestocausalitythatappeartolieoutsidethepresentframework.DefinitionsCausaltheoriesinvolveatleasttwoelements:acausalfactorandanoutcome.Sometimes,severalfactorsand/oroutcomesarecombinedinanabstract3Pearl(2000:345).SeealsoBunge(1959);Homans(1961);MacIver([1942]1964:5–11);Mackie(1974);Sloman(2005);Woodward(2005).198PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:44 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.012Cambridge Books Online © Cambridge University Press, 2012
theory.However,thattheorymustbetranslatableintospecifichypothesesinvolvingindividualcausalfactorsandanoutcome.Informalizingtheseelementsageneraltheoryistransformedintoacausalmodel.Confusingmatters,therearevarioussynonymsfortheseterms.Acausemaybereferredtoasacausalfactor,condition,covariate,exogenousvariable,explanatoryvariable,explanans,independentvariable,input,intervention,parent,predictor,right-sidevariable,treatment,orsimply“X.”Anoutcomemaybereferredtoasadependentvariable,descendant,effect,endogenousvariable,explanandum,left-sidevariable,output,response,or“Y.”(Ofcourse,therearesubtledistinctionsamongtheseterms.However,forpresentpur-posesthesimilaritiesaremoreimportantthanthedifferences.)Whatevertheterminology,tosaythatafactor,X,isacauseofanoutcome,Y,istosaythatachangeinXgeneratesachangeinYrelativetowhatYwouldotherwisebe(thecounterfactualcondition),givencertainbackgroundcondi-tions(ceterisparibusassumptions)andscope-conditions(thepopulationoftheinference).Thiswillserveasaminimaldefinitionofcausality.GiventheimportanceofvariationinXandY,itmaybehelpfultothinkofXas∆X(“deltaX”)andYas∆Y(“deltaY”).IftherelationshipbetweenXandYiscausal,achangeinXgeneratessomechangeinY:∆X→∆Y(atleastprobabilistically).Whenanoutcomeiscontinuous,∆XaffectsthevalueofYalongsomescale,whichmaybeunboundedorbounded.Whenanoutcomeisbinary(Y=0,Y=1)ormultichotomous(e.g.,Y=1,2,3,4,or5),∆Xaffectstheprobability(P)ofYachievingoneoftheseoutcomes.WhateverthenatureofXandYthereisalwaysanimpliedcounterfactual:ifXvaries,Yshouldalsovaryinsomemanner(atleastprobabilistically).Acausaltheorymustexplainwhyonethinghappened(happens)andsomeotherthingdid(does)not.Anotherwayofframingthisissueistosaythatacauseraisesthepriorprobabilityofanoutcomeoccurring.Letusassumetwofactors,XandY,eachofwhichassumesoneoftwopossiblevalues,0and1.WeshalldenoteX=1asXandX=0asx,Y=1asYandY=0asy.Inthenotationofprobabilitytheory,XcausesYif,andonlyif,P(Y|X)>P(Y|x),4withasetofunderstoodback-groundconditions.5Whilethisdefinitionofacausaleffectmayseemprejudicialtoprobabilisticcauses,itcanbeseenthatset-theoreticcausesalsofitwithintherubric.IfXis4TheprobabilityofYgivenXisgreaterthantheprobabilityofYgivennot-X.5Cartwright(1983);Dupre(1984:170);Guala(2005:82).199CausalargumentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:44 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.012Cambridge Books Online © Cambridge University Press, 2012
anecessaryconditionforY,thenP(Y|x)=0whileP(Y|X)>0.Thatis,thechangefromxtoXraisestheprobabilityofYfrom0tosomeundefinedprobabilitygreaterthan0,aslongasXisanontrivialnecessarycondition.IfXisasufficientconditionforY,thenP(Y|x)<1whileP(Y|X)=1.Thatis,thechangefromxtoXraisestheprobabilityofYfromsomethinglessthan1to1,aslongasXisanontrivialsufficientcondition.Furtherdiscussionofset-theoreticcausesispostponeduntilChapter12.Importantly,whenoneassertsthatXcausesYoneisassertingthattheactual(ontological)probabilityofaneventisincreasedbyX,notsimplyatheory’spredictivecapacity.Thisiswhatdistinguishesacausalargumentfromadescriptionorprediction.Tobecausal,thefactorinquestionmustgenerate,create,orproduceaneffect.Ofcourse,itisnotalwayspossibletospecifypreciselywhyXgeneratesY.YetinidentifyingXasacauseofYoneispresumingtheexistenceofsomecausalmechanism–understoodhereasthepathwayorprocessorchainofintermediaryvariablesbywhichXaffectsY,illustratedasMinFigure8.1.Causalrelationshipsoccuragainstabackgroundofotherfactors.Thesearetheconditionsthatmakeanycausalrelationshippossible.Notethatevenanexperimentconductedinaperfectvacuumpresumesabackgroundthatprovidestheconditionsfortheexperiment–inthiscase,thevacuum.BackgroundfactorsincludeallfactorsotherthanX(thefactoroftheoreticalinterest)thatmayinfluencetheoutcome,directlyorindirectly.Onesortofbackgroundfactor,labeledAinFigure8.1,liesantecedenttothecausalfactorofinterest.ItaffectsYindirectly,throughX.Unlessotherwisespecified,backgroundconditionsarepresumedtoholdconstant:theydonotvary.Thisisknownastheceterisparibus(allelseequal)assumption,andisimplicitinallcausalarguments.Forexample,whenconstructinganargumentaboutthecausalimpactofeconomicdevelopmentindemocratizationonemustassumethatotherfactorsaffectingdemocratiza-tion,suchasnaturalresourcewealth,religion,politicalculture,andinterna-tionalinfluences,areconstant.ThisisnotsimplyaproblemofempiricalX = Causal factor Y = Outcome M = MechanismA = AntecedentAXMYFigure8.1Asimplecausalgraph200PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:44 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.012Cambridge Books Online © Cambridge University Press, 2012 testing;itisinherentintheveryactofmakingacausalargument.Withoutceterisparibusconditions(implicitorexplicit),causalargumentsareimpos-sible.Ofcourse,onecanchangetheceterisparibusconditionsofacausalargumentbyspecifyinghowbackgroundfactorsinteractwithX,orbyalteringthescope-conditionsoftheargument.Weshallhavemoretosayaboutthisinlaterchapters.Causalfactorsarealsooftenclassifiedaccordingtotheirrelativedistancetotheoutcomeofinterest.FactorsclosetoYarereferredtoasproximate.FactorsdistantfromYaredistal(akaremote,structural).InFigure8.1,MismostproximateandAismostdistal.Ofcourse,causalregressispotentiallyinfinite.WecanimaginecausesofA,causesofthecausesofA,andsoforth.Likewise,wecouldinsertcausalmechanismsinbetweenMandY,whichwouldthenberegardedasmostproximaterelativetoY.Thenotionofadistalorproximatecauseisalwaysrelativetosomeothersetofpositedcausalfactors.Generally,distalcausesarethoseinwhichseveralmediatingfactorsseparateX(thevariableoftheoreticalinterest)fromYandinwhichXandYareseparatedbyastretchoftime.Relatedly,itisoftenimportanttodistinguishbetweenfactorsthatareindependentorexogenous(causalinnature),andfactorsthataredependentorendogenous(outcomes).InFigure8.1,AisexogenoustoX,M,andY;XisexogenoustoMandY;MisexogenoustoY.Likewise,YisendogenoustoA,X,andM;MisendogenoustoAandX;XisendogenoustoA.Endogeneity/exogeneityisalsoarelativematter.AllfactorsinFigure8.1maybetreatedasvariables.Bythis,wemeanthattheyareassumedtovary–evenifonlyhypothetically(asinacounterfactualthought-experiment).6Inthesimplestscenario,Xassumestwopossiblevalues(X/x)andYassumestwopossiblevalues(Y/y).VariationalongXandYmayalsobemulti-categorical(e.g.,Catholic/Protestant/Jewish/Muslim),ordinal(e.g.,aLikertscale),ornumeric(anintervalorratioscale).Inshort,XandY,alongwithothervariableslistedinFigure8.1,mayembodyanyofthescaleslaidoutpreviouslyinChapter7.Theymayrepresentevents(dynamic,swift,anddiscrete)orprocesses(dynamicandslow).Theymayalsotaketheformofstaticconditions,thoughtheymustbechangeableinprinciple.Thus,insayingthatageographicfactorsuchasaltitudeordistancefromtheequatorexertsacausalforceonsomeoutcomeoneisimplicitlyacknowledgingthepossib-ilitythatitcouldhavebeendifferentinthepastorcouldbedifferentinthefuture.6TetlockandBelkin(1996).201CausalargumentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:44 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.012Cambridge Books Online © Cambridge University Press, 2012 Itistruethatsomefactorsaredifficulttomeasureanddifficulttocon-ceptualizecounterfactually,andthereforedon’tconformtoourtraditionalsenseofa“variable.”Still,itmustbepossibletoconceptualizeaprocessassomethingthatvaries;otherwise,itcandonocausalwork.Ifsomethingcannotbeotherthanitisthenitcannotserveasacause.IntreatingfactorsinFigure8.1asvariableswearesayingthattheyvary,atleastpotentially.Wearenotimplyingthattheyareeasilyconceptualizedandmeasured,orthattheycanbedirectlymanipulated.AddingtothecomplexityofFigure8.1,eachfactormaybeunderstoodasanindividualvariableorasavector(set)ofvariables.Indeed,weareofteninterestedincombinationsofcausalfactors,combinationsofcausalpathways,combinationsofantecedentcauses.Occasionally,wemaybeinterestedinmultipleoutcomes.7Mostofourexampleswillconcernindividualfactors,buttheseexamplescanusuallybegeneralizedbytreatingthevariableasavector.Finally,itshouldbeclarifiedthatinemployingtheterminologyof“vari-ables”wearenotsupposingthatallcausalmodelsarestatisticalinnature.Nothingintheforegoingpassagesnecessitatesalargesampleoraprobabilisticmodel.Ifindthelanguageofvariablestobeaconvenientmeansofsimplifyingandunifyingourunderstandingofcausation.Itisnotmeanttoenforceauniformmethodofcausalanalysis(asubjectaddressedinsucceedingchap-ters).Norisitintendedtoeliminatetheuseofothernear-synonyms(e.g.,causes,conditions,factors,influences),whichwillbeemployedsporadicallyinthefollowingchapters.CausalcriteriaHavingdefinedcausationminimally,weturntotheideal-type.Whatisagoodcausalargument?RecallfromChapter3thatallargumentsstrivefortruth,precision,generality,boundedness,parsimony,coherence,commensurability,andrelevance.Iwillargueherethatcausalexplanationsinsocialsciencealsostriveforclarity,manipulability,differentiation,genesis,impact,andamechanism.Forconvenience,allfourteencriteriaapplicabletocausaltheories7Notethatsinceweareprimarilyconcernedinthisbookwithclassesofeventsratherthansingularevents(Chapter1),anumberofdifficultphilosophicalandpracticalproblemsofinferenceareminimized.Onethinks,e.g.,of“pre-emption”andvariousadditionalissuesthatarisewhenattemptingtodeterminethecauseofasingularevent.SeeBrady(2008);Lewis(1973).Thatsaid,itshouldbepointedoutthatinsofarasourknowledgeofcausalrelationsamongaclassofeventsbuildsuponourknowledgeofcausalrelationsofspecificevents,thesortsofpracticalandphilosophicalproblemsposedbysingularcausationarenotresolved.202PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:44 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.012Cambridge Books Online © Cambridge University Press, 2012 arereproducedinTable8.1.However,ourfocusinthischapterisonfactorsthatdistinguishcausalpropositionsfromdescriptivepropositions(Nos.9–14).8Table8.1Causalarguments:criteriaARGUMENTS(Chapter3)1.Truth(accuracy,validity,veracity)Isittrue?2.Precision(specificity)Isitprecise?3.Generality(breadth,domain,population,range,scope)Howbroadisthescope?4.Boundedness(population,scope-conditions)Howwellboundedisit?5.Parsimony(concision,economy,Occam’srazor,reduction,simplicity)Howparsimoniousisit?Howmanyassumptionsarerequired?6.Coherence(clarity,consistency;antonym:ambiguity)Howcoherentisit?7.Commensurability(consilience,harmony,logicaleconomy,theoreticalutility;antonym:adhocery)Howwelldoesitcumulatewithotherinferences?Doesitadvancelogicaleconomyinafield?8.Relevance(everydayimportance,significance)Howrelevantisittoissuesofconcerntocitizensandpolicymakers?CAUSALARGUMENTS(thischapter)9.Clarity(antonym:ambiguity)WhatistheenvisionedvariationonXandY,thebackgroundconditions,andthescope-conditionsoftheargument?CanXandYbeoperationalized?10.ManipulabilityIsthecausalfactormanipulable(atleastpotentially)?11.Separation(differentiation;antonym:tautology)HowseparableisXrelativetoY?12.Independence(foundational,original,prime,prior,structural,unmovedmover)IsXindependentofothercausesofY?13.Impact(effectsize,magnitude,power,significance,strength)HowmuchofthevariationinYcanXexplain?Isthecausaleffectsignificant(intheoreticalorpolicyterms)?14.Mechanism(intermediary,mediator,pathway,process)HowdoesXgenerateY?Whatarethecausalmechanisms(M)?8Otherattemptstospecifythedesiderataofcausalargument,uponwhichthiseffortbuilds,canbefoundinEckstein(1975:88);Hempel(1991:81);King,Keohane,andVerba(1994:ch.3);Kuhn(1977:322);Lakatos(1978);Laudan(1977:68,1996:18,131–132);Levey(1996:54);MariniandSinger(1988);PrzeworskiandTeune(1970:20–23);SimowitzandPrice(1990);Stinchcombe(1968:31);vanEvera(1997:17–21);Wilson(1998:216).203CausalargumentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:44 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.012Cambridge Books Online © Cambridge University Press, 2012 ClarityWehavedefinedcausalityasasituationinwhichachangeinX(thecausalfactoroftheoreticalinterest)generatesachangeinY(theoutcomeofinterest)relativetowhatYwouldotherwisebe,givencertainbackgroundconditionsandscope-conditions.Itfollowsthatagoodcausalargumentshouldprovideclarityalongeachofthesedimensions.Clarifying(akaspecifying,operatio-nalizing)causaltheoriesmakesthemmoreuseful,aswellaseasiertotest.Indeed,atheorythatishighlyambiguousisimpossibletoverifyorfalsify;itisneithertruenorfalse.Onemustwrestleattheoutsetwithterminologicalambiguities,forthereareagreatmanywaystoarticulateacausalclaim.Writersmaystatethatacausalfactor,X,leadstoanoutcome,Y,isrelatedtoY,isassociatedwithY,influencesY,resultsinY,andsoforth.Ofthese,onlythelasttwoareclearlycausalinthesenseinwhichwehavedefinedtheterm.Butallmaybecausal,dependinguponthecontext.Asimplesuggestionforwritersistoclarifywhetheranargumentisintendedtobecausalornot.Intuitiononthepartofthereadershouldnotberequired.AsecondissueisthespecificationofY.ToevaluateanargumentweneedtoknowthevariationinYthatisunderstoodastheoutcomeofinterest.Usually,thisisapparent;butsometimesitremainsambiguous.Thehumorous(andpresumablyapocryphal)taleistoldofapriestwhoqueriedthenotoriousbankrobber,WillieSutton,aboutwhyherobbedbanks.Tothis,themiscreantpatientlyexplainedthatthisiswherethemoneyis.Evidently,thepriestandthebankrobberhavedifferentideasaboutvariationinY.Forthepriest,itisrobbing(Y=1)versusnotrobbing(Y=0).ForSutton,itisrobbingbanks(Y=1)versusrobbingotherestablishments(Y=0).Ananalogousconfusionarisesinsomehistoricalargumentsoverspecificoutcomes,forexample,arevolution,awar,orthepassageofabill.Forsomewriters,theoutcomeofinterestmaybeunderstoodinadichotomousfashion(revolution/norevolution)andoveraperiodofmanyyears.Why,forexam-ple,didFranceexperiencearevolutionwhileSwedendidnot?Forotherwriters,theoutcomeofinterestmaybeunderstoodinmuchmorespecificterms.Why,forexample,didtheFrenchRevolutionoccurin1789andinpreciselythemannerinwhichitdid?Itiseasytoseehowtwostudiesofwhatisnominallythesamephenomenon(theFrenchRevolution)mayendupconstructingverydifferentarguments.Aswearedealinginthisbookwithclassesofevents,ratherthansingularevents,thisgenreofproblemissome-whatlessprevalent.Evenso,aclassofoutcomesmaybevariouslyinterpreted,204PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:44 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.012Cambridge Books Online © Cambridge University Press, 2012 andunlessthismatterisclarifiedtherewillbenoclarityabouttheoverallsetofclaims.AthirdissueisthespecificationofX,thatis,thechangeinXthatisenvisionedasacausalfactorortreatment.Thisisthecausalcounterfactual,anditmustbespecified,evenifitcannotbedirectlyobservedormanipulated(anissuediscussedinthenextsection).Ananalogousproblemisraisedbycausalinputsoroutputsthataredifficulttooperationalize(i.e.,tomeasure).Inthissituation,itmaybeconceptuallyclearwhatismeantbyachangeinXorY,butempiricallyambiguous.Aconceptthatcannotbemeasuredcannotbetested–atleastnotveryprecisely.Likewise,ifaconceptcanbeoperationalizedwithavarietyof(poorlyinter-correlated)indicators,thenthisambiguityimpairsitsfalsifiability.Itissomewhatproblematic,forexample,thatdemocracycanbemeasureddichot-omouslyorcontinuously,andthateachchoiceofmeasurementoffersanumberof(notsohighlycorrelated)indicators,asdiscussedinChapter7.Schoolvouchers,althoughseeminglymorespecific,canbeconstructedinanynumberofways(e.g.,byvaryingthemonetaryvalueofvouchersortheregulationsassociatedwithavoucherprogram).Vouchersmaybeappliedtochoicesamong“charter”schools(publicschoolswhoseenrolmentsarenotconstrainedtoaparticularneighborhood)ortoprivateandpublicschools.EachofthesedecisionsaboutXhasdifferentimplicationsforY.ThesimplepointisthatinordertoachieveclarityinacausalargumentitmustbepossibletolocateXandYempirically.Operationalizationisessential.9Afourthissueconcernsthebackgroundconditionsofanargument.Undernormalcircumstances,itisnotnecessarytospecifywhattheseare.Thus,ifoneisarguingthatcountriesaremorelikelytodemocratizewhenmoreeconomicallydeveloped,oneisassumingthatallotherfactorsimpactingXand/orYareheldconstant.Thiswouldincludeconditionssuchasmineralwealth,whichmanycommentatorsregardasanimpedimenttodemocracy.10Ifunmentioned,thesefactorsshouldberegardedasceterisparibusconditions:thatis,economicdevelopmentfostersdemocratizationforcountrieswithsimilarnaturalresourceendowments.Sometimes,however,thebackgroundconditionsofanargumentareimportantenough,andambiguousenough,thattheyreallyoughttobementionedexplicitly.Thiswillfurtherclarifythe9Notethatthisisdifferentfromtheproblemofmanipulation.Amanipulablecausemaynonethelessremainambiguous(as,e.g.,whenanargumentaboutvouchersdoesnotspecifyhowavouchersregimewillbeoperationalized).Likewise,anoperationalcausalfactormaybenonmanipulable(as,e.g.,inequality,whichcanbepreciselymeasuredbutisdifficulttomanipulate).10Dunning(2008a);Ross(2001).205CausalargumentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:44 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.012Cambridge Books Online © Cambridge University Press, 2012 natureoftheargument,andindicatestothereaderthattheauthorhasconsideredapotentiallyconfoundingfactor.Afifth(andcloselyrelated)issueconcernsthescope-conditions,orpopula-tion,oftheinference.Aswehavesaid,allcausalargumentshavescope-conditions,evenwhentheseremainimplicit.Typically,scope-conditionsarecontainedwithinthekeyconceptsthatarticulateacausalargument.So,anyargumentabouteconomicdevelopmentanddemocracypresumescertainthingsabouttheunitsofanalysis–thattheyarelargepoliticalunits,forexample.Supposesomeoneobjectsthatwhenfamiliesbecomewealthiertheydonotnecessarilybecomemoredemocratic.Whilethismaybetrue,itisnotreallyacounterargumentbecausethetopicliesoutsidethepresumedscopeoftheoriginalproposition.Thedistinctionbetweenabackgroundconditionandascope-conditionisnotalwaysclear,andthereforebearssomediscussion.Suppose,forexample,thateconomicdevelopmenthasdifferenteffectsondemocracywhendevel-opmentisspurredbynaturalresourcewealthasopposedtosomeothereconomicfoundation.Thisissuemaybetreatedasabackgroundcondition,inwhichcaseitisunderstoodasaceterisparibusassumption:incountrieswithsimilarlevelsofnaturalresourcewealth,economicdevelopmentwillhavesimilareffectsondemocratization.Oritmaybetreatedasascope-condition,inwhichcasetheauthormaystatethatthepositedrelationshipbetweeneconomicdevelopmentanddemocracyholdsonlyforcountrieswithlowlevelsofnaturalresourcewealth:resource-richcountrieslieoutsidethescope(population)oftheargument.(Athirdoptionistoexplicitlytheorizetherelationshipbetweenresourcewealth,economicdevelopment,andde-mocratization.However,thischangesthecausalargument–broadeningittoincludethreefactorsratherthantwo–andsoisnotaboutbackgroundconditionsorscope-conditions.)Readerscanreadilyseethetradeoffimpliedinthischoice–tomaintainalargescopewhileacceptingagooddealofbackgroundnoiseortoreducenoisebynarrowingthescope.Specifically,thetradeoffisbetweengenerality,ontheonehand,andprecisionandimpact,ontheother.Tobesure,thereisoftenadegreeofambiguitysurroundingthescope-conditionofacausalargumentinsocialscience.Forexample,theoriesofdemocratizationareusuallyunderstoodtoapplytosovereignnation-states.However,theargumentmightalsoapplytosubnationalunits(regionalandmunicipalgovernments),tosemi-sovereigncoloniesandprotectorates,andeventoothertypesoforganizations(e.g.,socialmovements,interestgroups,andsoforth).Theboundsofthisargument,likemanyothers,are206PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:44 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.012Cambridge Books Online © Cambridge University Press, 2012 notentirelyapparent.Typically,thereisahometurfwheretheargumentreallymustapply:thebestpossiblescenarioforXtoaffectY.Inthisinstance,itisprobablythesovereignnation-state.Beyondthishometurf,onemayencounteraseriesofconcentriccircleswherethelogicoftheargumentseemsmoreandmoretenuous–thoughstillplausible.Likewise,thetemporalscopeofanargumentdemandsconsideration.Typically,authorsapplythedevelopment/democratizationthesistothecon-temporary(twentieth-/twenty-first-century)era.Butitmightalsobeappliedtoearlierperiods,evenasfarbackasancientGreece.Norisitclearhowfarintothefuturethisrelationshipmighthold.Willdevelopmentenhancedemocracyintothetwenty-secondcentury?Asarule,thetemporalboundsofsocialscienceargumentsarelessclearthantheirspatialboundaries,pre-ciselybecauseoftheambiguityoffuturityandthecontinuousnatureoftime(whichstretchesbackwardalonganinfinitearc,withnoclearcut-offpoints).Furtherdiscussionofappropriatescope-conditionsisfoundinChapter3(see“Boundedness”).ManipulabilityIdeally,thetreatmentofprimarytheoreticalinterestshouldbeamenabletomanipulation,thatis,deliberatechangebytheresearcher(orsomeone).Ifitisnot,thentheargumentwillbeverydifficulttoevaluate(nottomention,totest).Manipulationistocausalargumentswhatoperationalizationistoconceptualarguments(seeTable5.1).Itclarifieswhatitiswearetalkingabout.Thisideaisimplicitinthecommonunderstandingofacauseassomethingthatchanges.Byaskingthequestionofmanipulabilityweareasking,ineffect,foraclarificationofwhataspectoftheworldchangesandwhatstaysconstant(thebackgroundconditionsofanargument).Asanexample,letusreturntoourperennialexemplars.Vouchersaremanipulableinprincipleandinfact.Assuch,anyargumentaboutvouchersisclearenoughwithrespecttoX;onehasonlytospecifywhatthevouchersregimeconsistsof(X=1)andwhatthenon-vouchersregimeconsistsof(X=0).Thisdefinesthetreatmentandcontrol,andhenceclarifiestheargument.Withdemocracy,however,considerableambiguitypersistsaboutthetreatment,andthisinturnisaproductofthefactthatdemocracyisdifficulttoimagineasamanipulabletreatment.Whataboutdemocracywouldbemanipulated(changed)?Notethatsomeelementsoftheconceptaredirectlychangeable,suchaselectorallaw.Onecanre-writeprovisionsofstatuteandconstitutionallaw.Otherelements,suchascompetitiveelections,arenot207CausalargumentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:44 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.012Cambridge Books Online © Cambridge University Press, 2012 directlymanipulable.Onecancreatetheconditionsforpartycompetition,butachievingcompetitiondependsonmanyfactorsthatareoutsideanyone’sabilitytodirectlycontrol.Ifpeoplecontinuetosupportthedominantparty(oftheirownfreewill)thereisnotmuchthatanexperimentercandotoalterthisfact.Inanycase,thethingsthatonecoulddirectlyalter–suchaselectorallaw–arethethingsthatareeasytoconceptualizeascauses.OnecanimagineXasacauseifonecanimaginechangingX,whileleavingeverythingelseasitis(theceterisparibusconditionsofthecausalargument).Ifitisnecessarytochangeotherthingsinorderto(possibly)getXtochange,thenoneisdealingwithanambiguouscausalargument.Impedimentstomanipulabilitystempartlyfromthefreewillofsocialactors.Recallthatsocialscienceisdistinguishedfromothersciencesbyitsfocusondecisionalbehavior,thatis,actionsinwhichthereisachoiceelement.Thisposesadilemmatoexplanation:namely,thatsomethingimportantliesinbetweenthingsthatwecanmanipulateandoutcomeswewanttoexplain.Thissomethingisactors’emotionalandcognitivestatesofmind,whichwecannotdirectlymanipulatepreciselybecausetheyaresubjecttofreewill.Wecandothingstofacilitatefeelingsofanger,love,ordesire,butwecannotmanipulatethesepsychologicalstatesdirectly.Likewise,wecandothingstofacilitateorimpairthedevelopmentofintelligence,butwecannotdirectlymanipulateintelligence.Asaconsequence,thereisanineffablequalitytoexplanationsthatrestonmentalstatesorconditions.Additionalexamplesincludetrust,legitimacy,cognitivedissonance,adaptivepreferences,or(thatoldstandby)rationality.Anothercategoryofexplanationrestsonthevolitionalbehaviorofgroupsofpeople.Conceptssuchascompetition,equilibrium,self-fulfillingprophe-cies,diffusion,threshold-basedbehavior,orreferencegroupsfallintothiscategory.Theproblem,again,isthatindividualstatesofmindarenotamen-abletomanipulation.Likewise,groupbehavior–becauseitrestsonindividualstatesofmind–isnotamenabletodirectmanipulation.11Additionalimpedimentstomanipulabilitystemfromcausalfactorsthatareprocessualinnature.Demographersassertthecausalimportanceofthedemographictransition,aphenomenonwithseveralphases.Inthepre-transitionphase,birthsanddeathsareinequilibrium;thepopulationregen-eratesitselfwithoutgrowingbecauseofMalthusianconstraints(land,food,11Likewise,sinceallsocialscienceexplanationsmustultimatelymakesenseoftheactionsofindividuals(eventhoughtheexplanationmayrestatahighlevelofabstractionandmaypertaintoorganizations),onemightsaythatallsocialscienceissubjecttoadegreeofindeterminacy.WecannotdirectlymanipulatethemechanismsbywhichXcausesY.208PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:44 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.012Cambridge Books Online © Cambridge University Press, 2012 healthcare,andsoforth).Inthesecondphase,mortalityratesdropwhilefertilityratesremainconstant,resultinginpopulationgrowth.Inthefinalphase,fertilityratesdropbringingpopulationbacktoequilibriumatanewlevel.Manyphenomenaaresaidtofollowfromademographictransition–including(accordingtoTimDyson)urbanization,expansionofgovernmentandsystemsofadministration,divisionoflabor,thegrowthofcivilsociety,increasedindependenceofwomenandreducedgenderdifferences,andwiderdistributionofpoliticalpower.12Theproblem,fromthepointofviewofexplanatorytractability,isthattheprocessoftransitioniscontinuous;onecannotintervenedirectlysoastoobservethecounterfactual.Naturally,onecanmanipulatesomeofthefactorsthataresupposedtoproducethedemo-graphictransition,forexample,suppliesoffoodandmedicalcare,sanitation,andsoforth.Thismirrorsthesituationofmentalstates:onecanmanipulatefactorsthatarethoughttoproducementalstates,butonecannotdirectlymanipulatethementalstateitself.Athirdimpedimenttomanipulabilityarisesinsituationswheremanipula-tionispossible,butdoingsointroducesproblemsofinterpretationorexternalvalidity.Considertheexampleofdemocracy.Whatwoulditmeantochangeabasicfeatureofacountry’sconstitution?Thisisaverybigandnecessarilycontentioussortofchangeandprobablynotreplicableinalaboratory.Soonemustthinkabouthowthissortofinterventionwouldhappeninarealsociety.Itmightbeimposedfromwithout,astheUnitedStates-ledforcehasattemptedtodoinIraq.Yetinvasionbyaforeignpowerintroducesallsortsofconfoundersthatproblematizeanycausalargumentaboutdemocracy.Whoistheoccupyingpowerandunderwhatcircumstancesdiditinvade?Wastheconquestlengthyorshort?Wastheconqueringpowersuccessfulinvanquish-ingopponentsandestablishingorder?Wasitviewedasaliberatororasanoppressor?Answerstoallofthesequestions(andmanyothers)willcompli-cateanyattempttoconceptualizetheimpactofXonY.Alternatively,letuspresumeahome-growntransitiontodemocracy(orsomethinglikeit)suchasoccurredinRussiain1991.Thisseemseasier,and,yet,itismoredifficultinotherrespects.Considerthatasocietythatisreadytochangethemostbasicfeaturesofitspolityisalsopresumablyundergoingfundamentalchangeonmanylevels.Indeed,thetransitionfromautocracytodemocracyinRussiawasaccompaniedbyanequallytransfor-mativechangefromcommunismtocapitalism.Howisonetoseparateonefromtheothersuchthatonecanmakeanargumentaboutdemocratization12Dyson(2001).209CausalargumentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:44 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.012Cambridge Books Online © Cambridge University Press, 2012 whilemaintainingceterisparibusconditions?Ifonecannotstateclearlywhattheceterisparibusconditionsofacausalargumentareonecannotstateclearlywhattheargumentis.Andinsituationswhereatreatmentcannotbemanipu-latedwithoutdisturbingceterisparibusconditionsitisunclearwhatoneistalkingabout.Letmegiveafewmoreexamplestoillustratetheubiquityofthissortofambiguityinsocialscience.Consideracausalfactorsuchasinequality.Unequalsocietiesarethoughttobepronetocivilconflict,autocracy,andunderdevelopment(relativetosocietiesthatfeatureamoreegalitariandis-tributionofwealth).Onecan,ofcourse,directlymanipulatewealth,atleastinprinciple.Onecouldconfiscatethewealthofrichpeopleandgiveittothepoor.Oronecouldtakeeveryone’swealth,leavingallcitizensofasocietyataverylowlevel.Likewise,onecouldintervenetodistributewealthunequally.Lotsofmanipulatedinterventionscanbeimagined.However,eachoneoftheseinterventionswouldbeassociatedwithenormousturmoil.Assuch,itisdifficulttoimaginehowceterisparibusconditionscouldbemaintained.Alternatively,onecouldimagineasituationinwhichrichmembersofasocietyvoluntarilygiveawaytheirmoneytothepoor,bringingthemselvesdowntothemedianwage.Hereisamechanismoftransferthatdoesnotrelyoncoercion.However,itasksustoimagineaverydifferenttypeofperson,thatis,analtruistwhocaresmoreaboutequalitythanaboutpersonalposses-sions.Thisdramaticalterationofceterisparibuscircumstanceschangesthescope-conditionsoftheargument,whichisnolongerabouttheworldinwhichwelive,butratheraboutsomeother,imaginedsociety–perhapssomewhereoffinthefuture.Inshort,onecannotask:whatwouldtheUnitedStatesbelikeifwealthweredistributedmoreequallywithoutaskingapriorquestion:howwouldwealthberedistributed,andhowwouldthismechanismaffecttheceterisparibusconditionsofthecausalargument?Whendealingwithnonmanipulablecausesoneisnecessarilydealingwiththecausesofthosecauses,thatis,thevariousthingsthatbringabout(in)equalityordemocracy.Afourthsortofobstacleisposedbynonmanipulablecausalfactorsthatserveaproxyrole.ConsidertheroleofraceineducationalattainmentintheUnitedStates.Weknowthatthereisapersistentgapintestscoresbetweenwhiteandblackstudents,13andoneisinclinedtosaythatracehasastrongcausalimpactoneducationalattainment.Fewwoulddisputethisclaim.However,itisanambiguousclaim,andthereasonforitsambiguityisthat13JencksandPhillips(1998).210PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:44 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.012Cambridge Books Online © Cambridge University Press, 2012 onedoesnotknow(withoutfurtherclarification)whatthemanipulablefeatureofthecausalargumentmightbe.Itcouldberaceitself,whichitispossibletoimaginemanipulatingthroughgenetics,eitheratthepointofconceptionoratsomelaterpointindevelop-ment.(Forheuristicpurposes,Ishallleaveasidediscussionofethicalcon-siderations.)Thissortofmanipulationenvisionsthefollowingcounterfactual:ablack(orwhite)childisborntowhite(orblack)parents.Thatchildhasallthegeneticendowmentsofhisorherparentsexceptthecoloroftheirskin.Andthoseparentsaresimilartoallotherwhite(orblack)parentsinallrespectsexceptthetreatment,thatis,theraceoftheirchild(theyarenotmore“progressive”thanotherparents).Anothersortofmanipulationfocusesonafeaturethatispresumedtofollowfromminoritystatus:namely,discrimination.Acounterfactualinthissettingwouldbethatablackchildismovedfromacommunityinwhichheorshefacesagreatdealof(raciallybased)hostilitytoonethatissimilarinallrespects,butisnothostiletopersonsofadifferentcomplexion.Thisisaverydifferentspeciesofargumentthanthepreviousone.Manyadditionalmanipulationscanbeimagined:forexample,thosebasedonsocioeconomics,theeducationalbackgroundofparents,familystructure,andsoforth.Thepointisthattheconceptof“race”–becauseitisopentomanypossiblemanipulations–ishighlyambiguous.Itisnotclearwhatonemeanswhenonesaysthatracecausessomeoutcome.Notonlyisthecounter-factualconditionambiguous,butsoalsoisthemechanism.(Ifwedon’tknowwhatareal-lifechangeinXentails,wecertainlydon’tknowmuchabouttheprocessesbywhichXmightaffectY.)Somemethodologistsviewmanipulabilityasanecessaryconditionofanycausalargument.14Bythisinterpretation,argumentsabouttheeffectofdemocracy,inequality,race,andotherabstractfactorsarenotreallycausalinnature.Thisseemsalittleextreme.Instead,Iwilltreatmanipulabilityasadesirabletrait,amongothers,andonethatisbestapproachedasamatterofdegrees.Aswehaveseen,causalfactorsthatseemnonmanipulablecansometimesbemanipulated,thoughittakessomeingenuitytodosoandthemanipulationmaynotbeethicallyorpracticallyfeasibleorgeneralizabletoreal-worldsituations.Onemayalsomanipulatetheantecedentcausesofatheoreticalfactorofinterest.Althoughthesemanipulationsmaybeimpossibletoimplementintherealworldtheynonethelesshelptoclarifythenatureofa14Holland(1986);Rubin(1975,2008:812).211CausalargumentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:44 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.012Cambridge Books Online © Cambridge University Press, 2012 causalclaim.Oneknowswhat“XcausesY”meansifonecandescribethemanipulationofXthatwouldachievetheenvisionedchangeinY.Thisiswhymanipulabilityisunderstoodhereasaformalelementofacausalargumentratherthansimplyasamatterofresearchdesign.15SeparationAcausemustbeseparablefromtheeffectitpurportstoexplain;otherwisetheargumentistautological.Thisseemsobvious.Yet,oncloserreflection,itwillbeseenthatseparationisamatterofdegrees.Tobeginwith,XsandYsarealwayssomewhatdifferentiatedfromoneanother.Aperfecttautology(e.g.,“TheCivilWarwascausedbytheCivilWar”)issimplynonsense,andneveractuallyencountered.Oneoccasionallyhearsthefollowingsortofargument:“TheCivilWarwascausedbytheattackoftheSouthagainstFortSumter.”Thisismoresatisfactory.Evenso,itisnotlikelytostrikereadersasaparticularlyacuteexplanation.Indeed,thereisverylittleexplanationoccur-ringhere,becausetheXisbarelydifferentiatedfromtheY(theattackagainstFortSumterwas,ofcourse,partoftheCivilWar).EquallyproblematicisanargumentthatlinkstheCivilWartoawarlikerelation-shipbetweenNorthandSouth,onethatpersistedfromthe1850stotheoutbreakoftheconflictin1861.Again,oneisatpainstodistinguishbetweencauseandeffect.Considerasecondexample,thisoneclassicalinorigin.Tosaythatthisman(X)isfathertothischild(Y)istoinferthatthefathercausedthechildtoexist;heisanecessary(thoughnot,ofcourse,sufficient)causeofthechild.(Onemightspeculatethatpresent-daynotionsofcausationarerootedintheprimordialquestionoflegitimacy.)Wearelessimpressed,however,bytheargumentthatafetusisthecauseofachild,orachildthecauseofanadult.Thereissomethingwrongwiththeseformulations,eventhoughXisclearlynecessaryforY(andpriortoY).WhatiswrongisthatthereislittleseparationbetweenXandY;theyarethesameobject,observedatdifferentpointsintime.Inshort,wehavetreateda“contin-uousself-maintainingprocess”asacausalfactor,andthisviolatesthepreceptofseparation.16Bycontrast,wemightaccepttheargumentthatanadultistheproductofhisorherchildhood,preciselybecausethenotionofachildhoodisseparablefromadulthood.(Evenso,theargumentlacksclarity.)15AngristandPischke(2009).16MariniandSinger(1988:364).212PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:44 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.012Cambridge Books Online © Cambridge University Press, 2012 IndependenceInadditiontoseparation,agoodcausalfactorischaracterizedbyindependencerelativetoothercausesofanoutcome.Ifoneproposedcauseofanoutcomeisexplainedbysomethingelse,thelatterhasbetterclaimtothestatusof“cause.”Typically,wedescribetheindependentcauseasstructural,andtheinterveningfactorassuperstructural,endogenous,orepiphenomenal.AsatisfactorycauseembodiesAristotle’squestforan“UnmovedMover,”afactorthataffectsotherthingsbutisnotexplained,oronlypartiallyexplained,byanysinglecause.Ofcourse,everygeneralcausalfactorisaffectedbysomething.Therearenounmovedmovers.Yetsomefactorsareentirely(oralmostentirely)explainedbysomethingelse.Here,wearedubiousaboutcallingthesuperstructuralfactoracause.Itdoesnotfulfillourexpectationsofagoodcausebecauseitlacksindependence.Itisentirelyendogenoustosomethingelse.Bycontrast,thefactorlabeledacauseisapttobeafactorthathasnosingleexplanation.Manythingsaffectit,someofwhichmaybepurelystochastic.Althoughitisnotanunmovedmover,itisanunexplained(ordifficulttoexplain)mover.ConsiderFigure8.1.IfXislargelyexplainedbyA(ifmostofthevariationinXisduetovariationinA),andbotharecausesofY,thenAisprobablymorecorrectlyregardedas“the”causeofY.XissubsumedbyA.OnceoneknowsthestatusofAonecanpredictthestatusofX,M,andY.XandMaddnofurtherinformationaboutthecausaleffect.(Ofcourse,theydoprovideinformationaboutcausalmechanisms,asdiscussedbelow.)If,ontheotherhand,AexplainsonlyasmallportionofX–whichisaproductofmanyfactors,someofwhichmaybepurelystochastic–thenXmayproperlyberegardedasthecauseofY.ItisnotsubsumedbyA.Generallyspeaking(andwiththeusualceterisparibuscaveat),themorefoundationalafactoris,thegreateritsstandingamongthevariouscausesofsomeoutcome.Indeed,debatesaboutcausalquestionsoftenrestonwhichcausalfactorisproperlyjudgedmostfoundational.WhichXexplainsalltheotherXs?Considerthevariousfactorsthathavebeenproposedasexplana-tionsoflong-termeconomicdevelopment,thatis,forexplainingwhysomenationsarerichandotherspoor.Ashortlistofsuchcausalfactorswouldincludegeography,colonialism,domesticpoliticalinstitutions,technology,humancapital,culture,population,anddemographictransitions.17Notethatargumentsamongpartisansofthesedifferentschoolsarenotsimplyabout17Workonthesevarioussubjectsincludes:geography(Diamond1992),colonialism(Grier),domesticpoliticalinstitutions(Acemoglu,Johnson,andRobinson2005),technology(Mokyr1992),humancapital(Clark2008),culture(Landes1999),population(Kremer1993),anddemographictransitions(Dyson2001).213CausalargumentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:44 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.012Cambridge Books Online © Cambridge University Press, 2012 whetherasinglefactor–say,demography–hasalargeimpactonlong-termeconomicdevelopment.Theyarealso,perhapsmoreimportantly,aboutrelationshipsamongthevariouscausalfactors,namely,whichareindepen-dentandwhicharedependent.Inthisargument,geographyhasanimpor-tantadvantage:itisnoteasilyexplained.Indeed,geographyapproximatesAristotle’sunmovedmover.Ofcourse,therearegeologicalexplanationsforwhylandmasseswereformedincertainways,whyriversappear,whysomearenavigableandothersarenot,andsoforth.However,theseexplanationswouldbequitecomplexandwouldinvolveaconsiderableamountofcon-tingency.Geographicexplanationswouldbedifficulttoexplainaway.Bycontrast,culturalexplanationsseemquitevulnerable,astheyareoftenen-dogenoustootherfactors.Thosewhowishtorestorethestatusofculturalexplanationmustshowthatasetofvaluesandpracticesthatimpactedeconomicdevelopmentisnotsuperstructural,thatithascausalindependenceinthelongsweepofhistory.Therelevanceoftheseconsiderationsmayescaperesearchersaccustomedtoexperimentalsettings.Wherethetreatmentofinterestismanipulateditisbydefinitionindependentrelativetoeverythingelse.However,ourtopichereistheformalpropertiesofcausalargumentationnotresearchdesign.Thepointisthatwhenconstructingcausalargumentswemustbeattentivetothewaythingsworkintheworld(asopposedtothelaboratory).Thereislittlepointindesigninganexperimentforacausalfactorthatis,intherealworld,controlledbysomepriorfactor.Thiswouldservetoelucidatecausalmechan-isms,butlittleelse.Ofcourse,ifthecausalfactorofinterestcanbemanipu-latedbyexperimenters,thenitcanprobablyalsobemanipulatedbypolicymakers,whichmeansitmayhavesomerelevancetotherealworld.Underthiscircumstance,itcanclaimcausalindependence;itisnotsimplytheproductofsomethingelse.ImpactCausalargumentsstrivetoexplainvariationinanoutcome.Themorevaria-tionthecausalfactorexplains–thegreatertheimpactofXonY–themoresignificantthatargumentislikelytobe.Thismayalsobearticulatedasaquestionofeffectsize,magnitude,power,orstrength.Necessary-and-sufficientcausalarguments(discussedinChapter12)arecompellingbecausetheyexplainallthevariationinY,whileremainingadmirablyconcise.Itisnowonderthattheycontinuetoserveincommonparlanceastheideal-typecausalargument.Bycontrast,whereverthereare214PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:44 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.012Cambridge Books Online © Cambridge University Press, 2012 exceptionstoacausalargument,orwheresomefactorotherthanXaccountsforvariationinY,wecanseethattheargumentisweakened:itnolongersufficestoaccountforY.18Thereareanumberofwaysinwhichthequestionofrelativeimpactcanbegauged.Inaregressionformat,wheretherelationshipbetweenXandYisassumedtobeprobabilistic,impactismeasuredbythecoefficient(slope)forXorbyamodel-fitstatisticsuchasR2forX,avectorofindependentvariables.Ofcourse,estimatesofcausalimpactfromanempiricalmodeldependuponthespecificsofthatsampleandmodel,andmayormaynotcorrespondtoreal-worldimpact.Ifthemodelisnotrealisticinthisrespect,thenaseparateevaluationofimpact–perhapsinamorespeculativemode–mayberequired.ItisoftenhelpfultoconsidertheimpactofXonYinpracticalterms,forexample,asamatterofpublicpolicy.CouldasignificantchangeinYbeachievedbymanipulatingX?Atwhatcostandwithwhatopportunitycosts?TheimpactofXonYmayalsobegaugedbycomparingitsimpacttootherfactors.Iftheimpactoftheseotherfactorsiswellunderstood,thismayprovideausefulmetricofsignificance(i.e.,relativeimpact).Whateverthemetricofevaluation,theimpactofXonYisakeymeasureofsuccess.Oneofthecriteriaofagoodcausalargumentisthatitexplainsalotaboutthephenomenonofinterest.Itshouldnotbetrivial.MechanismWehavesaidthatcausesgenerate–alter,change,condition,create,effect–outcomes.Itfollowsthattheremustbeacausalmechanism,ormechan-isms,atwork.Themechanismis“theagencyormeansbywhichaneffectisproducedorapurposeisaccomplished.”19Inmodel-basedterminologyitmaybeunderstoodasthecausalpathway,process,mediator,orintermedi-atevariablebywhichacausalfactoroftheoreticalinterestisthoughttoaffectanoutcome–illustratedbyMinFigure8.1.(Bycontrast,amod-eratorisaninterveningvariablethataltersthenatureofanX/Yrelationship.20)Toclarify,myuseofthetermmechanisminthisbookencompassesanyfactorthatisconsideredpartofthegenerativeprocessbywhichXaffectsY,18Avectoroffactorsmayalsoaccount,cumulatively,forallthevariationinY(thisisthegoalofacauses-of-effectstheory,discussedinChapter12),butatthecostofparsimony.19Webster’sUnabridgedDictionary(NewYork:RandomHouse,2006).20WuandMumbo(2008).215CausalargumentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:44 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.012Cambridge Books Online © Cambridge University Press, 2012 whetheritconsistsofaseriesofdiscretesteps(e.g.,dominoesfallingintooneanotheronatable)oracontinuousprocess(e.g.,abilliardballrollingacrossatableandhittinganotherball).Allthatisrequiredisthatthemechanismbefreetovaryinsomefashion–evenifthevariationisonlyhypothetical.Thus,inthedominoesexample,ifonedominoismissingthechainmaybebrokenandtheusualresult–runningfromthefirstdominotothelast–willnotoccur.Similarly,iftheeffectofavoucherstreatmentoneducationalattainmentrunsthroughacausalmechan-ismcenteredonteacherquality,andthelatterfactorisminimized,weexpecttheX/Yrelationshiptobealtered(voucherswillhavelessimpactoneduca-tionalattainment,orwillhavenoimpactatall).Thisiswhatjustifiesourunderstandingofcausalmechanismsasvariables.LikeXandY,theyvary.Sometimes,theworkingofacausalmechanismisobviousandcanbeintuitedfromwhatweknowabouttheworld.ThisislikelytobethecasewhentheX/Yrelationshipisproximateinnature.Supposethatanexperimentaltersthemonetaryincentivesofteachersandfindsthatthishasasignificantimpactonteacherperformance(byvariousmetrics).Itmaynotbenecessarytoprovidealong-windedexplanationofMsinceitseemsasafeassumptionthatthemechanismatworkisthemonetaryincentive.Enoughsaid.Alternatively,Mmaybeobscure.ThisislikelytobethecasewhentheX/Yrelationshipisdistaland/orwhenthecausalpathwaysconnectingXandYarecomplex:involvinglongcausalchains,diverseroutestravelingfromXtoY(equifinality),orthecombinedbutsimultaneouseffectofmultiplefactors.Here,thecausalmechanismsofatheoryrequireextensivediscussion,albeitinaspeculativemanner(asprocessesthatmayplausiblyconnectXwithY).Theimpactofeconomicdevelopmentondemocratization,orofdemocracyonpeace,aretwoexamplesofthissort.Althoughthischapterisfocusedontheformalpropertiesofacausalargument,itisimportantthatwesayafewwordsabouttheempiricalproper-tiesofacausalmechanism.Sometimes,thecausalmechanisminatheoryisdirectlymeasurableandhenceamenabletoempiricaltesting.Sometimes,itisnot–orissoonlythroughproxies.Sometimes,empiricaltestsmaybeconductedinaquantitativemanner(acrossalargesample).Sometimes,qualitativemodesaresufficient,orareallthatcanbemanagedgivendatalimitations.Theseissuesarediscussedatsomelengthinlaterchapters.Imentionthemnowonlytohelpclarifyourworkingdefinitionofamechan-ism,whichisnotintendedtotilttowardqualitativeorquantitativestylesofanalysisortowardassumptionsoftestabilityornontestability.AllImeanbyamechanismisapathwaythatrunsfromXtoY.216PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:44 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.012Cambridge Books Online © Cambridge University Press, 2012 ItshouldalsobepointedoutthatsomecausaltheoriesarecenteredonacoreX/Yrelationship,whileothersarecenteredonacausalmechanism(M).Duverger’stheoryabouttheroleofelectoralsystemsinpartyconflictisdrivenbyanX/Yhypothesis:thatdistrictsizeinfluencespartysystemsize.Marxism,bycontrast,isdrivenbyacausalmechanism:classstruggle.Notethateachtypeoftheorygeneratesitsownspeciesofconfusion.AtheorycenteredonacoreX/Ypredictionmaynotspecifyadeterminatesetofcausalmechanisms(indeed,agooddealofworkonDuverger’stheoryinsubsequentdecadeshasconcernedthepossiblepathwaysbywhichelectoralsystemrulesaffectthebehaviorofvotersandelites).21AtheorycenteredonacausalmechanismmaynotgenerateasetofspecificandtestablepredictionsabouthowXvarieswithY.InthecaseofMarxism,predictionsflowfromthecentralcausalmechanismofclassstruggleinalldirections,andnosinglepredictioniscriticaltothetheory–leadingsomecriticstoaccusethetheoryofunfalsifiability.Forourpurposes,whatbearsemphasisisthatallthreeelements–X,Y,andM–areimportantforcausalargumentation.AnX/Yhypothesiswithoutaclearcausalmechanismisanargumentinsearchofanexplanation.Itmaybetrue,butitwillbenotbeverymeaningful,willbedifficulttogeneralizeupon,andmayalsobedifficulttoproveinaconvincingfashion.Thus,itisincum-bentuponthewritertoclarifythecausalmechanism(s)atworkinacausalargument,ifitcannotbeintuitedfromcontext.22Thismaybeaccomplishedinprose,indiagrams,and/orinmathematicalmodels,andisimplicitintheveryactoftheorizing.2321Riker(1982).22Theimportanceofmechanismsingeneralizing(i.e.,extrapolating)afindingisdiscussedinSteel(2008).23Forfurtherdiscussionofcausalmechanisms,andalternatewaysofunderstandingthiskeyterm,seeGerring(2008,2010).217CausalargumentsDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:44 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.012Cambridge Books Online © Cambridge University Press, 2012 Cambridge Books Onlinehttp://ebooks.cambridge.org/Social Science MethodologyA Unified FrameworkJohn GerringBook DOI:http://dx.doi.org/10.1017/CBO9781139022224Online ISBN: 9781139022224Hardback ISBN: 9780521115049Paperback ISBN: 9780521132770Chapter9 - Causal analyses pp. 218-255Chapter DOI:http://dx.doi.org/10.1017/CBO9781139022224.013Cambridge University Press 9CausalanalysesWhenwelookaboutustowardsexternalobjects,andconsidertheoperationofcauses,weareneverable,inasingleinstance,todiscoveranypowerornecessaryconnexion;anyquality,whichbindstheeffecttothecause,andrenderstheoneaninfallibleconsequenceoftheother.Weonlyfind,thattheonedoesactually,infact,followtheother.Theimpulseofonebilliard-ballisattendedwithmotioninthesecond.Thisisthewholethatappearstotheoutwardsenses.Themindfeelsnosentimentorinwardimpressionfromthissuccessionofobjects:Consequently,thereisnot,inanysingle,particularinstanceofcauseandeffect,anythingwhichcansuggesttheideaofpowerornecessaryconnexion.DavidHume1SinceHume,writershavebeenawarethattheassessmentofcausalrelation-shipsisratherethereal.Onecanneverknowwithabsolutecertaintywhethersomefactorcausedanoutcometooccur,becauseonecannotgobackintimetore-playeventsexactlyastheyhappened,changingonlythefactorofinterestandobservingtheoutcomeunderthisalteredcondition.Thecausalcounter-factualcanneverbedirectlyobservedfortherearenotime-machines.Thisissometimesreferredtoasthefundamentalproblemofcausalinference.2Inrecentyears,socialscientistshavebecomeacutelyconsciousoftheinsubstantialnatureoftheevidencethattypicallyundergirdscausalproposi-tionsinanthropology,economics,politicalscience,sociology,andvariousoffshootsofthesedisciplines.Methodologistshavelittleconfidenceininfer-encesdrawnfromobservationaldata,andnostatisticalmachineryseemslikelytoprovidesecurefoundations.Thereis,somehaveinsinuated,a“crisisofcausality.”31Hume(2007:59).2Holland(1986).3McKimandTurner(1997).Ontheproblemsofstatisticalinferencebasedonobservationaldata,andthecorrespondingimportanceofresearchdesign,seeBerk(2004);BradyandCollier(2004);CloggandHaritou(1997);Freedman(1991,1997,2008,2010);Gerber,Green,andKaplan(2004);Gigerenzer(2004);Heckman(2008:3);Kittel(2006);Longford(2005);McKimandTurner(1997);Pearl(2009b:40,332);RobinsandWasserman(1999);Rodrik(2005);Rosenbaum(1999,2005);Seawright(2010);218Downloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:55 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.013Cambridge Books Online © Cambridge University Press, 2012 Whilecausalitymaybeincrisis,itiscertainlynotdead.Indeed,socialscienceismorefixatedoncausalquestionstodaythanatanypointinthepast.Fortunately,althoughcausalattributionisalwaysagamble,therearewaystomaximizevalidityandprecision,givenevidentiaryconstraints.4Itisinthisspirit–ofdoingthebestwecan–thatthebookiswritten.Beforejumpingintotheargumentitisimportantthatwedefinetheproblemofcausalanalysismoreprecisely.Ibeginbydiscussingthenotionofacausaleffect.Ithenlayoutacausalgraphshowingessentialresearchdesigncomponentsastheypertaintoquestionsofinternalvalidity.Next,Iproceedtothemainbusinessofthechapter:adiscussionofmethodologicalcriteriathatapplybroadlytoresearchdesignswhosepurposeistotestacausalproposition.CausaleffectsInChapter8,Iproposedageneraldefinitionofcausality.Akeypartofthatdefinitionisthecausal(akatreatment)effect:theeffectofsomechangeinacausalfactor(X)onanoutcome(Y),relativetowhatthatoutcomeotherwisewouldbe.Thishascometodefinecausalityinthesocialsciencesandiscentraltothe“potentialoutcomes”modelofcausation(discussedinChapter12).(Itisnot,ofcourse,theonlywayinwhichcausationcanbeunderstood.Forsomepurposes,forexample,inlegalsettings,itisimportanttodefinecausalitywithreferencetothecause-in-fact,asdiscussedbelow.)LetussaythatXisaschoolvoucherandYisschoolperformance,asmeasuredbyanachievementtest.Here,thecausaleffectistheimpact(onschoolperfor-mance)ofhavingavoucher(X=1)relativetonothavingavoucher(X=0).Notethatacausaleffectisunderstoodcounterfactually:whateffectwouldachangeinXhaveonY?ThecausalortreatmenteffectisthereforethechangeinYcorrespondingtoagivenchangeinX.Webeginbyintroducingavarietyofdifferenttreatmenteffects.InthenextsectionweproceedtodiscussvariousrelationshipsthatmightobtainbetweenXandY:varietiesofcausalrelationships.Inthethirdsection,Iintroduceanelaboratedcausaldiagram,buildingonFigure8.1.Summers(1991).Variousstudiescomparinganalysesofthesamephenomenonwithexperimentalandnonexperimentaldatashowsignificantdisparitiesinresults,offeringdirectevidencethatobservationalresearchisflawed(e.g.,BensonandHartz2000;FriedlanderandRobins1995;Glazerman,Levy,andMyers2003;LaLonde1986).Cook,Shaddish,andWong(2008)offeramoreoptimisticappraisal.4ForageneralformulationseeGuala(2005:136).219CausalanalysesDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:55 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.013Cambridge Books Online © Cambridge University Press, 2012 VarietiesoftreatmenteffectsTreatment(causal)effectscanbeunderstoodindifferentways,andthediffer-encessometimesmatterquitealotwheninterpretingresultsfromastudy.5Thus,ashortdigressionmaybewarranted(thosewhowishtoskipaheadmayreturnlatertothisrathertechnicaldiscussion).Notethatwhilethefollowingtermsareoftendefinedbytheirapplicationtoexperimentalresearchdesignstheirmostimportantapplicationsareofteninsettingswhereresearcherscannotrandomizethetreatmentofinterest.Here,aselsewhere,experimentsserveaheuristicrole.Anindividualtreatmenteffect(ITE)istheimpactofatreatmentcondition(X=1)onasingleunitrelativetothecontrolcondition(X=0).Inourvouchersexample,asingleunitmightbeasinglestudent.Thus,ITEforthatstudentishisorherperformanceinthetreatmentcondition(havingavoucher)versushisorherperformanceunderthecontrolcondition(withoutaschoolvoucher).Strictlyspeaking,anyestimationofITEmusttaketheformofacounterfactualthought-experiment,forwecannotdirectlyobservethetreatmentandcontrolconditionsforasingleunit.Thisisthefundamentalproblemofcausalinfer-ence,referredtoattheoutsetofthechapter.However,wecanobserveasingleindividualpre-andpost-treatment(withoutandwithavoucher),andmakeinferencesaccordingly.Alternatively,wecaninferITEfromthepropertiesofalargersample,asdiscussedbelow.Inanycase,ITEisusuallynotthemostinterestingpropertyofacausalanalysis,especiallyifourgoalistoelucidatepropertiesofalargerpopulation.Anaveragetreatmenteffect(ATE)isthemeanimpactofachangeinXonYacrossapopulation,thatis,theaverageITE.Theintuitionisthatindividualtreatmenteffectsarelikelytobedifferentfromunittounit.Indeed,causalheterogeneityaboundsinsocialsciencephenomena.Itseemsprobable,forexample,thatsomestudentswillrespondtoavouchersstimulusmoreposi-tivelythanothers.Somemaynotrespondatall,ormayrespondnegatively.ATErepresentstheaveragevalueoftheseheterogeneouseffects.Itistheusualgoaloflarge-sampleanalysis.Inanexperiment,ATEisestimatedbycomparingagroupofunitsrandomlyassignedtoreceivethetreatment(thetreatmentgroup)withagroupthatisrandomlyassignedtothecontrolcondition(novouchers).Unfortunately,thecorrectestimationofATEisnotalwayspossible;hence,thedevelopmentofasetofalternativetreatmenteffects,asfollows.5ForfurtherdiscussionseeHeckman(2000);Manski(1995);MorganandWinship(2007:ch.2);Rosenbaum(2002).220PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:55 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.013Cambridge Books Online © Cambridge University Press, 2012 Sometimes,averagetreatmenteffectrefersonlytorelationshipsfoundwithinachosensample,nottoalargerpopulation.Thisisreferredtoasasampleaveragetreatmenteffect(SATE).Anintent-to-treateffect(ITT)isawayofframingATEinsituationswhereitissuspectedthatsomeunitsassignedtothetreatmentgrouparenotactuallyexposedtothetreatment:aproblemofnoncompliance(discussedlaterinthischapter).Itmaybereadas“ATEwithprobablenoncompliance,”thatis,includingunitsinthetreatmentgroupthatarenotactuallytreated.Letusimagineanexperimentinwhichsomestudentsaregrantedvouchers,butnotallofthemtakeadvantageoftheopportunity;theycontinuetoattendtheirlocal(non-voucher)school.Onecanstillcompareschoolperformanceforstudentswhoreceivethevouchers(thetreatmentgroup,includingnoncom-pliers)andstudentswhodonot(thecontrolgroup),butthecomparisonhasadifferentinterpretation.ItmeasurestheITT.NotethatforsomepurposesITTmaybemorepolicy-relevantthanATE,forthereisoftensomedegreeofnoncomplianceassociatedwithapolicyinitiative.Inanycase,thesetwosample-basedcausaleffectssuggestdifferentinterpretationsaboutX’seffectonY.Whereinferencestoalargerpopulationareimprobable,onemayrefertoasampleintent-to-treateffect(SITT).Anaveragetreatmenteffectonthetreated(ATT)alsofocusesontheproblemofnonrandomassignmentand/ornoncompliance.ATTdesignatestheeffectofXonYforallunitsthatareactuallytreated–asopposedtoallthosethatareassigned,ormightbeassigned,tothetreatmentgroup.So,ifstudentsareallowedtoself-selectintoavouchersprogramitisprobablysafetoassumethattheyhavedifferentbackgroundcharacteristicsthanstudentswhodonotself-selectintotreatment.Theymaybemoreambitious,moreintelligent,withbetter-educatedparents,andsoforth.Thesefeatureswilllikelyaffecttheirperformanceonwhateveroutcomemeasureofschoolachievementisemployedasapost-test.Underthecircumstances,thereareseveralbasesformakingajudgmentabouttheATT.Wemightcomparethescoresofthesestudentswithotherswhodidnotreceivethetreatmentbutwhoseemsimilaronbackgroundcharacteristics.Wemightcomparethescoresofstudentsbeforeandaftertheyreceivethetreatment.Ineachoftheseanalyses,thecausaleffectofinterestisproperlyregardedasATT,ratherthanATE,becausewearedealingwithaspecialsubsetofthepopulationofinterest–thosewhoaretreated(perhapsbecausetheyelecttobetreated).AlthoughATTisusuallycorrectlyregardedasacorruptionoftheATEideal,insomecircumstancesitmaybemorerelevantthanATE.Considerthequestionofwhethercollegeprofessorssocializetheirstudentstoparticular221CausalanalysesDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:55 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.013Cambridge Books Online © Cambridge University Press, 2012 pointsofview:forexample,doliberalfacultycausestudentstobecomemoreliberal?6Inthissituation,letussupposethattherearearangeofuniversitiesavailabletomoststudents–someliberalinorientationandsomeconserva-tive.Studentscanthereforechoosewhatsortofideologicalclimatetheywishtoinhabitwhileattendingcollege.Letusfurthersupposethatnogovernmentprogramorregulationislikelytoinhibitthisfreedomofchoice.Here,wemaybemoreinterestedintheeffectofthetreatment(attendingclasseswithliberal/conservativefaculty)onthetreated(ATT)thantheaverageeffectofthetreatmentacrossthepopulation(ATE),weretheytoberandomlyassignedtoliberalorconservativeinstitutions.Forthelatterisunlikelyevertohappen.TheATEmaystillbeoftheoreticalinterest,butitisnotofgreatpracticalimport.WhereATTrefersonlytoasampleratherthanalargerpopulation,itishelpfultodesignateasampleaveragetreatmenteffectonthetreated(SATT).Alocalaveragetreatmenteffect(LATE)isamorespecializedtermusedinthecontextofinstrumental-variableanalysis(explainedinChapter10).Specifically,itreferstotheeffectofXonYforthoseunitswhosetreatmentstatus(treated/untreated)isaffectedbythechoseninstrument.Itexplicitlyexcludesthoseunitswhichwouldreceivethetreatmentconditionregardlessoftheinstrument(always-takers)andthosewhowouldbeinthecontrolconditionregardlessoftheinstrument(never-takers).Toreiterate,LATEdefinesthetreatmenteffectasreferringonlytothoseunitswithinthepopulationwhoseassignmenttotreat-mentisaproductoftheidentifiedinstrument.Abetterlabelforthiswouldbecomplieraveragetreatmenteffect(CATE);however,thistermisrarelyencoun-teredintheliterature,sowewillstickwithLATE.Althoughnotoftenacknowl-edged,onecanappreciatetheoccasionalutilityofanothersortoftreatmenteffectthatislocalandalsolimitedtothestudiedsample,ratherthanalargerpopulation,thatis,asamplelocalaveragetreatmenteffect(SLATE).7Thealphabetsoupofacronymsisconfusing,tobesure.Readersnewtothisliteraturearewelladvisedtostayfocusedontheconcepts,ratherthantheterminology.However,sincetheterminologyisbecomingubiquitous,andtheconceptstheyrepresentaresurelyimportant,somefamiliaritywithATEanditsvariantsisrecommended.Fortunately,thedistinctionsamongthesetermsmaybesummarizedinataxonomicform,asillustratedinTable9.1.Tobesure,notaxonomyofthissortcanclaimcomprehensiveness.Indeed,thereisnolimittothesortofthingthatcouldbedesignatedasatreatmenteffectaslongasitrespectsthedefinitionalcriterionofdescribingadifference6MarianiandHewitt(2008).7IamindebtedtoAdamGlynnforpointingthisouttome(personalcommunication,2010).222PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:55 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.013Cambridge Books Online © Cambridge University Press, 2012 Table9.1Treatmenteffects:anoncomprehensivetaxonomyAverageAverageamongunitsintendedtobetreatedAverageamongthetreatedAverageamongcompliersPopulationsamplePopulationsamplePopulationsamplePopulationsampleUnitATE(averagetreatmenteffect)XSATE(sampleaveragetreatmenteffect)XITT(intentiontotreateffect)XSITT(sampleintentiontotreateffect)XATT(averagetreatmenteffectonthetreated)XSATT(sampleaveragetreatmenteffectonthetreated)XLATE(localaveragetreatmenteffect)XSLATE(samplelocalaveragetreatmenteffect)XITE(individualtreatmenteffect)XDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:55 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.013Cambridge Books Online © Cambridge University Press, 2012 inYwhenXvaries.FollowingaBayesianapproachtocausalinference,onecanimagineafulldistributiontreatmenteffect(DTE)describingthediffer-enceinYacrossitsentiredistribution(bymeansofadensityfunction)–asopposedtoapointestimaterepresentingmeanvaluesofY.Inthissamespirit,onecanimagineaquartiletreatmenteffect(QTE),adoublefrontiertreatmenteffect(DFTE),8avariancetreatmenteffect(VTE),9andsoforth–eachwithsample-andpopulation-basedvariants.IhavenotincludedtheseinTable9.1becausetheyhavenotyetgainedcommoncurrency.AnothersortofcausaleffectisonethateschewspreciseestimatesofX’simpactonYinfavorofajudgmentaboutthegeneraldirectionofcausalimpact(positiveornegative).Thisisaplausiblereadingofmanyqualitativeandquantitativestudieswherethereispotentialmeasurementerror,wheretheinterventionofinterestisnotrandomized,andwheretheresearchdesignbearsscantresemblancetoanaturalexperiment.Thisbringsustoafinalpoint.Forpracticalandtheoreticalpurposes,theATEisusuallythemostdesirablewayofstructuringtheoutcome.Thisdoesnotmeanthatitcanalwaysbeachieved.However,othersortsoftreatmenteffectsmayberegardedasdeviationsfromATEinsofarasATEiswhatonewouldprefertoestimateinthebestofallpossibleresearchdesigns.ThisiswhyATEislistedfirstinTable9.1.Whereverthetermstreatmenteffectandcausaleffectareencounteredwithoutembellishment,thereadercanusuallyinferthatthewriterisinterestedinaveragetreatmenteffects.VarietiesofcausalrelationshipsAcausal(treatment)effectmaytakemanydifferentforms,asdiscussed.Likewise,thereisanimmensevarietyofdifferentrelationshipsthatqualifyascausal(i.e.,inwhichachangeinXgeneratesachangeinY).Althoughsomeofthefollowingtermsareratherarcaneitwillbehelpfultoreviewthembriefly,fortheyillustrateontologicalpossibilities(whatmaybegoingon“outthere”intheworld)andcommonmodelingstrategies.Conjuncturalcausalityreferstoasituationwhereaparticularcombinationofcausesacttogethertoproduceaneffect.Causalequifinalityiswhereseveralcausesactindependentlyofeachothertoproduceaparticulareffect.Monotoniccausalityiswhereanincrease(decrease)inthevalueofXcausesanincrease(decrease)ornochangeinY.Inotherwords,therelationshipbetweenXandY8Fried,Lovell,andSchmidt(2008),describedinRusso(2009:98–101).9Braumoeller(2006);Heckman(2005:21–22).224PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:55 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.013Cambridge Books Online © Cambridge University Press, 2012 iseitheralwayspositiveornulloralwaysnegativeornull.NonlinearcausalityiswheretheimpactofXonYvarieswiththevalueofX(butmaystillbemonotonic).Irreversiblecauses(e.g.,ratcheteffects)arethosewhoseimpactonYcannotbereversed.Constantcausesoperatecontinuallyuponanoutcomeratherthanthroughdiscreteinterventions.Proximalcausesoperateimmedi-atelyuponanoutcome.Distalcauses,bycontrast,havelong-termeffectsonanoutcome.SequentialcauseshavedifferenteffectsonYdependinguponthesequenceinwhichtheyareapplied.AcausalchaindescribesasituationinwhichmanyintermediatecausesliebetweenXandY.Path-dependencyreferstoasituationinwhichasinglecausalinterventionhasenduring,andperhapsincreasing,effectsovertimeonanoutcome.Causallawsusuallyrefertoperfect(exception-less)relationshipsbetweenXandY,observableacrossalargepopulation.ProbabilisticcausesarenotperfectlyrelatedtoY(thereareexceptions,whichmayberepresentedwithanerrorterm)eventhoughXisacauseofY.Set-theoretic(“deterministic”)causesarenecessaryand/orsuffi-cienttoproduceanoutcome.Thisclassofcausesencompassesthetechniqueknownasqualitativecomparativeanalysis(QCA),whichfocusesoncon-juncturesoffactorsthat,together,constituteasufficientcauseofanoutcome(Chapter12).Evidently,therearemanywaystothinkaboutcausation.TheunitaryconceptofcausationintroducedinChapter8sheltersapluralityofpotentialcausalrelationships.Indeed,onceoneheadsdownthisanalyticroaditisnotclearwhereoneoughttostop.Thereispotentiallyalwayssomenewwayinwhichtwofactorsmayco-varyorsomenewsetofcausalmechanismsthatmightexplaintheircovariation.Thetermsintroducedabove,summarizedinTable9.2,areasmallportionoftheinfinitevarietyofcausalrelationshipsthatmayexistintheuniverse.(Note:Table9.2isalistratherthanatypology,asitidentifiesneitherexhaustivenormutuallyexclusivecategories.)Nonetheless,thismenuisusefulasaquick-and-dirtycanvassofthefield.Thevastmajorityofcausalargumentsbandiedaboutincontemporarysocialscienceembodyoneoftheserelationships.DeparturesfromthetreatmenteffectHavingexploredvariationsintreatmenteffect,andvariationsincausalrelation-ships,itistimetodiscusssomedepartures.Thesewillbebrieflymentioned.Sometimes,investigationsofcausalityarefocusedoncausalmechanisms,asdiscussedinChapter8andagaininChapter11.Notethatmechanismicinvestigationsmaybeorientedtowardestimatingcausaleffects,inwhichcase225CausalanalysesDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:55 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.013Cambridge Books Online © Cambridge University Press, 2012 theyarenotdeparturesfromthetraditionalgoalofATE.Buttheyalsocompriseaseparateresearchagenda.WewanttoknowwhyXcausesY,notsimplythetreatmenteffectofXonY.Sometimes,theinvestigationofcausalityisfocusedonascertainingtheboundariesofaninference.ThequestionisnotwhatcausaleffectdoesXhaveonY,butratherwhere(acrosswhatsortofunits)doesithavethiseffect?Whatisthetruepopulationoftheinference?Sometimes,researchersareinterestedincalculatingtheprobabilityofanoutcomebasedonacausalmodel.Thatis,giventhataunithasaparticularvalueforX,whatistheprobabilityofY?Thismaybereferredtoasprediction(ifoneisinterestedinout-of-samplecases)ordescription(ifoneisinterestedonlyinfeaturesofthesample).Itiscentraltoset-theoreticcausalrelation-ships:whereXisnecessary,sufficient,ornecessary-and-sufficientforY(Chapter12).Anothersortofcausalargumentfocusesonestablishingthecause-in-fact,akaactualcause,singularcause,single-eventcause,ortoken-levelcause.10Thispointisoftenillustratedwithastylizednarrativeaboutamanwanderinginthedesertwithasmallcanteenofwater.ItsohappensthatthecanteenhasaTable9.2Causalrelationships:apartiallistConjunctures(akacompoundcause,configurativecause,combinatorialcause,conjunctivepluralityofcauses):Whereaparticularcombinationofcausesacttogethertoproduceaneffect.Equifinality(akamultiplecauses,multiplecausalpaths,adisjunctivepluralityofcauses,redundancy):Whereseveralcausesactindependentlyofeachothertoproduce,eachonitsown,aparticulareffect.Monotonicity:Whereanincrease(decrease)inXalwayscausesanincrease(decrease)ornochangeinY.Linearity/Nonlinearity:IftheimpactofXonYchangesacrossdifferentvaluesofXtherelationshipisnonlinear.Irreversibility:XaffectsYasXincreasesbutnotasitdecreases,orviceversa.Constancy/Delimited:Aconstantcauseoperatescontinuallyuponanoutcome;adelimitedcauseoperatesonlybriefly(thoughitmayhaveenduringeffects).Proximal/Distal:Aproximalcauseoperatesimmediatelyonanoutcome;adistalcausehaslong-termeffects.Sequence:TheeffectofX1–3onYdependsuponthesequenceinwhichX1,X2,andX3areapplied.Causalchain:Multiplemechanisms(M)formachainfromXandY.Path-dependency(akacriticaljuncture):Asinglecausalinterventionhasenduring,andperhapsincreasing,effectsovertime.Causallaws:Exception-lessrelationshipsbetweenXandY.Probabilisticcauses:Witherrors,i.e.,exceptions.Set-theoreticcauses:WhereXisnecessaryand/orsufficientforY.10HartandHonore(1959);Hitchcock(1995);Pearl(2009b:ch.10).226PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:55 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.013Cambridge Books Online © Cambridge University Press, 2012 holeinitandwaterleaksout,afterwhichhedies.Uponinvestigationitisdiscoveredthatthewaterispoisoned.Now,theITEof(a)theholeinthecanteenand(b)thepoisonedwateristhesameifbothareassumedtobefatal.EstimatingthisITEdependsuponassumptionsabouttheman’sconditionwithandwithoutthetreatmentsandwhetherthetreatmentsareadministeredsimultaneouslyorindependently.Forpresentpurposes,whatissignificantisthatifbothofthesecausalfactorsarefatal,andifbackgroundconditionsarethesame,theirITEsarethesame.However,thereisonlyonecause-in-fact.Thismightbetheholeinthecanteen(ifthemanactuallydiedofthirst)oritmightbethepoisonedwater(ifthemanactuallydiedofpoisoning).Itmightevenbeboth,ifitisdeterminedthatthesefactorsinteractedtocausehisdeath.Thekeypointisthatthissortofcausalityisnotdefinedbyacounterfactualandsodoesnotconformtothetraditionalunderstandingofacausaleffect.Ofcourse,thecause-in-factmightbecraftedinacounterfactualmannerifthecounterfactualisunderstoodinanextremelynarrowfashion(i.e.,whetherthemandiesinaparticularwayorataparticulartime)orifthevariouselementsofthecausalstorycanberepresentedinacausaldiagram(asclaimedbyPearl).11However,thepointofthecause-in-factisusuallytoassignmoralandlegalresponsibility,nottoshedlightonaclassofevents.Insofarasoneisinterestedingeneralizablearguments–akeypointofdepartureformostscientificinvestigations,asarguedinChapter3–thecause-in-factislikelytoplayasmallrole.Ifweareseekingtogeneralizefromthecasedescribedhere–thatis,tootherdesertwanderers–weareprobablymoreconcernedwiththeconclu-sionthatbothleaksincanteensandpoisonincreasetheprobabilityofdeath.Itisoflesssignificancewhatcausedthisparticularmantodieinaparticularwayandataparticularpointintime.Insum,aninvestigationintocausalrelationshipsneednotberestrictedtoquestionsaboutcausaleffects.SomeinterestingandimportantfeaturesofcausalityareleftoutofATEanditsvariants.12Thisdoesnotmeanthattheyareincontradictionwiththepotential-outcomesmodel(Chapter12),butitdoestestifytothediversemeaningsandpurposesoftheconceptofcausation.Inanycase,thetreatmenteffectretainsacentralpositionincausalinvestiga-tionswithinthesocialsciences.Consequently,mostofourdiscussioninthischapterandthenextfocusesonthistraditionalobjective.11AsPearl(2009b:311)notes,“themoreepisode-specificevidencewegather,thecloserwecometotheidealsoftokenclaimsandactualcauses.”SeealsoPearl(2009b:ch.10).12Onemayquibbleaboutwhetherthesealternativeformsofcausalinferencearefocusedon“causaleffects.”Evidently,thisdependsonhownarrowlyonewishestodefinethenotionofacausal(treatment)effect.227CausalanalysesDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:55 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.013Cambridge Books Online © Cambridge University Press, 2012 AnelaboratedcausalgraphCausalanalysisisnotamechanicalprocedure,forthedataneverspeaksforitself.Itmustbeinterpreted,andmanyassumptionsarerequired.Inordertoclarifywhattheseassumptionsareitisoftenhelpfultoconstructavisualrepresentationofwhatwethinkisgoingonintheworld.Acausalgraphshouldreplicatethedata-generatingprocess(DGP),thatis,theactualprocessbywhichthedataoneisexaminingwascreated.13Ofcourse,wecannotreallyknowwhatthetrueDGPis,soacausaldiagramiscorrectlyregardedasapresentationoftheauthor’sassumptions,someofwhichmaybetestablewhileothersmaynotbe.Acausaldiagramistheauthor’sbestguessaboutthenatureofsomereality.Drawinggraphsishelpfulregardlessofwhetherthesampleissmallorlarge,whetherthedatawasgeneratedexperimentallyorobservationally,andwhetherdataanalysisisquantitativeorqualitative.Allofthesesituationscanbequitecomplex;allrequireassumptions;andallaresubjecttosimilarthreatstoinference.Ourfirstcausalgraph,Figure8.1,includedthecauseoftheoreticalinterest(X),theoutcome(Y),themechanism(M),andanantecedentfactor(A).Figure9.1reiteratesthesefeatureswithtwoadditions.EachletterinFigure9.1(andinotherdiagramsthroughoutthebook)repre-sentsasinglevariableoravectorofvariables.(Thus,Xmightrefertoasinglecauseorasetofcauses.)Avariable(akacondition,factor,etc.)referstoanyfeaturethathasthepotentialtovary–whethermeasurableorun-measurable,qualitativeorquantitative,continuousorcategorical.Variablesareunderstoodascausallyrelatedifthereisadirectedarrowpointingfromonetoanother.Theyareunderstoodascorrelative(associational)ifalinewithoutarrowsconnectsthem.AcorrelativerelationshipbetweenAandBmightmeanthatAcausesB,thatBcausesA,thatathirdfactor,C,causesbothAandB,orthatthereisnodiscernablecausalrelationshipbetweenAandB(theyareaccidentallycorrelated).Now,letusexpandupontheelementsinthiscausalgraph.AcausalmechanismwasdefinedinChapter8asthepath(s)connectingXwithY,labeledMinFigure9.1.14Amechanismmediates,andinthissense13Readerswillnotethatthiscausaldiagramborrowscertainfeaturesfromthetraditionofcausalgraphs(e.g.,Pearl2009b),butisnotequivalenttoa“directedacyclicgraph”(DAG).Itissimplerinsomerespects(thoughfurtherelaborationisofferedinChapter10),andmoregeneralinsomerespects.Forexample,whileDAGsexpressallrelevantrelationshipsascausal,andfocusprimarilyontheassignmentproblem(theformalresearchdesign,ifyouwill),theframeworkofferedinFigure9.1shouldalsobeapplicabletoconfoundersthatcreepintoaresearchdesignaftertheassignmentoftreatment,e.g.,noncompliance,mortality,andthelike.Eachofthesethreatstoinferenceintroducesaspeciesofconfounder(somethingcorrelatedwithX,orwiththechangeofXovertime),thoughtheyareoftendifficulttoconceptualizeinacausalfashion.14ForfurtherdiscussionseeGerring(2008,2010).228PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:55 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.013Cambridge Books Online © Cambridge University Press, 2012 explains,X’srelationshiptoY.Letussupposethatvouchershaveapositiveimpactonschoolperformance;thus,studentsreceivingvouchersscorebetteronsomemeasureofperformancethanthosewhodonotreceivethetreatment(allotherthingsbeingequal).Mechanismsforthiscausaleffectmightinclude(a)higher-qualityinstruction,(b)smallerclasses,or(c)greatermotivationonthepartofteachersand/orstudents.Recallthatallcausalargumentstakeplaceagainstabackground,presumedtobeheldconstantsothatthetrueeffectofXonYcanbeobserved.Thisisthecontextual“noise”againstwhichthe“signal”(X’seffectonY)mustbeassessed.Threegenresofbackgroundfactorsareworthdistinguishing.ThefirstistheantecedentcauseofX,labeledAinFigure9.1.Generallyspeaking,anantecedentcauseisanyfactorlyingprior(causally)tosomeotherfactor.MisantecedenttoY,XisantecedenttoM,andAisantecedenttoXinFigure9.1.Inthisusage,antecedentissynonymouswith“prior,”“parentof,”or“exogenousto.”InFigure9.1,thefactorlabeledantecedent,A,hasadirecteffectonX,andanindirecteffectonMandY.15Inthecontextofvouchersresearch,antecedentcauseswouldincludefactorsthatinfluencewhichstu-dentsreceivevouchersandwhichdonot.Becausetheantecedentcause(A),asillustratedinFigure9.1,hasnodirecteffectontheoutcomeexceptthroughthedesignatedcauseoftheoreticalinterest(X),theadoptionofavoucherprogrammayimpactschoolperformance,butonlythroughtheworkingofthevoucherprogram.General features= Causal relationship= Covariation (possibly causal)Of theoretical interestBackground factorsX = Causal factorA = Antecedent Y = Outcome B = CovariateM = MechanismC = ConfounderAXMYCBFigure9.1Anelaboratedcausalgraph15If,bycontrast,acauseantecedenttoXhasanindependenteffectonanoutcomeotherthanthroughX,orifitiscorrelatedwithsomeotherfactorthathasanindependenteffectonthatoutcome,itisproperlyclassifiedasaconfounder,C.229CausalanalysesDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:55 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.013Cambridge Books Online © Cambridge University Press, 2012 Asecondbackgroundfactortakestheformofanorthogonalcovariate(B).ThisspeciesofcausalfactorhasaneffectonYbutisindependentofX.ThismeansthatBisorthogonalto(stochastic,random,uncorrelatedwith)thetreatment,evenwhenconditioningonotherfactorsinthemodel(anissuewhoseimportancewillbecomeapparentinChapter10).Inanexperimentalstudyofvouchers(wherethetreatment,vouchers,israndomizedacrossatreatmentandcontrolgroup),covariatesmightincludeindividual-levelfactorsrelatedtostudenttestperfor-mancesuchasage,race,sex,socialbackground,andyearsofschooling.Theymightalsoincludetrulyrandomfactorsthatwecanneitheridentifynormeasure.Afinalbackgroundfactor,andbyfarthemostimportant,istheconfounder(C).Generically,aconfounderisanyfactorthatmightcompromiseatrue(unbiased)estimateofX’seffectonYandthusposeathreattocausalinference.Aconfounderthereforeco-varieswithX,afactthatdistinguishesitfromanorthogonalcovariate(B).(ForfurtherdiscussionseeChapter11.16)ThesimplestandmostcommonstrategyofcausalinferencerestsonthecovariationofXandY.Iftheproblemofconfounderscanbesolved,oratleastmitigated,thepatternofcovariationbetweenXandYshouldprovidethebasisforavalid(unbiased)estimateofthecausaleffect.If,inaddition,thenoiseemanatingfromcovariates(B)islimited,controlledbyconditioningonthesefactors,oroutweighedbythesheernumberofobservationsinasample,thentheestimateofcausalimpactwillberelativelyprecise(stable,reliable).Allissuesassociatedwithestimationaretherebypartitionedintotwocategories:validity(theabsenceofconfounders)andprecision(noisefromcovariates),representedbyCandBinFigure9.1.Thisis,ofcourse,averybriefdiscussion,omittingagreatdealofimportancetocausalinference.Itshouldalsoberecognizedthatthegraphsrepresentahighlysimplifieddepictionofwhatmaybeanextremelycomplexreality.Notallcausalfactorsnecessarilyfitneatlyintooneofthesecategories(A,B,C,X,Y,orM).AfactormayexhibitelementsofMandC,forexample;thatis,itmaybepartlyendogenoustoX(acausalmechanism,M)butalsoexertanindependentcausaleffectonY(acommon-causeconfounder,C).However,theschematicfeaturesofFigure9.1areusefulforheuristicpurposes.Additionalproblemsofcausalinferencerequiremorecomplexcausalgraphs,asexploredinChapter11.16Theyinclude:thecommoncause(orclassicconfounder),whichhasacausaleffectonbothXandY;theincidentalconfounder,whichaffectsYandiscorrelatedwithX(butnotbyreasonofanyidentifiablecausalrelationship);thecompoundtreatmentconfounder,whichfailstodistinguishbetweenacausalfactoroftheoreticalinterestandaconfounder;theendogenousconfounder,inwhichaconditionedfactor(otherthanY)isendogenoustoX;thefeedbackconfounder,inwhichYaffectsX;theantecedentconfounder,inwhichaconditionedfactoraffectsYonlythroughX;andthecollider,inwhichaconditionedfactorisaffectedbybothXandY.230PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:55 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.013Cambridge Books Online © Cambridge University Press, 2012 CriteriaHavingclarifiedtheproblemofcausalassessmentandthenatureofpotentialthreatstoinference(withreferenceprimarilytoquestionsofinternalvalidity),weturnnowtogeneralcriteriaofcausalanalysis.Whatdesideratadoresearchdesignsandassociateddataanalysesstrivetoachievewhentestingcausalarguments?InChapter4,Iarguedthatagoodresearchdesignaddressesfourgenericcriteria:accuracy,sampleselection,cumulation,andtheoreticalfit,eachwithvariouscomponents.Atthispoint,Iwillintroducedimensionsofinferencethatpertainuniquelytocausality.Idividethissubjectintothreebroadcategories:thetreatment,theoutcome,andthesample,eachwithassociatedcriteria.Thesewillformthebasisforourdiscussion.Forconvenience,allsevendimensionspertainingtoresearchdesignincausalanalysisarelistedinTable9.3.17Itisvitaltobearinmindthathere,aselsewhereinthisvolume,everycriterionpresupposesaceterisparibuscaveat.Eachisgood,allotherthingsbeingequal.Whereotherthingsarenotequal,theresearchermuststriveforthebestpossibleadjustmentofcriteriasothattheneteffectmaximizesutilityalongthesemanydesiderata.Beforebeginning,Imustalertthereadertoastylisticdivisionoflaborinupcomingsectionsofthebook.Thischapterapproachesthetopicofresearchdesignfromanideal-type(maximal)perspective.Itasks,whatarethefeaturesofthearchetypal,canonicalresearchdesign?Ifonecouldwaveamagicmethodologicalwandatone’schosenproblemofcausalinference,whatresearchdesignfeatureswouldonewishintoexistence?Inlaterchapters,Iacknowledgetherealitythattherearenomagicwandsormethodologicalfairies;thecanonicalresearchdesignisthereforerarelyattainableinpractice–atleast,notwithoutsacrificingimportantfeaturesofaresearcher’stheoreticalagenda.Itishopedthatthesechapterswillbereadtogether.Researchersneedtoknowwhattostrivefor,buttheyalsoneedtoknowhowandwhentomakecompromises.Idealismisimportant,butsoispragmatism.Wecommenceinautopianmode.17Insofarascausalresearchemploysdescriptiveinferencessuchasindicators(whichalldo)ortypologies(whichsomedo)theyareliabletocriteriaspecifictotheseinferences,asdiscussedinChapter5.Ishallleavethesecriteriaimplicit.231CausalanalysesDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:55 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.013Cambridge Books Online © Cambridge University Press, 2012 TreatmentThecounterfactualassociatedwithanycausalquestionis,whatwouldhappentoanoutcome(Y)ifthetreatment(X)weretochange?Forpurposesoftesting,agoodtreatmentshouldbe:(a)exogenous(toY);(b)varying;(c)simple;(d)discrete;(e)uniform;(f)evenlydistributed;(g)strong;and(h)proximate(toY).ExogeneityAgoodtreatmentisexogenousrelativetotheoutcomeunderinvestigation.XshouldnotbeaffectedbyY.Thisisimplicitinthenomenclatureof“inde-pendent”(X)and“dependent”(Y)variables.(Sometimes,exogeneityhasaTable9.3Causalanalysis:criteriaANALYSIS(Chapter4)1.AccuracyAretheresults(a)valid,(b)precise(reliable),and(c)accompaniedbyanestimateofuncertaintywithrespectto(d)thechosensample(internalvalidity)and(e)thepopulationofinterest(externalvalidity,akageneralizability)?2.SampleselectionArethechosenobservations(a)representativeoftheintendedpopulation,(b)sufficientlylargeinnumber,and(c)attheprincipallevelofanalysis?3.Cumulation(a)Istheresearchdesignstandardizedwithothersimilarresearchonthetopic?(b)Doesitbeginbyreplicatingextantfindingsandendbyfacilitatingfuturereplicationsbyotherscholars?(c)Areresearchprocedurestransparent?4.Theoreticalfit(a)Doestheresearchdesignprovideanappropriatetestfortheinference(constructvalidity)?(b)Isthetesteasyorhard(severity)?(c)Isitsegregatedfromtheargumentunderinvestigation(partition)?(d)Arealternativeexplanationsruledout(elimination)?CAUSALANALYSIS(thischapter)5.TreatmentIsX(a)exogenous(toY),(b)varying,(c)simple,(d)discrete,(e)uniform,(f)evenlydistributed,(g)strong,(h)proximate(toY),and(i)scaleable?6.OutcomeIsY(a)varying,oratleastfreetovary?7.SampleArethechosenobservations(a)independent(ofoneanother)and(b)causallycomparable?232PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:55 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.013Cambridge Books Online © Cambridge University Press, 2012 broadermeaning,signifyingthatthetreatmentisrandomly,oras-ifrandomly,assigned,whichistosayXisnotcorrelatedwithpotentialconfounders.Iemploytheterminanarrowermanner,referringonlytotherelationshipbetweenXandY.)Ofcourse,weknowthatmanycausalrelationshipsintherealworldareprobablyreciprocal.Presumably,economicdevelopmentaffectspopulationhealth,andpopulationhealthaffectseconomicdevelopment.Presumably,socialclassaffectseducation,andeducationaffectssocialclass.However,informulat-ingacausalhypothesiswegenerallyidentifyonefactorasXandtheotherasY.Thus,aresearcherimposesaspecificconjectureuponthemanifoldcomplexitiesoftheworld.Allelsebeingequal,weaskwhateffect(ifany)achangeinXmighthaveonY.18Inordertotestthishypothesis,itisessentialthatXbeindependent(exo-genous)relativetoY–orthatanyremainingendogeneitiesbecorrectiblebystatisticallegerdemain(correctionsthatareusuallyopentoquestion).AnotherwayofphrasingthisproblemisintermsofendogeneitybetweenYandX,asdiscussedinChapter11.Oneempiricaltestofexogeneityistemporalprecedence.19However,itisbynomeansasufficienttestofexogeneity;indeed,itisoftenmisleading.SimplymeasuringXatsometimeperiodbeforeYdoesnotprovideafoolproofmethodof“exogenizing”X,andtestsofcausalityrestingonlyontemporalpriority(e.g.,Grangercausality),whileinformative,arebynomeansdefinitive.Forthisreason,Iviewtemporalityasasecondaryissue–oneofmanypossiblecluesaboutexogeneity.VariationEmpiricalevidenceofcausalrelationshipsislargelycovariationalinnature.InobservingtwobilliardballscollideweobservethatXandYareassociated:whereXhitsY,Yrespondsbymoving.PriortoX’sarrival,YwasstationaryandafterX’sdepartureYbecomesstationaryonceagain.Thisindicates(thoughitdoesnotbyitselfprove)thatXisacauseofY,anditalsosayssomethingaboutthenatureoftherelationship.Covariationcantakemanyforms,includingallthoselistedinTable9.2.Andtherearenumerousnear-synonymsforthisbasicidea,forexample,18Veryoccasionally,onemighttrytomeasurebothcausaleffectsatthesametime.However,thisismuchmoredifficulttodoand,inanycase,maybeapproachedastwoseparateunidirectionalcausalhypotheses:(a)doesXaffectY?;and(b)doesYaffectX?19Reichenbach(1956);Suppes(1970).233CausalanalysesDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:55 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.013Cambridge Books Online © Cambridge University Press, 2012 association,correlation,constantconjunction(Hume),concomitantvariation(Mill),andcongruity.Sometimes,thecovariationfollowsaperfect(exception-less,invariant,“deterministic”)pattern,thatis,Xisnecessaryand/orsufficientforY.Sometimes,itisprobabilisticinnature.20Whateverthenatureoftherelationship,XandYmustdisplaysomecovariationalpattern–atleasthypothetically.Withoutit,causationcannotbeatwork.Empiricalcovariationisthusappropriatelyregardedasanecessary(thoughbynomeanssufficient)conditionofacausalrelationship.VariationinX–theexplanatoryvariableofinterest–isespeciallycrucial.AnexperimentalstudyensuresvariationinXbymanipulatingthetreatment.AnobservationalstudylooksforcasesthatexhibitnaturalvariationinX.Forexample,astudyofvouchersmightincorporatevariationonthiskeypara-meterbycomparingschoolswithvoucherstoschoolswithout,orbycompar-ingstudentswithvouchersandstudentswithout(switchingtheunitofanalysisfromschoolstoindividuals).Oritmighttaketheformofatemporalcomparisonbetweenschools(orstudents)priorto,andafter,theinstitutionofvouchers.Ifwehavenosuchvariation,ouranalysismusttaketheformofacounterfactualthought-experimentinwhichsuchvariationisimagined–amuchweakerresearchdesign.21SimplicitySimpletreatmentsareeasiertotestthancomplextreatments.Thisiscom-monsensical.However,thecostsimposedbymorecomplextreatmentsdeserveattention.Evenifitisnotinthepoweroftheresearchertosimplifythetreatment(perhapsthetheorydemandsamorecomplextreatment),heorshewillstillhavetoreckonwiththesecosts.Thesimplesttreatmentinvolvesonlytwoconditions:atreatmentcondition(X=1)andacontrolcondition(X=0).Normally,itiseasytoidentifywhichiswhich,thatis,whichconditionexemplifiesthestatusquoor“null”hypothesisandwhichconditionexemplifiesthetreatment.Occasionally,however,twotreatmentconditionsarecomparedwithoneanotherwithoutapurecontrol.20Bennett(1999);Hume(1960:219);MariniandSinger(1988);Mill([1843]1972:263);Neuman(1997:50).Bowley(quotedinMorgan(1997:62),anearlypioneerofstatisticalmodeling,putitthisway:“Itisnevereasytoestablishtheexistenceofacausalconnectionbetweentwophenomenaorseriesofphenomena;butagreatdealoflightcanoftenbethrownbytheapplicationofalgebraicprobability...Whentwoquantitiesaresorelatedthat...anincreaseordecreaseofoneisfoundinconnectionwithanincreaseordecrease(orinversely)oftheother,andthegreaterthemagnitudeofthechangesintheone,thegreaterthemagnitudeofthechangesintheother,thequantitiesaresaidtobecorrelated.”SeealsoFrendreis(1983);Russo(2009).21Fearon(1991);Lebow(2007);TetlockandBelkin(1996).234PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:55 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.013Cambridge Books Online © Cambridge University Press, 2012 Forexample,aninvestigationofelectoralsystemsmustcomparedifferentelectoralsystems;thereisno“absence-of-treatment”condition(purecontrol).Inanycase,thesedistinctionsarelargelysemantic.ThemethodologicalissuesinvolvedwhencomparingX=0withX=1areidenticaltothoseinvolvedwhencomparingX=1withX=2.Botharesimpletreatments.Complexitymaymeanmanythings.Itmightmeanmultipletreatmentgroupsarrangedinanordinalscale(0,1,2,3,4,...).Itmayinvolvecategoricaldistinctionsthatarenominalratherthanordinal(e.g.,Catholic,Protestant,Jewish,Muslim).Itmayalsoinvolveinteractionsamongseveralcategoricalvariables(e.g.,Catholic+Male,Catholic+Female,Protestant+Male,Protestant+Female).Here,thenumberoftreatmentsisequaltothenumberofcombinations.Anotherkindofcomplexityinvolvescontinuoustreatments,whereXvariesacrosssomeinterval.Here,thetreatmentsareessentiallyinfiniteinnumbersinceaninfinitenumberofpointsliewithinanyinterval.Generally,contin-uoustreatmentsaremodeledmathematicallysoastoreducetheircomplexity.Thus,acontinuoustreatmentmightbemodeledasalinearfunction(Y=X+ε)orassomenonlinearfunction(e.g.,Y=X+X2+ε).Tobesure,thereare,inprinciple,aninfinitenumberofnonlinearfunctions(aninfinitenumberofnonlinearwaysforXtoberelatedtoY),socontinuoustreatmentsareinherentlycomplex,evenifthemathisparsimonious.Thegeneralpointisthis:complexityentailsagreaternumberoftreatments.Thismeansthattheresearcherwillhavetoeitherincorporatealargersampleinordertotestthesemultiplehypothesesorreducethenumberofhypothesesthroughsomemathematicalexpression.Thecostofthelatterapproachisthatonemustintroduceassumptionsaboutthetrueshapeoftheunderlyingrelationship,assumptionsthatcannotalwaysbefullytested–especiallyifthecompositionofXinthesampleisnotevenlydistributed(asdiscussedbelow).Discrete-nessThediscrete-nessofatreatmentpartlydeterminestheeasewithwhichcausalrelationswillbeobservable.Adiscretetreatmentisabrupt,thatis,shortinduration.Itcanbedescribedasadose.Asaresult,itiseasiertocompareunitspre-andpost-treatment,oracrosstreatmentandcontrolgroups,withoutalotofpotentialconfoundersenteringintothepicture.22Ifthetreatmentofastudyconsistsofavoucher,onehasonlytomarkthetimeatwhichthistreatment22Rosenbaum(2002:354–357).235CausalanalysesDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:55 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.013Cambridge Books Online © Cambridge University Press, 2012 wasadministeredandthetargetswhoreceivethevoucher(wepresumethattherearenoimportantanticipatoryeffects).If,however,thetreatmentisnondiscrete,theremaybenobaselineagainstwhichtheeffectofthetreatmentcanbecompared.Consideravouchersprogramthathandsoutmoneytostudentsatmonthlyintervals,withdifferentdisburse-mentsatdifferenttimes,andnoclearpointofcommencementortermination.Here,theanalysiswouldhavetodependuponsomesimplificationofthedata,forexample,alinearrelationshipbetweenmoneydisbursed(toastudentorschool)andresultsachieved.Whilethisexamplemayseemratherartificial,itdoesexemplifyacommontraitofmanyobservationalsettings.Becausetheexperimenterisnotincontrolofthetreatment,anaturaltreatmentislikelytobeintroducedinahaphazardfashion.Whatis“messy”aboutobservationaldataisnotsimplythenon-randomizedassignmentofthetreatment(asdiscussedbelow),butalsothenatureofthetreatmentitself.UniformityInordertotesttheimpactofacausalfactoritisessentialthattheinterventionberelativelyuniformacrossthechosenunits.Ifthetreatmentisbinary(0/1)ormultichotomous(0/1/2/...),thenachievinguniformityisasimplematterofmakingsurethatthedosesarecorrect.Ifthetreatmentiscontinuous,therequirementsofanintervalscalemustbeupheld.Insomerespects,theissueofuniformityisanissueofmeasurement,thatis,theconstructionofindicators(Chapter7).Sincecausalargumentsbuildonconceptsandindicators,allcriteriapertainingtoconceptsandindicatorsneces-sarilypertaintocausalanalysis.Inthisrespect,ourdiscussionisredundant.However,theproblemofnonuniformtreatmentsisworthmentioningagainbecauseinsofarasatreatmentisheterogeneousinnature–orhetero-geneouslyadministered–itscausaleffectwillbedifficult,ifnotimpossible,tointerpret.Considerwhathappensifweregardabinarymeasureofdemocracy(e.g.,asprovidedbyPrzeworskiandcolleagues)asacausalfactorinexplain-ingsomeoutcome(e.g.,economicgrowth).Incodingallcountriesas0=auto-craticor1=democraticweareassumingthatallcountriescodedas1receivethesametreatment,andallcountriescodedas0experiencethesame“control”status.Becausethetreatmentisobservational,andbecauseitisdifficulttoimaginewhatmanipulationofrealitywouldachievethistreatment,thisisadifficultmattertoevaluate.Sufficetosay,thereisastronglikelihoodthatallcountriescoded1arenotthesameonthedimensionoftheoreticalinterest236PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:37:55 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.013Cambridge Books Online © Cambridge University Press, 2012 (democracy)andcountriescoded0arenotthesameonthecorrespondingdimensionofinterest(autocracy).Inthisrespect,intervalmeasuresofdemocracymayseempreferable.Yetherewemustalsoworryaboutuniformityoftreatment.Considerthatacompositeindicator,suchaspolity,whichisconstructedthroughacomplexaggregationrulefromavarietyofcomponents,maynotbetrulyunidimensional.Specifically,acodingof“3”maymeansomethingdifferentindifferentcases:thatis,thevariouswaysofachievinga“3”maynotbetrulyinterchangeableintermsoftheircausaleffects.Ifso,theimpactofthistreatmentisfundamentallyambig-uous.23Again,theimportanceofauniformtreatmentisparamounttointer-pretingacausaleffect.EvendistributionInadditiontovariation,simplicity,anduniformity,itisalsodesirableforthefactoroftheoreticalinteresttoembodyanevendistributionacrosswhatevervaluesaredeemedtheoreticallysignificant.Thisissuemaybealsoexpressedasaproblemof“missingvalues.”24SupposeweareexaminingtheeffectofvouchersonschoolperformanceandwehaveahighlyskeweddistributionofvaluesforX.Letussaythatonlytwostudentshavebeengrantedvouchers(X=1),whiletheremaining10,000studentsinoursamplereceivethecontrolcondition(X=0).Thisisnotanidealsettingforresolvingquestionsofcausality,foranyresultsdrawnfromtheanalysisrestonthedispositionofthetwopositivecases.TheNofthestudyislarge,butitisnotveryinformativeandhaslittleclaimtogenerality.Similarly,withacontinuoustreatmentonewouldliketoseedoselevelsatalllevelsofX–high,medium,andlow,forexample.If,however,thetheoreticalaspirationsofthetheorysurpasstheactualvariationinXoneisinthepositionofintuitingvaluesforYwhenthereisnocorrespondingvalueforX–acounterfactualthought-experiment.IfXrangesonlyfrom0to5,onemustbewaryofpredictionsaboutYforvaluesofXthatsurpass5.25Likewise,ifvaluesofXincludeonlythetopandbottomofapresumeddistribution(e.g.,X=0orX=10),onemustbewaryofmakingpredictionsaboutYwhen0P(Y|x).BythiswemeanthatXraisestherealprobabilityofYoccurring–throughsomegenerativemechanism–notsimplyourenhancedabilitytopredictY.Itiscausalnotsimplycorrelative.Likewise,foranynecessary/sufficientargumentoneoughttobeabletocalculateacausaleffect,understoodasATEorsomevariationthereof(Chapter9).Thus,ifacauseisnecessaryorsufficientwewouldexpecttofindadifferenceinoutcomesacrosstreatment(X)andcontrol(x)groups.Morespecifically,ifXisanecessaryconditionforY,thenP(Y|x)=0whileP(Y|X)>0.Thatis,thechangefromxtoXraisestheprobabilityofYfrom0tosomeundefinedprobabilitygreaterthan0.IfXisasufficientconditionforY,22CausalconjuncturessufficienttocauseanoutcomemayalsobereferredtoasINUScausesinsofaraseachcomponentofaconjunctureisregardedas“aninsufficientbutnecessarypartofaconditionwhichisitselfunnecessarybutsufficientfortheresult”(Mackie1965:246).ThisisthehallmarkofQCA,asdiscussedbelow.337VaryingapproachestocausalinferenceDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:38:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.016Cambridge Books Online © Cambridge University Press, 2012
thenP(Y|x)<1whileP(Y|X)=1.Thatis,thechangefromxtoXraisestheprobabilityofYfromsomethinglessthan1toprecisely1.Allofthispresumesthatthecausalcondition(s)isnecessaryorsufficientinanontrivialfashion.Atrivialnecessarycauseshowsnodifference(ornoperceptibledifference)acrosstreatmentandcontrolgroups.Theyarenotthekindofcausesthatscholarsandlaypersonsgenerallyidentifyasnecessary.WhenonesaysXisnecessaryforYonepresupposesthatthereisasystematicdifferenceinthevalueofYbetweenunitswithxandunitswithX.Likewiseforsufficiency.23Sometimes,availableevidencedoesnotsatisfytheseassumptions.Forexam-ple,asamplemaylackvariationinthecausalfactorofinterest.LetussaythatallcasesareXandnonearex.Assumingsomevariationintheoutcome(Y/y),wemightinterpretthisasevidenceforanecessaryconditionargument:XcouldbenecessaryforY.However,reachingthiscausalinferencedependsuponacounterfactualthought-experimentratherthanonempiricalevidence.Wemustpresume,onthebasisofourknowledgeofaparticularcontext,thevalueofanoutcomewhenacaseassumesthevalueofx.Specifically,wemustassumethatthecombinationx/YislesslikelythanthecombinationX/Y.Andthis,inturn,fulfillstheassumptionsofatraditionalcausaleffect.Thatsaid,inmeasuringadifferenceofmeans–causaleffectsintheATEsense–onehasperhapsnotarrivedatthemostusefulstatementofcausalimpact.Foranecessarycause,thesalientresultisthatinallcaseswithx,onewillalsofindy.CasesexemplifyingXarelessuseful,forthevalueoftheoutcomeisinconsistent(sometimesY,sometimesy).Likewise,forasufficientcause,thesalientresultisthatinallcaseswithX,onewillalsofindY.Caseswithxarelessuseful,forthevalueoftheoutcomeisinconsistent(sometimesY,sometimesy).Considerapillthatpromisestopreventheartattacksamongthosewithhighcholesterol.Apotentialconsumeroftheanti-cholesterolpillwillprobablybelessinterestedintheaveragetreatmenteffect:thatis,thedecreaseinlikelihoodthatheorshewillhaveaheartattackwiththepill(X)asopposedtowithoutthepill(x).Instead,heorshewillprobablywanttoknowtheeffectivenessofthepill,toutcourt.Thatis,ifheorshetakesthepill(X)whatarethatperson’schancesofhavingaheartattack?Howsufficientisthepillinpreventingheartattacks?Thisisastatementofprobabilitybasedonapresumedcausalrelationship.Itisnotacausaleffect(atleastnotintheusualcounterfactualsense).Butit23BraumoellerandGoertz(2000:854–856).338PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:38:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.016Cambridge Books Online © Cambridge University Press, 2012 presumesacausaleffect,andalso(inset-theoreticlanguage)nontriviality.(Nontrivialityiswherecausaleffectsmeetset-theoreticrelationships.)Iftakingthepillissufficienttopreventaheartattackbutnottakingthepillisalso(forsomeotherreason)sufficientforpreventingaheartattack,theanti-cholesterolpillistriviallysufficient–andthecausaleffectisnull.Nooneshouldbothertakingthemedicationsinceithasnoeffectontheoutcome;itistriviallysufficient.Thepointiseasiesttoillustrateinanexperimentwithabinarytreatment(X/x)andbinaryoutcome(Y/y).Panel(a)inTable12.1showsresultsfromahypotheticalstudyinwhichatreatment(suchasananti-cholesterolpill)israndomizedacrosstwogroups,eachconsistingof500units.Thefirstcolumnillustratesthedistributionofoutcomesinthecontrolcondition(x).Here,all500casesclusterinthebottomcell.Thesecondcolumnillustratesthedistributionofoutcomesinthetreatmentcondition(X).Here,casesaresplitevenlybetweenbothcells.TheATEcanbecalculatedbycomparingtheprobabilityofYforthecontrolgroup(0)withtheprobabilityofYforthetreatmentgroup(0.5).Adifference-of-meanstestrevealsittobeahighlysignificantresult.However,iftheresearcherisinterestedinnecessitytherelevantdataiscontainedincolumn1,thecontrolcondition.Thiscontainsthefollowingfinding:nounitssubjecttothecontrolconditionachievetheoutcome(Y|xisanullset).Bycontrast,column2isirrelevantaslongasitisnotidenticalwithcolumn1(thatis,thereissomemeasurableATEacrossthetwogroups).Thesamepatternofrelevance/irrelevanceisfoundinthemeasurementofsufficiency,butinreverse,asillustratedinpanel(b)inTable12.1.Here,theTable12.1Necessary-and-sufficientcausalpatterns(a)NecessityY0[250]y500[250]xX(b)SufficiencyY[250]500y[250]0xXValuesincellsindicatethenumber(N)ofunitsthatassumeaparticularvaluefortheoutcomeundercontrol(x)andtreatment(X)conditions.339VaryingapproachestocausalinferenceDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:38:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.016Cambridge Books Online © Cambridge University Press, 2012 treatmentgroup(column2)containsthefinding,whilethecontrolgroup(column1)isirrelevant–again,presumingthatthereisanaveragetreatmenteffectacrossthetwogroups.Conceptualizingnecessary/sufficientargumentsasstatementsofprobabil-ityseemsheterodoxatfirst,especiallysincetheserelationshipsarecommonlyregardedas“deterministic.”Andyetthedeterministicclaimis,ofcourse,astatementaboutprobability.Foranecessarycause,P(Y|x)=0.Forasufficientcause,P(Y|X)=1.Ahelpfulaspectofthisinterpretationisthatitcanincorporateexceptions,thatis,degreesofnecessityorsufficiency.Recallthatinthecontextofmanyreal-lifesettings–suchasthepillthatretardsheartattacks–mattersofdegreeareoftencrucial.Wewanttoknowhownecessaryorhowsufficientanout-comeisifagiventreatmentisadministered,evenifitisnotaperfectcausallaw.WhatistheprobabilitythatIwillhaveaheartattackifItakeacholesterolpill?Thisis,ofcourse,quitedifferentfromATEbecauseitdoesnotcomparetreatmentandcontrolconditions(thecausalcounterfactual).Instead,itlooksonlyatoutcomevaluesinthecontrolgroup(necessity)orthetreatmentgroup(sufficiency).Thissuggeststhatclaimsofnecessityandsufficiencymaybeevaluatedintwosteps.First,thereisthequestionofcausality:isthereacausaleffect?Second,thereisthequestionofprobability.Ifacausalrelationexists,whatistheprobabilityofacertainoutcomewhenthecausalfactortakesonacertainvalue(X/x)?Thissuggests,finally,thatascertainingnecessaryandsufficientconditionsisnotsodifferentfromascertainingotherprobabilisticrelationshipsbasedoncausalmodels(modelspresumedtobecausal).Forexample,regressionmodelsarecommonlyemployedtopredictthelikelihoodofanoutcomegivenchosenvaluesforX(orsomevectorofXs).ThisisdirectlyanalogoustocalculatingprobabilitiesofY=0orY=1foragivenvalueofX(0or1)intheexampleillustratedinTable12.1.Thecaveatisthatinorderforthecalculatedprobabilitiestobeinterpretedascausal,themodeluponwhichtheprobabilitycalculationsarebasedmustrepresentatruecausalmodel.Soconceptualized,relationshipsofnecessityandsufficiencyarenotalientomainstreamcausalanalysisasunderstoodthroughthepotential-outcomesmodel(seebelow).Thisisadistinctadvantageinsofarasonemightwishtocreateasinglecommunityofscholarship,ratherthancontendingschoolsofcausation.2424Foracontraryview,emphasizingdifferencesacrossthesetraditions,seeGoertzandMahoney(2010).340PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:38:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.016Cambridge Books Online © Cambridge University Press, 2012 Now,letusquicklytakeupthequestionofcausalinference:howdoweknowwhenanecessary/sufficientcovariationalrelationshipistrulycausal?Ihavesuggestedthattheidealapproachproceedsintwostages:first,anexperimentalorquasi-experimentaltestforcausaleffects;next(ifthistestispassed),ameasurementofnecessityorsufficiency.Unfortunately,manysocialsciencesettingsdonotallowforthefirststage.Instead,oneisfacedwithobservationaldatafromwhichonemustinfercausality.Thus,ininvestigatingthedemocraticpeacehypothesisonefindsasampleofcases(nation-states)thatcanbeobservedovertime,butcannotbesubjectedtoarandomizedtreatment.Theobservedcasesexemplifypatternsofregimetype(democracy/autocracy)anddyadicoutcomes(peace/war)thatareconsistentwithcausalnecessity.But,ofcourse,appearancesmaybedeceptive;covariation(includingset-relations)doesnotequalcausation.25Oneapproachwouldbetointerrogatethedataforevidenceofacausaleffectwithanappropriatenonrandomizedresearchdesign(chosenfromthemenuofstrategiesoutlinedinChapters10and11).Apanelanalysisseemsideal,sinceitiscapableofintegratingbothtemporalandspatialevidence.Ifsuitableinstrumentsforregimetypecouldbeidentified,aninstrumental-variableanalysiswouldbepreferred–althoughinthisinstanceitseemsunlikelythatanysetofinstrumentswouldsatisfytheexclusionrestriction(Chapter11).Theresultingestimatormightbelogitregressionorsomeversionofmatching.26Inanycase,robustnesstestsshouldbeperformed,giventheconsiderableuncertaintyaboutproperspecificationofthemodel.Ifastrong(robust)causaleffectisconfirmed,onehasgreaterconfidencethattherelationshipbetweenregimetypeandwar/peaceiscausal.Then,theestimationofprobabilitiesforY|XandY|xareeasiertojustify.Hownecessaryisautocracyforwar(howsufficientisdemocracyforpeace)?Now,wemustcomplicatethingsfurther.Whenanalyzingobservationaldata,questionsofcausality(causaleffects)andprobability(necessity/suffi-ciency)arenotentirelysegregatedfromeachother.Thisisbecausepatternsofnecessity/sufficiency,ifconsistentacrossalargesample,alsoconstituteevi-denceofcausality.OneismoreinclinedtobelievethereisacausaleffectifX/xandY/yco-varyintheperfectmannerillustratedinTable12.1.Thus,inves-tigationsofnecessary/sufficientrelationshipsarealso,atthesametime,investigationsintocausality.25Yamamoto(2010)discussestheproblemofconfounding.26Beware:statisticalsoftwareusingmaximumlikelihoodmodelssometimesdiscards“perfect”predictors(Goertzforthcoming).341VaryingapproachestocausalinferenceDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:38:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.016Cambridge Books Online © Cambridge University Press, 2012 Thepointwillseemobvioustothoseengagedinthesestudies,butitisperhapsnotsoobvioustothoseapproachingcausalityfromatraditionalpotential-outcomesperspective.RecallfromourdiscussioninChapter4thatthebeliev-abilityofacausalconjectureisenhancedwheneveranespeciallyseveretesthasbeenpassed.Difficulthurdles,ifcleared,inspireconfidence.Aset-theoretichypothesis,ifunderstoodasexceptionless–P(Y|x)=0(necessity)orP(Y|X)=1(sufficiency)–isanextremely“risky”prediction.Ifthispredictionholdsupacrossalargenumberofcasesandthereisvariationinthetheoreticalvariableofinterest(X/x),thecausalconjectureisstronglycorroborated.NotethattheonlyplausiblealternativeexplanationforperfectnecessityorsufficiencyacrossalargesampleisaconfounderorsetofconfoundersthatisperfectlycorrelatedwithXacrossthesetofcasesthatexhibity(fornecessarycauses)orY(forsufficientcauses).Thishelpstomollifyconcernsaboutspuriouscausalclaims.27Ofcourse,ifthepositednecessary/sufficientrelationshipislessthanperfect(thereareexceptions),thepredictionislessrisky.Andifthesampleissmall,orifthereislittlevariationinXandY,thereislesssupportingevidence.Still,thepointremainsthatcausalinferencefornecessary/sufficientrelationshipsshouldideallybeconsideredatseverallevels–thesearchforacausaleffectandforcausalmechanisms(whichisequallyimportantforset-theoreticandnon-set-theoreticcauses),andtestsfornecessity/sufficiency.28Qualitativecomparativeanalysis(QCA)29Restingontheideaofsufficientcausalrelationsistheset-theoreticapproachpioneeredbyCharlesRagin,knownasqualitativecomparativeanalysis(QCA).30Thissectionthusbuildsdirectlyontheprevious.ThehallmarkofQCAistobefoundintheanalysisofmultipleconfigura-tionsoffactors,eachofwhich(i.e.,eachconfiguration)isconsideredasasufficientexplanationforaparticularoutcome.Moreconcisely,QCAisabout27Goertz(forthcoming).28BraumoellerandGoertz(2000).SeealsothecolloquyinPoliticalAnalysis(10:2)concerningwhatsortofcasesaremostusefulfortestingclaimsofnecessityandsufficiency(BraumoellerandGoertz2002;Clarke2002;Seawright2002).29ThissectionwaswrittenincloseconsultationwithCarstenSchneiderandwithinputfromJamesMahoney–thoughneithershouldbeimplicatedinmyconclusions.30Ragin(1987,2000,2008);RihouxandRagin(2009);SchneiderandWagemann(2007,2010).Reviews(sometimescritical)ofthemethodcanbefoundinCat(2006);Lieberson(2001);Yamamoto(2010),andthesymposiaonQCAinQualitativeMethods2(2)(2004):2–25(availableon-line)andinStudiesinComparativeInternationalDevelopment40(1)(2005):3–26.342PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:38:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.016Cambridge Books Online © Cambridge University Press, 2012 causalconjunctionsunderconditionsofequifinality.(Notethatregressionmeth-odscanalsodealwithcausalconjunctures,buttheseareanalyzedascausaleffectsratherthanasrelationsofsufficiency,andusuallyasmultiplicativetermsratherthanasset-theoreticrelationships.)QCAalsohandlesnecessaryconditions,thoughthesearerarelytheobjectoffocusandmaynotrequiretheadvancedmachineryofQCA.Thus,mostofthefollowingdiscussionfocusesonconfig-urationsofcausalfactorsunderstoodassufficienttoproduceanoutcome.Perhapstheeasiestwayofdescribingthistechniqueisbyexploringapar-ticularexample.Here,IrelyonadiscussionprovidedinarecentvolumeonQCAeditedbyBenoîtRihouxandCharlesRagin.31Thesubstantiveworkunderdiscussion,byDirkBerg-SchlosserandJeremyMitchell,examinespossibleexplanationsforthebreakdown/survivalofdemocracyininterwarEurope.32Myexpositionwillbebriefandschematic,focusingonthemostdistinctiveelementsoftheQCAtechniqueandomittingelementsofresearchdesignthatareheldincommonwithothermethods.33Foramoredetailedintroduction,withaguidetobest-practices,thereadershouldlookelsewhere.34cs-QCATheoriginal,crisp-set(cs)versionofQCAbeginswithabinarycodingofkeyvariables:theoutcomesoftheoreticalinterestandthefactorsthatmayhavecausedthem.Drawingontheliterature,theauthorsidentifyfivekeyfactors:development(percapitaGNP);urbanization;literacy;anindustriallaborforce;andgovernmentstability.Thesearecalibratedintosetmembershipscoresusingcut-offpointssuggestedbytheoreticalconsiderations.Developmentin1930iscodedas0ifpercapitaGNPisbelowUS$600,and1ifabove.Urbanization(populationintownsofgreaterthan20,000inhabitants)iscodedas0ifbelow50percent,1ifabove.Literacyiscodedas0ifbelow75percentoftheadultpopulation,1ifabove.Theindustriallaborforceiscodedas0ifbelow30percentoftheactivepopulation,1ifabove.Governmentstabilityiscodedas0iftenormorecabinetsgovernedduringtheperiodunderanalysis,1otherwise.Democraticsurvival=1,breakdown=0.31RihouxandDeMeur(2009)discussthecrisp-set(cs)versionoftheanalysisandRagin(2009)presentsthefuzzy-set(fs)versionoftheanalysis.32Berg-SchlosserandDeMeur(1994);Berg-SchlosserandMitchell(2000,2003).33So,e.g.,Iwillnotdwellontheneedforcausalcomparabilityinachosensample,amatterexploredinChapter9.Likewise,Ishallnotdiscusstheviabilityofrandomversuspurposivesamplingtechniques,amattertakenupinChapter4.34RihouxandRagin(2009);SchneiderandWagemann(2007,2010).343VaryingapproachestocausalinferenceDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:38:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.016Cambridge Books Online © Cambridge University Press, 2012 Basedonthiscoding,atruth-tableisconstructedinwhichcasesaregroupedtogetherinthesamerowiftheyshareidenticalscoresforall(potentially)causalfactors.Thistruth-tablereducesaplenitudeofcasesandvariablestoaparsi-miousgrid.Eighteencasesbecomenineconfigurations(combinationsofcausalfactors),asdepictedinTable12.2.Thenextstepistolookcloselyattheconfigurationsthatleadtopositiveout-comes.Survivalmightbeunderstoodasaproductofthreedistinctcausalpaths:1a.DEVELOPED*urban*LITERACY*indlab*GOVSTABCases:FI,IR1b.DEVELOPED*URBAN*LITERACY*indlab*GOVSTABCases:BE,CZ,NE,UK1c.DEVELOPED*urban*LITERACY*INDLAB*GOVSTABCases:FR,SWHere,upper-caselettersindicateapositivescoreonafactor(1)whilelower-caselettersindicateanegativescore(0)–oftentimesunderstoodasthepresence/absenceofafactor.Alternatively,democraticsurvivalmightbeunderstoodasaproductoftwocausalpaths:2a.DEVELOPED*urban*LITERACY*GOVSTABCases:FI,IR2b.DEVELOPED*LITERACY*indlab*GOVSTABCases:BE,CZ,FR,NE,UK,SWTable12.2cs-QCAtruth-tableCausalfactorsOutcomeConfigurationCasesDevelopedUrbanLiteracyIndustriallaborGovernmentstabilitySurvival1.FI,IR1010112.BE,CZ,NE,UK1111113.FR,SW1011114.ES0010105.AU1011006.GE1111007.GR,PL,SP0000008.HU,PO0010009.IT,RO000010AU:Austria,BE:Belgium,CZ:Czechoslovakia,ES:Estonia,FI:Finland,FR:France,GE:Germany,GR:Greece,HU:Hungary,IR:Ireland,IT:Italy,NE:Netherlands,PL:Poland,PO:Portugal,RO:Romania,SP:Spain,SW:Sweden,UK:UnitedKingdom.(Logicalremaindersnotincluded.)Codingexplainedinthetext.BasedonRihouxandDeMeur(2009:55).344PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:38:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.016Cambridge Books Online © Cambridge University Press, 2012 Restatedinprose:thesurvivalofdemocracyisfoundincountriesthatcombinehighGNP,highliteracy,anindustrializedlaborforce,andgovernmentalsta-bilityORincountriesthatcombinehighGNP,lowurbanization,highliteracy,andgovernmentalstability.Athirdinterpretation,droppingseveralfactors,viewssurvivalastheproductofasinglepathamongtheremainingfactors:3.DEVELOPED*LITERACY*GOVSTABCases:BE,CZ,FI,FR,IR,NE,UK,SWThesethreeconditions,incombination,aresaidtoassuretheoutcome.Afinalinterpretationofthetruth-table,evenmoreparsimonious,isthatsurvivalistheproductofonepathwithonlytwoconditions:4.DEVELOPED*GOVSTABCases:BE,CZ,FI,FR,IR,NE,UK,SWAsimilaranalysismightalsobeconductedonthenegativeoutcome,whereSurvival=0(breakdown)–thoughwewillnotfollowthedetailshere.Eachofthefoursolutiontermsreviewedaboveisconsistentwiththeempiricalinformationcontainedinthetruth-table.Whichtochooseforfurtherinterpretationdependsthereforeontheoreticalinterests,ontheusualdemandforparsimony(Chapter3),and–mostimportantly–onnon-Booleaninfor-mationaboutthecases(i.e.,evidencethatliesoutsidetherealmofformal,deductivelogic).Thisincludesassumptionsaboutoutcomesinso-called“logicalremainder”rows.Thesearepossiblecombinationsofcausalfactors(configurations)notfoundintheempiricaldata,andmaybeviewedascounterfactualthought-experiments.Considerthemostparsimoniousinterpretationofferedabove–thatdemocraticsurvivalintheinterwarperiodwastheproductofhighGNPcombinedwithgovernmentstability(No.4).Thesetwofactors,together,mayconstituteasufficientconditionofsurvival.However,makingthisargumentpresumesthatnoEuropeandemocracywiththistwo-factorconfigurationwouldhavebrokendown,evenifitsscorealongotherparameters(urbaniza-tion,literacy,andindustriallabor)weredifferent.Onlysomeofthesepotentialcases(combinationsofcausalconditions)areactuallyobserved.Indeed,thenumberofcasesismodestrelativetothenumberofpossiblecombinations.NotethataQCAwithfivecausalfactorsposesthirty-twopossibleconfigura-tions(25),onlynineofwhichareactuallyobservedinthehistoricaldata.(ThisisacommonpredicamentinQCA,thoughitalsoaffectsotherobserva-tionalresearch.)Evenso,ifonecanenlistin-depthcaseknowledgetomakeeducatedguessesaboutthesecounterfactualstheycanbeintegratedintotheQCA.Specifically,inordertoassertthatGNP*GOVSTABcomprisea345VaryingapproachestocausalinferenceDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:38:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.016Cambridge Books Online © Cambridge University Press, 2012 sufficientconditionfordemocracysurvival,wemustpresumethatallofthefollowingconfigurationswouldalsoleadtodemocraticsurvival,eventhoughtheyarenotobservedintheavailabledata:1.DEVELOPED*urban*literacy*indlab*GOVSTAB2.DEVELOPED*urban*literacy*INDLAB*GOVSTAB3.DEVELOPED*URBAN*literacy*indlab*GOVSTAB4.DEVELOPED*URBAN*literacy*INDLAB*GOVSTAB5.DEVELOPED*URBAN*LITERACY*indlab*GOVSTABInsofarastheseseemlikereasonableassumptions,baseduponwhatweknowaboutthecasesandabouttheworld,theargumentisbolstered.fs-QCAThefuzzy-set(fs)versionofQCAismorecomplicatedthanthecrisp-set(cs)version,whichiswhyitisintroducedithere,ratherthanearlier(eventhoughmanyQCApractitionersconsiderittobeasuperiorversionofthemethod).Tobeginwith,casesdonotneedtobecodedinaplainlycategoricalfashion.Acasemayoccupyapositionoffullorpartialmembershipinaset,codedfrom0to1–with0.0representingfullnonmembership,1.0representingfullmembership,and0.5representingthecut-offpointinbetweenthetwocategories.Followingourexemplar,theoutcome–democraticsurvival–maybere-scoredonthebasisofthePolity2variabledrawnfromthePolityIVindex,atwenty-one-pointindexstretchingfrom−10to+10,where0isdefinedasthecut-offbetweendemocracyandnondemocracy.Thus,Austria,withaPolity2scoreof−9,iscodedas0.05ontheoutcome–representing5percentmembershipinthecategory“democracy.”Bycontrast,Belgium,withascoreof10onthePolity2scale,iscodedas0.95onsurvival–95percentmembershipinthecategory.Similarre-codingsareconstructedforothervariables:developed;urbanization;literacy;industriallabor;andgovernmentstability.ThreeoftheseconditionsarerepresentedinTable12.3bywayofillustration.Theprocessofreducingthisinformationinto(possiblycausal)configura-tionsoccursthroughtheapplicationoftworules.Whencombiningmultipleconditionsintoasingleconfiguration(logicalAND),themembershipofeachcaseisdeterminedbytheminimalscoreacrossallfactors.Thus,Austria’smembershipintheconfiguration“DevelopedANDUrban”is0.12becausethisisthelowestscoreitreceivesacrossthetwoconditions,ascanbeseeninTable12.3.Itsmembershipintheconfiguration“Developed,Urban,AND346PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:38:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.016Cambridge Books Online © Cambridge University Press, 2012 Unstable”isalso0.12becausethisisthelowestscoreitreceivesacrossthethreeconditions.Whencodingalternativepathstoanoutcome(causalequifinality,imple-mentedbythelogicalOR),themembershipofeachcaseisdeterminedbythemaximalscoreacrossallfactors.Thus,Austria’smembershipintheconfig-uration“DevelopedORUrban”is0.81becausethisisthehighestscoreitreceivesacrossthetwoconditions,ascanbeseeninTable12.3.Itsmember-shipintheconfiguration“Developed,Urban,ANDUnstable”is0.89becausethisisthehighestscoreitreceivesacrossthethreeconditions.Table12.3includesonlythreeofthefivecausalfactorsinourexampleandonlytwoofthepossibleconfigurationsacrossthosethreeconditions.ButitisTable12.3Codingmembershipincausalfactorsandconfigurationswithfs-QCAConditionsConfigurationsOutcomeLogicalAND(setintersection)LogicalOR(setunion)CaseDevelopedUrbanUnstableDevelopedANDurbanDeveloped,urban,ANDunstableDevelopedORurbanDeveloped,urban,ORunstableSurvivalAU0.810.120.570.120.120.810.810.05BE0.990.890.020.890.020.990.990.95CZ0.580.980.090.580.090.980.980.89ES0.160.070.090.070.070.160.160.12FI0.580.030.420.030.030.580.580.77FR0.980.030.050.030.030.980.980.95GE0.890.790.690.790.690.890.890.05GR0.040.090.570.040.040.090.570.06HU0.070.160.870.070.070.160.870.42IR0.720.050.050.050.050.720.720.92IT0.340.100.420.100.100.340.420.05NE0.981.00.010.980.011.01.00.95PL0.020.171.00.020.020.171.00.12PO0.010.020.990.010.010.020.990.05RO0.010.030.160.010.010.030.160.21SP0.030.300.800.030.030.300.800.06SW0.950.130.090.130.090.950.950.95UK0.980.990.020.980.020.990.990.95AU:Austria,BE:Belgium,CZ:Czechoslovakia,ES:Estonia,FI:Finland,FR:France,GE:Germany,GR:Greece,HU:Hungary,IR:Ireland,IT:Italy,NE:Netherlands,PL:Poland,PO:Portugal,RO:Romania,SP:Spain,SW:Sweden,UK:UnitedKingdom.Codingexplainedinthetext.BasedonRagin(2009:97–98).347VaryingapproachestocausalinferenceDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:38:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.016Cambridge Books Online © Cambridge University Press, 2012 sufficienttoillustratethelogicbywhichalternativeconfigurationsareformedwithfuzzy-setcoding.Nowweturntotherelationshipbetweenconfigurationsandtheoutcomeofinterest.Thisrelationshipisdeemedoneofsufficiencywhenthescoreofacaseforaparticularconfigurationislessthanitsscoreontheoutcome.Supposewewanttotesttherelationshipoftheconfiguration“Developed,Urban,ANDUnstable”(column6inTable12.3)withtheoutcome(thelastcolumninTable12.3).ThisdatacanbegraphedinanX/Yscatter-plottotestrelation-shipsofsufficiency.Iftheconfigurationisasubsetoftheoutcome(andthereforeapossiblesufficientcondition)thenthecasesshouldlieabovethediagonal;thatis,thescoreforeachcaseontheoutcomeshouldbehigherthanthescoreforeachcaseontheconfiguration.AsonecanseefromFigure12.1,thisisonlypartiallyso.Thereisoneverydeviantcase:Germany.000.20.40.6Survival0.81UKBEFRSWNEIRCZFIHUROPLESAUITSPGRPO0.20.40.6GEDeveloped AND urban AND Unstable0.80.1Figure12.1Relatingconfigurationstooutcomeswithfs-QCA348PartIIICausationDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:38:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.016Cambridge Books Online © Cambridge University Press, 2012 Inordertotestallpossibleconfigurationsonesuggestedtechniquerevertstothetruth-tableapproachweexploredearlier.Casesarere-codedinabinaryfashionaccordingtotheirdegreeofmembershipinthecausalcondition,using0.5asthethresholdofmembership.Whileeachcasehaspartialmembershipinalllogicallypossiblecombinationsofconditions(akatruth-tablerows)ithasamembershipofhigherthan0.5inonlyoneofthem.Thisallowsfortheconstructionofatruth-table–representedinTable12.4–thatisidenticalincausalconditionstotheonepresentedinTable12.2.Theadditionalfeatureassociatedwithfuzzy-setanalysisisthecalculationofaconsistencyscore–thedegreetowhicheachofthecasesinthesamplefittheproposition–asreportedinthefinalcolumnofTable12.4.Theoutcomevalueforeachofthetruth-tablerowsisdeterminedbyrunningtestsoftheirconsistency(withthecasesinthesample)assufficientconditionsfortheoutcome.Iftheypassthistest,theyreceivethevalueof1intheoutcomecolumn;iftheyfailthetest,thevalueis0.Thethirdpossibilityisthatnotenoughcaseshaveamembershiphigherthan0.5inaparticularrow.Suchrowsaretreatedaslogicalremainders.Thegeneralformulaformeasuringtheconsistencyofasufficientconditionis:ðminðXi;YiÞÞ=ðXiÞ(12:1)Table12.4fs-QCAtruth-tablewithconsistencyscoresCausalfactorsConfigurationCaseswith>0.5membershipDevelopedUrbanLiteracyIndustriallaborGovernmentstabilityConsistencyassufficientcondition1.BE,CZ,NE,UK111110.902.FI,IR101010.803.FR,SW101110.714.ES001010.535.HU,PO001000.526.GE111100.457.AU101100.388.IT,RO000010.289.GR,PL,SP000000.22AU:Austria,BE:Belgium,CZ:Czechoslovakia,ES:Estonia,FI:Finland,FR:France,GE:Germany,GR:Greece,HU:Hungary,IR:Ireland,IT:Italy,NE:Netherlands,PL:Poland,PO:Portugal,RO:Romania,SP:Spain,SW:Sweden,UK:UnitedKingdom.349VaryingapproachestocausalinferenceDownloaded from Cambridge Books Online by IP 14.139.43.12 on Tue Oct 09 05:38:25 BST 2012.http://dx.doi.org/10.1017/CBO9781139022224.016Cambridge Books Online © Cambridge University Press, 2012
Thismeansthatonechooses,foreachcase,thelowervalueofX(themember-shipscoreofacaseinaconfiguration,asexplainedinTable12.2)andY(themembershipscoreofacaseontheoutcome),dividedbyX.Positivedeviance–whenY>X–amountsto0.Negativedeviance–whenYX1(probably)whenX1=1.Thus,withrespecttothecriterionofprecision(Chapter4),someversionsofQCAaremoreinformativethanothers.(7)Itissometimesarguedthattheintroductionoffuzzysets(fs-QCA)solvesproblemsassociatedwiththeoriginalcrisp-setversionofQCA(cs-QCA).Theobviouspointhereisthatthecodingoffactorsasfuzzy-setsisappropriateinsofarastheoryanddatawarrantit.Thatis,iftherearestrongreasonstosupposethatacausalrelationshipbetweeninterval(orratio)factorsisdiscontinuousinnature,andonecanintuitwhatthosebreak-pointsare,thenafuzzy-setcodingiswarranted.If,ontheotherhand,onecanneitherintuitnordiscover(empirically)plausiblebreak-pointsinanintervalscalethentheuseoffuzzysetsdoesnotsolvetheproblemofcoding.Imposedset-relationsarestillarbitrary–thoughperhapssomewhatlessarbitrarythancrisp-setcoding.(8)CausalinferencesbasedonQCAarevulnerabletospecificationerrors–theinappropriateinclusionorexclusionofcausalconditions/variables.46Suppose,forexample,thatdemocraticsurvivalintheinterwarerawasaffectednotonlybyGNPandgovernmentstabilitybutalsobyinequality.Theinclu-sionofthisnewcausalfactorwillintroducenewpathways(configurations)toY.Alternatively,supposethatoneoftheincludedfactorsturnsoutnottohaveacausaleffectondemocraticsurvival.Ifthisfactorisanelementofanidentifiedconfiguration–say,GNP–thennaturallythewholeargumentisfallacious.Ifitis,instead,understoodasabackgroundfactor–say,urbaniza-tion–thenithasnoeffectontheresults.InthisrespectQCAisnotfundamentallydifferentfromotheranalytictechniques.Thatsaid,oneimportantcontrastdeservestobeflagged.Inmostanalytictechniquesthespecificationproblemhingesonwhethertheincludedoromittedfactoriscorrelatedwiththetreatment(thecausalfactoroftheore-ticalinterest).Ifitisnot–asinallexperiments(ifpro